Neuromorphic computing

Lately I’ve been giving some thought on quantitative measures for the brain’s processing power. Let us take the number of brain neurons as $10^{11}$ and synapses as $10^{14}$, according to wikipedia. I am going to assume a very simplified computational model for a neuron:

$$y_i = \phi\left( \sum_j w_{ij} x_j \right)$$

With $\phi$ an activation function such as a step function or a sigmoidal function. So every time a neuron spikes, a computation is being performed. From our previous numbers, the sum in the $j$ has around $10^3$ components on average, which means there are one thousand additions and one thousand multiplications performed per spike! Let’s consider each of these a floating point operation on a computer. Then we have 2000 flops (I will call flops the plural of a flop and flop/s flops per second, because the terminology is really confusing) per spike. Let us assume an action potential (a spike) for a neuron lasting 1ms as an upper bound. Then a neuron can spike up to 1000 times per second. Thus we have $1000*10^{11}*2000=2*10^{17}$ flop/s, or 200 petaflop/s.

That is only an order of magnitude higher than the fastest supercomputers at the time of writing. It seems at least a first approximation to the brain would be achievable in realtime in the next few years correct? Not with the current architectures. An obvious oversight such a simplistic calculation makes is the concurrency of the calculation. The 2000 calculations for a given spike are performed simultaneously in a neuron, whereas a traditional von neumann architecture would need to do these calculations in steps: first perform all the multiplications, then sum all the results to a single value. Even a massively parallel architecture would need one clock cycle for the multiplications, and 10 clock cycles to integrate all values (by summing them in pairs you need $log_2 (1000)$ clock cycles). The number of flop/s is the same, but you need to run your machine 11 times faster, which destroys power efficiency (you don’t want your brain consuming megawatts of power).

An even greater problem is the memory bandwidth. At each clock cycle, you need to move $10^{11}$ numbers to your computational cores, compute their new values and move them back. If each is a double we are in the order of 800GB/s each way just for the main operation (i.e. not counting any temporary variables for the sum reduction above discussed), which does seem out of reach of current supercomputers, and also not very power efficient.

The brain is not affected by these problems since the memory is an integral part of the computational infrastructure. In fact synaptic weights and connections are both the software and the memory of the brain’s architecture. Of course, the way information is encoded is not very well understood, and there are likely many mechanisms to do so. The wikipedia page on neural coding is quite interesting. In any case it is clear that synaptic weights are not enough to fully describe the brain’s architecture, as glial cells might also play a role in memory formation and neuronal connectivity. However, spike timing dependent plasticity (STDP, associated with Hebb’s rule) seems to be an adequate coarse grained description of how synapse weights are determined (at the time of writing).

With this in mind, memristors seem to be an appropriate functional equivalent to our coarse grained description of the brain. In a memristor, resistance is proportional to the intensity of the current that flows through it. Thus you can engineer a system where connections which are often used are reinforced. By combining memristive units with transistors, it is in principle possible to create an integrate and fire unit similar to a neuron. A device to emulate STDP could also be implemented. The biggest hurdle seems to be connection density. In a planar implementation, only 4 nearest neighbor connections can be implemented straightforwardly. To reach an order of 1000 connections (not necessarily with nearest neighbors) per unit, 3D structures will need to be used. At the current time however there are no promising techniques to enable the reliable construction of such structures. I foresee that self assembly will play a large role in this field, again taking heavy inspiration from the way nature does things.

In spite of these hurdles, I am excited. With progress in science getting harder each year, the only way to continue to discover nature’s secrets will be to enhance our cognitive capabilities be it through biological or electrical engineering.

Rays for android

Another platform I had a go at porting the rays app for was android. This was probably the least enjoyable port I’ve done, not because of the Java language but because there is essentially no decent library to do the kinds of 2D effects the app requires.

I researched a bit on the topic and there were three ways to go about doing this:

  • Use android’s own openGL implementation.
  • Use the NDK to write native code which draws directly to the framebuffer.
  • Use libGDX.

At the time (well over a year ago, before the fancy new android developers redesign) documentation for the first two options was absolutely incomprehensible. Even now I find using the openGL implementation in android quite confusing, although it might be my fault as I never really learned low level openGL. I now feel tempted to use the much improved NDK and do some low level pixel manipulation, but other projects take priority…

I ended up using libGDX. It was also poorly documented (some random wiki pages + code comments) but the API made a lot more sense to me. This is because it was in the vein of some 2D libraries for the NDS such as the venerable PAlib and uLibrary which I used extensively previously (in hindsight, it appears they were not the best libraries in terms of code quality and the simple libNDS would have been a much better choice). The fact that you could instantly play the game on your computer natively without needing an emulator also helped. Though I would recommend just plugging in your android device and debugging directly there, Android’s debugger is truly excellent.

The library was chosen, the code was banged out, et voila! It lives! No antialiasing, no funky glow effects but it lives!

Rays app running on Galaxy S2
Rays app running on Galaxy S2

One idea I had to make the app more interesting would be to synthesize some musical instrument, something like a violin, where each instrument would be represented by a ray, with the intensity of the sound proportional to its speed. I could not find any satisfactory samples though, since they all come as single notes and I’d rather have some continuous sound produced. It might be more interesting to use some sort of synthesis algorithm to produce an interesting sound and modulate its amplitude, frequency or harmonics with the rays. How to go about it remains a mystery  There aren’t too many synthesis libraries available for android that do what I want. Libpd seems to be a decent option, now I just need to find the time to finish this project.

Since the application has only been tested on a handful of phones, I never did toss it on the play store. Get it here if you want to play with it. You’ll have to allow installation of non market apps.