Operational Amplifiers and the Neuron

Spiking Neuron Models

Artificial neural networks are all the rage right now, but current state of the art focuses on training models through brute force - vast swathes of data are processed on computers the size of buildings. While these methods can achieve impressive results on common benchmarks like MLPerf, their efficiency in achieving those results far from optimal. Many of their drawbacks surround the use of the perceptron as their theoretical basis. Perceptrons work by summing all inputs then comparing them to some threshold function. These networks are easily described mathematically through linear algebra and can be trained by gradient decent. They are easily (though not necessarily efficiently) implemented on digital computer architectures. However, perceptions have long been shown to be biologically inaccurate, hinting at the limitations of models that rely on them.

Within biology, the Hodgkin-Huxley model acts the gold standard for describing neuron behavior. Unlike the perceptron, the Hodgkin-Huxley model is dynamic, relying on differential equations rather than the aforementioned sum and fire. This leads to far richer behavior that can be observed in networks with just a few nodes. However, being based on a complex differential equation, it doesn't play nicely with synchronous digital hardware. Enter the Fitzhugh-Nagumo model. Between the two, Hodgkin-Huxley model has more terms, and it's phase diagram lives in a four dimensional space, which is hard to visualize. The Fitzhugh-Nagumo model, on the other hand, makes some assumptions, reducing the number of terms, and allowing it's phase space to live in two dimensions, which is very easy to visualize. Both accurately describe how activation potential (electrical signals) get started and move along the axon of neurons; the terms that are absent in the Fitzhugh-Nagumo model serve less of a computational role, and are more of a biological necessity.  While the Fitzhugh-Nagumo model still isn't as conducive to digital hardware as the perceptron, but it pairs very well with operational amplifier based analog circuits - a technology long proven in Integrated Circuits. Overall, the Fitzhugh-Nagumo model offers a good compromise between richness of emergent behavior and practical implementation and can provide excellent foundation for the next generation of neural networks.

Deriving the Circuit

Dendrite

This gets complex, quickly. For now an external function generator and hand tuned potentiometers will have to do

Integration

The standard inverting operational amplifier integration circuit, with a leakage resistor and reset switch

Threshold

Operational-amplifier based inverting Schmitt trigger

(Complete diagram)

Spice Model and Simulation (Soma)

Note: Text color corresponds to signal color and bounding loop. All signals are shown at the same scale

An input signal (blue waveform), represents the output of a dendrite tree is simulated.  Shown above is simple square wave, but this signal can be arbitrarily complex. That input is then fed into the into the operational amplifier leaky-integrating circuit (red waveform). The output node of the integration circuit connects to an inverting amplifier (teal waveform). Now, the signal is feed into a Schmitt trigger (magenta waveform). When the value of the signal exceeds its upper threshold, the Schmitt trigger will fire (dramatically change its output voltage), and it will only return to it's previous state once the input signal falls below it's lower threshold. In order to do this, the circuit must have some feedback element to reset the integrator, hence the reset switch. However, that MOSFET introduces a problem. Transistors have an inherent gate-to-source threshold voltage, meaning that integrator could not fully empty itself. To overcome this, a summing operational amplifier (black dotted bounding loop) is added to compensate for the the transistor gate-to-source threshold voltage. Finally, there is a transistor that serves as a both a NOT gate and buffer, isolating the Schmitt trigger output/feedback node.

Hardware Implementation

Structure

Peripherals

IC's

Discrete Components


Diagram bridging the gap between schematic and breadboard

Breadboard Circuit, Note: Va +5v, Vb -5v

Waveforms: Input (blue), Integration (green) and Output (yellow) 

Demonstration

Note the resilience to noise! I didn't expect this, but in hindsight it makes sense - it direct result of the Integrator and Schmitt Trigger

What next?

Well, a lot! Hopefully enough to move from this from a toy project to a useful education and research tool.

Soma circuit

Dendritic tree

Software upgrades

Hardware upgrades (to facilitate the study of networks)

Networks

Possible dendrite structure allowing a single neuron the faculty for complex logic

 Information flow diagram (top) and interconnectivity diagram (bottom)