Neuromorphic Hardware and Spiking Neural Networks

Brain-inspired computing: neurons fire spikes (events), asynchronous communication, dramatically lower consumption than a GPU for inference. State in 2026: mature research + niche commercial use (autonomous drones, always-on sensors, prosthetics).

Process note: this file is updated by /k-ia-compendium via Layer D (lines D11 and D12).

---

Why it matters

A GPU consumes ~300W for inference of a modest LLM. The human brain consumes ~20W with ~86B neurons. Neuromorphic hardware tries to harvest part of that efficiency via:

• Event-driven computing — only spends energy when there is a spike (sparse events)
• Memory next to computation (in-memory compute) — eliminates the von Neumann bottleneck
• Native massive parallelism — thousands of cores running asynchronously
• Low precision tolerated — analog/mixed-signal vs FP32

Trade-off: it can be 100-1000× more efficient in energy, but current algorithms (backprop on dense networks) don't fit naturally — hence the importance of SNN training methods and alternative algorithms like [[alternative-paradigms]] (Forward-Forward, Predictive Coding).

---

Neuromorphic Chips

Intel Loihi 2

• Launch: 2021; still canonical in 2026.
• Specs: 128 neuromorphic cores, 1M neurons per chip; up to 8M neurons per Kapoho Point system (8 chips).
• Process: Intel 4 (4nm).
• Programmability: customizable microcode per core (not only SNN — any local update rule).
• Framework: Lava (open source, github.comlava-nclava) — Python; Process / Channel abstractions.
• Access: Intel Neuromorphic Research Community (INRC) — academic + selected commercial; no commercial open-market chip yet.
• Cases: robotic olfaction, optimization (constraint satisfaction), real-time gesture recognition.

IBM NorthPole

• Announcement: Science 382, 329-335 (Oct/2023) — Modha et al.
• Specs: 256 cores; 224MB on-chip memory (no external DRAM for inference); 22B transistors in 12nm.
• Not pure SNN — inference-only chip for dense neural networks, but with memory-near-compute that eliminates data movement.
• Performance: ResNet-50 at 22× more efficient than H100 in energy, 5× less latency.
• Status 2026: still research silicon; no commercial product. Successor to TrueNorth (2014).

SpiNNaker 2

• University of Manchester + TU Dresden | launch 2024.
• Specs: 152 ARM Cortex-M4F cores per chip; target system of 10M chips = ~1B neurons.
• Differentiator: programmable digital (not analog) — flexible, supports any neuron model.
• Adoption: Human Brain Project (ended 2023) left a legacy; successors in Jülich and Manchester.

BrainScaleS-2

• Heidelberg University | 2nd gen 2020-2023.
• Mixed-signal analog/digital — analog neurons run 10,000× faster than biological ones (accelerated-time model).
• Adoption: computational neuroscience tool; less focus on applied ML.

Others (2026 panorama)

• Innatera Pulsar — Netherlands, startup; SNN for always-on sensor fusion in wearables.
• GrAI Matter Labs GrAI VIP — event-driven vision processing; integration with event cameras (Prophesee).
• SynSense Speck/Xylo — Switzerland, audio + vision always-on; ~µW.
• Mignon AI — Tsetlin-native (see [[alternative-paradigms]]); not SNN but same "post-GPU" family.

---

Spiking Neural Networks: Neuron Models

Leaky Integrate-and-Fire (LIF)

Standard model: the membrane accumulates input, fires a spike when it crosses the threshold, resets. Simple, efficient, dominant in modern SNN training.

Adaptive LIF / AdEx

Adds adaptation (threshold or current) — models biological accommodation.

Hodgkin-Huxley

Complete biophysical model (4 ODEs per neuron). Too realistic for ML — used in neuroscience.

Izhikevich

Trade-off between LIF and Hodgkin-Huxley: 2 ODEs, reproduces 20+ types of neuronal behavior.

---

Training of SNNs

Spikes are discontinuous → gradient not defined → you can't backprop directly. Modern solutions:

Surrogate Gradients

• Neftci et al. (IEEE SPM 2019) — replace the step function (Heaviside) with a smooth sigmoid only in the backward pass.
• Today it's the dominant method; it enables BPTT in SNN.

ANN-to-SNN Conversion

• Trains a normal ANN (ReLU) → converts activations into rate-coded spikes.
• Advantage: leverages all existing DL machinery.
• Limitation: latency (needs many timesteps to approximate rate); loses the appeal of event-driven.

Direct SNN Training with STBP

• Spatio-Temporal Backpropagation (Wu et al., Front. Neurosci. 2018) — BPTT with surrogate gradient.
• State of the art in accuracy + spike-count tradeoff.

Local Learning Rules

• Hebbian, STDP (Spike-Timing-Dependent Plasticity), Forward-Forward, Predictive Coding.
• Not competitive in accuracy today, but the only ones compatible with on-chip training in current neuromorphic hardware.

---

Software Frameworks

---

Applications where neuromorphic already wins

---

Applications where it does not win (yet)

• LLMs — dense, sequential, latency-bound; the GPU keeps dominating.
• Training at scale — no neuromorphic hardware capable of competing with an H100/B200 cluster.
• SOTA vision — ANN on GPU still ahead on ImageNet/COCO, even with ANN→SNN conversion.

---

For Kode

Short term: not actionable — no chip available for purchase + no internal use case that justifies the learning cost.

Medium term (24-36 months): if the Stack adds a wearable surface (Koru-watch, Hand glove, Eye glasses), neuromorphic becomes a serious candidate for:

• Permanent wake-word (see [[specsvoice/wake-word.kmd]])
• Always-on gesture recognition (Koder Hand)
• Eye gaze tracking (Koder Konsul XR variant)

To watch: Innatera Pulsar (commercial-friendly), SynSense (wearable deals), any LLM-in-spikes that shows parity with a 7B Transformer.

Cross-reference: [[alternative-paradigms]] covers learning algorithms (Forward-Forward, Predictive Coding) that fit this hardware.