Edge AI Accelerators (Edge / On-Device)

Inference silicon that is always-on, low-power, on-device — distinct from data center ASICs (gpus-other-asics.md). Here the budget is in milliwatts to a few watts, not hundreds of watts, and the goal is to run the entire model on the device, with no round-trip to the cloud. A category relevant to Koder: it connects with the small edge models (Liquid LFM, Tsetlin) in ../02-architectures/alternative-architectures.md and with the self-hosted-first philosophy (inference with no external dependency).

Google Coral NPU (2025) — full-stack, open-source, RISC-V platform

Announced by Google Research (Oct/2025) as a full-stack open platform for always-on AI on ultra-low-power devices (wearables, hearables, AR glasses, smartwatches, ambient sensing).

Spec Value
ISA RISC-V (open, extensible)
Performance (base) ~512 GOPS
Consumption a few milliwatts (battery-powered)
Architecture Scalar core (C-programmable RISC-V frontend, "run-to-completion" model) + Matrix execution unit (quantized outer-product MAC engine, dedicated to neural-network ops)
  • Open and extensible: SoC designers can modify the base design or use it as

    a pre-configured NPU — unlike closed IP (Hexagon, ANE).

  • Released software: the TFLM (TensorFlow Lite Micro) compiler, the

    MLIR compiler, a simulator, and custom kernels.

  • Co-design with Gemma: optimized together with the Gemma team for *mall

    transformers* ensuring the architecture supports GenAI at the edge.

  • SoC adoption: integrated into the Synaptics SL2610 / Astra Edge AI SoCs.

CoralBoard (Google I/O 2026) — dev board running Gemma 3 on-device

Presented by Google Research + Synaptics at Google I/O 2026 (May/2026). Successor to the Coral Dev Board line, now focused on multimodal GenAI at the edge. *(Origin of this entry: the video "Run Gemma on the edge with the Coral Board", Google for Developers channel, Jun152026 — verified against Synaptics + CNX Software + Google Developers Blog.)*

Spec Value
SoC Synaptics Astra SL2619 Edge AI (25×25 mm, LGA178)
NPU integrated Coral NPU, 1 TOPS
Demo model Gemma 3 270M, compressed to fit in 2 GB RAM, 100% on-device
I/O kit display, camera, microphones, LED
Open source demos available on GitHub
Availability limited edition at the I/O 2026 Gemma Pavilion; GA "later this year" (2026)

Demos shown (everything running on the board alone, no cloud):

  1. Voice translation — speech in, translate, speech out (on-device speech-to-speech).
  2. Action / physical control — natural language controlling physical hardware.
  3. Multimodal creative — vision + sound together; music generation from what

    the board sees and hears (a lightweight version of the pre-I/O show with a jellyfish aquarium).

Why it matters for the compendium: it is the first convincing public proof of a generative transformer LLM running entirely on a 1 TOPS edge chip without cloud. The historic edge bottleneck was fitting the model — Gemma 3 270M + a co-designed NPU + aggressive quantization (2 GB) closes the gap. It marks the transition from "edge = CNN/vision only" to "edge = multimodal GenAI".

Siblings — other edge accelerators (overview)

For context; they were not evaluated in depth here, only cataloged.

Accelerator Manufacturer Perf (~) Notes
Edge TPU (legacy Coral) Google 4 TOPS Previous generation, CNN/vision focus (TFLite int8); no GenAI
Jetson Orin Nano NVIDIA up to ~67 TOPS Ampere GPU + CUDA; the most "GPU-like" of the edge; runs small LLMs
Hailo-8 / Hailo-10 Hailo 26 / 40 TOPS Dataflow architecture; Hailo-10 targets on-device GenAI
Rockchip NPU (RK3588) Rockchip ~6 TOPS ARM SoC popular in SBCs; RKNN toolkit
Apple Neural Engine Apple tens of TOPS Closed IP; Core ML; runs on-device iOS/macOS models
Qualcomm Hexagon NPU Qualcomm tens of TOPS Closed IP; Snapdragon; QNN SDK; foundation of the "AI PC"/mobile

Coral NPU's differentiator vs. the siblings: the others (except Edge TPU/Jetson) are closed IP; the Coral NPU is the only one that is open-source + RISC-V — aligned with the Koder preference for an open, auditable stack.

Connections in the compendium