AI Compendium — Koder

Reference knowledge base for the development of Koder's AI (Kode). Compiled in April 2026. Update as new milestones emerge.

Structure

Directory Content
01-fundamentals/ Paradigms, books, foundational papers, theory (pre-Transformer), AGIASIAIXI
02-architectures/ Transformer, attention, MoE, SSMs, alternative architectures
03-models/ Complete catalog of LLM families (open-source and proprietary)
04-training/ Pre-training, post-training (SFTDPORLVR), datasets, synthetic data
05-inference/ Quantization, speculative decoding, KV cache, servers
06-hardware/ NVIDIA/AMD GPUs, ASICs, edge accelerators, HBM memory, interconnect, energy, cooling
07-frameworks/ PyTorch/JAX, distributed training, fine-tuning, agents, observability
08-benchmarks/ General benchmarks, code, agents, multimodal, sciences
09-applications/ Life, physics, mathematics, vision, audio, video, 3D, world models
10-security/ Alignment, interpretability, red teaming, safety
11-infrastructure/ Self-hosted cluster, MLOps for large models, GPU orchestration

Complete File Index

01-fundamentals

  • paradigms.md — Predictive vs generative, formal discriminative vs generative, supervision regimes, cross-map with the timeline
  • books.md — Essential ML/AI books (Goodfellow, Bishop, Murphy, Sutton & Barto)
  • foundational-papers.md — 60+ papers by era: 1943–2026
  • agi-asi-superintelligence.md — AGIASIAIXI definitions; From AGI to ASI paper (DeepMind 2026, arXiv 2606.12683); 4 paths, 6 frictions, limits of superintelligence

02-architectures

  • transformer-and-attention.md — Transformer, MQA, GQA, MLA, NSA, CSA+HCA, MSA, FlashAttention, RoPE, MoE
  • alternative-architectures.md — SSMs (Mamba), xLSTM, RWKV, Liquid (LTCCfCLFM), JEPA, dLLMs (language diffusion — DiffusionGemma), hybrids
  • alternative-paradigms.md — Neuro-symbolic (AlphaGeometry, DreamCoder, ∂ILP), Tsetlin Machines, HDCVSA, Forward-Forward, Predictive Coding, EBMModern Hopfield, Active Inference
  • multimodal.md — Flamingo, Chameleon, LLaVA-NeXT, cross-modal training

03-models

  • open-source.md — Llama 4, Qwen3.x, DeepSeek V4, Mistral Small 4, Gemma 4 + DiffusionGemma, Phi-4, Kimi K2.6, MiniMax M3, Nemotron 3, OLMo Hybrid, Command A
  • proprietary.md — GPT-5.x, Claude Mythos, Gemini 3.1 Pro, Grok 4.3, MAI (Microsoft, Build 2026); SWE-bench and Arena Elo

04-training

  • pre-training.md — Scaling laws, curation, parallelism, ZeRO, FSDP
  • post-training.md — SFT, DPO, GRPO, RLVR, PRMs, test-time compute
  • datasets.md — FineWeb, RedPajama, The Stack, MATH, UltraChat and more
  • synthetic-data.md — Self-Instruct, Alpaca, Orca, Phi Textbooks, MetaMath
  • data-pipeline.md — Crawler, dedup (MinHash/SimHash), quality filters, PII scrubbing
  • economics.md — Cost by scale (7B30B70B), build vs fine-tune vs API, roadmap

05-inference

  • quantization.md — GPTQ, AWQ, GGUF, BitsAndBytes, TurboQuant, BitNet, NVFP4
  • speculative-decoding.md — EAGLE-123, P-EAGLE, Medusa, Lookahead, MTP, QuantSpec
  • kv-cache.md — PagedAttention, RadixAttention, H2O, SnapKV, MLA, TurboQuant
  • servers.md — vLLM, SGLang, TGI, llama.cpp, Ollama, TensorRT-LLM, MLC-LLM

06-hardware

  • gpus-nvidia.md — A100, H100, H200, B100B200B300, GB200 NVL72, RTX 4090/5090
  • gpus-other-asics.md — AMD MI300XMI355XMI450, Google TPU v4–v7 IronwoodTPU 8t8i, AWS Trainium3, Intel Gaudi, Groq, Cerebras
  • edge-accelerators.md — Edge/on-device: Google Coral NPU (RISC-V, open-source) + CoralBoard running Gemma 3 270M on-device; Edge TPU, Jetson Orin Nano, Hailo, Rockchip NPU, Apple ANE, Qualcomm Hexagon
  • cpu-x86-extensions.md — AVX-512 VNNI, AMX, AVX10, ACE (AMD+Intel white paper, Apr/2026), x86 Ecosystem Advisory Group
  • memory-interconnect.md — HBM 1–4e, GDDR7, NVLink, NVSwitch, InfiniBand, PCIe, CXL
  • neuromorphic.md — Intel Loihi 2, IBM NorthPole, SpiNNaker 2, BrainScaleS-2, InnateraSynSense; SNN training (surrogate gradients, ANN-SNN, STBP); snnTorchNorse/Lava
  • energy-cooling-resources.md — Energy consumption, PUE, WUE, critical minerals, sustainability

07-frameworks

  • distributed-training.md — PyTorch, JAX, DeepSpeed, FSDP, Megatron-LM, ColossalAI
  • fine-tuning.md — PEFT, LoRA, QLoRA, TRL, Axolotl, LLaMA-Factory, Unsloth
  • local-inference.md — Local setup: Ollama, llama.cpp, vLLM, editor integration
  • agents.md — L1–L5 autonomy taxonomy (DeepSeek survey) + 4 architectural patterns + 6 open problems; LangChain, LangGraph, DSPy, smolagents, Pydantic AI, Strands, OpenAI Agents SDK, Claude Agent SDK
  • protocols.md — MCP, A2A, ACP, AGNTCY, NLIP, OpenAI-compat, Bedrock Converse, Computer Use, AP2, x402, OTel GenAI semconv, OASF
  • observability.md — LangFuse, Phoenix, W&B, MLflow, production metrics, OTEL

08-benchmarks

  • general-reasoning.md — MMLU, ARC, GSM8K, MATH, AIME, Humanity's Last Exam, GPQA, ARC-AGI-2
  • code.md — HumanEval, SWE-bench Verified, LiveCodeBench, BigCodeBench, RepoBench
  • agents-long-context.md — GAIA, τ-bench, WebArena, OSWorld, RULER, HELMET, NIAH, GDPval, SWE-Bench Pro, Agent-SafetyBench
  • multimodal-sciences.md — MMMU, DocVQA, MedQA, LegalBench, Video-MME, ChatBot Arena
  • continuous-evaluation.md — Private eval loop, LLM-as-a-judge, canary strings, contamination, drift

09-applications

  • life-sciences.md — AlphaFold 23, ESM-123, Evo 12, GNoME, drug discovery
  • physical-sciences-mathematics.md — GraphCast, AlphaProof, FermiNet, plasma fusion, quantum ML
  • vision-image.md — ViT, CLIP, SAM 1/2, Stable Diffusion, FLUX.1, DALL-E 3, VLMs
  • audio-speech.md — Whisper, Moshi, F5-TTS, EnCodec, MusicGen, GPT-4o Realtime
  • video-3d-world-models.md — Sora 2, Veo 3, HunyuanVideo, NeRF, Gaussian Splatting, DreamerV3, Genie 2, Cosmos
  • medical-imaging.md — MRIDICOMneuroimaging: FreeSurferFastSurferSynthSeg, nnU-NetTotalSegmentator, MedSAM2BiomedParseVISTA3D, MedGemmaMedImageParse, NeuroQuant/icobrain (clinical), TopCoW (MRA); empirical meningioma PoC

10-security

  • interpretability.md — SAEs, circuits, features, Gemma Scope, TransformerLens, MIT 2026, emotion vectors in Claude
  • alignment.md — Constitutional AI, RLHF, Constitutional Classifiers, red teaming, jailbreaks, HarmBench, EU AI Act (full applicability Aug/2026)

11-infrastructure

Quick entries — the most important

Unavoidable papers

  • Attention Is All You Need (2017) — arXiv:1706.03762 — the basis of everything
  • Scaling Laws (Kaplan 2020) — arXiv:2001.08361
  • Chinchilla (Hoffmann 2022) — arXiv:2203.15556 — compute optimal
  • DeepSeek-R1 (2025) — arXiv:2501.12948 — large-scale RLVR
  • TurboQuant (Google/ICLR 2026) — arXiv:2504.19874 — 6× KV cache compaction

Priority open models for Kode

  • Qwen2.5-Coder-32B — best size/performance ratio for code
  • DeepSeek-V3/R1 — frontier reasoning, MIT license
  • Llama 4 Scout — 10M context; native multimodal MoE; Llama License
  • Kimi K2.6 — open SWE-Bench Pro leader; 256K; agentic coding
  • Gemma 4 31B — best open per parameter; Apache 2.0; multimodal

Kode's development stack

  • Fine-tuning: Axolotl + TRL + Unsloth
  • Inference: vLLM (production) + SGLang (alternative)
  • Eval: lm-evaluation-harness + SWE-bench runner
  • Observability: Weights & Biases + LangFuse

Last updated: 2026-05-22 (added alternative-paradigms.md + neuromorphic.md + protocols.md; expanded alternative-architectures.md with Liquid NN and JEPA; Layer D added to /k-ia-compendium to keep alternative research lines up to date; split protocols out of the compendium into a dedicated file)

2026-06-04 (Microsoft Build 2026, 06/02): the *MAI (Microsoft AI) family in proprietary.md — MS's first own foundation models, trained from scratch without distillation; Autopilots / always-on agents with governed identity pattern (Scout/Entra) in agents.md; Microsoft IQ / Web IQ*(MCP-native grounding) in protocols.md. Source: "AI Revolution" video verified against primary Microsoft blogs.)*

2026-06-17: new *dLLM (language diffusion) paradigm in 02-architectures/alternative-architectures.md — dedicated section + a row in the comparative table, anchored on DiffusionGemma*(Google DeepMind, 0610, 26B-A4B MoE Gemma 4, discrete diffusion, 256-token blocks in parallel, 1000+ toks H100 / ~4× vs AR, Apache 2.0); corresponding entry in the 03-models/open-source.md catalog (Gemma family). Source: official post from the "Google for Developers" channel verified against ai.google.dev, DeepMind, and the model card on Hugging Face.*

2026-06-16: new 01-fundamentals/agi-asi-superintelligence.md — *AGIASIAIXI definitions, the From AGI to ASI*paper (DeepMind, arXiv 2606.12683, 06/10), 4 paths (scaling / paradigm shift / recursive self-improvement / multi-agent collectives), 6 frictions and physical limits of superintelligence; corresponding entry in foundational-papers.md (Era 11). Source: "AI Revolution em Português" video verified against arXiv and primary coverage (deepmind.google, alphaXiv).*