Part V · 7 — Case study: building Kode
A worked example of everything in this part applied to a real product: Kode, the AI programming assistant of the Koder Stack. It shows the frontier roadmap (levers → data → fine-tuning → RLVR → agent) grounded in a concrete case.
🎨 Figure
F-V.7— Kode's flywheel. Brief: a flow wheel (data flywheel): Kode usage → accepts/rejects → preference data → fine-tuning → better Kode → more usage. Compendium palette.
7.1 Why programming is the right position
Programming is the best domain for a small team (verifiable reward, filterable data, short feedback — see 06-ai-for-code.kmd) plus an asset that generalist labs don't have: proprietary interaction data. Every Kode suggestion that is accepted, rejected, or edited is an "almost-free" RL preference signal — the same one that feeds Cursor/Codeium against billion-dollar labs.
7.2 Applied pipeline (follows the frontier roadmap)
| Phase | Action in Kode |
|---|---|
| 0 — Domain | Go, FlutterDart, TypeScript, NASM + KMDKoder Koda. Differentiator: languages with scarce coverage in public datasets (NASM, KMD) |
| 1 — Baseline | Measure ClaudeGPT-5Qwen2.5-Coder-32B on the Koder monorepo; private benchmark of 50–200 real problems |
| 2 — Maximum API | Before training: prompt engineering + RAG over the codebase + tool use (already beats smaller fine-tuned models) |
| 3 — Data flywheel | Collect acceptsrejectsedits; generate synthetics from the monorepo (bug→fix from real commits, code Q&A, architecture CoTs); filter by build+tests |
| 4 — Fine-tuning | SFT → DPO with dev feedback → RLVR with build/test as reward; start with QLoRA of 32B |
| 5 — Reasoning RLVR | Apply the DeepSeek-R1 paradigm to the Koder code (2× RTX 4090/5090, 2–4 weeks) |
| 6 — Multimodality | Vision of code/UI screenshots; architecture diagrams |
| 7 — Agent | Kode as an agent: full tool use, repository memory, ReAct loop + sandbox |
7.3 Kode's technical stack
| Component | Choice |
|---|---|
| Base model | Qwen2.5-Coder-32B (or DeepSeek-Coder-V2-Lite for iteration) |
| SFT | Axolotl / LLaMA-Factory · DPO/GRPO TRL · 1 GPU Unsloth |
| Inference | vLLM (high concurrency) |
| Sandbox | Docker with limits (never outside a sandbox) |
| Eval | SWE-bench runner + private Koder benchmark |
| Observability | W&B (experiments), LangFuse (production) |
7.4 Private benchmarks — the defensible advantage
| Benchmark | Content | Why it matters |
|---|---|---|
| Kode-bench-go | 50–100 Go problems from the monorepo | real bugsrefactorsfeatures with a known solution |
| Kode-bench-flutter | equivalent in Flutter/Dart | — |
| Kode-bench-kmd | KMD generation/editing | proprietary format — no public model has data |
| Kode-bench-agent | long-horizon agent tasks | issue → implement → PR → pass CI |
The KMD benchmark is the most defensible: no public model was trained on Koder Koda. It is where Kode can genuinely beat GPT-5/Claude Opus 5+ in the proprietary domain.
7.5 PT-BR
Comments and docs in pt-BR in the monorepo are valuable training data: Kode should reason in English but understand and generate pt-BR naturally. Voice: XTTS v2 / Fish Speech cover PT-BR. Light fine-tuning of pt-BR over a solid code base is enough — do not train separately.
7.6 The three central messages (valid beyond Kode)
- Domain matters more than technique. The most valuable problems are not about
architecture — they are where a domain expert + an AI engineer build something neither would build alone.
- Proprietary data is the defensible advantage. Models and compute become
commodities.
- Open models + fine-tuning + specific domain is the winning formula
for a small team. Pick a niche, master it, let the labs handle the foundation.