Part IV · Ch. 01 — LLM (Large Language Model)

draft

Connectionist · Language/conversation · Transformer (decoder). The kind of AI that defined the current era: a network that learns to predict the next token and, at sufficient scale, exhibits language, reasoning, code and the ability to use tools. Origin card: ../02-types-of-ai/02-connectionist.kmd.

🎨 Figure F-IV.1.0 — Anatomy of an LLM. Brief: stylized cross-section of a Transformer: tokens coming in → embeddings + position → stack of blocks (multi-head attention + FFN, with residual connections) → projection → distribution over the vocabulary. Attention arrows linking tokens. Compendium palette.

Anatomy of an LLM

1. Definition and short history

An LLM is a neural network — almost always a Transformer decoder — trained on huge text corpora with a simple objective: given a prefix, predict the next token. From this self-supervised task, at scale, capabilities that were never explicitly programmed emerge: grammar, facts, translation, step-by-step reasoning, programming and dialogue.

Lineage (detail in Part III): n-gram models → word embeddings (word2vec, 2013) → seq2seq + attention (2014) → Transformer (2017) → GPT and the pre-training + scale paradigm (2018–2020) → instruction tuning and RLHF (2022) → multimodal, reasoning and agentic models (2023–present).

2. Foundations (sciences and mathematics)

Discipline Role in the LLM
Linguistics Distributional semantics ("the meaning of a word lies in the company it keeps"); morphology (tokenization); emergent syntax/pragmatics
Linear algebra Everything is a tensor; attention is the matrix product softmax(QKᵀ/√d)·V; embeddings are vectors
Probability The output is a distribution `p(token context)`; sampling controls generation
Information theory Cross-entropy/perplexity as loss and metric; compression as a proxy for learning
Calculus / optimization Backprop + AdamW tune billions of parameters
Learning theory Scaling laws: loss decreases predictably with parameters × data × compute
Neuroscience Loose inspiration (neuron, attention) — [metaphor] for "understanding"

Anchor formula — attention: Attention(Q,K,V) = softmax( Q·Kᵀ / √dₖ ) · V Each token "looks" at the others with learned weights — the central mechanism.

3. Algorithms and architectures

Transformer block (decoder), repeated N times:

  1. Tokenization — text → tokens (subwords) via BPE/SentencePiece.
  2. Embedding + position — vector per token + positional encoding (today

    RoPE is common).

  3. Causal multi-head self-attention — each position attends to the previous

    ones (causal mask); several "heads" capture distinct relations.

  4. Feed-forward (FFN/MLP) — per-position transformation (where much of the

    "knowledge" lives).

  5. Residual connections + LayerNorm — stabilize the training of deep networks.
  6. Final projection + softmax — distribution over the vocabulary.

Architecture variants and optimizations:

  • Mixture-of-Experts (MoE) — sparse routing: more parameters, nearly

    constant cost per token.

  • GQA/MQA — share keys/values to make inference cheaper.
  • FlashAttention — memory-efficient attention (the bottleneck is bandwidth, not FLOPs).
  • Long context — RoPE extensions, sparse/windowed attention, hybrid SSM.

Training algorithms:

  • Pre-training — self-supervised next-token objective at web scale.
  • Post-training — SFT (instructions) → alignment via RLHF / DPO / RLAIF

    (see ch. 29 — RLHF); distillation into smaller models.

Inference algorithms:

  • Decoding — sampling (temperature, top-k, top-p), beam search.
  • Efficiency — KV-cache, speculative decoding, quantization

    (int8/int4), continuous batching, paged attention.

4. Inputs

  • Hardware: clusters of GPU/TPU (Tensor Cores), HBM memory (the real

    bottleneck), interconnect (NVLink/InfiniBand, all-reduce); storage for checkpoints; power/cooling. Inference also uses NPUs and edge.

  • Data: web-scale corpora (text, code), deduplicated, quality-filtered,

    with a curated mixture (data mixture) and control of benchmark contamination.

  • Data structures: tensors; tokenizer/vocabulary (BPE);

    embedding tables; KV-cache (inference); vector index (HNSW) for RAG.

  • Systems / MLOps: PyTorchJAX + compilers (XLATriton); datatensorpipeline

    parallelism + FSDP/ZeRO; orchestration (RaySlurmK8s); serving (vLLM/TGI); model registry and versioning.

5. Specialized life cycle (the 11 stages)

The skeleton of Part I, filled in for LLMs.

Stage How it specializes in an LLM
0 · Problem Define target capabilities (chat, code, reasoning), languages, context window, and the success metrics/benchmarks
1 · Data Web crawl + code + curation; deduplication, quality/toxicity filtering, data mixture, tokenization; control of evaluation contamination
2 · EDA Corpus statistics (token, language, domain distribution), leakage and duplicate checks
3 · Modeling Choose size (scaling laws), depth, number of heads, context, MoE?, tokenizer; compute budget
4 · Training Pre-training next-token (massive parallelism, checkpointing, mixed precision) → post-training (SFT → RLHF/DPO)
5 · Evaluation Benchmarks (knowledge, reasoning, code), human evaluation, calibration; contamination caveats
5.5 · Acceptance Red teamingjailbreak, safety evaluations, system tests (APIcontract), shadow/canary
6 · Production Serving with KV-cache, batching, quantization, speculative decoding; context management; RAG/tools
7 · Monitoring Quality drift, refusal rate, latency/cost per token, abuse; correlation by trace_id
8 · Retraining New data, refreshcontinual, new RLHF iteration; versioning and AB of the new version
9 · Governance Model card, usage policy, alignment, evaluation transparency, privacy of training data

🎨 Figure F-IV.1.1 — The life cycle of an LLM. Brief: reuse the ring from F-I.1, but with stage-specific icons (web data funnel, scaling-laws scale, red-team shield, KV-cache in production). Shows the specialization of the generic cycle.

Life cycle of an LLM

6. Capabilities, modes and modalities

  • Linguistic — generation, translation, summarization, rewriting, dialogue.
  • Logical-mathematical — step-by-step reasoning (chain-of-thought),

    mathematics, code (related chapter: agents).

  • Intellectual and creative mode — from technical writing to fiction

    (controlled by sampling/conditioning).

  • Agentic — tool use, function calling, computer use

    (see ch. 30 — Agent).

  • Multimodal — LLMs become the base of models that also see/hear

    (see ch. 14 — Multimodal).

Types of intelligence (Gardner's grid, see ../01-life-cycle/04-nature-of-inputs.kmd): strong in linguistic and logical-mathematical; partial in interpersonal (dialogue); absent in bodily-kinesthetic (no body).

7. Limits, risks and ethics

  • Hallucination — generates plausible and false statements; mitigable (not

    eliminable) with RAG and grounding.

  • Bias — inherits biases from the corpus; requires evaluation and mitigation.
  • Opacity — billions of poorly interpretable parameters (interpretability

    is an open area).

  • Cost and energy — training and inference are intensive; environmental

    footprint.

  • Safety — jailbreaks, misuse; hence alignment (RLHF/RLAIF,

    Constitutional AI) and red teaming.

  • Data and law — provenance, copyright, corpus privacy.
  • [metaphor] — an LLM *oes not "understand", does not "want" and is not

    conscious* It models the statistical distribution of language; "reasoning" is the generation of steps, not formal deduction (see doc 4 of Part I).

8. State of the art and real examples

  • Frontier models (2025–2026): the Claude 4.X family (e.g. Opus 4.8,

    claude-opus-4-8) and Fable 5 (Anthropic); GPT (OpenAI); Gemini (Google); open weights such as Llama and Mistral.

  • Trends:
    • Reasoning models — long deliberation before answering.
    • Long context — windows of hundreds of thousands to millions of tokens.
    • MoE — parameter scale with controlled cost.
    • Agency — LLMs as the engine of agents that use tools and operate

      software (e.g. Claude Code).

    • Native multimodality — a single model for text, image, audio and video.

Cross-reference: for API details, model IDs, parameters and caching when building on top of Claude LLMs, use the claude-api skill — this chapter is conceptual, not an integration guide.

Template fixed. The next chapters (Diffusion, CNN, RL, Agents…) follow these 8 sections. Index: INDEX.kmd.