Part V · 1 — Glossary

Technical and conceptual terms used in the Compendium. Metaphorical concepts (which describe the perceived quality, not a mechanism) carry the [metáfora] mark — see Part I, doc 4.

A–C

• Alinhamento — adjusting a model's behavior to human values/objectives (see

  RLHF, ch. 29).

• Hallucination — a plausible, false statement generated by a model.
• Attention — mechanism that weighs the relevance of parts of the

  input; the core of the Transformer.

• Auto-supervisão — learning from data without labels (e.g., predicting the

  next token).

• Backpropagation — algorithm that propagates the error to adjust weights (chain

  rule).

• BPE (Byte-Pair Encoding) — subword tokenization.
• Benchmark — standardized test to compare models.
• Chain-of-thought (CoT) — generating reasoning steps before the answer.
• CNN — convolutional network (vision), ch. 03.
• Consciência — [metáfora]; not implemented in current AI.
• Cross-entropy — loss that measures the divergence between the predicted and

  the actual distribution.

D–G

• Difusão — generation by noise reversal, ch. 02.
• DPO (Direct Preference Optimization) — direct alignment by preferences,

  without explicit RL.

• Drift — change in the distribution of the data (data) or of the relation

  (concept) over time.

• Embedding — dense vector representation of an item (token, image, user).
• EDA — exploratory data analysis (step 2).
• ELBO — Evidence Lower Bound, the VAE objective.
• Fine-tuning — adjusting a pre-trained model to a domain/task.
• FlashAttention — memory-efficient attention.
• Loss function — quantity minimized during training.
• GAN — generative adversarial network, ch. 05.
• GNN — graph neural network, ch. 11.
• Gradiente — direction of steepest increase of the loss; used to adjust weights.

H–N

• HBM — high-bandwidth memory of GPUs; the real bottleneck of training.
• Heurística — practical rule that guides the search without a guarantee of

  optimality.

• Hiperparâmetro — configuration set before training (e.g., learning rate).
• In-context learning — learning the task from examples in the prompt.
• Inferência — (a) statistics: estimating parameters; (b) running the trained

  model (step 6).

• KV-cache — cache of attention keys/values, speeds up inference.
• Leis de escala — predictable relationship between performance and

  sizedatacompute.

• LLM — large language model, ch. 01.
• LoRA — low-rank adaptation (efficient fine-tuning).
• Lucidez / sagacidade / esperteza — [metáfora]; perceived quality, not a

  mechanism.

• MCTS — Monte Carlo Tree Search, ch. 27.
• MoE (Mixture-of-Experts) — sparse routing to scale parameters.
• MDP — Markov decision process (the basis of RL).

O–R

• Overfitting — the model memorizes the training set and generalizes poorly.
• Perplexidade — language-model quality metric (exp of the cross-entropy).
• Política (policy) — function that maps state → action (RL).
• Pré-treino — initial large-scale learning phase.
• Quantization — reducing numerical precision (int8/int4) to make inference

  cheaper.

• RAG — retrieval-augmented generation, ch. 31.
• Raciocínio — in current AI, the generation of steps (CoT), not formal

  deduction.

• Reflexão / *self-critique* — a model revising its own output (a real

  technique); ≠ conscious reflection [metáfora].

• RLHF / RLAIF — alignment by human/AI feedback, ch. 29.
• RoPE — rotary positional encoding.

S–Z

• Sampling — sampling from the output distribution (temperature, top-k, top-p).
• Scaling laws — see leis de escala.
• SSM — state space model (Mamba), ch. 13.
• SFT — Supervised Fine-Tuning (instruction tuning).
• Speculative decoding — speeds up generation by predicting several tokens

  ahead.

• Tensor — n-dimensional array; the central data structure of AI.
• Token — input/output unit (subword, patch, audio segment).
• Transformer — attention-based architecture; the basis of modern AI.
• VAE — variational autoencoder, ch. 06.
• ViT — Vision Transformer, ch. 04.
• World model — internal model of the dynamics of an environment.