Part II · Gallery — Probabilistic / Bayesian AI

draft

Intelligence as reasoning under uncertainty. It models the world with probability distributions and updates beliefs with Bayes' rule. The backbone of classical ML and still essential where uncertainty matters.

🔗 Bayesian Network Probabilistic · Causal inference/diagnosis · Directed acyclic graph (DAG)

  • What it is: represents conditional dependencies among variables and infers

    posterior probabilities.

  • Examples: medical diagnosis, spam filters, risk systems.
  • Algorithmic basis: belief propagation, variational inference, MCMC.
  • Peak: the 1990s–2000s (Judea Pearl; Turing Award 2011).
  • Capabilities / modes: intellectual; explainable causal reasoning.
  • → Part IV: chapter planned.

⛓️ Hidden Markov Model (HMM) Probabilistic · Sequences · Markov chain + emissions

  • What it is: models sequences with latent states that emit observations.
  • Examples: pre-deep-learning speech recognition, bioinformatics,

    PoS-tagging.

  • Algorithmic basis: Viterbi, Baum-Welch (EM).
  • Peak: the 1980s–2000s (speech, classical NLP).
  • Capabilities / modes: sequential/temporal.
  • → Part IV: chapter planned.

📈 Gaussian Process (GP) Probabilistic · Regression/optimization · Kernel methods

  • What it is: a distribution over functions; predicts with calibrated

    uncertainty.

  • Examples: Bayesian hyperparameter optimization, geostatistics.
  • Algorithmic basis: kernels, Gaussian inference, covariance algebra.
  • Peak: the 2000s–2010s.
  • Capabilities / modes: intellectual; natively quantifies uncertainty.
  • → Part IV: chapter planned.

🌲 Classical ML (Trees, Boosting, SVM) Probabilistic/statistical · Tabular data · Trees / margins / kernels

  • What it is: statistical models that still *eat deep learning on tabular

    data*

  • Examples: XGBoost, LightGBM, Random Forest, SVM.
  • Algorithmic basis: gradient boosting, bagging, margin maximization.
  • Peak: the 2000s–present (Kaggle champions; production in finance).
  • Capabilities / modes: intellectual/predictive; fast and robust.
  • → Part IV: chapter planned.

Salient sciences and mathematics: probability, statistics (inferential and Bayesian), information theory, linear algebra. Strength: calibrated uncertainty and interpretability; weakness: scaling on unstructured data.