AI & ML
Foundations through advanced, for a Python backend engineer moving into AI/ML work. Rebuild complete (phases 3.1-3.6). 44 files at the start of this effort, 124 now.
Baseline facts and current versions: Stack baseline — 2026-2027.
Built
| # | Folder | Covers |
|---|---|---|
| 00 | Math foundations | linear algebra, probability/stats, calculus & optimisation, information theory |
| 01 | ML foundations | problem types, splits, bias-variance, regularisation, lifecycle, when not to use ML |
| 02 | Classical ML | linear/logistic, trees, random forest, gradient boosting, SVM, kNN/NB, clustering, PCA |
| 03 | Feature engineering | encoding, scaling, selection, data leakage, imbalanced data, feature stores |
| 04 | Model evaluation | confusion matrix, precision/recall, ROC-AUC vs PR-AUC, regression metrics, calibration |
| 05 | Deep learning | networks, activations, training, CNN, RNN/LSTM, norms & residuals, transfer learning |
| 06 | Transformers and LLMs | architecture, GQA/MLA, tokenization, RoPE, KV cache, MoE, reasoning models, context |
| 07 | Training and fine-tuning | pretraining, SFT, DPO/GRPO/RLVR, quantization, fine-tune vs RAG |
| 08 | Inference and serving | prefill vs decode, continuous batching, PagedAttention, speculative decoding |
| 09 | RAG and embeddings | RAG, embeddings, chunking, hybrid search + reranking, agentic RAG, retrieval eval |
| 10 | Agents and orchestration | agent loop, LangGraph 1.0 durable execution, multi-agent, A2A, failure modes |
| 11 | Model Context Protocol | fundamentals, the 2026-07-28 stateless-core revision, building and securing servers |
| 12 | Context engineering | context budgeting, ordering, compaction, memory systems |
| 13 | Evaluation | why eval is hard, building eval sets, LLM-as-judge biases, online experiments |
| 14 | Guardrails and safety | prompt injection, output validation, PII, EU AI Act 2026 |
| 15 | MLOps and LLMOps | MLflow, registry, drift, OTel GenAI observability |
| 16 | ML frameworks | PyTorch, TensorFlow, scikit-learn, JAX, Optuna — the 2026 map |
| 17 | ML system design | the design framework, RAG assistant, fraud detection |
| 18 | Speech and realtime AI | STT/TTS, streaming vs batch, realtime voice agents, turn detection and barge-in |
Reading order
For a backend engineer preparing for an AI-facing role: 01_ml_foundations first (it’s the judgement layer and the most transferable), then 02_classical_ml for the tabular questions, then the LLM stack. 00_math_foundations is reference — dip into it when a specific question needs it rather than reading it front to back.