Agentic AI — Roadmap

The AI half of the stack, from token mechanics up to shipping agents. Same convention as Senior Python + AI — Technology Roadmap and Senior Python + LLM/Agentic Engineer — Depth Guide: tick the box once you can explain and defend it, bold is the primary note, the rest add depth.

Sections 0-3 are the retrieval track, 4-5 the agent track, 6-8 what makes either survive production. If you are short on time, the order that pays is 3 → 5 → 6 — evaluation, MCP, observability. Those are the ones most candidates cannot demonstrate.

0. LLM foundations

• Tokens and tokenization (BPE) — why token counts matter↳Tokenization·Classical NLP
• Context window, attention, KV cache↳Context windows·KV cache·Attention mechanisms·Transformer architecture
• Sampling: temperature, top-p, top-k↳Information theory·Inference basics
• Seed and determinism — why temperature 0 still isn’t reproducible↳Determinism
• Base vs instruct vs reasoning models↳Reasoning models·SFT and instruction tuning·Model routing
• Structured output — JSON mode, constrained decoding, tool schemas↳Function calling·LLM JSON validation
• Embeddings — what a vector encodes; cosine vs dot vs L2↳Embeddings
• Prompt caching, TTFT vs throughput, cost accounting↳Inference basics·Gateway and caching·The chat API·Cost attribution

1. RAG fundamentals

• What RAG solves — parametric vs non-parametric knowledge↳What RAG solves
• The naive pipeline — ingest → chunk → embed → index → retrieve → generate↳What RAG solves·RAG architecture patterns
• Chunking strategies — fixed, recursive, semantic, structure-aware↳Chunking
• Choosing an embedding model — dimensions, multilingual, Matryoshka↳Embeddings·Fine-tuning embeddings
• pgvector deep dive — HNSW vs IVFFlat, ef_search/m, build cost↳pgvector in production·HNSW and ANN indexes·Vector databases
• Metadata filtering — pre-filter vs post-filter, and the recall trap↳pgvector in production·Multi-tenant vector search
• Document ingestion — PDF, tables, images, OCR, layout↳Document parsing

2. Advanced retrieval

• Hybrid search — BM25 + dense, Reciprocal Rank Fusion↳Hybrid search and reranking·Query DSL and scoring
• Reranking — cross-encoders, ColBERT / late interaction↳Hybrid search and reranking
• Query transformation — HyDE, multi-query, decomposition, step-back↳RAG architecture patterns
• Contextual retrieval — prepend chunk context before embedding↳Hybrid search and reranking
• Small-to-big — parent-document and sentence-window retrieval↳Chunking·RAG architecture patterns
• GraphRAG and knowledge-graph retrieval↳GraphRAG
• Agentic RAG, Self-RAG, Corrective RAG↳Agentic RAG and eval·RAG architecture patterns
• Long context vs RAG — when retrieval stops paying for itself↳What RAG solves·Context windows
• Multimodal RAG — image + text↳Multimodal·Document parsing

3. Grounding, hallucination, evaluation

• Hallucination — intrinsic vs extrinsic taxonomy↳Hallucinations and tools
• Groundedness, faithfulness, attribution — and how they differ↳Output validation
• The four RAG metrics — faithfulness, answer relevancy, context precision/recall↳Agentic RAG and eval·Why LLM eval is hard
• RAGAS — what it measures, and where it misleads you↳Eval tooling
• LLM-as-a-judge — pointwise vs pairwise, rubrics, biases↳LLM-as-judge
• NLI and claim-decomposition grounding checks↳Output validation·Why LLM eval is hard
• Synthetic test sets and golden datasets↳Building eval sets·Agentic RAG and eval·Distillation and synthetic data
• Retrieval-only eval — recall@k, MRR, nDCG, MTEB/BEIR↳Agentic RAG and eval·Embeddings·Choosing the metric
• Offline suites in CI vs online eval on production traffic↳Online eval·Monitoring and drift
• Regression-testing prompts like code↳Building eval sets·Determinism

4. Agents

• Workflow vs agent — when you don’t need one↳What is an agent
• The ReAct loop — reason → act → observe↳The agent loop
• Tool design — schemas, idempotency, error surfaces, retries↳Function calling·Agent failure modes
• Planning patterns — plan-and-execute, reflection, tree-of-thoughts↳Advanced agent patterns·Reasoning models
• Memory — buffers, summarisation, episodic vs semantic↳Memory strategies
• LangGraph mental model — state, nodes, edges, routing, checkpointers↳State and reducers·Nodes and routing·Checkpointers·Why LangGraph
• Human-in-the-loop — interrupts, approvals, resumable state↳Human in the loop·Durable execution
• Multi-agent topologies — supervisor, swarm, handoffs, failure modes↳Multi-agent patterns·Subgraphs·Multi-agent frameworks
• Context engineering — compaction, context rot, sub-agent delegation↳Context management·Context windows
• Durable execution for long-running agents↳Durable execution·Long-running jobs·Workflows vs activities

5. MCP

• What MCP is — why a protocol beat ad-hoc tool wrappers↳MCP fundamentals
• Primitives — tools, resources, prompts, sampling, elicitation↳MCP fundamentals·The stateless-core spec
• Transports — stdio vs streamable HTTP↳The stateless-core spec·MCP fundamentals
• Authoring a server with FastMCP↳Building a server
• OAuth 2.1 for remote servers, resource indicators↳Securing MCP·OIDC and OAuth2 flows
• MCP security — tool poisoning, confused deputy, injection via results↳Securing MCP·Prompt injection

6. Production and observability

• Why LLM observability differs from APM — traces, spans, generations↳LLM observability
• OpenTelemetry GenAI semantic conventions↳Tracing platforms·OpenTelemetry
• Langfuse vs LangSmith vs Phoenix↳Tracing platforms·Eval tooling
• Prompt versioning, registries, A/B testing prompts↳Online eval·Experiment tracking·Prompt optimization
• Latency — streaming, prompt caching, parallel tool calls, batching↳Inference basics·Batching and serving engines·Server-sent events
• Reliability — timeouts, retries with jitter, fallbacks, routing, breakers↳Provider abstraction·Timeouts and retries·Circuit breakers
• Semantic caching vs exact caching↳Gateway and caching
• Async Python for LLM apps — asyncio, httpx, backpressure, asyncpg↳Agent backpressure·TaskGroup·Async SQLAlchemy
• Queueing agent runs — Celery/Redis, job status, cancellation↳Long-running jobs·Celery idempotency
• Feedback loops — implicit signals, thumbs, online evals on sampled traffic↳Online eval·Monitoring and drift
• Cost control — attribution, budgets, and cutting spend safely↳Cost attribution·Cutting LLM spend

7. Security

• Direct vs indirect prompt injection↳Prompt injection
• The lethal trifecta — private data + untrusted content + exfiltration↳Prompt injection·Agent failure modes
• Exfiltration via Markdown images and links↳Prompt injection
• Sandboxing tool execution, least privilege, allowlists↳Sandboxing agents·Output validation·RBAC and ABAC
• PII detection and redaction, data residency↳PII and compliance·Azure and vertex
• OWASP Top 10 for LLM applications↳Prompt injection·OWASP Top 10
• Red-teaming agents systematically↳Red teaming·Model risk management

8. Customization

• The decision tree — prompt → RAG → fine-tune↳Fine-tuning vs RAG·Prompt engineering
• SFT, LoRA/QLoRA, PEFT↳LoRA and PEFT·SFT and instruction tuning
• Preference tuning — RLHF, DPO↳RLHF and DPO
• Distillation into smaller models↳Distillation and synthetic data
• Quantization and self-hosted serving (vLLM, TGI)↳Batching and serving engines·Quantization·Model export and runtimes
• Fine-tuning embedding models and rerankers on domain data↳Fine-tuning embeddings

Coverage note

Checked 2026-08-21: every theme above resolves to at least one existing note. Where a theme lives inside a larger note rather than its own file, the section name is given after the link.

One article was written to close a real gap found while building this map:

• Determinism — seed and system_fingerprint, why temperature 0 is not reproducible, and how to test a non-deterministic system anyway.

Four sections were added to existing notes:

• MTEB and BEIR in Embeddings
• ColBERT / late interaction in Hybrid search and reranking
• Step-back prompting in RAG architecture patterns
• MCP sampling and elicitation in MCP fundamentals — the note previously described three primitives; there are five.

Plus the parametric / non-parametric framing in What RAG solves.