MLOps and LLMOps
| # | File | Covers |
|---|---|---|
| 01 | Experiment tracking and the model registry | MLflow, what makes a run reproducible, registry and rollback |
| 02 | Monitoring and drift | the four layers, data vs concept drift, PSI, retraining triggers |
| 03 | LLM observability | OTel GenAI conventions and their actual status, what to capture, content logging |
| 04 | Cost attribution | per tenant and per feature, cost per outcome, budgets as enforcement |
| 05 | Incident response for AI systems | failures that return 200, triage order, disabling the AI path |
Three things to have ready
Data version and git commit are what make a run reproducible. Metrics without them record that a number happened, not how to get it again.
Concept drift is invisible without labels. Data drift (P(X)) is measurable immediately; concept drift (P(y|X)) only shows up once ground truth arrives. That asymmetry is why input monitoring alone is insufficient.
The OTel GenAI conventions are still experimental. They moved to a dedicated repo in v1.42.0 for release cadence, not as a graduation to stable. Adopt them, pin your instrumentation, expect attribute names to change.