AI & ML / README.md

Training and fine-tuning

Updated 1 min read index source

Training and fine-tuning

From pretraining through post-training to the adaptation you’d actually do.

# File Covers
01 Pretraining next-token objective, data pipeline, scaling laws, base vs instruct
02 Supervised fine-tuning and instruction tuning SFT, data quality over quantity, LoRA defaults, synthetic data
03 RLHF and preference optimization PPO → DPOGRPO → RLVR, and why the field moved
04 Quantization INT4/FP8, GPTQ/AWQ, QLoRA, quality trade-offs
05 Distillation and synthetic data teacher to student, generating and filtering data, model collapse
07 Fine-Tuning vs RAG vs Prompt Engineering the decision most teams get wrong

The three that get asked

Compute-optimal is not inference-optimal. Chinchilla optimises training cost. If you serve a model heavily, over-train a smaller one — you buy cheaper inference forever.

GRPO dropped the critic. PPO needs a value network to compute advantages; GRPO samples a group of responses to the same prompt and uses the group mean as the baseline. Half the memory, still on-policy. That’s why reasoning training converged on it.

Fine-tuning teaches form, not facts. Knowledge in weights can’t be updated or cited. If the ask is “answer from our documents”, it’s retrieval.

Contents 7