AI & ML / README.md

Classical ML

Updated 2 min read index source
On this page6
  1. Linear models
  2. Trees and ensembles
  3. Distance and probability based
  4. Unsupervised
  5. Applied problem shapes
  6. The two comparisons worth memorising

Classical ML

Still what wins on tabular data in 2026. Gradient boosting beats deep learning on most structured problems, and interviews for AI-facing backend roles routinely check that you know the classics rather than only the LLM layer.

Linear models

# File Why it’s asked
01 Linear regression assumptions, and which ones matter for prediction vs inference
02 Logistic regression the default baseline; why cross-entropy not MSE; threshold != model

Trees and ensembles

# File Why it’s asked
03 Decision trees the base learner everything else is built from
04 Random Forest and bagging bagging attacks variance — why the trees are deep
05 Gradient boosting the one that wins on tabular data; XGBoost vs LightGBM vs CatBoost

Distance and probability based

# File Why it’s asked
06 Support vector machines the kernel trick
07 kNN and Naive Bayes kNN is vector search; Naive Bayes as the fast text baseline

Unsupervised

# File Why it’s asked
08 Clustering k-means assumptions, DBSCAN/HDBSCAN, evaluating without labels
09 Dimensionality reduction PCA vs UMAP, and compressing embeddings to cut RAG cost

Applied problem shapes

# File Why it’s asked
10 Anomaly detection fraud and intrusion: no labels, extreme imbalance, alert fatigue
11 Recommender systems the two-stage funnel, cold start, why offline metrics mislead
12 Time series forecasting no shuffling, seasonal naive, why MAPE betrays you

The two comparisons worth memorising

Bagging vs boosting. Bagging trains independent deep trees in parallel and averages them to cut variance; more trees never overfit. Boosting trains shallow trees sequentially, each fitting the last one’s errors, to cut bias; more trees eventually do overfit, so early stopping is mandatory. Same ensemble family, opposite mechanism, opposite base learner.

Why trees still beat neural networks on tabular data. Heterogeneous feature types, piecewise-constant relationships, robustness to uninformative features, and far lower data and tuning cost. Deep learning wins when you need to embed high-cardinality relations or fuse tabular data with text or images.

Contents 12