Time series forecasting

The one place where every default in the rest of ML is wrong. Rows are not independent, random splits leak, and the metric you reach for first is undefined on a zero.

The three rules that follow from ordering

1. Never shuffle. A random split trains on Thursday to predict Wednesday. Every score you produce afterwards is fiction. Split by time, and validate with a rolling or expanding window:

from sklearn.model_selection import TimeSeriesSplit

for tr, va in TimeSeriesSplit(n_splits=5).split(X):
    ...   # each fold trains on the past only

2. Only use what you would have had. A feature computed from the full series — a global mean, a scaler fitted on everything, tomorrow’s promotion flag — leaks the future. This is the dominant bug in time-series projects, and it produces beautiful backtests that collapse in production.

3. Match the horizon to the question. A model validated on one-step-ahead predictions tells you nothing about a 30-day forecast. Validate at the horizon you will actually serve.

Decompose before modelling

observed = trend + seasonality + residual

Most of a business series is calendar. Day of week, month, holidays and paydays explain more of a retail or traffic series than any model architecture, which is why the strongest cheap move is usually feature engineering rather than a better model.

Multiple seasonalities are normal — daily and weekly and yearly — and a model that only handles one will underfit.

The methods, in the order to try them

Seasonal naive — “last week, same day” — is the baseline. It is embarrassingly strong, and a model that cannot beat it is not ready. Reporting your model against it is the fastest credibility signal in this topic.

Gradient boosting on lag features is the practical default for business forecasting. Build lags, rolling means, calendar flags and known-future regressors, then let XGBoost or LightGBM handle it as a regression. It handles many series, missing data and exogenous variables in a way classical methods do not.

df["lag_7"] = df.groupby("sku")["units"].shift(7)
df["roll_28"] = (
    df.groupby("sku")["units"]
      .shift(1).rolling(28).mean()
)

Gotcha: the shift(1) before rolling is the whole game. Without it the rolling window includes the current row, and you have leaked the target into its own feature.

ARIMA is worth being able to discuss — differencing for stationarity, the p/d/q parameters — but reaching for it first on a multi-series business problem is a dated instinct.

Metrics: MAPE will betray you

MAPE is the default request from stakeholders and it breaks on intermittent demand — divide by a zero-sales day and the metric explodes. It also penalises over-forecasting more than under-forecasting, which quietly biases your model.

MASE is the one to propose: it divides your error by the seasonal naive’s error, so 1.0 means “no better than the baseline” and it is comparable across series of different scales.

Forecasting is not the whole job

Two things separate an engineer who has shipped one:

Prediction intervals, not point forecasts. A stock decision needs the range, not the mean. Quantile regression or conformal intervals give you that, and “we forecast the 80th percentile because stockouts cost more than holding” is a better sentence than any accuracy number.

Retraining cadence. Series drift; a model trained in January degrades by June. Decide the cadence, monitor error against the baseline, and alert when your model stops beating seasonal naive — that comparison is a working drift detector on its own.

Hierarchies and reconciliation

Forecasts at SKU, store, region and national level will not sum up. Choosing whether to forecast bottom-up, top-down, or reconcile the two is a real design decision, and naming it signals you have done this beyond one series.

Related

• Train / validation / test splits — why the split is different here
• Data leakage — the failure mode this topic is built around
• Gradient boosting — the practical default model
• Regression metrics — the metric background