Forecast contracts and naive models
A forecast begins with two clocks—when you know and when you predict—and a baseline that respects both.
Regional demand shifts through promotions, holidays, and drift.
This chapter moves from pin down origin and horizon to score by decision slice. We will make the idea concrete in a small code example before placing it back inside a production system with available before staffing decisions.
Report error separately by horizon, weekday, region, and high-demand periods so an easy average cannot hide the hours that drive staffing.
Verify the behavior illustrated in seasonal_naive.py, then measure decision utility, slice reliability, and system latency.
Training uses realized future weather and random folds, producing a forecast impossible to reproduce at origin time. An unsafe release must degrade to a named baseline or the last known-good version.
Build the idea one layer at a time.
Before choosing a forecasting model, freeze the moment at which the forecast is made and the action it must support. A prediction that arrives after staffing is locked is wrong operationally even when its numeric error is small.
A forecast contract names the target unit, forecast origin, horizons, publication deadline, aggregation level, and information available at origin. “Tomorrow’s demand” is ambiguous; “hourly completed orders for the next 24 hours, published at 16:00 local time” is testable.
Naive forecasts expose how much structure a sophisticated model must beat. Last-value works when levels move slowly; seasonal-naive copies the corresponding value from the previous cycle. These baselines are cheap, debuggable, and ideal fallbacks when upstream features fail.
- forecast origin
- seasonal naive
- lag features
- rolling backtests
Follow the reasoning before touching the code.
- 01
Pin down origin and horizon
For every prediction, record when it was issued and which future timestamp it targets; these two times prevent accidental use of later facts.
- 02
Build the cheapest credible baseline
Use last-value and a domain-aligned seasonal lag such as 24 hours or seven days, without tuning against the final test window.
- 03
Score by decision slice
Report error separately by horizon, weekday, region, and high-demand periods so an easy average cannot hide the hours that drive staffing.
actual = [18, 20, 23, 19, 21, 25]
season = 3
predicted = actual[:-season]
observed = actual[season:]
errors = [abs(y - yhat) for y, yhat in zip(observed, predicted)]
for horizon, (yhat, y) in enumerate(zip(predicted, observed), start=1):
print(horizon, 'forecast', yhat, 'actual', y)
print('MAE', sum(errors) / len(errors))Forecast the next three hours from the same hours one day earlier.
What to notice
- The seasonal baseline is available immediately because it uses only values observed before the forecast origin.
- Even in this tiny example, horizon-level errors reveal where the copied pattern misses instead of hiding everything inside MAE.
YOUR TURN Pause and predict before revealing the answer.
If a promotion is announced after the forecast origin, may its realized demand be used to improve this backtest?
Need a hint?
Ask whether the production system could have known the value at the recorded origin.
Check your reasoning
No. Realized promotional demand is future information; only a promotion plan known at origin may be used as a feature.
Evaluation, Data & features, Trees & boosting
Connect the chapter to the course’s quantitative spine.
A forecast begins with two clocks—when you know and when you predict—and a baseline that respects both. The quantile forecast objective below is the shared course-level lens that helps quantify the surrounding system. Name each symbol, check its units or shape, and use only values that were available at the production cutoff.
Quantile forecast objective
| Symbol | Meaning / shape / unit |
|---|---|
τ | target quantile |
q̂_τ | predicted quantile |
y | observed value |
Open derivation and numerical substitution
Start from the production quantity being optimized, substitute the observed values with their declared units, then isolate the model-controlled term. Preserve shape annotations at each step so broadcasting or aggregation cannot silently change the result.
- Write the named inputs: τ, q̂_τ, y.
- Substitute one small, hand-checkable batch before vectorizing.
- Calculate an independent reference value and compare within a declared tolerance.
# equation → code contract
inputs = validate_shapes_and_units(batch)
value = compute_c07(inputs)
assert is_finite(value)Change one assumption and make the tradeoff visible.
Use the worked chapter as your prediction, then change one input in the lab and explain why the output moves. The lab runs predefined TypeScript and never executes learner code, so the focus stays on the concept rather than setup.
Rolling-window backtest
Change history length and inspect bias, error, and interval coverage.
Assumption: Seasonal demand with one simulated regime change at week 30.
Open nonvisual data table
| Item | Computed state | Interpretation |
|---|---|---|
| h+1 | 12.5 MAE | 87.8% |
| h+6 | 13.6 MAE | 86.7% |
| h+12 | 15.0 MAE | 85.4% |
| h+18 | 16.3 MAE | 84.0% |
| h+24 | 17.6 MAE | 82.7% |
Trace the complete operating path.
Carry the chapter’s central idea into the system: A forecast begins with two clocks—when you know and when you predict—and a baseline that respects both. The path below shows where that reasoning needs an artifact, a test, an owner, or a fallback.
- 01
Turn “Pin down origin and horizon” into a written invariant and a failing test.
- 02
Promote seasonal_naive.py from illustration to a versioned implementation with representative fixtures.
- 03
Use “Score by decision slice” as the release checkpoint: Report error separately by horizon, weekday, region, and high-demand periods so an easy average cannot hide the hours that drive staffing.
- 04
Serve or materialize behind explicit available before staffing decisions.
- 05
Join telemetry to mature outcomes and retain a rollback path.
Observability
Join service health, input quality, prediction distributions, slice behavior, and mature outcomes by exact version.
Cost
Measure storage, preprocessing, compute, queueing, and human review under a representative arrival pattern.
Failure modes
Training uses realized future weather and random folds, producing a forecast impossible to reproduce at origin time. Add a detector, owner, mitigation, and stop condition for this class of failure.
Alternatives
Compare a rule, a simpler statistical baseline, and a different system boundary before adding model complexity.
Explain the contract, not just the vocabulary.
Regional demand forecast
Build a 24-hour pipeline with naive, statistical, and boosted baselines plus interval forecasts.
- Forecast contract
- Rolling-origin evaluation
- Horizon and coverage report
- Drift/retrain/fallback policy
Read primary material with a purpose.
Forecasting: Principles and Practice
90 min · last checked 26 Aug 2026
↗RECOMMENDED · Official docsscikit-learn lagged forecasting example
25 min · last checked 26 Aug 2026
↗DEEP DIVE · INDEXFind a primary implementation or benchmark
30–90 min · verify task, dataset, and evaluation match
↗Return to the opening failure.
Monitor horizon-level error, bias, interval coverage, freshness, reconciliation, and a seasonal-naive fallback.
For this lesson, the release evidence is the worked code example, a hand-checked formal result, deterministic simulation output, a ≥80% checkpoint, the production rubric, and a named fallback. The course resolves when the system can produce rolling validation, statistical baselines, boosted forecasts, and uncertainty bands.