Feature transformations that survive production
Ship the fitted representation contract with the model, in the same order and with the same edge cases.
Fraud labels arrive late and online features disagree with training data.
This chapter moves from fit on the training partition to test raw-request parity. We will make the idea concrete in a small code example before placing it back inside a production system with a declared latency, cost, and operator-capacity budget.
Feed the same raw fixture through offline and serving paths, then compare the final vector and model output.
Verify the behavior illustrated in fitted_transform.py, then measure decision utility, slice reliability, and system latency.
Joining only on customer ID attaches the present balance to every historical row. An unsafe release must degrade to a named baseline or the last known-good version.
Build the idea one layer at a time.
A transformation is part of the learned model whenever it estimates state from data. Means, vocabularies, bins, and category maps must travel with the weights.
Fit transformation state on training data only. Computing a mean or vocabulary over validation quietly lets future distributions influence the representation, even if the target column never appears in the transform.
Production parity requires a fixed feature order plus explicit behavior for missing values, unseen categories, and invalid ranges. Reimplementing “the same” preprocessing in a service is less reliable than serializing and invoking the fitted transform.
- availability time
- as-of joins
- transform state
- watermarks
Follow the reasoning before touching the code.
- 01
Fit on the training partition
Estimate imputation, scaling, encoding, and selection state without touching validation or test rows.
- 02
Freeze a feature contract
Store ordered feature names, learned parameters, version, input schema, and policies for unknown or missing values.
- 03
Test raw-request parity
Feed the same raw fixture through offline and serving paths, then compare the final vector and model output.
from math import sqrt
feature_order = ["age_years", "spend_usd"]
train = [[20.0, 10.0], [40.0, 30.0], [60.0, 20.0]]
means = [sum(row[j] for row in train) / len(train) for j in range(2)]
scales = [
sqrt(sum((row[j] - means[j]) ** 2 for row in train) / len(train))
for j in range(2)
]
def transform(raw):
values = [float(raw[name]) for name in feature_order]
return [(value - mean) / scale for value, mean, scale in zip(values, means, scales)]
artifact = {"features": feature_order, "means": means, "scales": scales}
print(artifact)
print(transform({"spend_usd": 25, "age_years": 50}))Fit scaler state once and reuse its declared feature order.
What to notice
- Dictionary insertion order does not affect the vector because feature_order is part of the artifact.
- The age and spend statistics come exclusively from train, so the held-out example cannot shape its own representation.
YOUR TURN Pause and predict before revealing the answer.
What happens if a training feature has the same value in every row, and how should a robust transformer respond?
Need a hint?
Its fitted scale becomes zero.
Check your reasoning
Division by zero would occur; validation should reject or explicitly drop/constant-encode that feature and persist the chosen policy.
Lessons 1, 2 in this course.
Connect the chapter to the course’s quantitative spine.
Ship the fitted representation contract with the model, in the same order and with the same edge cases. The feature availability invariant below is the shared course-level lens that helps quantify the surrounding system. Name each symbol, check its units or shape, and use only values that were available at the production cutoff.
Feature availability invariant
| Symbol | Meaning / shape / unit |
|---|---|
t_available | time the value became knowable |
t_decision | prediction cutoff |
f_i | feature i |
Open derivation and numerical substitution
Start from the production quantity being optimized, substitute the observed values with their declared units, then isolate the model-controlled term. Preserve shape annotations at each step so broadcasting or aggregation cannot silently change the result.
- Write the named inputs: t_available, t_decision, f_i.
- Substitute one small, hand-checkable batch before vectorizing.
- Calculate an independent reference value and compare within a declared tolerance.
# equation → code contract
inputs = validate_shapes_and_units(batch)
value = compute_c03(inputs)
assert is_finite(value)Change one assumption and make the tradeoff visible.
Use the worked chapter as your prediction, then change one input in the lab and explain why the output moves. The lab runs predefined TypeScript and never executes learner code, so the focus stays on the concept rather than setup.
Point-in-time freshness lab
Delay feature arrival and see which historical rows remain eligible.
Assumption: Five events have fixed prediction cutoffs separated by six hours.
Open nonvisual data table
| Item | Computed state | Interpretation |
|---|---|---|
| row 1 | cutoff +0h | not yet knowable |
| row 2 | cutoff +6h | eligible |
| row 3 | cutoff +12h | eligible |
| row 4 | cutoff +18h | eligible |
| row 5 | cutoff +24h | eligible |
Trace the complete operating path.
Carry the chapter’s central idea into the system: Ship the fitted representation contract with the model, in the same order and with the same edge cases. The path below shows where that reasoning needs an artifact, a test, an owner, or a fallback.
- 01
Turn “Fit on the training partition” into a written invariant and a failing test.
- 02
Promote fitted_transform.py from illustration to a versioned implementation with representative fixtures.
- 03
Use “Test raw-request parity” as the release checkpoint: Feed the same raw fixture through offline and serving paths, then compare the final vector and model output.
- 04
Serve or materialize behind explicit a declared latency, cost, and operator-capacity budget.
- 05
Join telemetry to mature outcomes and retain a rollback path.
Observability
Join service health, input quality, prediction distributions, slice behavior, and mature outcomes by exact version.
Cost
Measure storage, preprocessing, compute, queueing, and human review under a representative arrival pattern.
Failure modes
Joining only on customer ID attaches the present balance to every historical row. Add a detector, owner, mitigation, and stop condition for this class of failure.
Alternatives
Compare a rule, a simpler statistical baseline, and a different system boundary before adding model complexity.
Explain the contract, not just the vocabulary.
Point-in-time feature system
Design a versioned training set and online lookup from transaction, account, and outcome events.
- Contracts and lineage
- Temporal join specification
- Late-event/backfill policy
- Offline-online parity report
Read primary material with a purpose.
Return to the opening failure.
Version contracts, temporal joins, transform state, snapshots, backfills, and continuous offline/online parity checks.
For this lesson, the release evidence is the worked code example, a hand-checked formal result, deterministic simulation output, a ≥80% checkpoint, the production rubric, and a named fallback. The course resolves when the system can produce data contracts, point-in-time features, backfills, quality gates, and lineage.