Images are data contracts
Image preprocessing is model code; version and test it from raw bytes to final tensor.
A factory has few positive examples of costly visual defects.
This chapter moves from specify the byte-to-tensor path to prove raw-input parity. We will make the idea concrete in a small code example before placing it back inside a production system with one shift for inference and review.
Feed the same saved image bytes through training evaluation, batch scoring, and serving, then compare the final tensors and model outputs within tolerance.
Verify the behavior illustrated in image_tensor_contract.py, then measure decision utility, slice reliability, and system latency.
Random burst-frame splitting puts near-duplicates of one product in train and validation. An unsafe release must degrade to a named baseline or the last known-good version.
Build the idea one layer at a time.
A model never receives “a photograph.” It receives a tensor produced by decoding bytes, applying orientation, choosing channels, resizing, scaling, and normalizing. Every one of those operations is part of the learned function.
Image shape conventions such as HWC and NCHW describe where height, width, and channels live. A silent RGB/BGR swap, ignored EXIF rotation, or uint8-versus-float scaling change can preserve tensor shape while destroying semantics. Alpha channels, grayscale inputs, corrupt files, and unusual aspect ratios need declared behavior rather than lucky coercion.
Training and serving must share one versioned preprocessing implementation or pass golden parity tests at the raw-byte boundary. Augmentation belongs only to training and should represent plausible variation, while deterministic resize, color conversion, scaling, and normalization must be identical during validation and inference.
- NCHW tensors
- convolution
- receptive fields
- augmentation
Follow the reasoning before touching the code.
- 01
Specify the byte-to-tensor path
Declare decoder, orientation policy, color space, channel order, resize/crop rule, numeric range, normalization constants, layout, dtype, and corrupt-input behavior.
- 02
Separate augmentation from inference
Apply stochastic transformations only after the deterministic contract and only to training samples; record their probabilities and parameter ranges.
- 03
Prove raw-input parity
Feed the same saved image bytes through training evaluation, batch scoring, and serving, then compare the final tensors and model outputs within tolerance.
import numpy as np
image = np.array([[[255, 0, 0], [0, 255, 0]],
[[0, 0, 255], [255, 255, 255]]], dtype=np.uint8)
mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
float_hwc = image.astype(np.float32) / 255.0
normalized = (float_hwc - mean) / std
nchw = normalized.transpose(2, 0, 1)[None, ...]
assert nchw.shape == (1, 3, 2, 2)
print(nchw.round(2))Convert a tiny RGB uint8 image from HWC to normalized NCHW.
What to notice
- Scaling happens before normalization; subtracting ImageNet means directly from uint8 values would produce meaningless magnitudes.
- The batch dimension and channel-first transpose are explicit, so a serving runtime cannot guess the expected layout.
YOUR TURN Pause and predict before revealing the answer.
Reverse the three channels before normalization. Why can shape validation still pass while predictions fail?
Need a hint?
RGB and BGR tensors have exactly the same dimensions and dtype.
Check your reasoning
The contract is semantically wrong but structurally valid: learned red-channel filters receive blue values and vice versa, so shape checks alone cannot detect it.
Neural networks
Connect the chapter to the course’s quantitative spine.
Image preprocessing is model code; version and test it from raw bytes to final tensor. The convolution output size below is the shared course-level lens that helps quantify the surrounding system. Name each symbol, check its units or shape, and use only values that were available at the production cutoff.
Convolution output size
| Symbol | Meaning / shape / unit |
|---|---|
in, out | spatial input/output size |
k | kernel size |
s | stride |
p | padding |
d | dilation |
Open derivation and numerical substitution
Start from the production quantity being optimized, substitute the observed values with their declared units, then isolate the model-controlled term. Preserve shape annotations at each step so broadcasting or aggregation cannot silently change the result.
- Write the named inputs: in, out, k, s, p, d.
- Substitute one small, hand-checkable batch before vectorizing.
- Calculate an independent reference value and compare within a declared tolerance.
# equation → code contract
inputs = validate_shapes_and_units(batch)
value = compute_c10(inputs)
assert is_finite(value)Change one assumption and make the tradeoff visible.
Use the worked chapter as your prediction, then change one input in the lab and explain why the output moves. The lab runs predefined TypeScript and never executes learner code, so the focus stays on the concept rather than setup.
Receptive-field assembler
Stack kernels and see receptive field, parameter count, and output size.
Assumption: 3×3 stride-one convolutions, 32 channels, 64×64 input, no dilation.
Open nonvisual data table
| Item | Computed state | Interpretation |
|---|---|---|
| layer 1 | 3×3 | 62×62 |
| layer 2 | 5×5 | 60×60 |
| layer 3 | 7×7 | 58×58 |
| layer 4 | 9×9 | 56×56 |
Trace the complete operating path.
Carry the chapter’s central idea into the system: Image preprocessing is model code; version and test it from raw bytes to final tensor. The path below shows where that reasoning needs an artifact, a test, an owner, or a fallback.
- 01
Turn “Specify the byte-to-tensor path” into a written invariant and a failing test.
- 02
Promote image_tensor_contract.py from illustration to a versioned implementation with representative fixtures.
- 03
Use “Prove raw-input parity” as the release checkpoint: Feed the same saved image bytes through training evaluation, batch scoring, and serving, then compare the final tensors and model outputs within tolerance.
- 04
Serve or materialize behind explicit one shift for inference and review.
- 05
Join telemetry to mature outcomes and retain a rollback path.
Observability
Join service health, input quality, prediction distributions, slice behavior, and mature outcomes by exact version.
Cost
Measure storage, preprocessing, compute, queueing, and human review under a representative arrival pattern.
Failure modes
Random burst-frame splitting puts near-duplicates of one product in train and validation. Add a detector, owner, mitigation, and stop condition for this class of failure.
Alternatives
Compare a rule, a simpler statistical baseline, and a different system boundary before adding model complexity.
Explain the contract, not just the vocabulary.
Rare-defect vision system
Design grouped evaluation, transfer learning, calibration, batch-one export, and camera-drift monitoring.
- Grouped split and audit
- Transfer-learning ablations
- PR and calibration report
- Export/latency/monitoring
Read primary material with a purpose.
Return to the opening failure.
Version color, resize, orientation, normalization, class map, and threshold; monitor camera/site slices.
For this lesson, the release evidence is the worked code example, a hand-checked formal result, deterministic simulation output, a ≥80% checkpoint, the production rubric, and a named fallback. The course resolves when the system can produce cnn and transfer-learning baselines with augmentation, calibration, and monitoring.