Transfer learning for classification, detection, and segmentation
Transfer the representation in measured stages, while letting the required output and leakage-safe split define the task.
A factory has few positive examples of costly visual defects.
This chapter moves from choose output from the decision to evaluate the correct geometry. We will make the idea concrete in a small code example before placing it back inside a production system with one shift for inference and review.
Use class precision/recall for classification, IoU and mAP for localization, per-class mask metrics for segmentation, plus calibration and batch-one latency.
Verify the behavior illustrated in transfer_classifier.py, then measure decision utility, slice reliability, and system latency.
Random burst-frame splitting puts near-duplicates of one product in train and validation. An unsafe release must degrade to a named baseline or the last known-good version.
Build the idea one layer at a time.
Pretrained vision features can replace months of representation learning, but transfer starts with the product output: one label for an image, boxes for objects, or a class for every pixel.
Classification answers what is present globally; detection adds where each object is; segmentation gives dense boundaries. Choose the least complex output that supports the action because annotations, evaluation, latency, and failure modes grow sharply from classification to dense prediction.
A common transfer sequence freezes a pretrained backbone, trains a new task head, then gradually unfreezes later blocks with a smaller learning rate. Input normalization must match the pretrained weights. Group splits by physical object, capture session, site, or patient prevent near-duplicate images from overstating transfer quality.
- transfer learning
- IoU/mAP
- group leakage
- camera drift
Follow the reasoning before touching the code.
- 01
Choose output from the decision
If one image-level flag triggers review, classify; if location guides repair or measurement, use detection or segmentation and accept its annotation cost.
- 02
Adapt in stages
Replace the task head, train it while the backbone is frozen, then unfreeze selected layers only if grouped validation shows underfitting.
- 03
Evaluate the correct geometry
Use class precision/recall for classification, IoU and mAP for localization, per-class mask metrics for segmentation, plus calibration and batch-one latency.
import torch
from torchvision.models import ResNet18_Weights, resnet18
model = resnet18(weights=ResNet18_Weights.DEFAULT)
for parameter in model.parameters():
parameter.requires_grad = False
features = model.fc.in_features
model.fc = torch.nn.Linear(features, 3)
trainable = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.AdamW(trainable, lr=1e-3)
print(sum(p.numel() for p in trainable), 'trainable parameters')Freeze a torchvision backbone and replace its classification head.
What to notice
- Only the newly created three-class head is trainable, giving a fast baseline before the backbone is altered.
- The pretrained weight bundle implies a specific image preprocessing contract; reusing weights without its normalization is not equivalent transfer.
YOUR TURN Pause and predict before revealing the answer.
Validation plateaus because the source and target domains differ strongly. What controlled next step tests whether representation adaptation helps?
Need a hint?
Do not unfreeze everything at the original head learning rate.
Check your reasoning
Unfreeze the last backbone stage, use a smaller learning rate for it than for the head, and compare under the same grouped validation split.
Lessons 1, 2, 3 in this course.
Connect the chapter to the course’s quantitative spine.
Transfer the representation in measured stages, while letting the required output and leakage-safe split define the task. The convolution output size below is the shared course-level lens that helps quantify the surrounding system. Name each symbol, check its units or shape, and use only values that were available at the production cutoff.
Convolution output size
| Symbol | Meaning / shape / unit |
|---|---|
in, out | spatial input/output size |
k | kernel size |
s | stride |
p | padding |
d | dilation |
Open derivation and numerical substitution
Start from the production quantity being optimized, substitute the observed values with their declared units, then isolate the model-controlled term. Preserve shape annotations at each step so broadcasting or aggregation cannot silently change the result.
- Write the named inputs: in, out, k, s, p, d.
- Substitute one small, hand-checkable batch before vectorizing.
- Calculate an independent reference value and compare within a declared tolerance.
# equation → code contract
inputs = validate_shapes_and_units(batch)
value = compute_c10(inputs)
assert is_finite(value)Change one assumption and make the tradeoff visible.
Use the worked chapter as your prediction, then change one input in the lab and explain why the output moves. The lab runs predefined TypeScript and never executes learner code, so the focus stays on the concept rather than setup.
Receptive-field assembler
Stack kernels and see receptive field, parameter count, and output size.
Assumption: 3×3 stride-one convolutions, 32 channels, 64×64 input, no dilation.
Open nonvisual data table
| Item | Computed state | Interpretation |
|---|---|---|
| layer 1 | 3×3 | 62×62 |
| layer 2 | 5×5 | 60×60 |
| layer 3 | 7×7 | 58×58 |
| layer 4 | 9×9 | 56×56 |
Trace the complete operating path.
Carry the chapter’s central idea into the system: Transfer the representation in measured stages, while letting the required output and leakage-safe split define the task. The path below shows where that reasoning needs an artifact, a test, an owner, or a fallback.
- 01
Turn “Choose output from the decision” into a written invariant and a failing test.
- 02
Promote transfer_classifier.py from illustration to a versioned implementation with representative fixtures.
- 03
Use “Evaluate the correct geometry” as the release checkpoint: Use class precision/recall for classification, IoU and mAP for localization, per-class mask metrics for segmentation, plus calibration and batch-one latency.
- 04
Serve or materialize behind explicit one shift for inference and review.
- 05
Join telemetry to mature outcomes and retain a rollback path.
Observability
Join service health, input quality, prediction distributions, slice behavior, and mature outcomes by exact version.
Cost
Measure storage, preprocessing, compute, queueing, and human review under a representative arrival pattern.
Failure modes
Random burst-frame splitting puts near-duplicates of one product in train and validation. Add a detector, owner, mitigation, and stop condition for this class of failure.
Alternatives
Compare a rule, a simpler statistical baseline, and a different system boundary before adding model complexity.
Explain the contract, not just the vocabulary.
Rare-defect vision system
Design grouped evaluation, transfer learning, calibration, batch-one export, and camera-drift monitoring.
- Grouped split and audit
- Transfer-learning ablations
- PR and calibration report
- Export/latency/monitoring
Read primary material with a purpose.
Return to the opening failure.
Version color, resize, orientation, normalization, class map, and threshold; monitor camera/site slices.
For this lesson, the release evidence is the worked code example, a hand-checked formal result, deterministic simulation output, a ≥80% checkpoint, the production rubric, and a named fallback. The course resolves when the system can produce cnn and transfer-learning baselines with augmentation, calibration, and monitoring.