Casebook/Case 18
Drift & incidents

Monitoring ML models for bot detection

Observing an adversarial classifier when labels are partial and the traffic distribution fights back.

Reported by the primary sourceFACT LAYER

What we can attribute directly

Cloudflare published how it monitors machine-learning models used for bot detection.

Bot classification operates in a changing, adversarial environment.

Read Cloudflare — Bot model monitoring Primary source · last checked 26 Aug 2026
01 · Problem & constraints

The operating envelope

Delayed or imperfect labels, adversarial adaptation, massive volume, customer impact, and multiple traffic slices.

Actors

Model teams, platform owners, operators, downstream product systems, and people affected by decisions.

Evidence

Versioned data, configs, traces, artifacts, deployments, and outcomes aligned on one timeline.

Failure cost

Service metrics stay green while score distributions or decision quality change for a specific traffic class.

02 · Architecture reconstruction

Trace the system before naming the bug.

  1. 01

    Producers emit versioned data or model artifacts.

  2. 02

    A platform validates, computes, stores, schedules, or routes them.

  3. 03

    Training or inference consumes the exact declared version.

  4. 04

    Telemetry joins the decision to system, data, and model identity.

  5. 05

    Operators compare outcomes, stop conditions, and the last known-good path.

03 · Symptoms & investigation

Follow the evidence boundary by boundary.

Symptoms

Service metrics stay green while score distributions or decision quality change for a specific traffic class.

Investigation

Join model version, scores, rules, challenges, customer signals, and later outcomes by slice.

DIAGNOSTIC EXERCISE

Score mean is stable, but one browser family reports blocks. Which slice and outcome joins expose the issue?

Open investigation scaffold
  1. Write the earliest known-bad timestamp.
  2. Compare exact identities on either side of that boundary.
  3. Find the smallest affected slice and a known-good counterexample.
  4. Separate mitigation from root-cause confirmation.
04 · Root cause & fix

Repair the contract, not only the symptom.

ROOT CAUSE

Infrastructure-only monitoring cannot see a model’s semantic decision health.

FIX

Layer data, score, action, customer, and outcome telemetry with sampled investigations.

Rollout

Shadow decision changes, canary by controlled slice, and keep deterministic mitigation rules.

05 · Rejected alternatives

Reason about the tempting shortcuts.

  • A single global score mean.
  • Waiting for one perfect ground-truth label stream.
06 · Monitoring after the fix

Make recurrence visible early.

01

Score and action distributions by slice

Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.

02

Challenge/pass outcomes

Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.

03

Customer complaints and false-positive probes

Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.

REUSABLE PRODUCTION PATTERN

Adversarial ML requires joined telemetry and active probes, not passive averages.

Carry this pattern into assignments as a design constraint and into incident reviews as a hypothesis—not as proof about an unpublished system.