The assurance artefact
Mode A assurance artefactNot a threshold we picked. A miss rate we certified.
A distribution-free ceiling on how often a violation gets through, at a stated confidence, on a labelled corpus you can inspect.
The certificate, not the setting
The cut-off comes from the guarantee, not from tuning after the fact.
A distribution-free risk-control procedure, Learn-then-Test, RCPS, conformal risk control, runs over labelled held-out data. The threshold λ is derived from the target miss rate α at confidence δ, finite-sample, assuming only exchangeability of the calibration set.
one-directional, never backwards
The adversarial upgrade
Two kinds of certificate, know which one you're getting.
Base certificate
Bounds error against the honest distribution, and says nothing about an adversary.
Certified-detection lane
Randomized/denoised smoothing or a Lipschitz-bounded embedding extends the bound to a bounded adversary within a stated ℓ2 perturbation budget, the one construct that retires a failure surface rather than naming it.
Conformal bounds honest-distribution error. Certification bounds a bounded adversary. Say which one you bought.
Why it stays true
We keep checking the guarantee holds; we don't just assume it.
Calibration-validity monitor
Raises an attested guarantee-invalidation event on breach.
Weighted conformal
Coverage survives episodic decay with a bounded gap.
Adaptive conformal
Long-run coverage under continuous drift.
What you hold
A signed certificate, not a setting someone can turn.
Re-certification fires on model change, measured shift, or calendar, not discretion.
See a sample certificate →