Classifier validation
The ADI, ASDI, and ALWR classifiers are audited quarterly against a manually-labelled sample of 200 advertisements per series. This page reports precision, recall, and the false-positive rate per classifier, alongside inter-rater agreement between two labellers.
Latest audit
The first validation audit publishes alongside the first monthly report, which is in preparation. Until then, time-series charts are withheld rather than shown with a provisional band.
| Classifier | Precision | Recall | False positive |
|---|---|---|---|
| AI Diffusion classifier | Pending | Pending | Pending |
| AI-Specialist classifier | Pending | Pending | Pending |
| Workforce-reduction attribution | Pending | Pending | Pending |
How the audit works
- A random sample of 200 advertisements per series is drawn from the measurement window.
- Two labellers independently score each advertisement against the published definitions. Inter-rater agreement is reported alongside the headline accuracy figures.
- Classifier output is compared against the agreed labels to compute precision, recall, and the false-positive rate.
- The displacement classifier is audited continuously: every event passes through a human reviewer before publication.
- The labelled sample for each audit is committed to the repository under
audit/so the figures can be reproduced.