Evaluation protocol
- 01Pre-event freezing
- 02Same-cutoff controls
- 03Availability timestamps
- 04Outcome settlement
- 05Deterministic scoring
- 06Independent adjudication
Measures and controls
Brier score
Squared error for probabilistic forecasts.
Log loss
Penalizes confident incorrect probabilities.
Calibration
Checks whether stated probabilities match observed frequencies.
Coverage and abstention
Reports when the system speaks and when it declines.
Block bootstrap
Estimates uncertainty while respecting grouped or serial dependence.
Superiority boundary
Hit rate, paper return, or closing-line value cannot substitute for probability accuracy. A superiority claim requires the frozen prospective protocol, sufficient sample, deterministic replay, and independent adjudication.