Calibration by forecast horizon
A 7-day forecast and a 1-year forecast are different animals. A single headline Brier hides that. Below, every resolved forecast is bucketed by its horizon— the time between publishing it and the market resolving — and scored on its own. “Beats chance” means the bucket’s bootstrap Brier CI sits entirely below 0.25 (random / always-50%).
n_resolved=0 · bins=5 · min_bucket_n=10 · source=live · schema=theseus.horizon_calibration.v1
no resolved forecasts in the < 7 days bucket — useful horizon cannot be extended past the last measured bucket
limiting bucket: lt7
Per-bucket reliability
Mean Brier with a non-parametric bootstrap CI, the calibration slope (OLS of outcome ~ probability; ≈ 1.0 is well-discriminated), and the bucket’s own base rate. Below n=10 we print the sample size and nothing else — a slope over a handful of points is noise.
| Horizon | n | Mean Brier | Brier CI | Slope | Slope CI | Base rate | Verdict |
|---|---|---|---|---|---|---|---|
| < 7 days | 0 | — | — | — | — | — | no data |
| 7–30 days | 0 | — | — | — | — | — | no data |
| 30–90 days | 0 | — | — | — | — | — | no data |
| 90–365 days | 0 | — | — | — | — | — | no data |
| > 365 days | 0 | — | — | — | — | — | no data |
Implication for new forecasts
The firm has not yet established a useful horizon at any range — there is not enough resolved data to claim signal. Until that changes, everynew forecast is issued with the explicit “low confidence” framing, regardless of horizon.
Method × horizon
Some methods calibrate well on short horizons but decay fast on long ones. This cross-tab pairs each originating method with the horizon bucket its forecasts landed in.
No method × horizon cells. The resolved forecasts in this view carry no method→outcome attribution (Round 17 prompt 02 link) — this populates once the nightly manifest emits a horizon_calibration block with method linkage.
Notes
- Live fallback: the nightly manifest carries no horizon_calibration block. Per-bucket Brier, calibration slope, bootstrap CIs and the useful-horizon estimate are recomputed live from the database; the canonical estimator is noosphere.coherence.horizon_calibration.
- Method × horizon attribution requires the nightly manifest — published forecasts carry no method→outcome link on the live path.