Forecast
Cumulative Entries
Milestones and History
Key Milestones
Historical Comparison
Registration Curve and Fees
Registration Curve Pattern
Registration Fees
About This Model
About This Model
Ensemble model (N5v4): historical ratio + per-family Huber regression, predicting final count from current entries at each lead time. 80% confidence intervals (CIs) from lognormal fits with IQR outlier removal. Leave-one-out blind-tested on held-out 2023–2026 data.
Tournament Overview
Tournament Overview
| Tournament | Status | Event Date | Current | Predicted | Likely Range | Progress |
|---|
Daily Chess Puzzles
♔ This Day in Chess History
Tournament Results
CCA Tournament Entry Prediction Model
A plain-English explanation of how the model works, how it was tested, and where it falls short.
Model Health · last pipeline run, calibration vs claim, fallback rates
output/audit_warnings.json, output/performance_data.json, and output/website_data.json.
The Core Idea
Every CCA tournament follows a registration curve: entries trickle in slowly, then accelerate as the event nears. The model exploits this pattern. If a tournament has 180 entries with 60 days to go, and the same tournament historically had ~170 at that point and finished with ~900, we expect roughly 900 again. We compute the ratio of final-to-current entries at the same lead time across all prior years, average them, and multiply: 180 × 5 = 900.
Why Two Models?
The ratio approach works well with good history. For newer tournaments or unusual patterns, we also run a regression: a best-fit line through all data points. The final prediction blends both. Close to the event (last few days), we trust the ratio more (80% weight) because the current count is highly informative. Months out, we lean on regression (85% weight) because the count is still noisy.
Ensemble Equation
The predicted final entry count is a weighted blend of the two sub-models:
| T (days to end) | ≤ 3 | ≤ 7 | ≤ 28 | > 28 |
|---|---|---|---|---|
| w(T): ratio weight | 0.80 | 0.55 | 0.30 | 0.15 |
Historical Ratio Model (Primary)
For each tournament family at lead time T, compute final/current ratios from every prior year. Central estimate uses the harmonic mean (downweights outlier-high ratios):
When T falls between historical chop points, ratios are blended in log-space via inverse-distance interpolation.
Huber Regression Model (Secondary)
Per-family robust linear regression (Huber loss, ε = 1.35) trained on all historical (count, T) → final pairs. Falls back to a size-matched global model for unknown families:
Confidence Intervals & Sanity Checks
Every prediction includes a confidence interval: a range the actual result has fallen within about 75% of the time at two weeks out in backtesting. The interval targets 80%, so it runs slightly tight, and tighter still close to the event, where coverage drops further. Wider far from the event, narrower as it approaches.
s(T) calibrated via LOO for 80% coverage. Shrink(T) = 0.33 at T ≥ 60 → 0.75 at T < 5. σ floored via variance regularization for families with ≤ 3 editions. Small adjustments (1–5%) correct for year-over-year trends, historical withdrawals, and near-capacity dampening. Predictions that drift too far from historical norms are pulled back via plausibility bounds.
Sequential Adjustments
| Adjustment | Trigger | Effect |
|---|---|---|
| Late-surge damping | Scholastic families, T > 3 | Cap ratio at 1.1 + 0.4 · min(T/90, 1) |
| Fill-% shrinkage | Count > 60% of family mean | Shrink ratio toward 1.0 (max 20%) |
| Family anchor | Count < threshold, T ≥ 42 | Blend with 0.6 · recent + 0.4 · mean final |
| Growth trend | T ≥ 7 | Multiply by 1 + trend · 0.5 (capped ±15%) |
| Withdrawal correction | Historical data available | Reduce by median withdrawal rate |
| Plausibility bounds | Pred < 70% of family median | Blend toward historical median |
| Edition widening | 0–1 prior editions | Widen CI by 2.5× / 1.5× |
How We Tested It
We never test on data the model has seen. We train through (say) 2023, then predict every 2024 tournament as if living in early 2024: no future information leaks in, and each tournament is held out of its own training set. Across 136 blind-tested tournaments, at two weeks out the median miss was 7.4% and the 80% interval captured the true result 78% of the time. Coverage three days out sits near 76%. Earlier releases ran overconfident close to the event (three-day coverage as low as 67%) — exactly when people most want the number — and the 2026 recalibration widened short-lead intervals to close most of that gap. The full lead-time breakdown is on the Performance tab.
Validated via leave-one-out expanding-window blind test: train on years ≤ Y (holding out the target tournament), predict at each observed lead time. No future or in-sample data leakage. 133 tournaments across 2023–2026.
What We Tried and Rejected
We tested several alternatives on the same blind test. None improved accuracy. The figures below are the relative model-selection comparison and predate the leave-one-out leak fix, so the absolute coverage numbers run higher than the honest figures above; read them as a ranking of configurations, not current accuracy.
| Configuration | MedAPE | MAPE | 80% Cov | Verdict |
|---|---|---|---|---|
| Harmonic-mean ratio only | 8.2% | 14.8% | 89% | Good base, wide CIs |
| + Huber regression ensemble | 7.6% | 13.0% | 91% | Selected |
| + YoY pacing (direct blend) | 8.5% | 15.0% | 88% | Adds noise |
| + YoY pacing (dampened) | 7.9% | 14.1% | 89% | Marginal harm |
| + Recency-weighted ratios | 7.8% | 13.4% | 82% | CI coverage crash |
| Log-linear regression | 7.9% | 13.5% | 90% | No improvement |
| Aggressive fill-% shrink | 7.7% | 13.2% | 90% | Neutral → rejected |
Selection rule (at model-selection time): keep only changes that improve Median APE without degrading 80% CI coverage.
Prior Approaches
Earlier prediction methods used at CCA provided the foundation for the current model:
| Metric | Manual Multiplier Approach | Ratio Model | Current Ensemble Model |
|---|---|---|---|
| Method | Fixed multiplier × current entries; multiplier from days-to-event lookup, chosen subjectively | Historical ratio (entries_at_T / final), median across past 4 years | Ensemble: ratio-based + Huber regression, T-dependent weighting |
| Error Rate | ~20% median (est.) | ~12–15% median (est.) | 7.1% median APE at 2 wks (blind tested) |
| Prediction Intervals | None | MAE-based error bars; not calibrated intervals | Lognormal calibrated 80% CI; 75% coverage at 2 wks |
| Blind Testing | No formal backtesting | No formal backtesting | 133 tournaments, 2023–2026 leave-one-out expanding window |
| Limitations | Multiplier updated by feel; no systematic validation | Single-deadline events only; no multi-deadline handling | Needs 3+ yrs history; cannot anticipate one-off disruptions |
Each approach built on lessons from its predecessor. The prior models’ intuition, that historical ratios are the strongest predictor, remains the core of the current system.
Worked Example: Chicago Open 2026
Forward-looking estimate: 180 entries, 62 days to end (May 21–25, 2026).
Step 1: Historical Ratios at T ≈ 60
| Year | Count at T≈60 | Final | Ratio |
|---|---|---|---|
| 2022 | 168 | 944 | 5.62 |
| 2023 | 161 | 960 | 5.96 |
| 2024 | 191 | 860 | 4.50 |
| 2025 | 188 | 899 | 4.78 |
Harmonic mean = 4 / (1/5.62 + 1/5.96 + 1/4.50 + 1/4.78) = 5.15 → Fratio = 180 × 5.15 = 927
Step 2: Huber Regression
Freg = β0 · 180 + β1 · 62 + β2 ≈ 907
Step 3: Ensemble Blend
At T = 62 (> 28), ratio weight w = 0.15:
Fpred = 0.15 × 927 + 0.85 × 907 = 910
Step 4: Adjustments
| Adjustment | Calculation | Result |
|---|---|---|
| Growth trend (−3.4%/yr) | 910 × (1 + (−0.034) · 0.5) | 894 |
| Withdrawal corr. (1.2%) | 894 × (1 − 0.012) | 883 |
| Plausibility bounds | Below 70% of family median (916); blend | 900 |
The plausibility check prevents compounding small adjustments from pushing the prediction unreasonably far from historical norms. Final point estimate: ~908 (after CI re-centering in log-space).
Step 5: Confidence Interval
From lognormal fit on log-ratios, calibrated via LOO with T-dependent shrinkage:
σ = std(log(r)) = 0.127 → raw CI = [788, 1046] → × shrink(0.33 at T≈60) → CI = [856, 963]
Limitations
The model predicts from historical patterns. It cannot anticipate one-off events: venue changes, competing tournaments on the same weekend, entry fee changes, or external disruptions. It works best with 3+ years of history; brand-new events get wider confidence ranges. This model is a baseline; tournament directors should apply manual adjustments for known upcoming factors.