How the intervals are calibrated

The claim being tested

A 95% interval makes a frequency claim: across many published estimates, the true value should fall inside the stated range about 95 times in 100. That claim is checkable against held-out data — this page is the check.

How it is checked

Races are held out of the fit and the model is asked to predict them. Each prediction produces an interval; each held-out race produces an observation. The proportion of observations falling inside their intervals is the empirical coverage, and it is compared against the nominal 95%.

Coverage is reported per stratum — by age band, by number of races, by pool type, by federation — because a model can hit 95% overall while badly under-covering a subgroup, and the overall figure then conceals the failure a reader in that subgroup needs to know about.

What the check has found

Against 41 902 held-out season figures, 2019–2026, 94.2% fell inside their stated 95% interval — eight-tenths of a point below nominal, and not evenly spread. By race count in the window: 96.1% for swimmers with ten or more races (n = 28 774), 91.8% for swimmers with five to nine (n = 9 950), and 84.3% for swimmers with fewer than five (n = 3 178). Below five races the stated 95% interval is, in practice, closer to an 84% one. Rather than widen every interval to hide the difference, Swimetrica reports the race count beside every estimate, so a reader can see which regime a figure is in.

When a figure is wrong

Corrections are published, dated, and kept. A figure that changes carries the date it changed and the reason; both stay attached to the corrected figure permanently. See corrections for the record.