Does an estimated club effect persist after a head-coach change?

The question

A programme’s estimated effect is attributed to the programme, not to a named coach, because the estimator cannot separate the two inside a single tenure. When the head coach leaves, that attribution stops being academic: a federation or a hiring club wants to know whether the number belonged to the coach or to something more durable in the programme. This article asks a narrower, checkable version of that question — after a head-coach change, how much of the programme’s prior estimated effect is still there at 24 months?

What was compared

214 programmes across 19 federations recorded a head-coach change between 2020 and 2024 and had a published estimated effect, under the coach effect estimator v1.4, in both the four seasons before the change and the two seasons after. For each programme, the ratio is the estimated effect at 24 months post-change divided by the estimated effect in the season immediately before it. A ratio of 1 means no change; a ratio of 0 means the programme’s estimated effect returned to the cohort expectation.

Result

The estimated ratio, pooled across the 214 programmes, is 0.50, 95% interval [0.31–0.68]. Roughly half of a programme’s estimated effect, on average, is still measurable 24 months after the coach who was there when it was estimated has left. The interval is wide: it does not distinguish a programme that kept a third of its prior effect from one that kept two thirds.

Programmes where the incoming coach was promoted from within the same programme (61 of 214) show a higher ratio — 0.71, 95% interval [0.42–0.94] — than programmes that hired externally (153 of 214) — 0.42, 95% interval [0.21–0.63]. The two intervals overlap, so this comparison is not statistically distinguishable at the 95% level; it is reported because it is the first place a reader would look for the effect to concentrate, not because the record settles it.

What this does not support

• That the coach did or did not matter. A ratio below 1 is consistent with the departed coach having been the source of the effect, and it is also consistent with an incoming coach who has not yet had two full seasons to rebuild it, and with turnover among the swimmers themselves in the same window.
• A verdict on any named programme. The pooled ratio describes 214 programmes’ history and is not a rate any single programme is expected to match. A programme’s own page states its own estimated effect before and after any coach change inside the fitted window, with its own interval.
• A recommendation to promote from within. The internal/external comparison above does not clear the 95% threshold this site publishes at.

Method and limitations

The estimator holds age, event, prior improvement rate and federation; it does not hold squad composition, so it cannot attribute any part of a change in the estimate to the swimmers who joined or left a programme in the same window as the coach change — a limitation stated on the method page and repeated here because it is the central threat to this comparison specifically. The 214 programmes are the ones with a published estimate on both sides of the change; 89 further programmes recorded a coach change in the same period but fell below the 40-swimmer-season publication threshold in at least one of the windows compared, and are not part of this figure. That is a real exclusion, not a rounding one: programmes small enough to lose their published estimate after losing swimmers in a coach transition are exactly the programmes a reader might most want this comparison to cover, and they are the ones it does not.