The Lab What a surprise QB start really costs
What a surprise QB start really costs
We tested four feature layers against our NFL margin model, one at a time, across four development seasons. Two made it worse and one barely moved it. The one that survived says a team starting anyone but its real starter costs about 2.8 points.
Our NFL margin model has one line that reads minus 2.7700 times qb_diff. That single number is the entire cost of a surprise quarterback start, and it is the only one of the four layers we tested that moved the number enough to mean anything. Rest differential did not. A per team home field adjustment did not. Roster age barely did, and barely is being generous. Here is how we tested each one and what the survivor is actually saying.
The layers we tested one at a time
Our base model rates all 32 teams from a ridge regression on game margins, then converts that rating gap into a predicted margin for any matchup. The question we asked next was simple: does anything else about a game, known before kickoff, sharpen that prediction on games the coefficients were not fit on?
We added four candidate layers one at a time to the base rating gap, fit each coefficient on 2004 through 2021 only, then froze it and scored the whole model on 2022 through 2025. A layer stayed only if it lowered the average margin error across those four seasons. That rule matters for how to read the table: because 2022 through 2025 decided which layers stayed, those are development seasons, not an untouched test. We said so in a correction to the model notes on 2026-09-21, and the clean forward test of our NFL board started with week 3 of this season.
| Model | Avg margin error, 2022-2025 (development seasons) | Kept |
|---|---|---|
| Rating gap only | 10.0335 | base |
| + rest differential | 10.0397 | no |
| + roster age differential | 10.0322 | yes |
| + QB disruption flag | 10.0000 | yes |
| + per team home field | 10.0869 | no |
Two of the four made the 2022 through 2025 predictions worse. Rest differential, the thing you’d assume matters most after a short week or a bye, moved the error the wrong way. So did letting each team carry its own home field number instead of one league constant. Both got cut, no matter how reasonable they sound at a bar.
What made the cut, and how much it’s worth
Roster age cleared the bar, but only by 0.0013 points of error, which is noise that happened to land on the right side of zero. We keep the coefficient in the model because the rule says any improvement keeps a layer, but we don’t read anything into its sign or size. Older rosters were very slightly better in this sample, and that’s a fact about 2022-2025, not a mechanism to trust.
The quarterback flag is a different story. It takes the error from 10.0322 to 10.0000, a full 0.032 points of average margin error, by itself the largest improvement of any layer we tested. The final model reads:
margin = 2.122 + 1.0453 * rating_diff + 0.0153 * age_diff - 2.7700 * qb_diffwith a residual sigma of 13.553 points from the pre-2022 fit. Each team’s flag is binary: it fires when that team’s starter for the game is not its expected starter, defined as the modal starter across that team’s prior weeks this season, or its primary starter from the previous season in week 1. qb_diff is the home flag minus the away flag, so two teams both starting someone unexpected cancel out. The source is edgelabs.nfl_qb_starts, built off nflverse’s listed starters across 2002 through 2025 with zero missing game sides. When the flag fires for one side only, 2.77 points come off that team’s predicted margin. Nothing about the backup’s own quality enters the number. The model doesn’t know if the backup is good. It only knows he isn’t the guy the offense was built around.
What 2.77 points does to a coin flip game
Hold everything else fixed, rating gap and home field included, and put two teams in a game the model would otherwise call even. Win probability comes from the normal curve of the predicted margin over the residual sigma: a 0 point margin gives exactly 50 percent. Flip on the QB flag for one side and the predicted margin moves to minus 2.77, which prices out to about 41.9 percent to win, an 8.1 percentage point swing from one binary fact about who’s starting under center. That’s the entire mechanism. No adjustment for scheme, no adjustment for which backup, just the flat cost of not having your starter.
The 2026 walk, as of August
In the 2026-08-17 inputs behind our preseason numbers, 6 of the league’s 32 teams projected a week 1 starter who was not that team’s 2025 primary passer: Atlanta (Michael Penix to Tua Tagovailoa), Cleveland (Shedeur Sanders to Deshaun Watson), Las Vegas (Geno Smith to Kirk Cousins), Miami (Tua Tagovailoa to Malik Willis), Minnesota (J.J. McCarthy to Kyler Murray), and the Jets (Tyrod Taylor to Geno Smith). Those flags came from a hand built table of projected week 1 starters against 2025 starts of record. Every later week of that preseason walk assumed each team’s expected starter played, because no in-season absence is knowable in August.
That is also why the flag is a small part of any preseason projection. In a game the model would otherwise call even, the 8.1 percentage point swing above is worth about 0.08 expected wins, and in the August walk it fired for those six teams in week 1 only: a fraction of one game out of seventeen. Week 1 has been played, so no remaining 2026 game carries the flag in that walk today. Most of what separates one team’s preseason number from another’s comes from the rating itself, built on how each team actually played in the seasons before.
What this doesn’t claim
This margin model beat its own predecessor on average margin error in all four of the 2022 through 2025 development seasons, and the quarterback layer is a real piece of why. It does not beat the closing spread on that same margin test, and across those same four seasons it went 504-554 against the spread, 47.64 percent, below 50 percent in three of the four. So this model fronts expected wins and margin context, the way it’s used above, never a spread pick. The game number on our board, which our analyst panel weighs as one input, is a simpler calibrated margin that leaves the age and QB layers out, so the 2.77 in this article is a finding about the full model, not an adjustment sitting inside any pick.
Preseason injuries at any position other than the week 1 QB spot, trades and cuts after the roster snapshot, rookies who aren’t a projected starter, and any in-season QB change past week 1 stay outside what the preseason walk sees. So do coaching changes, since the new_hc flag never became a model input, along with weather, roof, surface, altitude, travel distance, suspensions and holdouts, and anything that changed after the 2026-08-17 capture. It prices exactly one known fact well. It doesn’t pretend to know the ones that haven’t happened yet.