By: Patrick W. Zimmerman

France saved the gol-o-mático’s bacon.  In a World Cup full of upsets, busts, and little nations that could, the team that we thought would be at the top of the heap at the beginning of the tournament took home the trophy.  In pretty much any tournament model, if you can get the champion right, everything else kind of gets forgiven (and distracts people from asking about Poland, who we grossly overrated).

In the end, our model performed pretty admirably.  We got 14 of the 16 knockout stage teams correct, and, once the tournament reseeded, 3 of 4 semifinalists, calling the correct 90-minute result (W-D-L) a decent (though improvable) 54.8% of the time.

The question

So, does it look like historical performance relative to player pool size matters?

The short-short version

It seemed to matter at the extremes of the “punching above their weight” scale, at least in this tournament,.  That is to say countries that have done well with smaller-than-expected player pools were correctly given a bump (Belgium, Uruguay, Croatia), and countries that have done badly in spite of large player pools (Egypt!) were docked in the ratings.

The middle seems to be more of a muss.  That makes some sense in a new model, and means that we need to work on the scaling.

Most importantly, however, we need to apply this to as many international matches (non-friendlies) as possible, as running it on just World Cups (obviously) restricts our sample size.

The final results

It’s not every day that your model beats the FiveThirtyEight, and we really, really wish we could claim that the one-pick difference was super meaningful and gave us bragging rights for the next 4 years.  But….ah….nope.  The random variation of the tournament (particularly one with so many late winners and equalizers) more than accounts for such a small difference in accuracy.

That said, it definitely does feel like a validation that the gol-o-mático performed comparably to the modeling experts and the odds-makers.  Also that the eternal truism that the FIFA rankings seriously suck held up (though it did better in the knock outs than in the group stage).

Model performance since the beginning of the tournament

In bracket form, things also look pretty good.  Yeah, we had a couple of whiffs indicated by the red mistakes, but so did everyone else.  It was inconceivable that Germany was going to go out with losses to México and (especially) South Korea, and Spain’s inability to turn 71% possession into a single goal against Russia in the round of 16 was almost equally shocking.  Those two, combined with Argentina’s barely scraping through their group, left the right half of the bracket wide open, and allowed Croatia to advance to the final never facing a team stronger than Gareth Southgate’s good-but-not-great young English Lions.  Yes, Modric was a worthy winner of the Golden Ball as tourney MVP, having dragged a mediocre team with one world-class unit (the midfield) through to the final.  But we really doubt even his performance would have been good enough had they not won their group and ended up having to go through some combination of France, Uruguay, Belgium, Portugal, and Brazil.

What next?

We’re already tossing around ways to improve the model, such as weighting teams more heavily on the more extreme ends of the performance per available player scale and using a combined poll metric as our baseline for current team rating (normalizing a bunch of different polls / rating systems and then taking the average of the z-scores, basically).

And what do you know, there are a handful of perfect test cases coming up: The Women’s World Cup in France and the Copa América in Brazil next year as well as the Euros in 2020 (spread out over 12 countries, semis & final in London).