World Cup Predictions: Most models underestimate the chance of a tie.

By: Patrick W. Zimmerman

Soccer has a lot of ties.  This is not a mystery.

So why do most models severely under-account for them?

Looking at the modeling experts at thefivethirtyeight, the composite betting line from oddsportal, and Principally Uncertain’s own model all treat ties as rare events, even though more than a quarter of World Cup games all time have ended in draws after 90 minutes.  Through games of Monday 6/25, this World Cup looks as if ties are a little down from their normal rate (probably related to the ridiculous number of late late late goals), but at 19.44% (vs. 25.7% all time, 26.6% in 2014), that’s not really way out of the normal with only 36 games played so far.

Ties are a thing.  Get used to it.


The question

Are we, collectively as soccer analysts, under-weighting the possibility of a tie? 


The short-short version

Oh, yeah we are.  Big time.

If one looks at only results after 90 minutes (to compare games on an even playing field), models based on head-to-head matchups produce predictions that are way below the historical rate for ties.  Even if you separate out knock-out games (where extra time and penalty wins technically count as 90min draws), it’s still way way low

Principally Uncertain’s model suspected this might be the case, so we doubled the weight on ties.  Nnnoppppeeee.  We’re still at around 50% of what should be the expected historical result.


Ties over time

Mouseover for details.

Here is just group games (1934 & 1938 did not have a group stage, 1950 had only group stages).

Mouseover for details.

So how are we doing compared to the smart people elsewhere in the world?  Not bad!  Much better than the FIFA rankings, with the betting line in the lead, FiveThirtyEight a tick behind, and us just behind that.  Remember, the FIFA rankings’ “tie” result is us eyeballing a 150pt difference in FIFA team rating and calling that a draw (which apparently was pretty close).

Model comparisons after 36 group games

Model scoreboard
Model Points Points % Correct results Correct % Ties predicted Tie %
Betting Markets 24⅓ 0.676 22 0.611 0 0.00%
The FiveThirtyEight 23⅔ 0.657 21 0.583 1 2.78%
Principally Uncertain 23 0.639 20 0.556 4 11.11%
FIFA rankings 20 0.556 16 0.444 7 19.44%


Checking your work: Comparing models to reality

This is endemic.  While it’s understandable that the betting line would tend to eschew ties, as it’s based on the collective wisdom of the betting public (and the public clearly expects draws less often than it should).  That makes sense.

FiveThirtyEight, though, has the same problem we do.  They’re actually even lower, only forecasting one single tie during the group stages as a max-probability result from any game (Uruguay v. Russia.  Which they got wrong). 

This brings up the issue of tournament aggregate patterns v individual game forecasting.  If one is producing results based on a game-by-game basis, a tie that is only a little less probable than one or another team winning isn’t counted, whereas over the course of a whole tournament, that result will happen a fair amount.  It’s just obscured at the single-game level.

Stage Game P? 538 Odds FIFA Actual result
Group A Russia v. Saudi Arabia RUS RUS RUS TIE RUS, 5-0
Group A Egypt v. Uruguay URU URU URU URU URU, 1-0
Group A Uruguay v. Saudi Arabia URU URU URU URU URU, 1-0
Group A Russia v. Egypt RUS RUS RUS EGY RUS, 3-1
Group A Uruguay v. Russia URU TIE URU URU URU, 3-0
Group A Saudi Arabia v. Egypt RSA EGY EGY EGY RSA, 2-1
Group B Portugal v. Spain TIE ESP ESP TIE TIE, 3-3
Group B Morocco v. Iran MAR MAR MAR TIE IRN, 1-0
Group B Portugal v. Morocco POR POR POR POR POR, 1-0
Group B Iran v. Spain ESP ESP ESP ESP ESP, 1-0
Group B Iran v. Portugal POR POR POR POR TIE, 1-1
Group B Spain v. Morocco ESP ESP ESP ESP TIE, 2-2
Group C France v. Australia FRA FRA FRA FRA FRA, 2-1
Group C Peru v. Denmark DEN DEN DEN TIE DEN, 1-0
Group C France v. Peru FRA FRA FRA TIE FRA, 1-0
Group C Denmark v. Australia DEN DEN DEN DEN TIE, 1-1
Group D Argentina v. Iceland ARG ARG ARG ARG TIE, 1-1
Group D Croatia v. Nigeria CRO CRO CRO CRO CRO, 1-0
Group D Argentina v. Croatia ARG ARG ARG ARG CRO, 3-0
Group D Nigeria v. Iceland ISL ISL ISL ISL NIG, 2-0
Group E Brazil v. Switzerland BRA BRA BRA BRA TIE, 1-1
Group E Costa Rica v. Serbia TIE SRB SRB TIE SRB, 1-0
Group E Brazil v. Costa Rica BRA BRA BRA BRA BRA, 2-0
Group E Serbia v. Switzerland TIE SUI SUI SUI SUI, 2-1
Group F Germany v. Mexico GER GER GER GER MEX, 1-0
Group F Sweden v. South Korea SWE SWE SWE SWE SWE, 1-0
Group F Germany v. Sweden GER GER GER GER GER, 2-1
Group F South Korea v. Mexico TIE MEX MEX MEX MEX, 2-1
Group G Belgium v. Panama BEL BEL BEL BEL BEL, 3-0
Group G Tunisia v. England ENG ENG ENG TIE ENG, 2-1
Group G Belgium v. Tunisia BEL BEL BEL BEL BEL, 5-2
Group G England v. Panama ENG ENG ENG ENG ENG, 6-1
Group H Colombia v. Japan COL COL COL COL JPN, 2-1
Group H Poland v. Senegal POL POL POL POL SEN, 2-1
Group H Poland v. Colombia POL COL COL POL COL, 3-0
Group H Japan v. Senegal SEN JPN SEN SEN TIE, 2-2

What’s next?

Soccer, coffee, and beer.  Probably not in that order. 

Also, our favorite mantra: observe and refine, test and tweak.  Keep working on this, because we honestly didn’t think that it would hold up so well this far (except against the FIFA rankings.  We saw that one coming.

About The Author

Architeuthis Rex, a man of (little) wealth and (questionable) taste. Historian and anthropologist interested in identity, regionalism / nationalism, mass culture, and the social and political contexts in which they exist. Earned Ph.D. in social and cultural History with a concentration in anthropology from Carnegie Mellon University and then (mostly) fled academia to write things that more than 10 other people will actually read. Driven to pursue a doctorate to try and answer the question, "Why do they all hate each other?" — still working on it. Plays beer-league hockey, softball, and soccer. Professional toddler wrangler. Likes dogs, good booze, food, and horribly awesome kung-fu movies.

No Comments on "World Cup Predictions: Most models underestimate the chance of a tie."

Leave a Comment

Your email address will not be published. Required fields are marked *