How Good are Reddit’s Predictions?

It’s the eve of a new LCS Split and that means prediction threads and articles  and even the occasional prophets are appearing everywhere. This made me curious. What are these predictions worth? Is reddit any good at guessing how the LCS will turn out? What about the articles that are posted every few weeks offering insight and foresight?

To answer this I combed through old reddit LCS prediction threads from just before the beginning of the Spring Split, about 50 from NA and EU each and compared them to what actually happened. Each prediction was a list of the 10 teams in each region ranked based on how well they would do. The difference between tehse predictions and the final results produced a score for each list. Below are the results of this analysis. The teams are listed based on how well Reddit predicted they would do.

Some terms:

ME: Mean Error. The average absolute difference between the predicted ranking and the actual ranking (based on playoff results).

Difference from Average: This represents the average amount by which guesses were different from the typical guess. 

Strangest/Most Normal Predictions: These are the predictions that deviated the least/most from the most typical guess

Most Certain/Uncertain: The team in each league the predictions of which had the lowest/highest standard deviation.

First off, good job Bigrash and Purealpha for having the best predictions over all. Purealpha in particular did a good job with a chaotic league and took some hilarious, in hindsight, abuse for it

No one got the EULCS right. Almost everyone though Elements would finish top-2 (in fact they were the most certain team in EU) and that Fnatic would suck. This is an important reminder of something that we should all know by now: there is very little serial correlation in League. On the other hand most people had a good grasp of NA. Team8 and TIP were a bit underestimated and Winterfox overestimated, but overall the predictions were reasonably accurate. It’s worth noting that the worst NA prediction was more accurate than all but the very best predictions about EU.

In both leagues the team’s about which people were least certain were MYM and Impulse. Impulse is understandable, they were basically an entirely new team. But the analogous logic did not apply to EU where people were relatively certain where Fnatic would end up (5th-7th). Instead people placed MYM all over the place reflecting both their decent performance the previous split (5th) and also their roster swaps during the break. And speaking of certainty there was actually a slight negative correlation between the accuracy of Reddit’s prediction and how certain people were about it. The most certain predictions ended up being most wrong. 

The above implies that there may be a little of mob mentality in how people form their opinions; discussions that drive out new information and alternative viewpoints lead to inaccuracy. This is corroborated by the fact that in both leagues peoples guesses were more similar to each others than to the truth; in EU this was true by a mile. Also, shout-out to Sabrewolf and Bakanino who had the most unusual predictions. Sabrewolf said H2K wold finish 9th, but also placed Fnatic higher than anyone else at 3rd; Bakanino thought Impulse would end up 9th and Team8 third. So maybe being different is always best.

Among the dozen or so non-Reddit thread predictions I found the best, in fact the best overall, was found right here on GP10. Paravine also had decent EU predictions. But overall Reddit did about as well as anyone else.