The Minecraft community was in turmoil early in December after Speedrun.com announced that it had officially rejected a fifth-place run from popular Minecraft speedrunner Dream.
Dream is one of the largest YouTube content creators for Minecraft with 12 million subscribers. He competes in the Any Percent Random Seed Glitchless category on Minecraft 1.16. The run in question, which was posted in October, was removed after an investigation from Speedrun.com used a statistical method to conclude that he cheated by altering the probability of key events in the game.
The YouTuber repeatedly denied this claim throughout the investigation and argued that Speedrun.com, and the community that first brought this issue to the site’s attention, hasn’t provided conclusive evidence that he cheated. After the official report was released, he even went as far as hiring his own third-party statistician to dispute the official analysis.
Here’s a summary of all the events that have taken place throughout this controversy to date.
The rumors and investigation
On Oct. 16, speedrunner MincrAvenger collected and published data from six consecutive Dream livestreams. Throughout these separate runs, the odds for piglin barters to produce ender pearls appeared to be far better than they should be.
After streamers and viewers started signal-boosting these findings, Speedrun.com opened its own investigation into the situation that lasted two months. Throughout the time in which Speedrun.com carried out its investigation, Dream repeatedly denied the allegations of cheating that were coming from YouTubers, viewers, and other speedrunners.
Dream’s perspective
Dream first responded to cheating allegations on Oct. 29 when the rumors of his cheating first started to surface on forums. This was before the full report was released, but the content of the accusations was the same.
Dream argued three primary points in his defense:
- He argued that there’s no reason why he should cheat since he’s a successful and popular streamer.
- He argued that he doesn’t have the coding knowledge to change some simple parameters in the game code.
- He claimed the data that was being used in the report was cherry-picked to make it look like he was getting better drop rates than he actually was.
The first two arguments can’t be proven one way or the other. They’re claims about his personal abilities and temperament, which no one can actually examine using material evidence. Further, his accusers don’t have to establish his motive or technical ability to come to the conclusion that his six-run “luck streak” was statistically impossible. This means that his final claim about the cherry-picking of data was the only material argument he had in his defense, at least initially.
“Rather than taking a truly random sample, the sample starts when the luck seems off, and the sample ends as soon as I hit a personal best goal of 19 minutes, because that’s when I said I would stop streaming,” Dream argued in his response. “Although this can be touted as a ‘random set of data,’ it is most definitely not. There are five other 1.16 streams that I streamed on Twitch that can be rummaged through that are not included in the data, and there are also thousands of offline runs that I have done myself that also have data.”
This argument was less than satisfying to his critics and was eventually addressed directly in the report by Speedrun.com, which went out of its way to take into account sampling bias.
Speedrun.com releases official report
The official investigation concluded on Dec. 11 when the Minecraft speedrunning team published a 29-page official report detailing the background of the case, the statistical analysis, and the final rationale for removing Dream’s speedrun from its site. The investigation found that Dream’s piglin barter rate for producing ender pearls and his blaze drop rates both appeared to have been tampered with since he received exceptional results in this area across multiple runs despite near-impossible odds.
In data collected from a sample of Dream’s livestreams, the team found that 42 out of 262 barters with piglins resulted in ender pearls and 211 out of 305 blazes ended up dropping blaze rods. Although there’s some possibility of exceptional odds in a single run, these odds across multiple runs look suspect. It’d be like flipping a coin 100 times and it coming up heads 99 out of 100 times—it just shouldn’t happen based on the probability. His outlier data far outpaces even the luckiest runs from other streamers.
“Dream’s observed drop rates are substantially greater than those of other top-level RSG runners—including, Illumina, Benex, Sizzler, and Vadikus,” the team explained in its report. “If nothing else, the drop rates from Dream’s streams are so exceptional that they ought to be analyzed for the sake of it, regardless of whether or not any one individual believes they happened legitimately.”
The probability of Dream’s luck streak across the data that was examined was one in 177 billion, according to the team’s final analysis. You don’t have to be a math wizard to realize that those odds are extremely suspect at best. Thus, the team concluded that Dream must have tampered with the probability values in the game code and the site subsequently removed his time from the leaderboards for cheating.
As mentioned above, the report also addressed the sampling bias argument that constitutes Dream’s only initial defense.
“The reader should also note that the data presented is extensively corrected for the existence of any bias,” the report reads. “It would lack rigor and integrity for the conclusions made in this report to substantiate the moderation team’s decision if they were merely based on a surface-level analysis of the data.”
“Indeed, these corrections inherently skew the analysis in Dream’s favor. We aim to calculate not the exact probability that this streak of luck occurred if Dream is innocent, but an upper bound on the probability; that is, we will arrive at a value which we are certain is greater than the true probability.”
Dream’s rebuttal of the report
After the publication of Speedrun.com’s report, Dream published his own paper and video disputing the statistical analysis. The paper was written by a third-party statistician who wasn’t named in the document. The report does very little to establish Dream’s innocence. At most, it claims that it isn’t impossible that Dream got that lucky streak, even while acknowledging how unlikely it is.
“Even in the worst case, the probabilities are not so extreme as to completely rule out any chance that Dream used the unmodified probabilities,” Dream’s response paper reads. “If you have independent high-probability reasoning to suppose that the game was modified by Dream before his final six runs, then the lower probability of that hypothesis even after correcting for other biases suggests an alternative explanation. There are responsible explanations for Dream’s ender pearl and blaze rod probability, potentially excluding extreme ‘luck,’ but the validity and probability of those explanations depend on explanations beyond the scope of this document.”
After the release of Dream’s paper and video, the Speedrun.com mods stood by their decision. They released their own rebuttal that thoroughly answered every criticism made in Dream’s defense paper and video.
“Most of the direct criticisms of our analysis in the response paper are blatantly incorrect, disputing the accuracy of extremely standard statistical techniques firmly grounded in Probability Theory,” the Minecraft speedrunning team said in a rebuttal released in late December. “The only criticism of our analysis which even arguably holds any water is the critique of our choice of 10 as the number of RNG factors to correct for. We strongly disagree that 37 is a suitable number, but even if, despite that, it were used, it would not change our conclusion.”
This refutation was the final official response to this controversy. Speedrun.com will stand by its decision and Dream will have to abide by it.
The final fallout
Since the release of that final rebuttal, there’s been further criticism of Dream online from YouTube creators and the speedrunning community. The deluge of negative feedback even prompted Geosquared, one of the mods who worked on the Speedrun.com report, to request that people abstain from sending Dream hateful messages.
“Negativity is not justice, and being vindictive gets us nowhere,” he said on Twitter. “In fact, it alienates people from informing their opinions.”
For his part, Dream has continued to stand by his claim that he didn’t cheat. Over the past few days, however, his tone has changed. Although he still won’t admit to doing anything wrong, he’s stopped demanding that his suspect run be verified and he apologized for his poor handling of the situation.
“I want to apologize to the mod team for some harsh things I’ve said since the video came out,” the streamer said. “Although I have reason to be upset, I have no reason to act like a baby. I tend to act before I think when I receive intense criticism.”
On Dec. 31, Dream indicated in a tweet that he accepts the results of the investigation. And this is where this situation ends, at least for now.
Published: Jan 1, 2021 12:43 pm