Mediatek defends against benchmark cheating claims by calling it “industry standard”

Earlier this week, a report from Anandtech uncovered the “mobile benchmark cheating” mechanisms within many Mediatek devices, highlighting how Mediatek-powered devices entered higher power states within benchmarking applications to achieve higher performance ratings. 

Since then, Mediatek has responded to these claims with a “Tek Talk” article titled “Why MediaTek Stands Behind Our Benchmarking Practices“. In short, Mediatek has responded by using the weakest excuse in the book, saying that “our key competitor has chipsets that operate in the exact same way”. That’s right, their defence for cheating is that somebody else is doing it. 

Adding insult to injury, Mediatek has suggested that Anandtek is biased because they called out Mediatek specifically. We will note here that within its article, Anandtech references past instances of Samsung, Huawei, and other mobile phone manufacturers for implementing similar benchmarking cheats in the past. Their report also makes it clear that calling out these companies usually results in them removing their benchmarking cheats, or developing user-accessible performance modes to enable their devices peak performance levels. 

Benchmark cheats often turn the SOCs used in mobile phones to their highest power levels to enable higher performance levels. The result of this often will cause overheating, significant increases in power consumption and benchmark performance that isn’t representative of the device under normal circumstances. 

Many benchmarks are designed to emulate real-world system performance, which means that these benchmark cheats deliver users, and reviewers, with results that are not representative of real-world device usage. Simply put, these cheats make certain mobile devices look better on paper, allowing device makers to make performance claims that don’t hold true within the real world.  

If Mediatek knew that its competitors were cheating in their benchmarks, they could have given this information to the press. They could have helped to reveal the cheating within the industry and clean up the world of mobile benchmarking. Instead, Mediatek started to cheat themselves, knowing that device makers have been caught doing this in the past. Just because a competitor cheats doesn’t make you any less of a cheater, and calling its benchmarking practices “accepted industry standards” shows how muddy the situation has become. 
 
Why Cheating Matters

Consumers rely on reviews and benchmarking tools to judge the real-world performance of their devices, and benchmark cheats make these tools less useful by generating results that aren’t representative of system performance under normal circumstances. At a minimum, cheated benchmark results are dishonest, and I don’t think anyone wants their performance analysis tools or device makers to be dishonest. 

We don’t want consumers to buy products based on false or misleading information, which means that mobile benchmark cheating is a huge black mark against any brand or company. Mediatek should know that the answer to cheating isn’t to cheat themselves, but based on the company’s statements, it cares more about cheating its benchmarks.