Skip to main content

Report finds almost all Android OEMs, not just Samsung, cheat on benchmarks

Android-Benchmarks-cheaters

When a story earlier this week discovered Samsung was artificially inflating benchmark scores for its new Galaxy Note 3, many were quick to point out it wasn’t the first time Samsung had been caught engaged in such a practice. The same issue was discovered by AnandTech for the Galaxy S4 back in July, and today the site has an extensive report showing that almost every Android smartphone manufacturer is shipping devices that do the same.

As pictured in the chart above, that includes the HTC One, HTC One mini, LG G2, Galaxy Tab 10.1, and many others. In fact, the only companies that appear to not be using the method is Apple and Motorola, as well as Google with its Nexus 4 and Nexus 7 devices:

We started piecing this data together back in July, and even had conversations with both silicon vendors and OEMs about getting it to stop. With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization. It’s possible that older Motorola devices might’ve done the same thing, but none of the newer devices we have on hand exhibited the behavior. It’s a systemic problem that seems to have surfaced over the last two years, and one that extends far beyond Samsung…  None of the Nexus do, which is understandable since the optimization isn’t a part of AOSP. This also helps explain why the Nexus 4 performed so slowly when we reviewed it – this mess was going on back then and Google didn’t partake.

As noted in the report, the gains that OEMs are experiencing from the inflated scores are probably not worth the press they’ve been receiving. AnandTech points out that most of the inflated scores provide under a 10% increase in GPU and CPU performance benchmarks:

The hilarious part of all of this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 – 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can’t stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors.

You can check out the full report here, which analyzes the optimizations it’s found for several devices across various benchmark tests.

FTC: We use income earning auto affiliate links. More.

You’re reading 9to5Google — experts who break news about Google and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Google on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel

Comments

Author

Avatar for Jordan Kahn Jordan Kahn

Jordan writes about all things Apple as Senior Editor of 9to5Mac, & contributes to 9to5Google, 9to5Toys, & Electrek.co. He also co-authors 9to5Mac’s weekly Logic Pros series and makes music as one half of Toronto-based Makamachine.