Samsung pulls an ARod, caught juicing its S4 benchmark results

Update: The Verge reports that Samsung has responded with a semi-convincing case for how some of it may be innocent, but no comment on the more damning evidence of code referencing specific benchmark apps:

The Korean manufacturer says certain “full screen apps” (any app in which the status bar isn’t present) such as the camera, browser, video player, and benchmarking tools, are classified as requiring the highest performance available. Many games don’t require the maximum clock speed to run, the company notes. Samsung doesn’t address AnandTech‘s discovery of strings of code that implied specific benchmark apps were being targeted for higher clock speeds, but the site did note that other benchmark apps that are not explicitly mentioned in code were also behaving the same way.

For years now, people have been using benchmarks to analyze how devices compare to each other in terms of raw power. As most people know, however, benchmarks don’t usually mean much in terms of real world performance. According to a new report from AnandTech, Samsung has been performing some trickery to make its devices perform better during benchmarking.

Essentially, Samsung has set the GPU of the Exynos 5 Octa Galaxy S4 variant to run at a higher frequency when being benchmarked than during normal, day-to-day usage. AnandTech tested this and found that Samsung sets the device to run at 533MHz during benchmarking tests,  as opposed to the 480MHz during normal usage.

The same trickery also appears to be true when it comes to the CPU speed. While running the GLBenchmark 2.5.1, AnTuTu, Linpack, and Quadrant benchmarking apps, the device was set to use the Cortex A15 cores clocked at 1.2GHz. When using the GFXBench 2 app, which is apparently not subject to Samsung’s benchmark trigger, the device ran at  the lower 500MHz speed.

Finally, the report also points to some interesting code within the Galaxy S4, dubbed “BenchmarkBooster” that essentially orders the device to kick up its clock speed when certain apps (read: benchmarks) are being ran.

It really all comes down to knowing that you should never base your purchase of a smartphone on benchmarking. It wouldn’t shock me to find out that other manufacturers are doing something similar, either. It really only matters how the device runs in day-to-day usage, not in benchmarking apps.