Published: Sun, September 09, 2018
Electronics | By Kelly Massey

Huawei caught out over benchmark boosting

Huawei caught out over benchmark boosting

On Tuesday, hardware review and benchmarking publication AnandTech disclosed that Huawei has been caught cheating in the performance tests to overperform in them.

When tested under the benchmark app that wasn't recognized by the phone's operating system, they showed slower and less efficient performance, ending in the dethroning of the company from its status of "Best Smartphones" by UL, the company which developed the benchmark app, 3DMark.

Huawei has stated that the performance peak observed during the run of 3DMark was intuitive and determined by Artificial Intelligence.

To prevent confusion around current benchmarking results, after discussion, UL and Huawei have temporarily delisted the benchmark scores of a range of Huawei devices, and will reinstate them after Huawei grants all users of Huawei handsets access to the Performance Mode.

"Whenever confronted, Huawei's Wang Chenglu revealed to AnandTech that it needed to grasp standardized benchmarks that were nearer to genuine experience, and that it was fudging existing benchmark results since" others do the same testing".

Pence Pushed Paraguay to Keep Embassy in Jerusalem
The announcement angered Israel has triggered a diplomatic rift between the two countries. Israel closed its embassy in Paraguay and recalled its ambassador.

It has been reported that Huawei had been tampering with benchmark tests to showcase higher performance than the actual performance. As a punishment, 3DMark has removed these phones' rankings from its leaderboard and updated their listings on its website with a note that the phone's "manufacturer has not complied with UL benchmark rules". Huawei has a path to redemption in the eyes of UL after this sanction, at least.

Modified benchmarks can be considered as a fake representation of a smartphone's actual performance.

That detection and boosting behavior breaks the benchmarking rules UL sets for manufacturers.

According to a top Manager, benchmarks have long ceased to be a real indicator of performance and computational power of mobile devices regardless of usage scenario.

"Huawei always prioritizes the user experience rather than pursuing high benchmark scores - especially since there isn't a direct connection between smartphone benchmarks and user experiences". Huawei smartphones use advanced technologies such as AI to optimize the performance of hardware, including the CPU, GPU and NPU. Thus, when opening a benchmarking app on a new Huawei phone, the AI will recognise the same and adapt to an enhanced performance mode to deliver more power. The company “is planning to provide users with access to Performance Mode, so they can use the maximum power of their device when they need to.”.

Like this: