Stats: 3,192,285 members, 7,947,355 topics. Date: Thursday, 12 September 2024 at 05:24 PM |
Nairaland Forum / Science/Technology / Phones / The Problem With Benchmarks (1182 Views)
Best Processor Ranking And Benchmarks / TECNO Camon X Pro Vs Phantom 8 - Speed, Benchmarks and Ram Management Test / Iphone 8 Benchmarks - By Far The Most Powerful Phone Ever (2) (3) (4)
The Problem With Benchmarks by atheistandproud(m): 9:38pm On Dec 09, 2021 |
Inquisitive Universe: Oya this evening Let's discuss a hotly contested topic. Hotly contested for those who know about it sha. Benchmarks. Whether we like it or not, everybody get bias. Serious biases dey. Some people try to put aside their bias and try to be objective while others want you to accept their bias as fact. The second group I mentioned is usually more forceful in discussion and do not accept others challenging their position. In our social media sphere, everything that is debatable is up for debate and should be up for debate. One area where people do not seem to agree is processors. Especially mobile processors. It is rare to find people arguing about PC processors like AMD 7000 vs Core i9 for example. This is because many people don't really know or care about PCs that much. But when it comes to mobile processors. The Snapdragons vs Exynos vs Mediatek sphere. Omo! It is a bloody battlefield where everyone seems to have an opinion. By everyone, I mean everyone. Both the people that know and the ones who don't. Everybody wan talk. And I have read some messed up stuff in the last few years that is enough to make my eyes fall out. Like what the f"ck did I just read. I have had people argue that the number of cores are what makes one processor better than the other. For these people, nothing else matters. The extent of their knowledge na Octa Core and Quad core. This is what they actually check when they say that they're reading SoC specs. Another group of people who really need to tied and flogged are those who use Clock speed to determine if one SoC is better than another one. The only thing that they check is clock speed. Nothing more. I had to ask one, that the Helio G90T is clocked at 2.05GHz and the Snapdragon 660 is clocked at 2.2GHz. Which SoC is the better one? With all confidence, he replied SD660. Why? Because SD660 has a faster clockspeed. Another set of people will tell you that it is by name that you judge an SoC. A few years back, the common opinion was that Exynos is better. I follow my roommate for NYSC drag this thing for days. He was far older than me and used Samsung phones. To him, Exynos will always be the best. The opinion that is held now is that Snapdragon SoCs are the best and that is what a lot of people live by. Imagine if two people with this mindset jam. War!!! To these people, the idea that each SoC should be examined based on individual merit is not even remotely present in their thinking. Snapdragon good. Everything else bad! Exynos good. Everything else bad! This last set of people no even join at all. Na RAM size them take dey judge SoC ability. If you give them a Snapdragon 450 with 6GB RAM and a MediaTek Helio G96 with 4GB RAM... They will happily pick the SD450 because of RAM Since a lot of people don't know how to or do not want to read SoCs specs, especially the type of cores used e.g. Cortex A55, A72 etc. There has to be a quick and easy way for people to be able to compare SoCs so as to see which ones are better than others. A very easy and quick way (if you noticed I didn't say good) to do so is by using benchmarks. What is a benchmark? A benchmark is a standard or reference point against which other things are compared. It is mostly used as a tool of comparison. For example... If you apply for any paramilitary job like Immigration, FRSC, NSCDC etc. They usually have a height benchmark. 170cm (5'4" ![]() ![]() If you as a guy no tall reach that 170cm, you are automatically disqualified. That height is supposed to reflect the average height of Nigerian males. So if you pass 170cm, congratulations. If you no tall reach am, well, ��� You may be tall in other places ��♂️��♂️��♂️��♂️ In the smartphone area, benchmarks are used to compare everything from the performance of entire phones to its individual components such as cameras, SoCs (CPUs and GPUs), storage speed, battery life etc. Why should we use benchmarks? Simple Benchmarks provide a fair and universal way of comparing the performance of different smartphones and their components. This is because there are many specs associated with smartphones and these are all built differently. With benchmarks, you do not have to understand these details. You just have to look at the benchmark performance and see how good it is. You can use it to compare the performance of the other SoC. For example Mediatek Dimensity 900 has a score of 701 and Snapdragon 732G has a score of 560. (Geekbench single core test) You can easily tell which one is better. How are Benchmarks done? A benchmark usually consists of a series of standardized tests designed to test the performance of the thing to be compared. The results of these tests are universal. They are designed to be uniform for all items being tested. Let us use PUBG mobile as a benchmark. To use PUBG as a benchmark, we will download PUBG into the phones to be tested and then score it based on the following criteria: Ability to max out all settings (both graphics and fps) - 100 (A+) Able to play the game at high settings with low fps - 60 to 69 (B) Able to install game and play at medium settings - 50 to 59 (C) Able to install game and play at low settings - 45 to 49 (D-) Able to install but unable to run the game - 40 to 44 (E) Unable to install game - 0 (F) So basically, if we bring out my old Tecno Pouvoir 2, that can install PUBG but cannot run the game. We are going to score it E. If I bring a Redmi 9A that can install the game and play it at low settings, we will give the 9A, a score of D- So in other words, the Redmi 9A performance better than the Tecno Pouvoir 2. This is how benchmarks work. Benchmarking is usually associated with assessing the ability of a CPU and it’s ability to run software. It provides a fair way to compare the performance of two or more CPUs/SoCs. The major generally accepted benchmarks for smartphones include: Antutu Geekbench 3D mark PC mark GFX etc. For a benchmark to be accepted as a standard for comparing phones. They have to fulfill certain criteria. They have to be: 1. Relevant 2. Fair 3. Transparent 4. Verifiable or Repeatable 5. Cost effective 6. Scalable By now, I am sure that we know what Benchmarks are and what they are used. So why do people hate Benchmarks? I mean something that is supposed to make life easy for all of us. Why would anyone hate it? We will look at it from two angles. 1. From the side of phone users i.e. us 2. From the side of the benchmark companies/phone manufacturers. Why do people hate Benchmarks? 1. They don't understand it and do not want to understand it. 2. They usually disagree with it 3. It puts their favourite brands down. 3. They distrust it What are some of the problems with benchmarks? 1. Different interpretation: different benchmarks can sometimes put out different scores. Thus it is sometimes left to the user to interprete these scores by themselves or to rely on those who can. This situation makes it possible for people to misinterpret benchmarks. Sometimes the score from Antutu may counter the one from Geekbench and so on and so forth. 2. Vendors can and will optimize SoCs to run better on benchmark software. These SoCs would then score high in benchmark tests but would perform poorly on other tasks. 3. Outright cheating: SoC companies have been known to add code to the firmware of their SoCs. If these codes detect a benchmark test, they are designed to overclock the SoC to reach maximum performance. This way, they can get unrealistically high benchmark scores which they would then use for advertising. 4. Overly streamlined tests that focus only on processing power and ignore other stuff like photography, videography, network speed, heating levels, power consumption, throttling etc. 5. User perception: Sometimes average day to day use does not really reflect benchmark scores. It could be possible for a phone that did well in benchmarks to flop in real life. This has led to some distrust for benchmark scores. 6. No account for CPU/OS load: Most phone are usually tested using the best possible conditions. They usually do not take into account what would happen if the storage of a phone if filled up to a certain point with several apps running in the background. These are some of the plenty problems that one would run into when trying to use Benchmarks. So should we stop using benchmarks then? Well no. Absolutely not. Despite their shortcomings, benchmarks are still the fastest and most objective way that we can compare SoCs. Until another method is devised, they are the best that we can rely on for now so they are still very useful. The organizations owning these benchmarks have to step up and keep their benchmarks free and fair as well as looking for ways to eliminate cheating. So that's it on this topic from me. As usual, contributions, additions, corrections, comments, recommendations etc are also very very much welcome. Good evening you all. Stay safe. https://inquisitiveuniverse.com/2021/11/24/what-is-a-benchmark/ 3 Likes
|
Re: The Problem With Benchmarks by Freelancerx: 11:09pm On Dec 09, 2021 |
Here for the comments |
Re: The Problem With Benchmarks by AREWAELENU: 12:15am On Dec 10, 2021 |
Following ![]() |
Re: The Problem With Benchmarks by skywalker240(m): 1:58am On Dec 10, 2021 |
The average Nigerian that are majorly tech illiterates would pick a pouvoir 2 over Redmi 9a That said, mediatek na Baba for that coding stuff, hence why am always having reservation's about their G series 4 Likes 1 Share |
Re: The Problem With Benchmarks by atheistandproud(m): 3:00am On Dec 10, 2021 |
skywalker240: No be only you lol ![]() ![]() 2 Likes 2 Shares |
Re: The Problem With Benchmarks by Newman20(m): 3:29am On Dec 10, 2021 |
How come faster Soc like the snapdragon 888 show reduced performance on phones like samsung s21 ultra and show increased performance on other phones like mi 11 ultra? Is it a software issue? |
Re: The Problem With Benchmarks by atheistandproud(m): 5:02am On Dec 10, 2021 |
Newman20: Optimization is the word you're looking for. 5 Likes |
Re: The Problem With Benchmarks by Nobody: 6:57am On Dec 10, 2021 |
This thread deserve to be on the front page |
Re: The Problem With Benchmarks by scarycuteface(m): 8:38am On Dec 10, 2021 |
Newman20:he talked about it in the thread. And it is one of the reason why people don't trust benchmark scores 1 Like |
Re: The Problem With Benchmarks by skywalker240(m): 9:28am On Dec 10, 2021 |
atheistandproud:As in enh |
Re: The Problem With Benchmarks by Deicide: 3:05pm On Dec 10, 2021 |
benchmark wouldn't also account for long term use |
Re: The Problem With Benchmarks by atheistandproud(m): 4:17pm On Dec 10, 2021 |
Deicide: Bless you. It doesn't account for drop offs in performance as the CPUs age and can no longer run as fast as they used to. 1 Like |
Re: The Problem With Benchmarks by Darkseid(m): 8:42pm On Dec 10, 2021 |
I sell computers and it gets worse. One question customers always ask is " what is the speed". They'll tell you that your 10th generation core i5 laptop @1.10GHz is slow and that an old school core 2 duo laptop clocked at 3.0GHz would be faster. The thing dey tire me seriously. 1 Like |
Re: The Problem With Benchmarks by KallmemrB0: 9:23pm On Dec 10, 2021 |
Beautiful.. Love the write up 1 Like |
Re: The Problem With Benchmarks by atheistandproud(m): 10:15pm On Dec 10, 2021 |
Darkseid: Foto! No be juju be that! 1 Share
|
Re: The Problem With Benchmarks by atheistandproud(m): 10:15pm On Dec 10, 2021 |
KallmemrB0: Thank you |
Re: The Problem With Benchmarks by Flamees(m): 4:27am On Dec 11, 2021 |
atheistandproud:
|
Re: The Problem With Benchmarks by Hakeem12(m): 4:31pm On Dec 11, 2021 |
Darkseid:A seller showed a friend of mine two laptops, one was a 5th gen core i7, U variant with 1tb HDD and 12gb of RAM, the other was an 8th gen core i5, also U variant, with 256gb SSD with extra space for HDD and 8gb ram. Both were going for 175k, I told him the i5 was better, and he looked at me and said; "guy, you well so, you no see the numbers?". He burst into laughter and bought the i7. I wondered why he needed my opinion in the first place. |
Re: The Problem With Benchmarks by atheistandproud(m): 4:46pm On Dec 11, 2021 |
Hakeem12: Eeeeh! The i5 was the better option na. Ignorance!!! 1 Like |
Re: The Problem With Benchmarks by Hakeem12(m): 4:55pm On Dec 11, 2021 |
atheistandproud:Yeah, but you could say that cos you understand better, my guy was looking at numbers. I'd have taken time to break it down to him, but he was laughing so hard, I felt like punching him. So I just shrugged and let him buy it, it's his money in the end. He doesn't know PC processors are even more complicated. There's generations to compare, there's variants, U, K, H and so on. Even I don't know that much about AMD processors yet. 1 Like 1 Share |
Re: The Problem With Benchmarks by Darkseid(m): 6:02pm On Dec 11, 2021 |
Hakeem12:I can totally relate to this ![]() ![]() 1 Like |
Re: The Problem With Benchmarks by OwoukoUrua(m): 4:11am On Jan 09, 2022 |
Hakeem12: Please I'll like to know why the i5 is better Thanks |
Re: The Problem With Benchmarks by OwoukoUrua(m): 4:13am On Jan 09, 2022 |
Re: The Problem With Benchmarks by atheistandproud(m): 5:02am On Jan 09, 2022 |
OwoukoUrua: Newer generations of CPUs are always better than the older versions. Newer generations have better architecture, fab sizes, production styles, newer technology, better optimization etc. In the scenario he painted... The i5 was the newer CPU and the distance between an old i7 and a new i5 is kinda wide. In addition, even though the SSD was smaller in size, it is way faster and would help the CPU perform even much better. 2 Likes |
Re: The Problem With Benchmarks by RuneKingThor: 9:11am On Jan 09, 2022 |
Hakeem12:Numbers don deceive the guy. And truly it's the same with most Nigerians. 1 Like |
Re: The Problem With Benchmarks by RuneKingThor: 9:15am On Jan 09, 2022 |
OwoukoUrua:If you check the benchmarks of both you'd be shocked at the gap the i5 gave the i7 though it shouldn't be surprising since it's ages newer. Another reason is the specs the laptop itself has. SSD with an option to expand through HDD. It's too good to let i7 big name pass you by. 2 Likes |
Re: The Problem With Benchmarks by atheistandproud(m): 12:59pm On Jan 09, 2022 |
RuneKingThor: This is the truth lol. To add to this, Nigerians like simple explanations without nuance or substance hence they're easily manipulated. |
Re: The Problem With Benchmarks by OwoukoUrua(m): 1:48pm On Jan 09, 2022 |
atheistandproud: Thank you |
Re: The Problem With Benchmarks by RuneKingThor: 1:50pm On Jan 09, 2022 |
atheistandproud:I really hope the narrative changes really soon. 1 Like |
Re: The Problem With Benchmarks by OwoukoUrua(m): 2:56pm On Jan 09, 2022 |
RuneKingThor: Thanks |
Re: The Problem With Benchmarks by Hakeem12(m): 5:12pm On Jan 09, 2022 |
OwoukoUrua:The processors are three generations apart, the i5 being the latest one. The i5 has also has a faster ram speed so the 8gb to 12gb is negligible. Then the storage, SSD is way faster than HDD, wouldn't touch a laptop with HDD these days. Faster loading times is noticeable on SSD, so real life performance is miles apart. 2 Likes |
Samsung Galaxy A51 London Used Open Box / Is It Safe To Order Phone From Aliexpress / Phone For Sale Redmi Note 4 25k (clean)
Viewing this topic: 1 guest(s)
(Go Up)
Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health religion celebs tv-movies music-radio literature webmasters programming techmarket Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) Nairaland - Copyright © 2005 - 2024 Oluwaseun Osewa. All rights reserved. See How To Advertise. 58 |