Mlcommons endeavors to establish AI benchmarks specifically designed for laptops, desktops, and workstations

MLCommons, a group focused on AI-related hardware benchmarks, aims to simplify comparing consumer PCs' AI performance with new benchmarks for laptops, desktops, and workstations.



They announced the formation of MLPerf Client, a working group tasked with establishing AI benchmarks for various operating systems like Windows and Linux, focusing on real end-user scenarios.



The first benchmark will concentrate on text-generating models, particularly Meta's Llama 2, optimized for Windows-running devices through collaborations with Qualcomm and Microsoft.



The group includes major tech players like AMD, Arm, Asus, Dell, Intel, Lenovo, Microsoft, Nvidia, and Qualcomm, but not Apple. While disappointing for macOS users, these benchmarks will likely shape future device-buying decisions.



MLPerf Client's benchmarks could provide insights similar to PC build comparison tools, indicating AI performance expectations. With Qualcomm and Arm involved, expansion to cover phones and tablets may be possible in the future.