NPUs in processors are all well and good, but not if they make them power-hungry.
In a keynote speech at Computex 2024, Cristiano Amon, the CEO of Qualcomm, talked about how its new Snapdragon X chips have plenty of AI performance, up to 45 TOPs from its NPU. But he also pointed out that while talking about "industry-leading performance" is all well and good, people should also be talking about how much power such chips are using.
"People want to talk about TOPS, but they also need to talk about watts, because that's actually very, very important." In case you're wondering what on Earth the CEO of Qualcomm is referring to, TOPS is the now-standard metric for the performance of NPUs—neural processing units, that handle basic AI routines, leaving the CPU and GPU free to do other tasks.
One TOP equates to a trillion operations per second, which sounds like a huge number. Well, it is, but compared to even a budget-level graphics card, it's not very much at all. However, NPUs are designed to do AI tasks in the background, so that means they need to fulfil their performance requirements using as little energy as possible. And since all electrical energy used by a chip gets converted into heat, power-efficient NPUs help to keep the whole processor cooler.
Amon then compared the NPU in Qualcomm's Snapdragon X to those made by AMD and Intel, claiming a performance-per-watt 2.6 times better than AMD and 5.4 better than Intel's Core Ultra 7 chips. Those are some pretty bold claims but as there are no independent reviews of Snapdragon processors yet, they cannot be verified.
But even if the actual performance per watt of power consumed is only 50% of what Qualcomm claim it is, that's a sizeable improvement over the competition.
And it's not just about heat, even though Amon went on to point out a Snapdragon-powered laptop running 20 degrees Celsius cooler in an AI task than an equivalent laptop using its GPU. Getting more performance per unit of energy means you could turn things down a notch, to use even less power, and get a handy boost to a laptop's battery life.
This is something we reported on recently, with Dell's Snapdragon XPS 13 laptops having a claimed 27-hour battery life compared to the Intel Meteor Lake version's 18 hours, running a constant video stream. A 50% increase in battery life is nothing to sniff at and as with the performance-per-watt claims, even if it's not really that long, anything that makes a laptop run a bit longer is welcome.
Personally, I'd go one step further than what Cristiano Amon was saying. Forget about TOPS altogether—it's a pretty meaningless number, as it's a peak throughput under ideal circumstances, and we've already seen that certain AI tools run perfectly well on older, non-NPU processors. Power usage is far more important and I want to see the processor market wake up to the fact that the majority of PC users would gladly give up a bit of performance, for better thermals, noise, and battery life.
If Qualcomm can really deliver with its Snapdragon X range, then we could finally see some proper competition in the CPU market, especially when it comes to having low-power, high-performance chips. However, I suspect the TOPs war isn't going anywhere, just yet.
