AndreyKrav/iStock, Michael M. Santiago/Getty, Tyler Le/BI
Nvidia stock jumped 6% on Wednesday, helping fuel a tech-led rebound in the broader market after the CPI report failed to excite the market about imminent interest rate cuts.
The gains in Nvidia stock came as CEO Jensen Huang addressed investors at a Goldman Sachs conference in San Francisco Wednesday morning.
Talking to Goldman Sachs CEO David Solomon, Huang answered key questions related to the ongoing buildout of AI infrastructure, including whether the return on investment was worth it for its customers.
"How would you assess customer ROI at this point in the cycle?" Solomon asked.
Huang noted that because efficiency gains in CPUs have nearly come to a halt, effectively ending Moore's Law, the cost of data computations was poised to soar in a world that is creating exponentially more data.
But Nvidia's GPU-based accelerators have meant massive power and efficiency gains in processing data computations, leading to immediate savings for its customers.
In other words, in a world where Nvidia's AI-enabled GPUs didn't exist, data centers would cost a lot more money due to the sluggish nature of CPUs.
"You reduce the computing time by about 20 times, and so you get a 10x savings," Huang said of running Nvidia's GPU accelerators relative to traditional CPUs.
He added: "That's the instant ROI you get by acceleration."
While Nvidia's next-generation GPU racks for data centers cost millions of dollars, Huang said the cost pales in comparison to the materials costs for a setup built around CPUs.
"Nvidia server racks look expensive and it could be a couple of millions of dollars per rack, but it replaces thousands of nodes. The amazing thing is just the cables of connecting old general purpose computing systems costs more than replacing all of those and identifying into one rack," Huang explained.
In the Gen AI world, where popular consumer-facing products like ChatGPT and Claude exist, Huang said the ROI for its customers is strong.
"The return on that is fantastic because the demand is so great that for every dollar they spend with us translates to $5 worth of rentals. And that's happening all over the world and everything is all sold out," Huang said.
Finally, Huang noted that investors need to take productivity gains that are unlocked via Nvidia's GPU systems into account.
"The productivity gains are just incredible," Huang said. "There's not one software engineer in our company today who don't use cogenerators."
He added: "And so I think the days of every line of code being written by software engineers, those are completely over."
