How Nvidia is overcoming slowdown issues in GPU clusters
Claims it has cracked the scalability problem with incoming H100
GTC If you're training a large deep learning model, and you want it to train it faster, you should just throw more GPUs at it, right? Well, that works for a while, but there is actually a limit to the amount of proportional performance you can expect from adding additional GPUs in a server cluster.…