Comment by pests
1 day ago
You say that like I d a bad thing. Nvidia architectures keep changing and getting more advanced as well, with specialized tensor operations, different accumulators and caches, etc. I see no issue with progress.
1 day ago
You say that like I d a bad thing. Nvidia architectures keep changing and getting more advanced as well, with specialized tensor operations, different accumulators and caches, etc. I see no issue with progress.
That’s missing the point. Things like tensor cores were added in parallel with improvements to existing computer and CUDA kernels from 10 years ago generally run without modification. Hardware architecture may change, but Nvidia has largely avoided changing how you interact with it.
Modern CUDA programs that hit roofline look absolutely nothing like those from 10 or even 5 years ago. Or even 2 if you’re on Blackwell.
But for research you often don't have to max out the hardware right away.
And the question is what do programs that max out Ironwood look like vs TPU programs written 5 years ago?
1 reply →
They don't have to, CUDA is a high-level API in this respect. The hardware will conform to the demands of the market and the software will support whatever the compute capability defines, Nvidia is clearer than most about this.
And yet current versions of Whisper GPU will not run on my not-quite-10-year old Pascal GPU anymore because the hardware CUDA version is too old.
Just because it's still called CUDA doesn't mean it's portable over a not-that-long of a timeframe.
Portable doesn't normally mean that it runs on arbitrarily old hardware. CUDA was never portable, it only runs on Nvidia hardware. The question is whether old versions of Whisper GPU run on newer hardware, that'd be backwards compatibility.