Nvidia announces native GPU programming in Rust

21 hours ago (developer.nvidia.com)

I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.

The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.

  • > I strongly dislike CUDA. Once you have allowed that proprietary cr*p

    Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.

    • IMO cr*p and crap are both valid but separate swear words. People have a wide option to choose from when they want to swear, and people like variety (much much more than LLMs do). People also tend to influence each other with their usages: cr*p is popular because it is popular.

      Otherwise cr*p is just as good as crap, shit, horseshit, poopoo or such.

      edit: * replaced with \* as HN interprets asterisks for emphasis. Thx latexr for informing me

      1 reply →

    • My guess is that jacobgorm will not reply. I would love a reply, because I want to understand how others think.

      I believe we'll be left to wonder.

  • > Once you have allowed that proprietary cr*p into your C++ codebase

    People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.

    Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice.

    Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, use some sort of HAL if they are available, or you don't have a job. Even simpler.

  • Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity. Having the compiler give an error when you mess up is a huge benefit. But having the compiler allow you to express "I want to launch this kernel over a grid with these dimensions, with these arguments" as a single expression is where the vast majority of the value comes from.

    The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".

    • > Having the compiler give an error when you mess up is a huge benefit.

      Shouldn't this be alleviated by the current code generation machines?

      2 replies →

  • Having also played with Metal and WebGPU (at least years ago), I would say that CUDA is, amazingly, the best GPGPU API we have. Do I wish we had an open source parallel programming language as good or better than it? Yes. But asymmetrically hating on CUDA like this is how we continue to lag behind it in UX.

    > The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually

    Not to mention that this is a completely sane way to use CUDA as well.

    • People that attack proprietary APIs always miss the point why most devs outside FOSS circles prefer them.

      Turns out when one isn't ideologically against something they aren't willing to put up with a lesser experience just for the cause.

      1 reply →

  • from what i can tell, you're going to be stuck with that no matter what you do

    i'm currently using vulkan, and HLSL via dxc. which should be portable but it's not.

    apple refuses to support vulkan, and relies on moltenvk and there's a bunch of OS/hardware/driver differences no matter what you do, that you'll probably have to feature test for, and compile a few different versions of your code no matter what you do

    i think if you're doing something that you don't have to distribute to customers, just picking one stack and getting locked in has some appeal.

    it leaves you vulnerable to lockin. but, especially in the age of ai, "claude, port this to vulkan" seems like a good enough defense against that

  • > The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually

    Isn't that how CUDA code is normally written?

    • No. CUDA allows you to write all the code in a single file, and uses a preprocessor to split it back out and pass it through separate compilers, one for host and one for device.

      1 reply →

  • I don't mind CUDA, I do mind that all of the SDKs don't dynamically load the various CUDA shared libraries at runtime.. intertwining itself into your application linking process makes for extreme binary portability inconvenience.

  • ? I find it hard to see the issue here. Just put it in a separate file and call it?

  • yeah, just write a stub/wrapper around it and abstract. it's the classic coupling problem. nothing to do with CUDA

Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.

[1] https://github.com/huggingface/candle

  • Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.

Really exciting but it reads like Claude instead of what Nvidia posts have generally been like in the past. I don't need nor want my tech blogs to sound like a young adult novel.

  • I’ve had this happen to me several time over the past weeks and it’s gone from quaint to humorous to farcical to outright “is-the-world-gaslighting-me” insane.

    Just today I was reading Stanley Druckenmiller’s op ed in WSJ. This dude is like 80 and has made billions of dollars, and he got Claude to write his op ed???

    Unbelievable. And the tells are so obvious, yet people still love the Claude-like quips and odd grammatical choices that read like halfway asshole halfway mid-sentence confusion.

    • That op ed was absurd. I respect Druckenmiller a lot and am always impressed with his lucidity in interviews. The Claude “ick” was all over his writing.

  • Yeah definitely Claude. Lazy authors, if you're going to get AI to write for you please use Astra instead - it makes way less annoying prose than Claude.

Anyone know when Rust's std::autodiff will become stable? Assuming this Rust support expands to other GPU vendors, autograd will probably be the only reason to use Slang instead of Rust anymore.

  • I was told in the 2025 LLVM dev meeting that it will always stay in nightly because it's not practical for them to provide long-term stability guarantees that is expected of stable Rust.

Not a cuda programmer, but since they’re making a new API, why would they already make it inconsistent at start? :-/ I’m referring to the examples a,b,c vs z,x,y (different ordering of output elements)

How does this compare to vectorware? (https://www.vectorware.com/blog/)

  • Towards the end of the post, we (NVIDIA) mention that this work was done in collaboration with Vectorware and others in the Rust community. And we can't wait to build further with the community.

Interesting direction from Nvidia. Anything that makes writing reliable GPU code less painful is definitely a good thing.

Does this mean I can write shaders in Rust for use with WGPU or Vulkan?

I read this the other day - definitely think it is the right direction Nvidia is taking.

Thank you NVIDIA - for once (not twice though - you've given us nothing but despair for Linux+GPU).

I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.

Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.

I haven't tried the tile API yet; looking forward to it.

The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.

  • cudarc been great for me, because it's easy to look up existing examples and references, and it maps 1-to-1 with what I see. I'm already having a tough time with CUDA itself, a dialect of it makes a tad harder to rely on previous work.

    Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.

I wonder how many parallels there are between CUDA's Tile abstraction and that of Metal.

> The launch is checked rather than trusted.

Damn even Nvidia is putting out fully Claude-written articles.

  • > even Nvidia

    Why "even Nvidia"?

    They are fully behind using AI for basically everything.

    What's next? "Damn, even McDonald's is putting out unhealthy food"

    • I think implication being organizations with 40,000+ employees and even more consultants and contractors plus a lot of budget are also using LLMs to draft public facing content instead of paying for content writers or even just proof readers .

      It points to friction rather than cost economics. Same reason we are always surprised why multi billion dollar product companies with millions of install base prefer electron instead of a native app.

      4 replies →

    • Is Jensen Huang still all-in on OpenClaw? That moment feels more like a flash in the pan.

    • I believe most of their marketing videos use fairly convincing text to speech too, not voice actors.

    • McDonald's food is not even that unhealthy. I just tried a Burger King burger the other day and it's terrible. I think it's like 2000 calories in a single burger or something.

      4 replies →

  • I get the impression that Nvidia employees don't care too much - I started seeing fully AI-written "documentation" on some of their smaller projects more than a year ago (i.e., before it was even slightly a good idea).

  • What are we for, I ask? What the hell are we now. Chatters to LLMs now? Is this our future? It really is starting to feel like it now.

    • Do you have the stomach to walk into a high school in the USA these days? Teachers use AI to generate assignments. Students feed the assignments to AI and submit the responses. Teachers feed the student submissions to an AI for grading.

      11 replies →

    • Dude I am in slop fucking hell right now. There is still room for a human touch, without which the agents will lever us harder and faster into a world of incomprehensible garbage.

      7 replies →

  • I hadn't read the article and read this comment as though NVIDIA themselves were implying that this library was checked but not trusted by them since it was fully LLM generated.

  • Another of those AI is bad for articles, great for coding.

    Plenty of us share the same opinion on doing reviews of AI generated code.

  • Yeah. I think if the text is written for other machines, then by all means have an LLM generate it, but if it is intended for a human audience, have a human being write it.

    We are still much better at writing in a way that doesn't waste other people's time.

  • It is the number 1 thing I cannot stand with Claude slop. It's a sort of anthropomorphization of language. Every "thing" does, produces, feels, wants, asks, answers, etc....

    - "Launch is checked"

    - "Question is asked"

    - "The implementation answers"

    - "The model wants"

    - "The results name"

    - "The connection surfaces"

    - "The prompt wires"

    - "The feature rides the mechanism"

    Every single fucking thing is alive, wants things, and does things.

    It's terrible. Infuriating. I want to rip my eyeballs out reading this filth. All. The. Time. "The anger is real".

    • Create any page with a file uploader. They all look the same now. It's like the Twitter Bootstrap days of responsive design. You'll get an icon which looks like ones on (on the drop space) those sites which are like "you must wait 60 seconds for this file to download".

      It's so horrible. The human element has been completely removed and replaced by..... mediocre.

      3 replies →

  • Is that your honest load bearing assessment you’re going to flag?

In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!

  • I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher. I've been re-learning a lot of higher-level math and it has been an knowledgeable, infinitely patient, always-available tutor. Of course, I could just have it do just about any math I want for me, but that's not the point.

    Kinda the same with language/technology stuff - it can be a great tutor and it can scaffold other parts of a project for you. It can give you feedback and let you focus on the interesting parts.

    I guess the motivation itself may be hard because of the fear of it taking over much of our jobs, but having this kind of help/feedback is pretty cool for the sake of learning things just because they are interesting!

    • > I've found sorta the opposite - in any area, it can just do everything for you, or it can be an incredible teacher.

      Please don't. I've had all of Codex, Claude and Gemini convincingly tell me absolutely wrong stuff, pointing it out with easily verifiable example they come up with more and more weird reasons.

      Things don't become correct simply because most sources are again - easily and logically verifiable - wrong. This already was a plague when people "just googled" stuff and effectively returned with the most SEO optimized answer. Now we have very convincingly written instances all over the place.

      If these were singular instances I wouldn't be so worried, but if you are learning it already is very easy to learn something wrong. This is why back in the days when people still used physical books to learn new things it was a good idea to check first which books are actually recommended. There have been a lot of "experts" that wrote things they clearly misunderstood but worked for all the examples in their books.

      To give a common example for both the backend and frontend devs, that isn't about a specific projects. LLMs and Google searches frequently turn out wrong results regarding CORS caching and how it works in relation to domains/hostnames. The circumstances under which Content-Disposition work are another example. I think a lot of wrong statements that LLMs are "convinced" about are due to wrong statements (sometimes in otherwise correct response) of popular Stack Overflow answers.

      It's saddening how much wrong "common knowledge" exists in the industry. I have been bitten by a lot of these, but it feels when people don't even actually code and think anymore this will just rise forever.

      1 reply →

  • LLM don't need to be trained in a library to use it well. It's just Rust which they know well.

  • I was learning Rust slowly when the LLM enabled coding became good enough. I switched from learning to full on building with Rust. I still learn high level concepts as needed but I will not be able to write Rust on my own at all.

    And that sounds scary but the way I got over the fear is by realizing there are many things that I do very well but I do not know their internals very well. Driving is an example. I barely understand what the steering wheel, clutch or brake pedals do. I have driven over 130,000 Kms and I will perhaps drive more than double that in the next many years.

    I have been building software since PHP/Drupal days. Got into AWS S3 as a beta user. Adopted Memcached (and MQ) in 2008 out of necessity. Then Python/Django for 10 years. Then Rust. And tons of JS/TS. I owe a lot to my curiosity. I believe we can keep learning what we need and still delegate most of programming to agents.

    • There are two types of programmers: the pragmatists who see programming as a chore and would gladly never write a line of code again given the right tools, and the gardeners who don't want their enjoyable and rewarding garden-tending work taken away from them.

      1 reply →

  • And what prevent you exactly ?

    There were humans far superior than you for writting Rust before LLM, now there's a LLM. The only difference is price and time execution.

    You get an awesome teacher (LLM) ready to answer all your questions about Rust.

    And you still find excuses not to learn it ?

    At some point, just realize you've been lazy to learn it and LLMs are just an excuse.

    • I think OP’s point is that the payoff in learning a new language has diminished in this AI era. You can call that lazy, I’d consider it smart to consider whether you could be doing other, better, things with your time.

      2 replies →

  • Sad to break it to you, but...

    I had LLMs write a pile of cuda-rust code and they were quite competent at it. Ported a bunch of (C++) CUDA kernels over, and ground away on them til they got equivalent performance

    https://github.com/rdaum/eider/tree/main/backends/cuda-oxide

    And mostly just DeepSeek 4.1 Flash, too. Not even a frontier model.

    Sorry.

    • Wtf does that have to do with learning, you should be encouraging people to learn.

      Just because an LLM can do something doesnt mean we shouldnt learn it. Imagine if your math teacher had this attitude with calculators. Or your engliah teacher with audio books, "no need to learn to read we have audio books and llms".

      I sometimes also struggle with the motivation to learn these days, but I realize its a crutch and super bad for my future to fall for that trap.

      You having access to an llm does not make you special, you will not get farther than the person who also understands the thing they can do with the LLM, just like how the kid who did well on math in high school didnt rely on a calculator (and is doing far better in life than those who did).

      That code looks like garbage btw...

      1 reply →

The recent circular moves that Nvidia is making is designed to wrap things around them, anything to keep the AI model party going.

First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.

Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).

Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.

what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.

Rust for GPU programming? My CUDA debugging sessions just got a whole lot less painful, hopefully.

Been waiting for something like this. CUDA C++ is a pain; Rust's safety for kernel programming could be a game changer.