Comment by dllu
14 hours ago
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
14 hours ago
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
noob here - what's the benefit of this? Will using Rust lead to more optimal LLMs or code or both?
Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
And quite a few reasons that something better than C should be used. Rust seems a good candidate.
That is exactly why OpenCL failed adoption, focusing on C, instead of being polyglot like CUDA.
SYCL is the natively polyglot counterpart, with practical implementations of it compiling down to the same sort of SPIR-V kernels as OpenCL. (OTOH, much of the current adoption on the open standards side seems to target the more widely supported SPIR-V compute shaders, via Vulkan compute.)
1 reply →
I have never seen a comment this gray
in all of hackernews' shitty UX decisions, gray unreadable comments is one of the worst ones
I haven't felt this popular since then 1990s when I was opposing Visual J++ and IIS.
oh no no no, this is going to break the CPP hold on AI and game dev.
Nah, Rust compiler still needs C++ to be built in first place, and everyone on AI uses LLVM as infrastructure.