Comment by andy_xor_andrew
1 year ago
This gave me lots of confidence in Unsloth when I first read it.
I'll admit I was a little skeptical of Unsloth, since anything that boasts free perf improvement, just by dropping in some middleware, makes me suspicious. Especially from such a small team.
I assumed it was just introducing some hacks that create an inexact implementation of attention or some faster-but-inaccurate cuda kernels or something.
But now I believe this small team really knows their stuff :)
The founder I know personally, he interned at Nvidia and contributed many performance improvements, he's the real deal - just really enthuasitic so it may come off as boastfulness ;)
Apologies on the enthusiasm!! :) And hi!!
Daniel is of the best engineers I have ever worked with. Engineer in the true sense of wanting to know how something works and figuring out ways to improve it !
2 replies →
No need to apologise, it's a great trait to have :)
1 reply →
They’ve had their work applauded by Karpathy and Jeremy P. Howard as well, which are about the best credentials you could ever get for open source AI stuff:
https://twitter.com/karpathy/status/1765473722985771335
I’ve been using the library since it started out and it works really well. Daniel is also super helpful and responsive in their Discord, assisting everyone from the most basic users to breaking down complex ML math stuff.
Thanks to Andrej and Jeremy as well :) And also thanks to community members like you! It makes me super happy to keep making Unsloth better so appreciate it a lot!
real recognizing real
:)
The article mentioned in the comment: https://unsloth.ai/blog/gemma-bugs
Whoops I think I might have forgotten to add it to the Colab!!
Oh thanks! I get that a lot :) But ye there's no approximations at all! Just special maths hacks with no degradations, rewriting everything and creating a custom backprop engine, sprinkling Triton / CUDA everywhere and more :)
But thanks you believe in me + my bro more :) Appreciate it a lot!