Hey this is my work, although the exploits etc aren't my ideas by any means.
This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
Yeah I hate the patched drivers too, I have a CMP170 as well I want to see if I can find a way to get fully working the same way without or minimal driver patching.
4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)
So yeah, buy an Nvidia card then download and run all the software needed to make it work to its full potential so Jensen can buy his 500 millionth leather jacket.
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
It's quite nuanced. In a nutshell the unlock just flips the 2 bits that halves how often the core can issue the specific instructions Nvidia locks to full rate on the professional and datacenter cards.
Obviously doing more compute requires more power but what it does is shift the card down on the frequency curve to keep the wattage in check.
So you're right that at stock power limits it hits a cap where the full issuing rate isn't reached but it depends heavily on the exact workload how much juice is left.
The frequency curve also takes heat into account so in general you end up with a nice perf boost for a slightly hotter card.
I've been running this on my 2 3090s for a month now with no issues but they are also water cooled :-)
Ah, that's a twist I didn't see coming, all my GPUs are aircooled. I can see how with watercooling you'd be able to get more performance out of the GPU before hitting the limits.
There 3090 is from the era of overclocking, anything but the reference design should be fine even on air cooling (especially if you have a 3 connector model)
My EVGA came with stock firmware that started at a 420W and allowed up 500W, of course 500W did nothing for gaming
> - you will need to run your card hotter than you might be comfortable with
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.
This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
As a person who uses consumer level hardware to do neural network research, this unlock is truly INVALUABLE.
This unlock is super useful as it improves the GPU performance on computation-bounded neural network workloads like LLM prefilling, image/video generation, and general neural network training, in a way of disabling deliberate software segmentation on the same hardware imposed by Nvidia.
I'm on rtx 50 series, and no free performance uplifting for now. Waiting for more reverse engineering efforts to reveal whether the unlock is possible for 50 series.
I'm aware that long-lasting reverse engineering efforts on nvidia gpus, like cmp gpus, will lead to something impactful. I just didn't see it clearly until your project came out. Good job on that.
For now, I think reverse engineering efforts on 50 series gpus are just beginning. There are projects like reading sensor temperatures directly from the gpu itself instead of using deliberately DOWNGRADED software like nvidia-smi. However, I don't really see these efforts spreading, and people behind them really needs more recognition. Props to these people, too.
It worked for me, and I'm getting about 6% faster training on nanochat with my 3060 card. Thanks for this!
However, it required a few extra steps, as the repo is set up for Linux kernel 6.x and I'm on 5.15. Claude Code was able to get me past the hurdles, including code modifications and startup scripts.
Right now, I'm clearly power limited at 170 watts and I have no desire to brick my card or set it on fire.
The real gains are far less than the advertised.
Tests ton TFTOPS show 25% gains at 200Watt cap. While mostly prefil benefit from that less decoding which make the real gains 7-8%.
Using at full power:
This is a benchmark test with Qwen 27B Int4 with 4x3090 and using full batching on VLLM to squeeze every bit of the GPU's.
These firmwares are already fully encrypted but nvidia has been publishing encrypted debug images that turns out could be decrypted with one of their public test keys. And those debug firmwares turned out to be close enough to the production ones that they could be used to construct exploits from.
In this case it's a DMA buffer overflow that writes to the stack which in turn can be exploited with Return Oriented Programming. So no unsigned code is ever executed but we still have code execution control.
Can we unlock FP64 so we can do serious work on second order methods? Even at enterprise grade the last good fp64 gpu was the H100. B100 focused on lower fp due to llms.
I don't believe so. As I understand it, the only Ampere GPU with a full complement of FP64-capable cores is the GA100 used in the A-series server cards (which, as you suggest, still underperforms the H-series for FP64).
Certain (mostly AI) workloads. Depends on how hot you're comfortable running your card. The real-world improvement is considerably less due to other things being bottlenecks.
Honestly if Nvidia is going to be short on cards for years there is no reason to keep this locked. People are going to buy what they can get and this in no way is going to influence sales at this point by making it official.
Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-)
There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
Hey this is my work, although the exploits etc aren't my ideas by any means. This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
Happy to answer questions!
Thanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics.
great to hear, which quant + KV quant + KV size are you on? is it llama.cpp?
1 reply →
Super well done, I wished all mods took your approach. Patched drivers are a nuisance, especially if you have more than one kind of card. Thank you.
edit: now do the 4090 ;)
Yeah I hate the patched drivers too, I have a CMP170 as well I want to see if I can find a way to get fully working the same way without or minimal driver patching.
4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)
19 replies →
https://github.com/JustAResearcher/gpu-unlock-ada
Any idea if something like this is happening on GB10 (DGX Spark) and needs similar treatment?
edit: seems you have provided some clarity that even if, unlikely soon unless China has and does the world a favor
DGX Spark is going to be very hard.
A semi related: a patch for those with multiple cards. You can unlock around 20% increased card to card speed on consumer GPUs using this library:
https://github.com/aikitoria/open-gpu-kernel-modules
The hardware supports it, but then Nvidia disabled it if the driver detects cheaper cards. This bypasses the NVidia slowdown.
Indeed, I already posted it earlier but this definitely deserves a top level comment.
There is also the unlock for Nvidia encoding which "removes restriction on maximum number of simultaneous NVENC video encoding sessions"
https://github.com/keylase/nvidia-patch
So yeah, buy an Nvidia card then download and run all the software needed to make it work to its full potential so Jensen can buy his 500 millionth leather jacket.
This one is real, but there are some caveats:
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
It's quite nuanced. In a nutshell the unlock just flips the 2 bits that halves how often the core can issue the specific instructions Nvidia locks to full rate on the professional and datacenter cards.
Obviously doing more compute requires more power but what it does is shift the card down on the frequency curve to keep the wattage in check. So you're right that at stock power limits it hits a cap where the full issuing rate isn't reached but it depends heavily on the exact workload how much juice is left.
The frequency curve also takes heat into account so in general you end up with a nice perf boost for a slightly hotter card.
I've been running this on my 2 3090s for a month now with no issues but they are also water cooled :-)
Ah, that's a twist I didn't see coming, all my GPUs are aircooled. I can see how with watercooling you'd be able to get more performance out of the GPU before hitting the limits.
There 3090 is from the era of overclocking, anything but the reference design should be fine even on air cooling (especially if you have a 3 connector model)
My EVGA came with stock firmware that started at a 420W and allowed up 500W, of course 500W did nothing for gaming
> - you will need to run your card hotter than you might be comfortable with
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.
4 replies →
These number claim 240W locked power draw on 4070:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.
I've tested myself and the results are legit, this is incredible, even if a little risky (it relies on exploiting firmware): https://github.com/Zagitta/gpu-unlock/blob/main/docs/How%20A...
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada
2080 Ti also getting some love:
https://github.com/KohakuBlueleaf/KohakUnlock
Those work but don't play nice with other such mods, in contrast to TFA. They require a pinned version of the driver.
As a person who uses consumer level hardware to do neural network research, this unlock is truly INVALUABLE.
This unlock is super useful as it improves the GPU performance on computation-bounded neural network workloads like LLM prefilling, image/video generation, and general neural network training, in a way of disabling deliberate software segmentation on the same hardware imposed by Nvidia.
I'm on rtx 50 series, and no free performance uplifting for now. Waiting for more reverse engineering efforts to reveal whether the unlock is possible for 50 series.
I wouldn't hold my breath for 50 series, hopper & blackwell has a different boot architecture with more security layers
I'm aware that long-lasting reverse engineering efforts on nvidia gpus, like cmp gpus, will lead to something impactful. I just didn't see it clearly until your project came out. Good job on that.
For now, I think reverse engineering efforts on 50 series gpus are just beginning. There are projects like reading sensor temperatures directly from the gpu itself instead of using deliberately DOWNGRADED software like nvidia-smi. However, I don't really see these efforts spreading, and people behind them really needs more recognition. Props to these people, too.
Also exists on turing apparently: https://github.com/nlzy/turing-unlock
It worked for me, and I'm getting about 6% faster training on nanochat with my 3060 card. Thanks for this!
However, it required a few extra steps, as the repo is set up for Linux kernel 6.x and I'm on 5.15. Claude Code was able to get me past the hurdles, including code modifications and startup scripts.
Right now, I'm clearly power limited at 170 watts and I have no desire to brick my card or set it on fire.
The real gains are far less than the advertised. Tests ton TFTOPS show 25% gains at 200Watt cap. While mostly prefil benefit from that less decoding which make the real gains 7-8%. Using at full power:
This is a benchmark test with Qwen 27B Int4 with 4x3090 and using full batching on VLLM to squeeze every bit of the GPU's.
The next version is sure to have signed firmware which sets these configurations to prevent this sort of thing.
These firmwares are already fully encrypted but nvidia has been publishing encrypted debug images that turns out could be decrypted with one of their public test keys. And those debug firmwares turned out to be close enough to the production ones that they could be used to construct exploits from.
In this case it's a DMA buffer overflow that writes to the stack which in turn can be exploited with Return Oriented Programming. So no unsigned code is ever executed but we still have code execution control.
Can we unlock FP64 so we can do serious work on second order methods? Even at enterprise grade the last good fp64 gpu was the H100. B100 focused on lower fp due to llms.
I don't believe so. As I understand it, the only Ampere GPU with a full complement of FP64-capable cores is the GA100 used in the A-series server cards (which, as you suggest, still underperforms the H-series for FP64).
So if I'm reading this chart correctly, my 3090 card can get a 65-80% boost in speed generally? Or, is this only for certain models and quantizations?
25%-80%.
Certain (mostly AI) workloads. Depends on how hot you're comfortable running your card. The real-world improvement is considerably less due to other things being bottlenecks.
Really interesting.
On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).
I also wonder how this affects power consumption.
Training is compute bound.
If its memory bandwidth bound you need to raise your batch sizes to leverage more parallelism.
It's also useful for prefill in inference, VLLM sees up to 30% boost depending on the exact kernels and prefill size
Where's the 5090 version?
Honestly if Nvidia is going to be short on cards for years there is no reason to keep this locked. People are going to buy what they can get and this in no way is going to influence sales at this point by making it official.
cool
Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.
It's the FP/BF16 FMA to FP32 instructions.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-) There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
The repo has fp16 numbers.
Also this one for 4070s shows that its for fp8 as well:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
it's the widening (for example: fp16 * fp16 = fp32) instructions, which are useful on some loads.
2 replies →
The repo has a lot of reports of people applying it to other 3xxx card, see https://github.com/Zagitta/gpu-unlock/issues
There's at least 3080 Ti, 3070 Ti, 3070, 3060
There's also a Windows version:
https://github.com/impacifist/Windows-GPU-UNLOCK
[flagged]