Comment by refibrillator
7 hours ago
There are roughly 3 ways to give a KVM guest a GPU.
1) VFIO passthrough: host binds entire GPU to guest as PCI device, which only allows one VM to use the GPU, thus you sacrifice your host display too (unless you fallback to integrated graphics on cpu etc). Strongest isolation because host kernel module driver not involved.
2) virtio-gpu: guest sees paravirtual GPU and loads virgl/venus mesa driver which serializes graphics API calls and replays them on the host driver. This allows multiple VMs to use the GPU, but performance overhead can be significant, and guests can’t practically leverage lower level primitives eg NVENC without paying price of CPU readback.
3) virtio-nvgpu (this repo): guest loads standard NVIDIA user mode driver (closed source), a fake /dev/nvidia* kernel module copies ioctl bytes + handle onto queue for host kernel mode driver to execute. This also allows multiple VMs to use a GPU, but is near native speed due to low overhead. Unfortunately the tradeoff is this project has the weakest isolation, eg every guest ioctl is forwarded to the host by default, the VMM holds read/write FDs, no seccomp/caps/allowlist. With respect to There is basically no GPU related security measures here, the exposure is the same as running multiple processes using the GPU with no VM. Only caveat is these guests can’t drive a physical display, so there is some restriction of surface area but it feels incidental rather than intentional in this case.
Anyways this is a tough problem OP, I don’t want to discourage you.
Without hardware/driver support for isolation (MIG) on consumer grade NVIDIA GPUs, it won’t be possible to solve this properly for a long time.
Also a factor is that NVIDIA has no open Mesa driver to support a native context approach (guest owns GPU command buffers, host maps them) like we have for AMD/Intel.
A week or so ago when I stumbled on this he at least seemed to be clear about the potential security implications, which gave me pause. But it doesn't seem any worse than just running software on your main OS which is what most people do. Sure, it's DoA for a hypervisor in a data center but that isn't the only use case out there.
What kinds of things can a guest running undesirably applications (viruses, malware, LLM escaping a sandbox, etc) get up to with shared GPU access?
vGPU support on consumer cards is more or less just drivers, you can patch it back in and there exist a few git repos to help do so.
I do think at the very least the domain specific workarounds are neat too some, even if not solving every problem. Such as ffmpeg-over-ip, pytorch with remote gpu usage, etc.
What is the state of virtio-gpu these days? I have a system76 "pang14" (Ryzen 7 7840U laptop), is that a valid option for me to have a VM with 3d acceleration? I'm fine with having some overhead if it is better than the current software GPU.
If you don't need Mac or Windows guests then the existing virtio-gpu works in my experience.
This is not a completely novel approach. This has been a thing in virtio-gpu for a while, it's called "DRM native context".
Putting Security aside, do you know how this might complicate bot detections that use GPU or canvas indicators?
4th solution: nvidia vgpu
If I remember correctly, the idea is that you have a physical GPU and you split its memory (with, eventually, time-budget) to create multiple virtual GPUs, which can then be associated with a KVM guest and use by it
(not available legally on consumer-grade GPU)
I'm more interested how this would effect providing VM graphics with migrations between hosts.
The dream would be put the user OS in a VM in a lab, and then be able to suspend and resume seamlessly if you need to push it to a new workstation, with locally accelerated graphics available.