Comment by pcwalton

It's not really practical to do this outside of simple cases. For example, a modern userspace GPU driver must allocate lots of memory in hundreds if not thousands of places to implement a standard API and there's no way around that.