Comment by zozbot234

Arguably it makes more sense technically to get the model support into llama.cpp, which provides many options for GPU+CPU split inference already.