Comment by tarruda

I'm only interested in the local, single user use case. Plus I use a Mac studio for inference, so vLLM is not an option for me.