Comment by buyucu

I just use llama.cpp server. It works really well. Some people recommend llama-swap or kobold but I never tried them.