Comment by dgb23
13 days ago
Didn't set it up myself so my info will be meagre, but I think it's litellm and a big enough nvidia card. We use it for development but also for internal applications (where smaller models might escalate a task to it when needed).
Obviously not feasible for (just) personal use, the model is too large for consumer hardware.
No comments yet
Contribute on Hacker News ↗