← Back to context

Comment by santiago-pl

8 hours ago

I wish all the best to LiteLLM team with handling it again. I can imagine how challenging are situations like this one.

However, if anyone is looking for 20x faster and less "targeted" LiteLLM alternative - I'm work on one in Go. That's why GoModel it's called. https://github.com/ENTERPILOT/GoModel/

6 comments

santiago-pl

Reply
ghrl  8 hours ago

Sounds like a reasonable choice of a language.

I myself have written a small llm proxy in Go, doing nothing but injecting the real API key to outgoing requests while itself validating virtual keys it generates, and it works great.

TZubiri  7 hours ago

If you don't use any LLM router, it's infinity times faster, and infitesimally as targetted.

  • verdverm  5 hours ago

    I have GoModel running at home, being deployed at work. There are many good reasons to run a gateway, some include

    - being resilient to upstream provider downtime

    - centrally managed config/controls

    - visibility and allocation into usage and costs

    We are deploying a gateway at work primarily for these three.

    At home, GoModel's virtual models have been super helpful for testing out new open weights. I can change the model deployed in vLLM without having to change any of my downstream tools, which get a stable name.

    https://gomodel.enterpilot.io/docs/features/virtual-models