Show HN: GoModel – an open-source AI gateway in Go; 44x lighter than LiteLLM
3 hours ago (github.com)
Hi, I’m Jakub, a solo founder based in Warsaw.
I’ve been building GoModel since December with a couple of contributors. It's an open-source AI gateway that sits between your app and model providers like OpenAI, Anthropic or others.
I built it for my startup to solve a few problems:
- track AI usage and cost per client or team
- switch models without changing app code
- debug request flows more easily
- reduce AI spendings with exact and semantic caching
How is it different?
- ~17MB docker image
- LiteLLM's image is more than 44x bigger ("docker.litellm.ai/berriai/litellm:latest" ~ 746 MB on amd64)
- request workflow is visible and easy to inspect
- config is environment-variable-first by default
I'm posting now partly because of the recent LiteLLM supply-chain attack. Their team handled it impressively well, but some people are looking at alternatives anyway, and GoModel is one.
Website: https://gomodel.enterpilot.io
Any feedback is appreciated.
Does this have a unified API? In playing around with some of these, including unified libraries to work with various providers, I've found you are, at some point, still forced to do provider-specific works for things such as setting temperatures, setting reasoning effort, setting tool choice modes, etc.
What I'd like is for a proxy or library to provide a truly unified API where it will really let me integrate once and then never have to bother with provider quirks myself.
Also, are you also planning on doing an open-source rug pull like so many projects out there, including litellm?
Nice one! Let's say I'm serving local models via vllm (because ollama comes with huge performance hits), how would I implement that in gomodel?
Expectable, given that LiteLLM seems to be implemented in Python.
However kudos for the project, we need more alternatives in compiled languages.
It’s also badly implemented - everything is a global import. Had to stop using it
Agree and thank you! Please let us know if you'd like to give it a try and if you miss any feature in GoModel.
Curious how the semantic caching layer works.. are you embedding requests on the gateway side and doing a vector similarity lookup before proxying? And if so, how do you handle cache invalidation when the underlying model changes or gets updated?
Hey, contributor here. That's right, GoModel embeds requests and does vector similarity lookup before proxying. Regarding the cache invalidation, there is no "purging" involved – the model is part of the namespace (params_hash includes the LLM model, path, guardrails hash, etc). TTL takes care of the cleanup later.
Any plans for AI provider subscription compatibility? Eg ChatGPT, GH Copilot etc ? (Ala opencode)
You are not the first person who has asked about it.
It looks like a useful feature to have. Therefore, I'll dig into this topic more broadly over the next few days and let you know here whether, and possibly when, we plan to add it.
I don't see any significant advantage over mature routers like Bifrost.
Are there even any benchmarks?
how does this compare to bifrost - another golang router?
First of all, GoModel doesn't have a separate private repository behind a paywall/license.
It's more lightweight and simpler. The Bifrost docker image looks 4x larger, at least for now.
IMO GoModel is more convenient for debugging and for seeing how your request flows through different layers of AI Gateways in the Audit Logs.
That would be valuable if there's a commitment to never have a non-opensource offering under GoModel? If so, you can document it in the repo.
1 reply →
This is really useful. I've been building an AI platform (HOCKS AI) where I route different tasks to different providers — free OpenRouter models for chat/code gen, Gemini for vision tasks. The biggest pain point has been exactly what you describe: switching models without changing app code.
One thing I'd love to see is built-in cost tracking per model/route. When you're mixing free and paid models, knowing exactly where your spend goes is critical. Do you have plans for that in the dashboard?
This comment looks like AI-generated.
However IIUC what you're asking for - it's already in the dashboard! Check the Usage page.