Comment by irthomasthomas

You can spin up a version of this at home using simonw's LLM cli with the llm-consortium plugin.

Bonus 1: Use any combination of models. Mix n match models from any lab.

Bonus 2: Serve your custom consortium on a local API from a single command using the llm-model-gateway plugin and use it in your apps and coding assistants.

https://x.com/karpathy/status/1870692546969735361

  > uv tool install llm
  llm install llm-consortium
  llm consortium save gthink-n5       -m gemini-pro -n 5 --arbiter gemini-flash --confidence-threshold 99 --max-iterations 4

  llm serve --host 0.0.0.0

  curl http://0.0.0.0:8000/v1/chat/completions \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gthink-n5",
    "messages": [{"role": "user", "content": "find a polynomial algorithm for graph-isomorphism"}]
  }'

You can also build a consortium of consortiums like so:

  llm consortium save gem-squared -m gthink-n5 -n 2 --arbiter gem-flash

Or even make the arbiter a consortium:

  llm consortium save gem-cubed -m gthink-n5 -n 2 --arbiter gthink-n5 --max-iteration 2

or go openweights only:

  llm consortium save open-council -m qwen3:2 -m kimi-k2:2 -m glm-4.5:2 -m mistral:2 --arbiter minimax-m1 --min-iterations 2 --confidence-threshold 95

https://GitHub.com/irthomasthomas/llm-consortium