← Back to context Comment by tealcod 2 months ago Would it be possible to run something like vLLM or TensortRT-llm with tinfoil? 1 comment tealcod Reply FrasiertheLion 2 months ago We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.
FrasiertheLion 2 months ago We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.
We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.