Comment by tealcod 1 year ago Would it be possible to run something like vLLM or TensortRT-llm with tinfoil? 1 comment tealcod Reply FrasiertheLion 1 year ago We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.
FrasiertheLion 1 year ago We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.
We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.