← Back to context Comment by nshotton 4 hours ago This looks really great! Quick question, is there any support for using local models? 3 comments nshotton Reply segmenta 4 hours ago Yes! You can use local models through Ollama and LM Studio. We do some special handling when you use local models, such as suppressing background agents when you chat, so the model is not overwhelmed. nshotton 3 hours ago sweet, is there any plans to implement VLLM in the future? segmenta 2 hours ago vLLM works today - it exposes an OpenAI-compatible endpoint, and you can point Rowboat at a custom base URL in model settings.
segmenta 4 hours ago Yes! You can use local models through Ollama and LM Studio. We do some special handling when you use local models, such as suppressing background agents when you chat, so the model is not overwhelmed. nshotton 3 hours ago sweet, is there any plans to implement VLLM in the future? segmenta 2 hours ago vLLM works today - it exposes an OpenAI-compatible endpoint, and you can point Rowboat at a custom base URL in model settings.
nshotton 3 hours ago sweet, is there any plans to implement VLLM in the future? segmenta 2 hours ago vLLM works today - it exposes an OpenAI-compatible endpoint, and you can point Rowboat at a custom base URL in model settings.
segmenta 2 hours ago vLLM works today - it exposes an OpenAI-compatible endpoint, and you can point Rowboat at a custom base URL in model settings.
Yes! You can use local models through Ollama and LM Studio. We do some special handling when you use local models, such as suppressing background agents when you chat, so the model is not overwhelmed.
sweet, is there any plans to implement VLLM in the future?
vLLM works today - it exposes an OpenAI-compatible endpoint, and you can point Rowboat at a custom base URL in model settings.