Comment by oblio 7 months ago What kind of hardware are you using? 4 comments oblio Reply Abishek_Muthian 7 months ago I use a laptop with 4090 16GB VRAM, core i9 and 96GB RAM for low latency work and Mac mini M4 for tasks which doesn’t require low latency.I had written a blog on how I run LLM locally a while back[1] I’ll update the information on models & Mac mini soon.[1] https://abishekmuthian.com/how-i-run-llms-locally/ oblio 7 months ago Have you tried to hook up your local setup to Cline/Roo or similar stuff? Abishek_Muthian 7 months ago I haven’t heard of them, can you explain how it could be useful for those run LLMs locally ? 1 reply →
Abishek_Muthian 7 months ago I use a laptop with 4090 16GB VRAM, core i9 and 96GB RAM for low latency work and Mac mini M4 for tasks which doesn’t require low latency.I had written a blog on how I run LLM locally a while back[1] I’ll update the information on models & Mac mini soon.[1] https://abishekmuthian.com/how-i-run-llms-locally/ oblio 7 months ago Have you tried to hook up your local setup to Cline/Roo or similar stuff? Abishek_Muthian 7 months ago I haven’t heard of them, can you explain how it could be useful for those run LLMs locally ? 1 reply →
oblio 7 months ago Have you tried to hook up your local setup to Cline/Roo or similar stuff? Abishek_Muthian 7 months ago I haven’t heard of them, can you explain how it could be useful for those run LLMs locally ? 1 reply →
Abishek_Muthian 7 months ago I haven’t heard of them, can you explain how it could be useful for those run LLMs locally ? 1 reply →
I use a laptop with 4090 16GB VRAM, core i9 and 96GB RAM for low latency work and Mac mini M4 for tasks which doesn’t require low latency.
I had written a blog on how I run LLM locally a while back[1] I’ll update the information on models & Mac mini soon.
[1] https://abishekmuthian.com/how-i-run-llms-locally/
Have you tried to hook up your local setup to Cline/Roo or similar stuff?
I haven’t heard of them, can you explain how it could be useful for those run LLMs locally ?
1 reply →