← Back to context Comment by XCSme 2 days ago Why slow? I see ~50tps on a single 3090 7 comments XCSme Reply apitman 2 days ago Yeah make sure you're using MTP and potentially tensor parallelism. app13 2 days ago Whats your setup? I have a single 3090 and am struggling to get it purring XCSme 2 days ago 5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama.I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com shdh 2 days ago How did you undervolt the 3090? 1 reply → swalsh 2 days ago yeah, i consider that slow. XCSme 2 days ago Oh, ok, that's like the average tps for most AI providers
app13 2 days ago Whats your setup? I have a single 3090 and am struggling to get it purring XCSme 2 days ago 5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama.I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com shdh 2 days ago How did you undervolt the 3090? 1 reply →
XCSme 2 days ago 5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama.I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com shdh 2 days ago How did you undervolt the 3090? 1 reply →
swalsh 2 days ago yeah, i consider that slow. XCSme 2 days ago Oh, ok, that's like the average tps for most AI providers
Yeah make sure you're using MTP and potentially tensor parallelism.
Whats your setup? I have a single 3090 and am struggling to get it purring
5900x, 3090 24gb (slightly undervolted), 128gb ddr4, running via Ollama.
I am benchmarking it now locally, will put the results and speed/tps on aibenchy.com
How did you undervolt the 3090?
1 reply →
yeah, i consider that slow.
Oh, ok, that's like the average tps for most AI providers