← Back to context Comment by qeternity 2 months ago It's not impressive nor efficient when you consider batch sizes > 1. 3 comments qeternity Reply p12tic 2 months ago All of this is for batch size 1. qeternity 2 months ago I know. That was my point.Throughput doesn't scale on CPU as well as it does on GPU. p12tic 2 months ago We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.
p12tic 2 months ago All of this is for batch size 1. qeternity 2 months ago I know. That was my point.Throughput doesn't scale on CPU as well as it does on GPU. p12tic 2 months ago We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.
qeternity 2 months ago I know. That was my point.Throughput doesn't scale on CPU as well as it does on GPU. p12tic 2 months ago We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.
p12tic 2 months ago We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.
All of this is for batch size 1.
I know. That was my point.
Throughput doesn't scale on CPU as well as it does on GPU.
We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.