← Back to context Comment by KaiserPro 2 months ago From my estimation each second of gpt eats about 0.5-1.5 watthours 3 comments KaiserPro Reply YetAnotherNick 2 months ago You can say it takes 1800-5400 W. Not sure where you are estimating it from. KaiserPro 2 months ago Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries. YetAnotherNick 2 months ago Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.
YetAnotherNick 2 months ago You can say it takes 1800-5400 W. Not sure where you are estimating it from. KaiserPro 2 months ago Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries. YetAnotherNick 2 months ago Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.
KaiserPro 2 months ago Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries. YetAnotherNick 2 months ago Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.
YetAnotherNick 2 months ago Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.
You can say it takes 1800-5400 W. Not sure where you are estimating it from.
Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.
Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.