Comment by bredren
2 years ago
The speed of GPT-4 via chatgpt varies greatly on when you’re using it.
Could the data have been collected when the system is under different loads?
2 years ago
The speed of GPT-4 via chatgpt varies greatly on when you’re using it.
Could the data have been collected when the system is under different loads?
Unless they captured many different times and days, that is very likely a factor. GPU resources are constrained enough that during peak times (which vary across the globe) the token throughput will vary a lot.
The speed data is an average over 30 days.
Clearly OpenAI is throttling their API to save costs and get more out of fewer GPUs.