← Back to context Comment by hkt 3 days ago Out of interest, roughly how many tokens per second do you get on that? 2 comments hkt Reply edude03 3 days ago Like 4. Definitely single digit. The P40s are slow af coolspot 2 days ago P40 has memory bandwidth of 346GB/s which means it should be able to do around 14+ t/s running a 24 GB model+context.
edude03 3 days ago Like 4. Definitely single digit. The P40s are slow af coolspot 2 days ago P40 has memory bandwidth of 346GB/s which means it should be able to do around 14+ t/s running a 24 GB model+context.
coolspot 2 days ago P40 has memory bandwidth of 346GB/s which means it should be able to do around 14+ t/s running a 24 GB model+context.
Like 4. Definitely single digit. The P40s are slow af
P40 has memory bandwidth of 346GB/s which means it should be able to do around 14+ t/s running a 24 GB model+context.