Comment by jtbaker

4 hours ago

Ok, but prefill/prompt processing was definitely the weak point before. They were already solid in raw tokens/sec after TTFT

0 comments