Comment by fancyfredbot
8 hours ago
Normally people refer to the compute-bound phase as "prefill". Nothing wrong with saying it's building the kv cache though, it's accurate just unusual.
8 hours ago
Normally people refer to the compute-bound phase as "prefill". Nothing wrong with saying it's building the kv cache though, it's accurate just unusual.
No comments yet
Contribute on Hacker News ↗