Comment by hedgehog
6 hours ago
Not strange, for the kind of applications models at that size are often used for the prefill is the main factor in responsiveness. Large prompt, small completion.
6 hours ago
Not strange, for the kind of applications models at that size are often used for the prefill is the main factor in responsiveness. Large prompt, small completion.
No comments yet
Contribute on Hacker News ↗