Comment by GreenWatermelon
20 hours ago
yeah they mention:
> Required for a complete inference service (not included in this release):
> A KV cache system
> VE/Prefill/Decode disaggregation (most required engine functionality is already implemented)
> Load balancing across data-parallel replicas for both prefill and decode
Yes, that's why I commented. I'm not sure I understand your point.
I was just adding a citation to your point, not arguing against you :)
also adding what other stuff aren't included in the release, and implicitly pointing out that they explicitly mention it's not included rather than it being stealthily missing.