Comment by 4by4by4

17 hours ago

Does anyone have rough numbers on how this compares to vLLM in terms of price performance? Maybe with something like Dynamo for disaggregated serving and KV-aware routing.

I'm curious if it's a modest increase in performance or a significant one.