Comment by vessenes

2 hours ago

In any batch inference environment that includes experts, expert routing may vary depending on what else is in the batch. For one thing.

0 comments