Comment by XCSme
6 hours ago
Yes, I know you can have both.
My point was that this is just a different way of creating specialised task solvers, the same as with MoE.
And, as you said, with MoE it's about the model itself, and it's done at training level so that's not something we can easily do ourselves.
But with agent swarm, isn't it simply splitting a task in multiple sub-tasks and sending each one in a different API call? So this can be done with any of the previous models too, only that the user has to manually define those tasks/contexts for each query.
Or is this at a much more granular level than this, which would not be feasible to be done by hand?
I was already doing this in n8n, creating different agents with different system prompts for different tasks. I am not sure if automating this (with swarm) would work well in my most cases, I don't see how this fully complements Tools or Skills
MoE has nothing whatsoever to do with specialized task solvers. It always operates per token within a single task, you can think of it perhaps as a kind of learned "attention" for model parameters as opposed to context data.
Yes, specific weights/parameters have be trained to solve specific tasks (trained on different data).
Or did I misunderstand the concept of MoE, and it's not about having specific parts of the model (parameters) do better on specific input contexts?