In a general sense, you see this crop up in SKILLS.md files and other places, as LLMs have to deal with broad contexts. People try and drill into some taxonomy in a naive fashion using plaintext as a directive, which is not particularly optimal.
I wonder if one could build a 'mixture of experts' at the model level that leveraged a variety of small models "within" a larger model...
In a general sense, you see this crop up in SKILLS.md files and other places, as LLMs have to deal with broad contexts. People try and drill into some taxonomy in a naive fashion using plaintext as a directive, which is not particularly optimal.
I wonder if one could build a 'mixture of experts' at the model level that leveraged a variety of small models "within" a larger model...
because you have a different vector store for each categorization?
What if the question crosses categories?