Comment by qumpis

That sounds like having a mixture of experts model (at high scale popularly developed by Google): train multiple specialised models (say embedders from text to a representation) that could be fed into a single model at the end. Each expert would be an adapter of sorts, activating depending on the type of input