Comment by vektormemory

Great idea, I do a lot of work testing small LLM's in this space and agree if you add some guardrails and train a small embedder, you won't need these large models, as your info is static, you can ship it all as one.

Or try these with your trained embedder: https://ollama.com/library/minicpm-v4.6 https://ollama.com/library/bonsai-2