Comment by chasd00

yeah i really like this idea too, I don't need the entire internet indexed I only need the stuff i'm interested in indexed. I can imagine like a small agent i can task with "find out as much as you can about <subject>" and what it does is search the web, download the content, and index it for later retrieval. Then I can add a skill for the main agent to search the knowledge base if needed. Kind of like a rag pipeline but using agents to build a curated data source of stuff i'm interested in.