Yeah - the search demo (https://search.notte.cc/) is just an attempt to showcase what one can do with a scraping agent embedded in a served LLM with an MCP server. We have a sys prompt pushing the LLM to request an URL to make sure it's scraping what you want to scrape - but I've played with it a bit and it can also come up with it's own URLs to be scraped.
I've got to say - and this in a way makes sense: The model seems to be very "URL-oriented" ...
... as in (at least in the demo, the search one) if you ask it to visit a site by name (ie. "Go go Altavista and ...," ...
... it will complain "no URL" has been provided.-
It probably has to do with the "special instructions" given to the AI for this one program/demo ...
... still.-
Yeah - the search demo (https://search.notte.cc/) is just an attempt to showcase what one can do with a scraping agent embedded in a served LLM with an MCP server. We have a sys prompt pushing the LLM to request an URL to make sure it's scraping what you want to scrape - but I've played with it a bit and it can also come up with it's own URLs to be scraped.