Comment by kgritesh
18 hours ago
Working on Strot - an AI agent that reverse-engineers website APIs for scraping.
Instead of DOM scraping, it intercepts AJAX calls and figures out which API endpoint gives you the data you need. Uses visual analysis + fuzzy matching to identify the right call.
The use case: scraping product reviews, paginated listing data (products), etc. Existing AI scrapers either didn't work or were very slow and costly. A product with 1000 reviews takes 10+ minutes with Playwright, costs $10 with LLM scrapers. With Strot? 10 seconds via direct API calls.
Being used in production by a couple of clients. Would love feedback!
Blog: https://blog.vertexcover.io/strot-is-a-api-scraper GitHub: https://github.com/vertexcover-io/strot
Very neat - I imagine you could even use this as a web scanner to identify security misconfigurations in API implementations (e.g. broken access control)