Comment by ICodeSometimes
6 days ago
High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)
Almost all requests goes through a proxy to be honest.
You pay per GB transferred basically which works ok for me at the moment.
Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.
What was the scraper written in? Python? Node? Go?
It indeed does.
My reasoning for creating this are the alternatives are WAY TOO EXPENSIVE (5-10x per api call).
Since it's a one man show, i don't need much margin and i'm happy to just keep it alive since my friends and i use it for other projects that have more significant margin.