← Back to context Comment by ICodeSometimes 6 days ago Yep scraping alot of sites, what specifically would you like to know? 4 comments ICodeSometimes Reply someuser54541 5 days ago What's the tech stack? How did you get around issues with your scrapers IP getting blocked? ICodeSometimes 5 days ago High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)Almost all requests goes through a proxy to be honest.You pay per GB transferred basically which works ok for me at the moment. someuser54541 5 days ago Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.What was the scraper written in? Python? Node? Go? 1 reply →
someuser54541 5 days ago What's the tech stack? How did you get around issues with your scrapers IP getting blocked? ICodeSometimes 5 days ago High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)Almost all requests goes through a proxy to be honest.You pay per GB transferred basically which works ok for me at the moment. someuser54541 5 days ago Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.What was the scraper written in? Python? Node? Go? 1 reply →
ICodeSometimes 5 days ago High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)Almost all requests goes through a proxy to be honest.You pay per GB transferred basically which works ok for me at the moment. someuser54541 5 days ago Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.What was the scraper written in? Python? Node? Go? 1 reply →
someuser54541 5 days ago Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.What was the scraper written in? Python? Node? Go? 1 reply →
What's the tech stack? How did you get around issues with your scrapers IP getting blocked?
High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)
Almost all requests goes through a proxy to be honest.
You pay per GB transferred basically which works ok for me at the moment.
Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.
What was the scraper written in? Python? Node? Go?
1 reply →