Comment by zimpenfish
1 day ago
I use iocaine[0] to generate a tarpit. Yesterday it served ~278k "pages" consisting of ~500MB of gibberish (and that's despite banning most AI scrapers in robots.txt.)
1 day ago
I use iocaine[0] to generate a tarpit. Yesterday it served ~278k "pages" consisting of ~500MB of gibberish (and that's despite banning most AI scrapers in robots.txt.)
Can't seem to access this.
It flashes some text briefly then gives me an 418 TEAPOT response. I wonder if it's because I'm on Linux?
EDIT: Begrudgingly checked Chrome, and it loads. I guess it doesn't like Firefox?
Doesn't work on my firefox either.
Friendly fire, I suppose.
Works on my Firefox. Mac and Linux
Nor Safari on iOS.
Works fine on my iOS Safari - maybe there's some extension that's tickling it just the wrong way?
1 reply →
Unfortunately and you kind of have to count this as the cost of the Internet. You've wasted 500Mb of bandwidth.
I've had colocation for eight years+. My monthly b/w cost is now around 20-30Gb a month given to scrapers where I was only be using 1-2Gb a month, years prior.
I pay for premium bandwidth (it's a thing) and only get 2TB of usable data. Do I go offline or let it continue?
> You've wasted 500Mb of bandwidth.
Yep, it sucks, but on the positive side, I'm feeding 500Mb of garbage into them every day and that feels like enough of a small win for me.
> My monthly b/w cost is now around 20-30Gb a month given to scrapers [...] 1-2Gb a month
That definitely sucks.
> Do I go offline or let it continue?
Might be time to start blocking entire IP ranges and ASNs and see if that helps.
i have no idea what this does because the site is rejecting my ordinary firefox browser with "Error code: 418 I'm a teapot". Even from a private browser.
If I hit it with Chrome, now I can see a site.
Seems pretty not ready for prime time as a lot of my viewers use Firefox