← Back to context

Comment by pvab3

5 hours ago

its going to get worse though as cloudflare keeps blocking more and more

I‘m using mostly the browser automations for things like that now. Same for research - ChatGPT is banned from reading many pages, but Codex can read anything I can.

But isn’t it funny that Cloudflare is blocking AI on their pages, but on the other hand is researching and marketing things like „you can put a browser in a CF worker“

  • Their browser workers won’t get blocked. Same with all big vendors, keep out small competitors enjoy access yourself and sell it to a select few partners.

The sites that do that won't be getting money from my and others' agents. Guessing that's going to become more and more of a problem for those sites.

  • I expect a small site that undercuts the top Google results by 18% with a sign-up discount probably isn't profitable on those orders, so blocking agents would save them money - it's not like someone using agents in that way is going to have any loyalty to shopping from that site in the future.

  • And yours and other's agents will probably remain an insignificant and invisible customer base anyway.

    My crystal ball is as good as anyone's, but if "agentic shopping" ever becomes mainstream, you can be sure that the vast majority will ask their phone (i.e. Google, i.e. Google Shopping) what the best price is anyways.

    • Sure, Google could be their agent, or ChatGPT, or Claude, or whatever local model the person is running. The big guys might all have the results cached so they don't have to rerun the crawl. Whatever their choice of agent, it seems pretty clear that almost everyone's going to use them, they're way too useful not to.