Comment by MyNameIsNickT
5 months ago
Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
A big reason we invest in this is because we want to keep free and logged-out access available for more users. My team’s goal is to help make sure the limited GPU resources are going to real users.
We also keep a very close eye on the user impact. We monitor things like page load time, time to first token and payload size, with a focus on reducing the overhead of these protections. For the majority of people, the impact is negligible, and only a very small percentage may see a slight delay from extra checks. We also continuously evaluate precision so we can minimize false positives while still making abuse meaningfully harder.
That still doesn’t explain why you can’t even start typing until that check proceeds. You could condition the outbound request from being processed until that’s the case. But preventing from typing seems like it’s just worse UX and the problem will fail to appear in any metrics you can track because you have no way of measuring “how quickly would the user have submitted their request without all this other stuff in the way”.
Said another way, if done in the background the user wouldn’t even notice unless they typed and submitted their query before the check completed. In the realistic scenario this would complete before they even submit their request.
I developed the first version of Google's equivalent of this (albeit theirs actually computes a constantly rotating key from the environment, it doesn't just hard-code it in the program!).
The reason it has to block until it's loaded is that otherwise the signal being missing doesn't imply automation. The user might have just typed before it loaded. If you know a legit user will always deliver the data, you can use the absence of it to infer something about what's happening on the client. You can obviously track metrics like "key event occurred before bot detection script did" without using it as an automation signal, just for monitoring.
That doesn't make sense. The server would wait to process anything until after you received the signal. If it doesn't arrive within a reasonable period of time that tells you something, the same as right now.
If you mean that you can infer client side tampering with the page contents you could still do that - permit typing but don't permit the submit action on the client. The user presses enter but nothing happens until the check is complete. There you go, now you can tell if the page was tampered with (not that it makes much difference tbh).
9 replies →
This perfectly explains the trade-off. But from a pure UX perspective, freezing the input pipeline feels uniquely hostile. They could buffer the keystrokes invisibly in the background instead of locking the cursor, which creates the jarring perception that the site is actively fighting the user.
can you reformulate your message?
2 replies →
[dead]
[dead]
Many cloud products now continuously send themselves the input you type while you are typing it, to squeeze the maximum possible amount of data from your interactions.
I don’t know whether ChatGPT is one of those products, but if it is, that behavior might be a side effect of blocking the input pipeline until verification completes. It might be that they want to get every single one of your keystrokes, but only after checking that you’re not a bot.
It's still possible to let users already type from the beginning, just delay sending the characters until checks are complete. Hold them in memory until then.
4 replies →
I just checked the network inspector, the only thing it does per key press is to generate an autocomplete list. It doesn't seem too hard to wait with the autocomplete generation until after whichever checks you run pass.
I wondered if ChatGPT streams my message to the GPU while I type it, because the response comes weirdly fast after I submit th message. But I don't know much about how this stuff works.
1 reply →
You cannot know what verifications they use. I could argue the disabled textbox is some sort part of the verification process. Humans will click on it while bots won't.
Seems like a trivially simple verification to defeat.
1 reply →
Because the way they have the server architecture setup and how it loads the screen. You don’t even want all the bots hitting servers
Keyboard response feels 10x slower in ChatGPT Projects (possibly for reasons other than react state).
Sounds like anti-distillation to me. But, know what? Meh.
I’d be inclined to agree with the “meh” if their entire product weren’t built off pirated content
[flagged]
Remember you’re talking to a vibe coder who just stares at code being printed out by AI.
That’s a big assumption. It’s a brand new account, might be a bot. PR/astroturfing is a great use case for agentic AI
It's interesting to me that OpenAI considers scraping to be a form of abuse.
It’s funny because the first AI scraper I remember blocking was from OpenAI’s, as it got stuck in a loop somehow and was impacting the performance of a wiki I run. All to violate every clause of the CC BY-NC-SA license of the content it was scraping :)
Quite sure even literal thieves would consider thievery a form of abuse.
Engineers working on AI and AI enthusiasts are seemingly incapable of seeing the harm they cause, so I disagree.
It is difficult to get a man to understand something, when his salary depends on his not understanding it.
What’s being stolen? AI output isn’t copyrightable, and it’s not like they’re ripping pages out of a book
2 replies →
Yeah, they know it's bad, they just don't think the rules apply to them.
20 replies →
" Integrity at OpenAI .. protect ... abuse like bots, scraping, fraud "
Did you mean to use the word hypocrisy. If not, I'm happy to have said it.
I just want to note, that it is well covered how good the support is for actual malware...
They don't want anyone to take that which they have rightfully stolen.
Well at least they have 1 person working on "Integrity" so can't be too bad
Exactly! How dare you have access to their stolen content in the midst of them doing the same.
The levels of irony that shouldn't be possible...
The irony is thick
Seriously. The hypocrisy is staggering!
Church, politicians, moralists are all the biggest hypocrites that want to teach you something.
I agree on politicians, no idea what a "moralist" is supposed to be but there are good and bad churches and church goers; lumping all church goers into one category calling them hypocrites is wrong. There are many good churches and church goers who help people and their communities.
And have absolutely no reservations about making such an obvious statement on a public forum
"You're trying to kidnap what I've rightfully stolen!"
I interpreted scraping to mean in the context of this:
> we want to keep free and logged-out access available for more users
I have no doubt that many people see the free ChatGPT access as a convenient target for browser automation to get their own free ChatGPT pseudo-API.
> I have no doubt that many people see the free ChatGPT access as a convenient target for browser automation to get their own free ChatGPT pseudo-API.
Not that hard - ChatGPT itself wrote me a FF extension that opened a websocket to a localhost port, then ChatGPT wrote the Python program to listen on that websocket port, as well as another port for commands.
Given just a handful of commands implemented in the extension is enough for my bash scripts to open the tab to ChatGPT, target specific elements, like the input, add some text to it, target the relevant chat button, click it, etc.
I've used it on other pages (mostly for test scripts that don't require me to install the whole jungle just to get a banana, as all the current playright type products do). Too afraid to use it on ChatGPT, Gemini, Claude, etc because if they detect that the browser is being drive by bash scripts they can terminate my account.
That's an especially high risk for Gemini - I have other google accounts that I won't want to be disabled.
This is bad why? Well yeah for openai because all they want it to be is a free teaser to get people hooked and then enshittify.
Morally I don't see any issues with it really.
This
[dead]
[flagged]
Very few websites are truly static. Something like a Wordpress website still does a nontrivial amount of compute and DB calls - especially when you don't hit a cache.
There's also the cost asymmetry to take into account. Running an obscure hobby forum on a $5 / month VPS (or cloud equivalent) is quite doable, having that suddenly balloon to $500 / month is a Really Big Deal. Meanwhile, the LLM company scraping it has hundred of millions of VC funding, they aren't going to notice they are burning a few million because their crappy scraper keeps hammering websites over and over again.
It's not scraping they're concerned about, it's abusing free GPU resources to (anonymously) generate (abusive) content.
Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things.
The former relies on fairly controversial ideas about copyright and fair use to qualify as abuse, whereas the latter is direct financial damage – by your own direct competitors no less.
It's fun to poke at a seeming hypocrisy of the big bad, but the similarity in this case is quite superficial.
> Scraping static content from a website at near-zero marginal cost to its server, vs scraping an expensive LLM service provided for free, are different things.
I bet people being fucking DDOSed by AI bots disagree
Also the fucking ignorance assuming it's "static content" and not something needing code running
46 replies →
I understand why OpenAI is trying to reduce its costs, but it simply isn't true that AI crawlers aren't creating very significant load, especially those crawlers that ignore robots.txt and hide their identities. This is direct financial damage and it's particularly hard on nonprofit sites that have been around a long time.
7 replies →
That is ridiculous.
You imply that "an expensive llm service" is harmed by abuse, but, every other service is not? Because their websites are "static" and "near-zero marginal cost"?
You have no clue what you are talking about.
1 reply →
Interesting how other people's cost is "near-zero marginal cost" while yours is "an expensive LLM service". Also, others' rights are "fairly controversial ideas about copyright and fair use" while yours is "direct financial damage". I like how you frame this.
Lets not try to qualify the wrongs by picking a metric and evaluating just one side of it. A static website owner could be running with a very small budget and the scraping from bots can bring down their business too. The chances of a static website owner burning through their own life savings are probably higher.
5 replies →
Have you not seen the multiple posts that have reached the front page of HN with people taking self-hosted Git repos offline or having their personal blogs hammered to hell? Cause if you haven't, they definitely exist and get voted up by the community.
The cost is so marginal that many, many websites have been forced to add cloudflare captchas or PoW checks before letting anyone access them, because the server would slow to a crawl from 1000 scrapers hitting it at once otherwise.
It's not like those models are expensive because the usefulness that they extracted from scraping others without permission right? You are not even scratching the surface of the hypocrisy
It's more ironic because without all the scraping openai has done, there would have been no ChatGPT.
Also, it's not just the cost of the bandwidth and processing. Information has value too. Otherwise they wouldn't bother scraping it in the first place. They compete directly with the websites featuring their training data and thus they are taking away value from them just as the bots do from ChatGPT.
In fact the more I think of it, I think it's exactly the same thing.
5 replies →
Getting scraped by abusive bots who bring down the website because they overload the DB with unique queries is not marginal. I spent a good half of last year with extra layers of caching, CloudFlare, you name it because our little hobby website kept getting DDoS'd by the bots scraping the web for training data.
Never in 15 years if running the website did we have such issues, and you can be sure that cache layers were in place already for it to last this long.
"near-zero marginal costs". For whom exactly????
https://drewdevault.com/2025/03/17/2025-03-17-Stop-externali...
I don't think a rule along the lines of "Doing $FOO to a corporate is forbidden, but doing $FOO to a charitable initiative is fine" is at all fair.
What "$FOO" actually is, is irrelevant. I'm curious how you would convince people that this sort of rule is fair.
The corp can always ban users who break ToS, after all. They don't need any help. The charitable initiative can't actually do that, can they?
You’re describing the tragedy of the commons. No single raindrop thinks it’s responsible for the flood.
1 reply →
It is direct financial damage if my servers not on an unmetered connection — after years of bills coming in around $3/mo I got a surprise >$800 bill on a site nobody on earth appears to care about besides AI scrapers.
It hasn’t even been updated in years so hell if I know why it needs to be fetched constantly and aggressively, - but fuck every single one of these companies now whining about bots scraping and victimizing them, here’s my violin.
5 replies →
60% of our traffic is bot, on average. Sometimes almost 100%.
Lol, you single-handedly created a market for Anubis, and in the past 3 years the cloudflare captchas have multiplied by at least 10-fold, now they are even on websites that were very vocal against it. Many websites are still drowning - gnu family regularly only accessible through wayback machine.
Spare me your tears.
> Scraping static content from a website at near-zero marginal cost to its server
It's not possible to know in advance what is static and what is not. I have some rather stubborn bots make several requests per second to my server, completely ignoring robots.txt and rel="nofollow", using residential IPs and browser user-agents. It's just a mild annoyance for me, although I did try to block them, but I can imagine it might be a real problem for some people.
I'm not against my website getting scraped, I believe being able to do that is an important part what the web is, but please have some decency.
AI providers also claim to have small marginal costs. The costs of token is supposedly based on pricing in model training, so not that different from eg your server costs being low but the content production costs being high. And in many cases AI companies are direct competitors (artists, musicians etc.)
(TBH it's not clear to me that their marginal costs are low. They seem to pick based on narrative.)
> Scraping static content
How do you know the content is static?
My website serving git that only works from Plan 9 is serving about a terabyte of web traffic monthly. Each page load is about 10 to 30 kilobytes. Do you think there's enough organic, non-scraper interest in the site that scrapers are a near-zero part of the cost?
Absolutely not, the former relies on controversial ideas to qualify as legal.
Stealing the content from the whole planet & actively reducing the incentive to visit the sites without financial restitution is pretty bad.
You are, of course, ignoring the production costs of the static content that OpenAi is stealing.
Stop justifying their anti-social behavior because it lines your pockets.
And yet I have to pay in my time and cash to handle the constant ddos'es from the constant LLM scraping
Because you say it is?
I obviously disagree. I mean, on top of this we are talking about not-open OpenAI.
It’s not for techbros to decide at what threshold of theft it’s actually theft. “My GPU time is more valuable than your CPU time” isn’t a thing and Wikipedias latest numbers on scraping show that marginal costs at scale are a valid concern
I'm sure the copyright holders would consider your use of their content as direct financial damage
Are they, actually?
Speak for yourself.
I don’t know what world you live in but it’s not this one.
> Scraping static content from a website at near-zero marginal cost to its server
The gall. https://weirdgloop.org/blog/clankers
Bait or genuine techbro? Hard to say
[flagged]
[dead]
The issue is that there are so many awful webmasters that have websites that take hundreds of milliseconds to generate and are brought down by a couple requests a second.
1 reply →
It's getting to the point where a user needs at minimum two browsers. One to allow all this horrendous client checking so that crucial services work, and another browser to attempt to prevent tracking users across the web.
Nick, I understand the practical realities regarding why you'd need to try to tamp down on some bot traffic, but do you see a world where users are not forced to choose between privacy and functionality?
Local models for privacy.
You want to go to the world's best hotel? You are gonna be on their CCTV. Staying at home is crappier but private.
Unfortunately for the first time moores law isn't helping (e.g. give a poor person an old laptop and install linux they will be fine). They can do that and all good except no LLM.
> You want to go to the world's best hotel? You are gonna be on their CCTV.
ironically, in high end hotels, there's often a lot less cctv. not none. just less. rich people enjoy privacy
3 replies →
> Staying at home is crappier but private.
Doesn't make sense, my home is much more preferable to a hotel
3 replies →
Meet me in a cafe and I will sign a JWT saying you're not a bot. You can submit this to whoever will accept it.
If apple approves it, ive got a solution: A keyboardthat attests to your humanity https://typed.by/magicseth/2451#2NyGLfAQxmqRiAOTlaX7ma3G4d1o...
34 replies →
Sounds like we’re bringing back the PGP key signing parties
5 replies →
Doesn’t really make sense, because any service can just say “you must paste your human-attestation JWT here to use this service” and plenty of people will.
5 replies →
I've been doing that for years. Cloudflare is slowly breaking more and more of the web.
What if I run a website and OpenAI produces bot traffic? Do they also consider it abuse when they do it?
This is indeed what I do. And you also should. Separate browser for banking, trusted shipping sites etc, and the normal one.
Make sure not to browse the Internet without adblock and/or similar.
Firefox multicontainers are pretty cool. But it’s an advanced process that most people wouldn’t do or do correctly.
I love the containers too. My current use case is to keep my YouTube account separate from my Google one. Google doesn't need all that behavioural data in one place.
It's a pity Firefox doesn't get the praise it deserves half as much as it cops criticism.
It is absolutely not an advanced process. It's clicking a gui. It's not advanced thinking to understand profiles. It's a basic ability to hold multiple things in your mind at once. Telling people that's difficult only increases the societal problem that being ignorant is ok.
3 replies →
The possibilities with Firefox multi containers and automation scripts as well are truly endless.
It's also possible to make Firefox route each container through a different proxy which could be running locally even which then can connect to multiple different VPN's. I haven't tried doing that but its certainly possible.
It's sort of possible to run different browsers with completely new identities and sometimes IP within the convenience of one. It's really underrated. I don't use the IP part of this that I have mentioned but I use multi containers quite a lot on zen and they are kind of core part of how I browse the web and there are many cool things which can be done/have been done with them.
I am not Nick, but there's a few ways that world happens: the free tier goes away and what people pay for more correctly reflects what they use, this all becomes cheap enough that it doesn't matter, or we come up with an end to end method of determining usage is triggered by a person.
Another way is to just do better isolation as a user. That's probably your best shot without hoping these companies change policies.
> It's getting to the point where a user needs at minimum two browsers. One to allow all this horrendous client checking so that crucial services work, and another browser to attempt to prevent tracking users across the web.
Every time I try this, I end up crossing wires (ie using the browser that 'works' for most things, more than the one that is 'broken')
i am increasingly moving towards a model of 'no browser'.
search for me is now a proprietary index (like exa) that filters rubbish, with a zero data retention sla. so we don't need google profiling.
the content is distilled into markdown pulled from cloudflare's browser rendering api.
i let cloudflare absorb the torrent of trackers and robot checks, i just get md from the api with nothing else. cloudflare is poacher and gamekeeper.
an alternative is groq compound which can call browsers in parallel.
for interactive sites, or local ai browsing, i sometimes run a browser in a photon os docker with vnc, which gives you the same browser window but it runs code not on your pc.
that said little of my use is now interacting with websites, its all agentic search and websets so i don't have to spend mental energy on it myself
is this bad?
There is also the browser I use to get Claude to route around people blocking its webfetch. Both Playwright and chrome-mcp.
Camoufox?
>It's getting to the point where a user needs at minimum two browsers. One to allow all this horrendous client checking so that crucial services work, and another browser to attempt to prevent tracking users across the web.
What are you talking about? It works fine with firefox with RFP and VPN enabled, which is already more paranoid than the average configuration. There are definitely sites where this configuration would get blocked, but chatgpt isn't one of them, so you're barking up the wrong tree here.
Is your interlocutor barking up the wrong tree, or are you missing the forest for the trees?
According to the OP:
> The program checks 55 properties spanning three layers: your browser (GPU, screen, fonts), the Cloudflare network (your city, your IP, your region from edge headers), and the ChatGPT React application itself (__reactRouterContext, loaderData, clientBootstrap).
I guess Firefox VPN will hide the IP at least. But what about the other data, is it faked by RFP? Because if not, the so-called privacy offered by this configuration is outdated.
You might be fingerprinted by OpenAI right now, as “that guy with all the Firefox anti-fingerprinting stuff enabled, even though it breaks other sites”.
2 replies →
Don’t know if it’s related to the article, but the chats ui performance becomes absolutely horrendous in long chats.
Typing the chat box is slow, rendering lags and sometimes gets stuck altogether.
I have a research chat that I have to think twice before messaging because the performance is so bad.
Running on iPhone 16 safari, and MacBook Pro m3 chrome.
In the good old days Netflix had "Dynamic HTML" code that would take a DOM element which scrolled out of view port and move it to the position where it was about to be scrolled in from the other end. Hence he number of DOM elements stayed constant no matter how far you scroll and the only thing that grows is the Y coordinate.
They did it because a lot of devices running Netflix (TVs, DVD players, etc) were underpowered and Netflix was not keen on writing separate applications. They did, however, invest into a browser engine that would have HW acceleration not just for video playback but also for moving DOM elements. Basically, sprites.
The lost art of writing efficient code...
> Hence he number of DOM elements stayed constant no matter how far you scroll and the only thing that grows is the Y coordinate.
This is generally called virtual scrolling, and it is not only an option in many common table libraries, but there are plenty of standalone implementations and other libraries (lists and things) that offer it. The technique certainly didn't originate with Netflix.
10 replies →
This is how every scrolling list has been implemented since the 80s. We actually lost knowledge about how to build UI in the move to web
4 replies →
Almost certainly running some sort of O(n^2) algorithm on the chat text every key press. Or maybe just insane hierarchies of HTML.
Either way, pretty wild that you can have billions of dollars at your disposal, your interface is almost purely text, and still manage to be a fuckup at displaying it without performance problems.
Same. It’s wild how bad it can get with just like a normal longer running conversation
OpenAI sites are the only ones that do this to me. I have to keep a separate browser profile just for my OpenAI login with absolutely nothing installed on it or it'll end up being dogshit slow and unusable.
Yeah just had this earlier today, I had to write my response in vscode and paste it in, there were literal seconds of lag for typing each character. Typical bloated React.
Just because a web application uses React and is slow, it does not follow that it is slow because of React.
It's perfectly possible to write fast or slow web applications in React, same as any other framework.
Linear is one of the snappiest web applications I've ever used, and it is written in React.
5 replies →
That's how eating your own dogshit works, or whatever was that saying
Hi Nick, first of all, very cool of you to respond here instead of letting us all sit in the dark. I think that's what makes HN special.
That said, is it not a little bit weird that you want to protect yourself from scraping and bots, when your entire company, product, revenue, and your employment, depends on the fact that OpenAI can bot and scrape literally every part of the internet? So your moat is non-hydrated react code in the frontend?
Don't beat up an engineer for decisions made by company leadership. It's really inappropriate.
Yeah, no one is responsible for what they do as long as someone else tells them to do it.
They decided to work at this company, I think it's a reasonable discussion to have?
While I would generally sympathize on that front, it doesn't really apply here.
None of the management-level desiderata he appealed to require that the user experience be broken this bad. There is very little bot deterrence from prevention of typing at that stage, while it heavily impacts user experience, especially on mobile.
I elaborate here: https://news.ycombinator.com/item?id=47575982
Great to hear from a first-party source. I'm a Pro subscriber and my team spends well over two thousand dollars per month on OpenAI subscriptions. However, even when I'm logged in with my Pro account, if I'm using a VPN provider like Mullvad, I often have trouble using the chat interface or I get timeout errors.
Is this to be expected? I would presume that if I'm authenticated and paying, VPN use wouldn't be a worry. It would be nice to be able to use the tool whether or not I'm on a VPN.
> even when I'm logged in with my Pro account, if I'm using a VPN provider like Mullvad, I often have trouble using the chat interface or I get timeout errors
Heard from a founder who recently switched his company to Claude due to OpenAI's lagginess–it's absolutely an OpenAI problem. Not an AI problem in general.
> Hey! I'm Nick, and I work on Integrity at OpenAI. These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
How can first-party products protect themselves from abuse by OpenAI's bots and scraping?
This is a completely in-scope question.
How do we defend against your scraping, OpenAI?
I dont want any of my content scraped or seen by you all. Frankly, fuck you all for thinking my content is owned by you.
I use nginx conditionals and useragent checking, then respond with 418 or 410.
Probably too late now but my list needs updating
It's documented here: https://developers.openai.com/api/docs/bots
2 replies →
robots.txt bro https://developers.openai.com/api/docs/bots/
8 replies →
Hi! It's all perfectly understandable - after all, we use things like Anubis to protect our services from OpenAI and similar actors and keep them available to the real users for exactly the same reasons.
>These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
Can you share these mitigations so we can mitigate against you?
It's just Cloudflare. Bypassing it is a whole industry.
I read the comment as “use it to mitigate against OpenAI bots scraping the web” and not to mitigate Cloudflare.
1 reply →
Flaresolverr is one way. Isn’t perfect but bypasses a lot.
"we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform"
The scary part is that you don't even see the irony in writing this.
Or, are you just okay "misusing" everyone for your own benefit?
Brand new account with 2 comments in this thread. How can we be sure you're not a bot deployed to defend OpenAI?
Please run Cloudflare's privacy invasive tool and share all the values it generates here so we can determine if you're a real person.
> because we want to keep free and logged-out access
But don't you run these checks on logged-in users too?
Yep, on logged-in users too. The reason is basically the same: we want scarce compute going to real people, not attackers. Being logged in is one useful signal, but it doesn’t fully prevent automation, account abuse, or other malicious traffic, so we apply protections in both cases.
> The reason is basically the same: we want scarce compute going to real people, not attackers.
You are defining "Bots" and "Scrapers" as a subset of attackers, though.
Is this really fair? The value in your product came from people who wrote for other people, not bots, but your bot scraped them anyway.
There is no way to determine if a request that is coming from my browser is typed in by me or automated with a browser extension. Your only way to win this "war" on "attackers" is by forcing users into using your own application to access your product.
My browser extension (see my previous reply on this story) automates the existing open tab I have to all the different chat AIs (GPT, Claude, Gemini, etc).
I suppose all you can do is rate-limit each user.
Nothing you do can fully prevent automation. Someone who wants to automate requests badly enough will be able to do it, especially when the “protections” are as easy to decrypt and analyze as the OP proved.
Meanwhile, the rest of us (well, not me, because I don’t use your garbage product, but lots of others do) have to suffer and have our compute resources used up in the name of “protection.”
2 replies →
More like "We want your money, but don't want to provide service." Are you sure OpenAI isn't morphing into a finance/insurance company?
2 replies →
I'm glad you guys at least went with CloudFlare. LMarena went with Google's ReCaptcha, which is plain evil. It'll often gaslight you and pretend you failed a captcha of identifying something as simple as fire hydrants. Another lovely trick is asking you to identify bridges or busses, but in actuality it also wants you to identify viaducts or semi-trucks.
Can't have those bots or scrapers running amok can we...
Would OpenAI also consider renumerations to every site they have scraped that had a robots.txt file and they chose to ignore it anyway? Feel free to not answer this question.
I have kind of lost count of how many content creators have said personally to me traffic is meaningfully down because of all these chatbots. The latest example is this poor but standup guy: moneyfortherestofus.com.
I'm really glad Hacker News disallows AI generated comments. The response I got from asking that question really is quite enlightening. Short answer: "no", long answer: "no -- fuck off", longer answer: "no -- fuck off -- if you want I can dig into whether or not you should fuck off harder"
Y'all just salty that DeepSeek et al are training their LLMs on yours
"abuse like bots, scraping, fraud, and other attempts to misuse the platform"
This has to be a joke, right?
I really can't tell for sure (new user posting a ridiculously hypocritical corporate message on a Sunday) but if GP actually works for OpenAI the lack of self-awareness is seriously striking
How?
6 replies →
Still feels very anti-consumer.
If every company behaved like you do, the internet would be a much worse place.
In fact, OpenAI has already made the Internet a much worse place, already much, much less open and much less optimistic about its own future than it was even five years ago...
"Integrity at OpenAI"
Basically an oxymoron at this point.
> A big reason we invest in this is because we want to keep free and logged-out access available for more users.
Thank you for the reply, Nick. It wouldn’t be a problem to disable the tracking for authenticated users then, would it?
It would because someone's KPI depends on number of tracked users lol
If logging in disabled all checks, all bots would just spam-create users first. Of course it needs to run for all users, without it being necessarily nefarious.
1 reply →
> OpenAI: These checks are part of how we protect products from abuse like bots, scraping, and other attempts to misuse the platform.
This would be fucking HILARIOUS if it wasn't so tragic.
Manifest destiny for me, border enforcement for thee.
This kind of flawed thinking again. Like the natives didn't fight and lose wars against the manifest destiny types.
6 replies →
It can be both
> These checks are part of how we protect our first-party products from abuse like bots, scraping,
Do you guys see the irony here?
They obviously get it. They just do not care.
But is the title true, is typing specifically blocked? Or does it just block submitting the text?
I ask because I have seen huge variations in load time. Sometimes I had to wait seconds until being able to type. Nowadays it seems better though.
> [...] we protect our first-party products from abuse like [...] scraping [...]
what an odd thing to say for someone whose product is built entirely on exactly that
sometimes I paste giant texts (think summarization) in the chatgpt (paid) webapp and I noticed that the CPU fans spin up for about 5 seconds after, as if the text is "processed" client side somehow. this is before hitting "submit" to send the prompt to the model.
I assumed it was maybe some tokenization going on client side, but now I realize maybe it's some proof of work related to prompt length?
Paying customer since inception here.
I presume the local ChatGPT.app has even more measures to prevent automation, right? Presumably privacy-invasive ones as it is customary these days?
Is there a way I can opt out? I really, really, really don't like it.
The way I use the products something like this. My main account on my MacBook - ChatGPT website, codex cli. Then, a Mac VM running via UTM with shared writable dir - anything more ‘shady’ in terms of permissions and for playing with new ai apps - eg ChatGPT/Codex standalone apps, Atlas, Claude desktop app etc. Seems to work decently enough. And I do totally agree that there should be a way to opt out of all these privacy invasive measures, especially after paying $200/mo
I don't trust what OpenAI says. Sam Altman gives shivers, and these kinds of blog posts make things look even worse.
Can you fix the resizing text box issue on Safari when a new line is inserted? When your question wraps to a newline Safari locks up for a few seconds and it's really annoying. You can test by pasting text too.
> how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
Are you applying the same standards to your own scraper bots?
It has not been negligible for me, and, however you're doing this, there is significant room for improvement.
There have been times when, across about ten minutes of usage, most of which is me typing on iOS Safari, it drained 15% of my battery. There is no functional justification for this beyond poor code quality. (It was on a long conversation FWIW.)
This when I'm logged in, with a paid (Plus) account, connected to a very old email address with a real user profile. That can't be the result of super-clever bot defense measures, because it's merely an inconvenience on desktop. And if you genuinely believe that email has been compromised, why aren't you reaching out the to the account owner, as the account isn't otherwise connected to fraud by your heuristics?
However brilliant the LLM agent it is, I'm seeing a lot of unforced errors regarding how you implement a web interface to it. If it makes you feel any better, it doesn't really register compared to all the bloat I see on other sites.
Chatgpt banned me after I said disparaging things about Sam Altman in a chat.
When I appealed the ban, I was told that I couldn't be told exactly why I was banned, but if I wrote a written apology and "promised to never do it again" my ban could be appealed.
I asked for an update on the ban via email every month for over a year.
Maybe you could tell me a little bit about that process?
> we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform
Have you just described the dilemma facing all the content sites used to train LLMs?
Fwiw, I stopped using ChatGPT and went to a competitor because the checks slow down ChatGPT so much that the webapp becomes unusable in anything but a new short chat. CPU usage goes to 100%, you can't type, the entire tab freezes, etc. It's a miserable experience to use and I'm on a relatively new MacBook not some old computer. If you read around it's a very common problem people have been having for a while now.
Its your business and your call. But my opinion is that I wish you would quit offering free services. I'm pretty concerned about the horrible effect your free services are having on education. Yes, AI can be an incredible tool to enhance education. But the reality is that it is decimating children's will to learn anything.
I don't want to blame AI for all the world's problems. And I don't want to throw the baby out with the bath water. But I think you should think really hard about the value of gates. Smart people can build better gates than cash. But right now, cash might be better than nothing. Clearly you have already thought about how to build gates, but I don't think you have spent enough time thinking about who should be gated and why. You should think about gates that have more purpose than just maximizing your profit.
"We want to hook as many people as possible without letting in our competitors" is a pretty crummy thought to use as a public justification.
(Edited for typos.)
Tangential question: are there chatgpt app devs on X? There are a few from Codex team but I couldn’t find guys from “ordinary” chatgpt.
Also if you could pass this over: it takes 5 taps to change thinking effort on ios and none (as in completely hidden) on macos.
If I were to guess it seems that you were trying to lower the token usage :-). Why the effort is only nicely available on web and windows is beyond me
Earnest question: if I was feeling lazy and security-conscious at the same time, would I be better off...
(A) opening chatgpt.com in qubes (but staying logged out, i.e. never creating a chatgpt account)
-or-
(B) creating a freemium chatgpt account
?
(Obviously, the "best" answer would be something like running a local LLM from an airgapped machine in a concrete bunker :) But that's not what I'm after).
>abuse like bots, scraping
10/10, I've got no notes
<protect our first-party products from abuse like scraping>
Abuse from scraping has long been a serious problem for many, good job!
Hi Nick, the lag is quite bad in the field, honest. In desktop app in this case/datapoint. There was that "halt and catch fire" episode where they spoke about a millisencod threshold of delay that separated usability and non. Solvent hw and fiber connection.
But why block the ui until then? Surely you can just not make any requests until the checks are complete?
I shouldn't be giving ideas to your boss, but I bet he would be interested in making ChatGPT available only by paying customers or free for those whose who gets their eyes scanned by The Orb. Give 30 days of raised limits and we're all set to live in the dystopia he wants.
I always wondered why you even have logged out access. I'm glad I can use ChatGPT in incognito when I want a "clean room" response, but surely that's not the primary use case.
Is user base that never logs in really that significant?
This episode proves they know who you are, even when you're logged out. If they didn't know, they wouldn't let you use the service.
Why send the Turnstile bytecode encrypted ? Surely people savvy enough to abuse the system will find out how to decrypt it, see OP, and it gives the impression that you are trying to hide stuffs you're not proud about.
Because they want to make it as hard as possible to reverse engineer. If they wanted it to be easy, they'd use <input type="checkbox" name="ishuman">I am a human
Why are all these checks still performed on an authenticated, paid user?
> we want to keep free and logged-out access available for more users
How does this comport with OpenAI's new B2B-first strategy?
> We also keep a very close eye on the user impact
Are paid or logged-in users also penalised?
> These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
The lack of self awareness...
> we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
Isn't that how you build your service from the very start? How ironic.
As a free tier user I only get like three queries in now without model quality reduction, so I'd say your bases are covered as far as GPU costs around misuse.
> "abuse like bots, scraping"
You what, mate? Would you please use that on yourselves first? Because it comes off as a GROSS hypocrisy. State of the art hypocrisy.
>> behavioral biometric layer
But this one, especially, takes the cake.
Quite disgusting.
> A big reason we invest in this is because we want to keep free and logged-out access available for more users.
Are these checks disabled for logged-in, paid users?
I really appreciate the free options, without even needing a login. Wish they would also keep the small free weekly allowance for Codex.
Have you given any thought to what we trade when big tech elects one corporation as the gatekeeper for vast swaths of the Internet?
For what it's worth, I switched to Gemini because of the long ChatGPT load time. Gemini loads as fast as Google Search.
The reason why you did it is clear, why you guys settle down for such a poor implementation is why this thread exists
No, leave it. Surely the mighty OpenAI can deal with the scraping. At least, it seems to think everyone else can
> we want to keep free and logged-out access available for more users.
And THANK YOU for that!
Being able to use ChatGPT and Grok without signing in is a big part of why I like those services over Gemini etc.
Hell, dummy Claude won't even let me Sign-In-with-Apple on the Mac desktop, even though it let me Sign-UP-with-Apple on the iPhone! BUT they do support Sign-In-with-Google!!? What in the heavenly hell is this dumbassery
Do you do those checks only for users without accounts or also for those with accounts?
the company that scrapes every until it collapses really needs to protect itself from scraping. Lol.
Kudos for trying
This whole thread was like watching a swarm of ants try and take a grasshopper down
In long threads in chatgpt, it grinds to a halt in both Chrome and Firefox. Please fix
Thanks. I've used ChatGPT a million times and never had any input issues.
> protect our first-party products from abuse like bots, scraping
You do see the irony here?
History will not be kind to you and your ilk. Quit your job.
Hey Nick, I find it concerning this account is. Frayed just to comment on this thread. And never even reply back to any of the real concerns.
Here to hoping this is real person and actually created account out of concern and sharing.
Are you disabling them for paying subscribers?
Then make sure they only target the free tier!
You do not ever trust the client side. Sometimes being simple is good enough. The maximum you can do is put rate limits on the IP address and/or user account. You just do not want some one to use the product at machine speeds.
Hi Nick, your software is a horrendous encroachment on users' privacy and its quality is subpar to those of us who know what we're working with. We don't use your product here.
> Hi Nick, your software is a horrendous encroachment on users' privacy and its quality is subpar to those of us who know what we're working with. We don't use your product here.
It’s ok, OpenAI is cooked.
Feel bad for anyone who joined OAI in the past 12 months. Their RSU ain’t going to be worth much later this year. IPO is too late.
We really need ZKPs of humanity
No, we really don't. We don't need worldcoin, we don't need papers, please. We just don't.
"Prove your humanity/age/other properties" with this mechanism quickly goes places you do not want it to go.
No, it doesn't go places we "do not want it to go". What part of zero knowledge doesn't make sense? How precisely does a free, unlinkable, multi-vendor, open-source cryptographic attestation of recent humanity create something terrible?
It would behoove people to engage with the substance of attestation proposals. It's lazy to state that any verification scheme whatsoever is equivalent to a panopticon, dystopia as thought-terminating cliche.
We really do have the technology now to attest biographical details in such a way that whoever attests to a fact about you can't learn the use to which you put that attestation and in such a way that the person who verifies your attestation can see it's genuine without learning anything about you except that one bit of information you disclose.
And no, such a ZK scheme does not turn instantly into some megacorp extracting monopoly rents from some kind of internet participation toll booth. Why would this outcome be inevitable? We have plenty of examples of fair and open ecosystems. It's just lazy to assert right out of the gate that any attestation scheme is going to be captured.
So, please, can we stop matching every scheme whatsoever for verifying facts as actors as the East German villain in a cold war movie? We're talking about something totally different.
2 replies →
> quickly goes places you do not want it to go.
Which places?
Sure. I'll provide an API to provide mine to your bot for $1 each time.
Protecting your site from bots and scraping is absolutely hilarious considering how you acquired (read: stole) the data you trained your bot on dude.
Just yank that ladder up behind you.
> Just yank that ladder up behind you.
You would be an irresponsible entrepreneur if you didn't. Don't forget your legal obligation to maximise shareholder value.
> I work on Integrity at OpenAI
Irony is truly dead. Show you have integrity by quitting your job
> These checks are part of how we protect our first-party products from abuse like bots, scraping, fraud, and other attempts to misuse the platform.
Isn't this the same behavior used by AI companies to gather training data? Pot, meet kettle.
You’re doing gods work sir, thank you!
I understand it's not your area, but can you please politely tell your colleagues that the clickbait-type teaser questions from the latest model are absolutely infuriating and are quickly leading to me abandon the platform entirely?
If you'd like, I can write a two-sentence paragraph to send to your colleagues. It contains a special phrase which most colleagues will find difficult to ignore. Would you like me to do that?
Hi Nick, do you believe what you say? You scraped the shit out of everyone
Fake Account
the irony of your statement is hilarious, disappointing, and infuriating.
lol, hypocrites.
[dead]
[dead]
[dead]
[dead]