Comment by bluegatty
5 hours ago
The AI is not remotely ready for this - this is an absolute delusion they are selling.
"By the way don't do this again" <- as if the AI has the ability to ingest and systemically diffuse this.
I think Zuckerberg himself is deeply into the Koolaid, and is likely himself unaware of the limits of this tech.
He's probably surrounded by enablers.
If it uses "memory" files like Claude Code there's a good chance it works.
If it was Claude I'd expect it to now put some form of this into every output even when not really related to the task: "No offers were accepted without consulting you and I haven't shared your address or availability."
Is it guaranteed to work? No.
And obviously it's a terrible idea to set up a chatbot to communicate, negotiate deals, and handle logistics on your behalf.
"If it uses "memory" files like Claude Code there's a good chance it works."
You have explained literally why it would not work.
The AI can absolutely not depend on 'arbitrary statements in some file' as operational policy.
For a very, very narrow scope of work, when it's well defined, when the information is rigorously applied, sure ...
But they don't have that.
The are throwing agents out there like they can handle this degree of complexity and nuance, when they cannot.
100% failure rate over any period of time.
I didn't say it would work with high reliability, or that it would fix this clearly unsuitable use case.
However, it is untrue that it doesn't have the capability to memorize an instruction and diffuse it to new sessions.
Simply saying "do not ever do this again" can result in the behavior not reoccurring with any likelihood.
You'd have to benchmark whether with the instruction in place it would violate it, and in how many cases, so that you can understand the risk better.
4 replies →
Had that problem here. Our product lead thinks it’s infallible and is surrounded by yes men.
Day one our agentic platform goes live it causes a reportable compliance issue. Massive clean up. Reputational impact. Turned off.
No one held accountable still. Assuming that adding more guardrails will fix everything.
You should write about that, because it's the primary systems failure of AI right now.
'Managers Delusion' - which includes techies as well, to be fair, at least they have the excuse they are one step removed.
I'd rather everyone burned for it than heeded a warning.
They are morally bankrupt and will try and do it again and again if it's stopped too early.
1 reply →
They're building for future capabilities. It's a logical approach to leveraging the data they have.
The AI is not ready for this _yet_, but it will be, and FB wanting getting ahead of the game here is potentially good business. It’s all in the public perception of utility vs fuck-up, and it’s far too early to say Zuck got that wrong, and indicators are he got that right.
> The AI is not ready for this _yet_
It should be ready in about two more weeks! How many models have a Ph.D level intelligence now? I feel like I’ve been hearing that for about a year at this point.
I’m guessing you’re not a programmer. If you were, you’d have seen models go from “kinda helpful for programming” to “usable as a daily driver” about 9 months ago, for example.
“Models aren’t improving incredibly fast” seems a very odd point to be making.
2 replies →
[flagged]