Comment by sunaurus
10 days ago
The question posed sounds like "why should we have deterministic behavior if we can have non-deterministic behavior instead?"
Am I wrong to think that the answer is obvious? I mean, who wants web apps to behave differently every time you interact with them?
Because nobody actually wants a "web app". People want food, love, sex or: solutions.
You or your coworker are not a web app. You can do some of the things that web apps can, and many things that a web app can't, but neither is because of the modality.
Coded determinism is hard for many problems and I find it entirely plausible that it could turn out to be the wrong approach in software, that is designed to solve some level of complex problems more generally. Average humans are pretty great at solving a certain class of complex problems that we tried to tackle unsuccessfully with many millions lines of deterministic code, or simply have not had a handle on at all, like (like build a great software CEO).
> Because nobody actually wants a "web app". People want food, love, sex or: solutions.
Talk about a nonsensical non-sequitur, but I’ll bite. People want those to be deterministic too, to a large extent.
When people cook a meal with the same ingredients and the same times and processes (like parameters to a function), they expect it to taste about the same, they never expect to cook a pizza and take a salad out of the oven.
When they have sex, people expect to ejaculate and feel good, not have their intercourse morph into a drag race with a clown half-way though.
And when they want a “solution”, they want it to be reliable and trustworthy, not have it shit the bed unpredictably.
Exactly this. The perfect example is Google Assistant for me. It's such a terrible service because it's so indeterministic. One day it happily answers your basic question with a smile, and when you need it most it doesn't even try and only comes up with "Sorry I don't understand".
When products have limitations, those are usually acceptable to me if I know what they are or if I can find out what the breaking point is.
If the breaking point was me speaking a bit unclearly, I'd speak more clearly. If the breaking point was complex questions, I'd ask simpler ones. If the breaking point is truly random, I simply stop using the service because it's unpredictable and frustrating.
> When they have sex, people expect to ejaculate and feel good, not have their intercourse morph into a drag race with a clown half-way though.
speak for yourself
Ways to start my morning...reading "When they have sex, people expect to ejaculate and feel good, not have their intercourse morph into a drag race with a clown half-way though."
Stellar description.
This thing of 'look, nobody cares about the details really, they just care about the solution' is a meme that I think will be here forever in software. It was here before LLMs, they're now just the current socially accepted legitimacy vehicle for the meme.
In the end, useful stuff is built by people caring about the details. This will always be true. I think in LLMs and broadly AI people see an escape valve from that where the thinking about the details can be taken off their hands, and that's appealing, but it won't work in exactly the same way that having a human take the details off your hands doesn't usually work that well unless you yourself understand the details to a large extent (not necessarily down to the atoms, but at the point of abstraction where it matters, which in software is mostly about deterministically how do the logic flows of the thing actually work and why).
I think a lot of people just don't intuit this. An illustrative analogy might be something else creative, like music. Imagine the conversation where you're writing a song and discussing some fine point of detail like the lyrics, should I have this or that line in there, and ask someone's opinion, and their answer is 'well listen, I don't really know about lyrics and all of that, but I know all that really matters in the end is the vibe of the song'. That contributes about the same level of usefulness as talking about how software users are ultimately looking for 'solutions' without talking about the details of said software.
1 reply →
> Because nobody actually wants a "web app". People want food, love, sex or: solutions.
Okay but when I start my car I want to drive it, not fuck it.
Most of us actually drive a car to get somewhere. The car, and the driving, are just a modality. Which is the point.
26 replies →
But do you want to drive, or do you want to be wherever you need to be to fuck?
4 replies →
I feel like this is the point where we start to make jokes about Honda owners.
2 replies →
Sadly, this is not true of a (admittedly very small) number of individuals.
Christine didn’t end well for anyone.
...so that you can get to the supermarket for food, to meet someone you love, meet someone you may or may not love, or to solve the problem of how to get to work; etc.
Your ancestors didn't want horses and carts, bicycles, shoes - they wanted the solutions of the day to the same scenarios above.
6 replies →
Even if it purred real nice when it started up? (I’m sorry)
1 reply →
Weird kink
Food -> 'basic needs'... so yeah, Shelter, food, etc. That's why most of us drive. You are also correct to separate Philia and Eros ( https://en.wikipedia.org/wiki/Greek_words_for_love ).
A job is better if your coworkers are of a caliber that they become a secondary family.
> Average humans are pretty great at solving a certain class of complex problems that we tried to tackle unsuccessfully with many millions lines of deterministic code..
Are you suggesting that an average user would want to precisely describe in detail what they want, every single time, instead of clicking on a link that gives them what they want?
No, but the average user is capable of describing what they want to something trained in interpreting what users want. The average person is incapable of articulating the exact steps necessary to change a car's oil, but they have no issue with saying "change my car's oil" to a mechanic. The implicit assumption with LLM-based backends is that the LLM would be capable of correctly interpreting vague user requests. Otherwise it wouldn't be very useful.
6 replies →
There would be bookmarks to prompts and the results of the moment would be cached : both of these are already happening and will get better. We probably will freeze and unfreeze parts of neural nets to just get to that point and even mix them up to quickly mix up different concept you described before and continue from there.
I think they're suggesting that some problems are trivially solvable by humans but extremely hard to do with code - in fact the outcome can seem non-deterministic despite it being deterministic because there are so many confounding variables at play. This is where an LLM or other for of AI could be a valid solution.
When I reach for a hammer I want it to behave like a hammer every time. I don't ever want the head to fly off the handle or for it to do other things. Sometimes I might wish the hammer were slightly different, but most of the time I would want it to be exactly like the hammer I have.
Websites are tools. Tools being non-deterministic can be a really big problem.
Companies want determinism. And for most things, people want predictability. We've spent a century turning people into robots for customer support, assembly lines, etc. Very few parts of everyday life that still boil down to "make a deal with the person you're talking to."
So even if it would be better to have more flexibility, most business won't want it.
Why sell to a company when you can replace it?
I can speculate about what LLM-first software and businesses might look like and I find some of those speculations more attractive than what's currently on offer from existing companies.
The first one, which is already happening to some degree on large platforms like X, is LLM powered social media. Instead of having a human designed algorithm handle suggestions you hand it over to an LLM to decide but it could go further. It could handle customizing the look of the client app for each user, it could provide goal based suggestions or search so you could tell it what type of posts or accounts you're looking for or a reason you're looking for them e.g. "I want to learn ML and find a job in that field" and it gives you a list of users that are in that field, post frequent and high quality educational material, have demonstrated willingness to mentor and are currently not too busy to do so as well as a list of posts that serve as a good starting point, etc.
The difference in functionality would be similar to the change from static websites to dynamic web apps. It adds even more interactivity to the page and broadens the scope of uses you can find for it.
1 reply →
The issue with not having something deterministic is that when there's regression, you cannot surgically fix the regression. Because you can't know how "Plan A" got morphed into "Modules B, C, D, E, F, G," and so on.
And don't even try to claim there won't ever be any regression: Current LLM-based A.I. will 'happily' lie to you that they passed all tests -- because based on interactions in the past, it has.
So basically you say the future of web would be everyone gets their own Jarvis, and like Tony you just tell Jarvis what you want and it does it for you, theres no need for a preexisting software or to even write a new one, it just does what's needed to fulfill the given request and give you the results you want. This sounds nice but wouldn't it get repetitive and computationally expensive, life imagine instead of Google maps, everyone just asks the AI directly for the things people typically use Google maps for like directions and location reviews etc. A centralized application like maps can be more efficient as it's optimized for commonly needed work and it can be further improved from all the data gathered from users who interact with this app, on the other hand if AI was allowed to do it's own thing, it could keep reinventing the wheel solving the same tasks again and again without the benefit of building on top of prior work, while not getting the improvements that it would get from the network effect of a large number of users interacting with the same app.
You might end up with ai trying to get information from ai, which saves us the frustration..
knows where we’d end up?
On the other hand the logs might be a great read.
We're used to dealing with human failure modes, AI fails in so unfamiliar ways it's hard to deal with.
But it is still very early days. And if you have the AI generate code for deterministic things and fast execution, but the ai always monitors the code and if the user requires things that don't fit code, it will jump in. It's not one or the other necessarily.
Determinism is the edge these systems have. Granted in theory enough AI power could be just as good. Like 1,000,000 humans could give you the answer of a postgres query. But the postgres gonna be more efficient.
No, I wouldn’t say that my hypothesis is that non-deterministic behavior is good. It’s an undesirable side effect and illustrates the gap we have between now and the coming post-code world.
AI wouldn't be intelligent though if it was deterministic. It would just be information retrieval
It already is "just" information retrieval, just with stochastic threads refining the geometry of the information.
1 reply →
Web apps kind of already do that with most companies shipping constant UX redesigns, A/B tests, new features, etc.
For a typical user today’s software isn’t particularly deterministic. Auto updates mean your software is constantly changing under you.
I don't think that is what the original commenter was getting at. In your case, the company is actively choosing to make changes. Whether its for a good reason, or leads to a good outcome, is beside the point.
LLMs being inherently non-deterministic means using this technology as the foundation of your UI will mean your UI is also non-deterministic. The changes that stem from that are NOT from any active participation of the authors/providers.
This opens a can of worms where there will always be a potential for the LLM to spit out extremely undesirable changes without anyone knowing. Maybe your bank app one day doesn't let you access your money. This is a danger inherent and fundamental to LLMs.
Right I get tha. The point I’m making is that from a users perspective it’s functionally very similar. A non deterministic llm or a non deterministic company full of designers and engineers.
4 replies →
The rate of change is so different it seems absurd to compare the two in that way.
The LLM example gives you a completely different UI on _every_ page load.
That’s very different from companies moving around buttons occasionally and rarely doing full redesigns
And most end users hate it.
I think it's actually conceptually pretty different. LLMs today are usually constrained to:
1. Outputting text (or, sometimes, images).
2. No long term storage except, rarely, closed-source "memory" implementations that just paste stuff into context without much user or LLM control.
This is a really neat glimpse of a future where LLMs can have much richer output and storage. I don't think this is interesting because you can recreate existing apps without coding... But I think it's really interesting as a view of a future with much richer, app-like responses from LLMs, and richer interactions — e.g. rather than needing to format everything as a question, the LLM could generate links that you click on to drill into more information on a subject, which end up querying the LLM itself! And similarly it can ad-hoc manage databases for memory+storage, etc etc.
Or, maybe, just not use LLMs?
LLM is just one model used in A.I. It's not a panacea.
For generating deterministic output, probably a combination of Neural Networks and Genetic Programming will be better. And probably also much more efficient, energy-wise.
Every time you need a rarely used functionality it might be better to wait 60s for an LLM with MCP tools to do its work than to update an app. It only makes sense to optimize and maintain app functionalities when they are reused.
For some things you absolutely want deterministic behaviour. For other things, behaviour that adapts to usage and the context provided by the data the user provides sounds like it could potentially be very exciting. I'm glade people are exploring this. The hard part will be figuring out where the line goes, and when and how to "freeze" certain behaviours that the user seems happy with vs. continuing to adapt to data.
Like, for sure you can ask the AI to save it's "settings" or "context" to a local file in a format of its own choosing, and then bring that back in the next prompt ; couple this with temperature 0 and you should get to a fixed-point deterministic app immediately
There still maybe some variance at temperature 0. The outputted code could still have errors. LLMs are still bounded by the undecidable problems in computational theory like Rices theorem.
Why wouldn't the llm codify that "context" into code so it doesn't have to rethink through it over and over? Just like humans would. Imagine if you were manually operating a website and every time a request came in you had come up with sql queries (without remembering how you did it last time) and manually type the responses. You wouldn't last long before you started automating.
> couple this with temperature 0
Not quite the case. Temperature 0 is not the same as random seed. Also there are downsides to lowering temperature (always choosing the most probable next token).
Llms are easily made deterministic by choosing the selection strategy. More than being deterministic they are also fully analayzable and you don't run into issues like the halting problem if you constrain the output appropriately.
Why do good thing consistently when we can do great thing that only works sometimes??? :(
Designing a system with deterministic behavior would require the developer to think. Human-Computer Interaction experts agree that a better policy is to "Don't Make Me Think" [1]
[1] https://en.wikipedia.org/wiki/Don%27t_Make_Me_Think
That book is talking about user interaction and application design, not development.
We absolutely should want developers to think.
As experiments like TFA become more common, the argument will shift to whether anybody should think about anything at all.
2 replies →
This is such a massive misunderstanding of the book. Have you even read it? The developer needs to think so that the user doesn't have to...
My most charitable interpretation of the perceived misunderstanding is that the intent was to frame developers as "the user."
This project would be the developer tool used to produce interactive tools for end users.
More practically, it just redefines the developer's position; the developer and end-user are both "users". So the developer doesn't need to think AND the user doesn't need to think.
1 reply →
..is this an AI comment?
> who wants web apps to behave differently every time you interact with them?
Technically everyone, we stopped using static pages a while ago.
Imagine pages that can now show you e.g. infinitely customizable UI; or, more likely, extremely personalized ads.
Small anecdote. We were releasing UI changes every 2 weeks making app better more user friendly etc.
Product owners were happy.
Until users came for us with pitchforks as they didn’t want stuff to change constantly.
We backed out to releasing on monthly cadence.
No.
When I go to the dmv website to renew my license, I want it to renew my license every single time
Ah, sure; that's why everyone got Adblock and UBo in first place. Even more under phones.
> infinitely customizable UI; or, more likely, extremely personalized ads
Yeah, NO.