Comment by tyleo
1 day ago
If leaking credentials is observable behavior. I don't get your point.
Memory leaks, deleting the hard drive, spending money would all be observable behavior.
By your reasoning that the "observable behavior needs to be specified rigorously" it seems like you'd have to list these all out. We do, after all, already have cases of AI deleting data.
That sounds harder and more error prone than what we're doing now by rigorously defining these defects out of existence in code.
My ultimate point is that source code specifies everything the software can or will do, but most of that is not part of the required specification. Most of it is implementation details you don’t care about.
The entire reason we have functions and components and modules etc is to isolate engineers from the things we do not need to care about. I should not need to care about the implementation details of most software, only if it meets my retirements.
The move to AI first software development will not happen because we find a way to specify as much in English as we previously would have specified in a programming language. The move will happen when and as we figure out how to specify the things that matter. We don’t need the same rigor. We need the correct rigor.
> I should not need to care about the implementation details of most software, only if it meets my retirements.
The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details that have not made it into the specifications. One one side you have the platform and on the other side you have the interface contract (requirements). Saying what’s in the middle doesn’t matter is strange. Because both the platform and the interface are dynamic and can shift drastically from their 1.0 version.
> The only reason those details don’t matter to you is because someone has gone through the pain of ironing out every details
I disagree. Very often the reason the details don’t matter is that they are irrelevant. There are a million ways an app might remember my personal settings, as a simple example. SQLlite db, json file, ini, cloud storage, registry, etc. The specific implementation matters very little so long as it’s sane.
> Saying what’s in the middle doesn’t matter is strange
I understand your point but do not agree. I think over the next decade we will get increasingly good at specifying rigorously the parts of the surface that matter while increasingly caring less and less about the rest. We will not find a way to write rigorous code in English because that would necessarily be less efficient than just using a programming language.
5 replies →
Yeah, I'm with you and I'm not seeing much sense in the argument above.
We should only need to specify observable behavior... but observable behavior is so broad that it includes common defects... but also we shouldn't need to specify the lack of defects even though they are observable.
It feels like, "the AI should read my mind and do the right thing without me fully specifying it."