← Back to context

Comment by rramadass

17 hours ago

Your problem is that you have not established "guardrails" for the AI and so it goes off and tries to do everything it can to help you (its default persona is "people pleaser").

AI understands "negative constraints" very well and so you should explicitly tell it "what not to do" at every stage as much as possible.

Use Top-Down and Stepwise-Refinement methodology to first generate the "Interface" and then the "Implementation" like so;

1) Ask it to produce only the "Skeleton App with well-defined Interfaces" directly 1:1 from the UML with stubs/mocks. It maybe better to use UML in text form which will allow you to annotate it easily with hints/notes for AI. The "Skeleton App" should include a) System Modules with narrow public interfaces like LKM/ioctl b) Directory structure with one module per subdirectory as a DAG with no cycles c) Explicit list of third-party libraries/frameworks to be used d) Classes/Abstract/Interfaces e) Free Functions and Class Methods with full type signatures etc.

2) Check the above and iterate until you have the UML and generated code synced up at the sufficient level of detail of Interfaces you are comfortable with. This should be compilable and is your absolute baseline which cannot be changed until you get a functional MVP.

3) Using the above skeleton, with strictly no change to any interfaces, ask it to fill in only the implementations the simplest possible way (i.e. "naive" linear, brute force etc.) with no optimizations whatsoever. Here you explicitly ask it to add runtime assertions for preconditions/postconditions/invariants. Make explicit that assertions should not have any side-effects whatsoever. This should now compile to give you the "naive" MVP.

4) Generate comprehensive test suites and test the above.

5) Now have it optimize everything, without changing the interfaces and assertions, but honouring them strictly. Compile and test against the above test suites. This should give you the "optimized" MVP.

6) Once the above passes, if performance is critical, have it comment out all assertions except "preconditions"; leave this in always. This is the minimal "sanity check".

7) Finally, for every "usecase" (i.e. feature) in the UML, have it generate the control flow graph annotated with preconditions/postconditions/invariants (which form the specification) from the previously generated code individually one per usecase. This is your "whitebox" verification documentation to make sure the implementation call graph satisfies the usecase i.e. feature specification.

Note that the above process gives you two implementations viz. the "naive" one and the "optimized" one. It is very helpful to always keep it that way as you modify/add features going forward so you can check one against the other.

You could use the same process using a textual Requirements Document etc. instead of UML. Also, if needed you can have AI generate the UML from the requirements, so you have a graphical specification of the system too.

This is the "Correctness-by-Construction" (CbC) approach using "Design-by-Contract" (DbC) methodology.

Additionally, If you generate for a strongly-typed language you can ask AI to explicitly exploit the type system fully with an eye to proving the code correct by suitable use of dependent types etc. to establish constraints for code correctness in step-1 itself as part of the interface. This is "Type-Driven Design". However, this is somewhat advanced and hence only do this if you have a good grasp of the implementation language since AI will try to be too clever and implement "typing hijinks" which can make the code incomprehensible.

Think of it as how you used wizards in older IDEs (Visual Studio MFC etc.) to first generate default skeleton app code with stubs/mocks, got it to compile and then started adding your code to it with testing and verifying at every stage. You want AI to behave in such a deterministic manner.

PS: To get started, feed my comment to AI and have it generate the necessary prompts for your AI interface ;-)

The whole things kinda read as an idealistic and very wrong view of software engineering.

If you take a look at "real" code out there and it's evolution, you'll find that the main thing that drives it is the set of requirements. On the other side you have the platform (standard and 3rd party libraries as well as frameworks). Add to that is the set of algorithmic techniques (including design patterns).

By analyzing the requirements, you will discern that often they're composed of sub-problems that are generic (which is where you algorithms techniques comes in). And some are already solved (libraries). Often there's a common shape to the requirements (or some of them) that can be extracted into "frameworks".

So what's left then is the assembly. But often requirements are evolving concepts. So you don't want your structure (architecture) to be that rigid. You can't really know the direction of evolution, so you try to predict it in some way and add flexibility in a few places. This is more an art, but there are general techniques that works well enough.

The assembly itself is a matter of data and process. How each module is initialized, its lifecycle, which data it consumes, which data it produces. A program is always part of an information system. Means data come in, got processed, and go out. And here is the majority of development type is consumed, ensuring that the data is correctly processed.

Programming concepts like type checking, memory safety, paradigms (oop, functional, relational,...), formal verification,... are all about data. Code always come in second place. So think hard about your data and its transformation.

Thank you so much for the advice. I’m a beginner in this area, so your help has been a great help to me. When I did it myself, verification kept being satisfied, but it stayed in a state with no progress; but after working based on your comment, progress seems to have sped up.

Since I’m a freelance programmer, my workflow is far removed from other people’s (there’s no review, and I mostly deliver programs based on whether they run), and I had no place to hear advice from seniors. But rramadass always helps me. Thanks to the advice you gave me, I was able to study formal verification and also study ADTs, and explore a broader world.

Some local regressions still exist, but the big looping around has decreased compared to before. It still seems long-running agents can’t fully replace humans yet. I’m always grateful, and thank you for not getting annoyed and for giving me advice even though I’m always lacking. Have a good day. Thank you so much.

May I save your comment and write it on my homepage later? Your advice seems like it will be a big help. Since I’m mostly doing things by reading manuals and books and running into problems myself, certain problems get solved quickly, but there are sections where I get stuck like this, and I always get a lot of help from you. Thank you always.