Comment by bredren

I also use "do not overengineer and do not gold plate." I also sometimes tell it to "avoid bike shedding." These are high signal words to the model(s) and they work!

I also have found what I believe is a bad training bias in the design of release related CI workflows toward proof of release artifact provenance.

Both major frontier models love provenance programming in CI, so much that they will spin endlessly trying to solve basic CI functionality at the same time as ensuring SHA's match up across lengthy (often already complex) cross-system pipelines.

I had thought some of my durable context was causing this, and sought to strip anything that might be triggering this behavior.

But then I come upon some more work in release workflows comes up, and boom its back! I couldn't believe it, I called the AI out on it and it agreed it had been told specifically not to do this but was doing it anyway. It did kindly stop and remove the commit(s).

Somewhere, something was oversampled in training because the AI will try their damndest to build this stuff. The worst of it is that it can often involve lengthy, sometimes resource-heavy CI runs so the validation of this unnecessary stuff can have very long feedback loops.

And, sometimes you actually need the provenance. In this case, I've had success forcing the AI to split the work up into functional capability completely devoid of artifact ~chain of custody and get that right before attempting any kind of provenance work.

Bit of a rabbit hole on this, but the above cost me a lot of burned tokens so hopefully helps someone...or some AI.