Comment by leobg

I find it odd that your engineer would make the system rely on instructions (“Do this. Never do that.”). This exposes your system to inconsistencies from the instruct tuning and future changes thereof by OpenAI or whoever. System prompts and instructions are maybe great for demos. But for a prod system where you have to cover all the bases I would never rely on such a thin layer of control.

(You can imagine the instruct layer to be like the skin on a peach. It’s tiny in influence compared to what’s inside. Even more so than, in humans, the cortex vs. the mammalian brain. Whoever tried to tell their kids not to touch the cookies while putting them in front of them and then leaving the room knows that relying on high level instructions is a bad idea.)