Comment by Terr_

My point is that most such comparisons are already flawed because the "machine" people are referring-to is an illusion.

It's like people are debating the cellulose-quality of playing cards, comparing cards in a TV broadcast of a (real) poker tournament versus the cards that show up through a magical spy window caused by solitaire.exe. The comparison is already nonsense because the latter set of cards has no cellulose, or any mass at all.

Similarly, the recipient of your "now do X" command in an LLM chat doesn't really exist, so can't have source-code or variables or goals. The illusion may sometimes be useful (esp. for marketing and getting investor money), but software engineers can't afford to fall for it when trying to diagnose problems.

The real "constraints" are that each remotely-generated append to a hidden document statistically fits what came before with a certain amount of wiggle-room. Maybe that means you see text about "HAL-9000" opening the pod bay doors, and maybe you don't, but the document-generator is the thing in charge.