Comment by lukevp

Have you used SOTA models to code in the last 2 months or so? This reads like someone who has given up a year ago and made their impressions based off GPT-3.

AI can absolutely have creativity. You just have to engage it like that. The article itself talks about that. You don’t just say “hey AI, go write this code.” You write a spec along with the AI. You tell it what parts are open to its interpretation. Tell it if you want it to be creative or to follow common practices. What level of abstraction is appropriate, etc.

If all you do is give it directions then it just follows the directions.

Also context doesn’t have much to do with a data schema. It’s more like a document database with no schema, if anything. It’s a collection of tokens that it refers back to. Schema implies some structured data with semantic meaning and hierarchies or relationships. That might exist as an emergent property, but for example if I just had a folder full of PDFs, I wouldn’t consider that a schema. That’s kinda what context is like.