Comment by asdff
6 hours ago
With enough model drift that won’t even work over time. These files would have to be pinned to the intended model version and that is either used directly or emulated with a faithful emulator in a larger model.
6 hours ago
With enough model drift that won’t even work over time. These files would have to be pinned to the intended model version and that is either used directly or emulated with a faithful emulator in a larger model.
I doubt that will be a problem. Models are different enough right now. If you can write a spec with enough detail and constraints today to get valid and comparable output from say GPT-6 Astra, Opus 5.5, DSV4F, Kimi K3, GLM 5.3, etc... Then I think there's a good chance that whatever SoTA coding LLMs everyone is using 3 years from now will also be able to implement that same spec.
Again, I think it heavily depends on if people are writing comprehensive specs with sufficient detail.
That depends on if you are really getting what you expect from the spec or you are actually relying on undefined behavior for your expected output from the spec. This is why we often pin software library versions: we may very well be relying on unintended or a bugged behavior to get out expected output.
llmpm