Comment by fooker
1 month ago
> non-deterministic
You can control the amount of non determinism.
And also, it is interesting that you think modern compilers are deterministic.
1 month ago
> non-deterministic
You can control the amount of non determinism.
And also, it is interesting that you think modern compilers are deterministic.
While an LLM with a fixed seed is technically deterministic, it’s still unpredictable.
> And also, it is interesting that you think modern compilers are deterministic.
A compiler, unless it has a bug, will always produce output matching the specification of the language.
> the specification of the language
Guess what language the specification is written in (if it exists at all) ?
It's usually natural language, and even then, compilers deviate from the specification all the time, and the specification has bugs.
Formal specifications are a thing, but the science is nowhere mature enough to dictate what compilers do.
> unless it has a bug
Compilers are not magic, the way it follows specifications is up to the interpretation of compiler developers like yours truly. And there are tens of thousands of compiler bugs. There was a notorious LLVM pass that had more bugs than lines of code ;)
https://github.com/llvm/llvm-project/issues?q=is%3Aissue
This is a list for just the last few years after LLVM switched to the using github for tracking issues.
Please do not engage in bad-faith arguments centered around pedantry. You know very well what I mean.
I know what you mean, and it was correct a year or more ago. Now, you are wrong.
AI is reliable enough to not mess up this translation now, especially if you configure it right (top p, and temperature parameters).
This abstraction lifting is absolutely happening in front of our eyes now. For the exact same reason the C for loop is less readable.
The different is that you don't yet store the prompts, just the generated code. That difference is not going to last too long. Storing prompts and contexts along with generated code is likely how we are going to be doing software engineering for a few decades before whatever the next leap in technology works out to be.
You could just lock the seed to get "deterministic" behaviour, but you are missing the point of programming languages completely. Programming languages are a set of rules that ~guarantee predictable behaviour down to the bit level. If you were to try to recreate that with LLMs, you run into two problems: one, your LLM is now a programming language where you have to put exact specific inputs in to get the correct outputs, except you don't have a specification for the inputs and you don't know what the correct outputs even look like because you aren't a software engineer. Two, even with a locked seed, the LLM is still going to output different code based on the exact order of letters in your prompt and any change in that will change the output. Compilers can execute a variety of optimizations on your code that change the output, but in the end they are still bound to hard rules and you can learn the rules when you get output that does not match what your expectations were from the input. And if there is a bug in the execution of the rules by the compiler, you can fix it; that is not possible with an LLM.
This talk about replacing software engineering by people who have no idea what software engineering is gets unbelievably tedious. The advent of Javascript did nothing to replace software engineers, it just created an entirely new class of developer. It lowered the barrier to entry and allowed anybody to write inefficient, bloated, buggy, and insecure programs. Our hardware is advanced enough that for many trivial applications there is sufficient overhead for inefficient and bloated programs to exist and be "good enough" (although they are causing untold damage in the real world with security breach after security breach). However, lowering the barrier to entry does not replace the existing engineers. You still need a real software engineer to develop novel applications that use the hardware efficiently. The Duchies of Javascript and Python are simply a new country founded adjacent to, and depending upon, the Software Engineer Kingdom. Now a new duchy is being founded, one that lowers the barrier to entry further to make even more inefficient, even more bloated, even more buggy, and even more insecure programs easier than ever. For some use cases, these programs will be good enough. But they will never replace the use cases that require serious engineering, just as Javascript never did.
6 replies →