Comment by staticassertion
6 days ago
Writing a complex parser or certainly a compiler is a 1 - 3 month project, for example.
Again, I'm not trying to downplay this, but to frame this accurately. I think an AI being able to build a parser/ compiler is cool too.
> One of the more strange phenomena with machines getting better and the incessant need (seemingly driven by human exceptionalism) to downplay each result, is that you just end up belittling humans in the process.
I don't believe in human exceptionalism at all, don't attribute positions to me.
>Writing a complex parser or certainly a compiler is a 1 - 3 month project, for example.
1. Estimating time completion of something that has been done multiple times before and an open problem that has not yet been solved is a different matter entirely. 1 to 3 months is an educated guess and more likely than not, an underestimate.
2. I do not think months long complex compilers and parsers are being routinely completed by LLMs as your original comment implied. Regardless, they are different classes of problems.
I don't get what either of your points is intended to demonstrate. Let's revisit the first post I replied to:
> It's deeply surprising to me that LLMs have had more success proving higher math theorems than making successful consumer software
As far as I can tell, they absolutely have not had more success in this area relative to making successful consumer software.
Well we are kind of arguing past each other aren't we ?
"More success" is a bit vague in this instance but building a compiler that would take a programmer 1 to 3 months is not comparable to this result regardless of whatever similarity exists in time completion estimates. That's the point.
You can publish a paper (and in fact the researchers plan to) off this result. A basic compiler is cool but otherwise unremarkable. It's been done many times before.
You are leaning too hard on how long the researchers (who again did not manage to solve the problem in their attempts) estimated this would take and the "moderately interesting" tag of again, what was still an open research problem.
This, alongside a few math and physics results that have cropped up in the last few months is easily more impressive than the vast majority of work being done with LLMs for software.
5 replies →