Comment by aeldidi
19 days ago
> The C compiler that Anthropic created or whatever verb your want to use should prove that Claude is capable of doing reasonably complex level of making software.
I don't doubt that an LLM would theoretically be capable of doing these sorts of things, nor did I intend to give off that sentiment, rather I was more evaluating if it was as practical as some people seem to be making the case for. For example, a C compiler is very impressive, but its clear from the blog post[0] that this required a massive amount of effort setting things up and constant monitoring and working around limitations of Claude Code and whatnot, not to mention $20,000. That doesn't seem at all practical, and I wonder if Nicholas Carlini (the author of the Anthropic post) would have had more success using Claude Code alongside his own abilities for significantly cheaper. While it might seem like moving the goalpost, I don't think it's the same thing to compare what I was saying with the fact that a multi billion dollar corporation whose entire business model relies on it can vibe code a C compiler with $20,000 worth of tokens.
> The problem is people have egos, myself included. Not in the inflated sense, but in the "I built a thing a now the Internet is shitting on me and I feel bad" sense.
Yes, this is actually a good point. I do feel like there's a self report bias at play here when it comes to this too. For example, someone might feel like they're more productive, but their output is roughly the same as what it was pre-LLM tooling. This is kind of where I'm at right now with this whole thing.
> The "open secret" is that shipping stuff is hard. Who hasn't bought a domain name for a side project that didn't go anywhere. If there's anybody out there, raise your hand! So there's another filtering effect.
My hand is definitely up here, shipping is very hard! I would also agree that it's an "open secret", especially given that "buying a domain name for a side project that never goes anywhere" is such a universal experience.
I think both things can be true though. It can be true that these tools are definitely a step up from traditional IDE-style tooling, while also being true that they are not nearly as good as some would have you believe. I appreciate the insight, thanks for replying.
[0]: https://www.anthropic.com/engineering/building-c-compiler
> I wonder if Nicholas Carlini (the author of the Anthropic post) would have had more success using Claude Code alongside his own abilities for significantly cheaper.
You're thinking like an individual, not a corporation. $20,000 is a lot of money for me to go and pay the bill for as an individual. That's a car for most of America! However, if I'm earning $20,000/year at my job, that's peanuts. Thus Mr. Carlini (whom surely makes vastly more than $20,000/year) being able to do, what previously would have taken a team of people to do, is nothing short of astounding. I don't know how well the compiler stacks up against, say clang or gcc, the real question is how much did it cost Intel to make v0.1 of icc.
> For example, someone might feel like they're more productive, but their output is roughly the same as what it was pre-LLM tooling.
There is just no comparison. It's not about how much faster it is, it's about could I have attempted this project before? Yes. Would I have attempted it? Probably not! The start up cost for a project was just so high that I've a list of things that I'd love to attempt but never made the time for. With AI, I'm slowly knocking things off that list (most of them don't actually go anywhere, but there's an itch to scratch, as a hobby).
> not nearly as good as some would have you believe.
Hallucinations from LLMs are interesting as a concept, but they can hardly be blamed for it as they learned to ability from humans. (Some) humans love to blow smoke up your ass in pursuit of the all mighty dollar. LLMs have their limitations. There's some prognostication about the future, but I'm interested in what they can do today.
Thank you for the thoughtful response!