Filed a compiler bug related to dwarf tables that screws up debugging and line of code coverage that they completely ignored, just because I mentioned that I had every LLM check it to confirm it's a bug, since all I know is what kcov and every coverage tool generates incorrect coverage data for my repo, for 100% certain.
Reading their comment it sounds like they wanted to confirm the bug existed with AI, not sure they ever said they had AI write the code. Why have such a dumb policy?
Wow I thought surely they wouldn't object to using AI to confirm bugs, but they really do.
Tbf I guess as a popular open source project not using AI to fix bugs, they probably already have more open bugs than they can ever fix so it doesn't really help them for people to find more.
I would imagine his bug was actually ignored just because Zig has 2700 open bugs, rather than some AI policy violation.
Encountering something that seems incorrect, and proving something is a bug and not just YOUR user error, AND reproducing it minimally - turns out to not be easy when it's a low-level compiler issue, and YOU'RE not an expert, especially when it relates to Dwarf tables...
Typically, any time you think you've found a compiler error, you're using it wrong...
Seems like "parallel construction" where you show you actually understand and can explain the issue independent of an LLM would be the way to go. No need to mention how you found the bug as long as you can explain what the bug is, why it matters, and how to repro.
Lying to get around a project policy you disagree with is immature. They have the right to run things the way they want, either accept their rules or leave the project be.
If they really insist on no LLM involvement at all then they're going to fall behind and lose out. Your LLM use case seems really very conservative - you didn't write any code with it, you just use it to confirm the bug. There are many OSS projects that are also taking a similar hardline against LLMs - people are going to fork them and then move on. I've had LLMs fix bugs/add features to a couple of projects like this and, well, they're missing out on the fixes/added features that I'm using locally.
Can you offer any specifics? The ToC looks quite extensive, which to me indicates a lot of design decisions needed to be made (probably involving many people) and unclear how that process might be accelerated by AI.
Unless you're suggesting the language design should also be vibed together?
> Unless you're suggesting the language design should also be vibed together?
Sounds like a fun little project, have a bunch of AI pushers fork Zig and see if they can do a better job. I want to see results, not snarky HN comments. After all this progress, ChatGPT should be able to one-shot a better language since AI is so good now... right?
One-shot, no, but there are a bunch of people pretty much solo-building their personal ideal language with AI and it's going quite well. You need to know just enough about language design to be dangerous, but you don't need to be a seasoned pro.
None of these systems can. They need enormous training. They need alignment and reinforcement. They need harnesses. And most importantly they need a human that knows how to write and develop a C compiler.
The ISO specifications are not sufficient. Neither are the System V guidelines. Not even spec tests and compcert.
You're not going to one-shot it, but over the course of a couple of months of evenings you could come up with something usable/interesting if you manage the LLM well.
Yep, it's true, but the reality of what it would take to actually fork and commence with healthy use of AI alongside with the elusive soft/hard stance needed to lead technically and socially is probably a bigger challenge than anyone is willing to take on! Soo.. slow and (rather?) well it goes for Zig.
Filed a compiler bug related to dwarf tables that screws up debugging and line of code coverage that they completely ignored, just because I mentioned that I had every LLM check it to confirm it's a bug, since all I know is what kcov and every coverage tool generates incorrect coverage data for my repo, for 100% certain.
Sounds like you violated the clearly stated project rules and admitted to it. Are you surprised that they're not engaging with you?
Reading their comment it sounds like they wanted to confirm the bug existed with AI, not sure they ever said they had AI write the code. Why have such a dumb policy?
It's a violation of project rules to confirm bugs with AI?
15 replies →
Pointing out something that is not good doesn't require surprise at the fact that it happened
Yeah, the maintainers of a DOA hobby project should certainly be allowed to run it however they want imo
1 reply →
Wow I thought surely they wouldn't object to using AI to confirm bugs, but they really do.
Tbf I guess as a popular open source project not using AI to fix bugs, they probably already have more open bugs than they can ever fix so it doesn't really help them for people to find more.
I would imagine his bug was actually ignored just because Zig has 2700 open bugs, rather than some AI policy violation.
8 replies →
If it's generating detectably incorrect code, why do you need an LLM to tell you it's a bug?
Encountering something that seems incorrect, and proving something is a bug and not just YOUR user error, AND reproducing it minimally - turns out to not be easy when it's a low-level compiler issue, and YOU'RE not an expert, especially when it relates to Dwarf tables...
Typically, any time you think you've found a compiler error, you're using it wrong...
3 replies →
> breaks the project rules
> gets ignored
Who could have forseen this.
Their policy is immature. Clicking little upvote and downvote buttons is immature. Better to just lie and duplicate the issue.
Seems like "parallel construction" where you show you actually understand and can explain the issue independent of an LLM would be the way to go. No need to mention how you found the bug as long as you can explain what the bug is, why it matters, and how to repro.
Lying to get around a project policy you disagree with is immature. They have the right to run things the way they want, either accept their rules or leave the project be.
If they really insist on no LLM involvement at all then they're going to fall behind and lose out. Your LLM use case seems really very conservative - you didn't write any code with it, you just use it to confirm the bug. There are many OSS projects that are also taking a similar hardline against LLMs - people are going to fork them and then move on. I've had LLMs fix bugs/add features to a couple of projects like this and, well, they're missing out on the fixes/added features that I'm using locally.
People keep promising me that I'll be left behind, but somehow it never seems to happen.
3 replies →
They are doing fine, all this "left behind" talk has no basis other than cope.
Can you offer any specifics? The ToC looks quite extensive, which to me indicates a lot of design decisions needed to be made (probably involving many people) and unclear how that process might be accelerated by AI.
Unless you're suggesting the language design should also be vibed together?
> Unless you're suggesting the language design should also be vibed together?
Sounds like a fun little project, have a bunch of AI pushers fork Zig and see if they can do a better job. I want to see results, not snarky HN comments. After all this progress, ChatGPT should be able to one-shot a better language since AI is so good now... right?
One-shot, no, but there are a bunch of people pretty much solo-building their personal ideal language with AI and it's going quite well. You need to know just enough about language design to be dangerous, but you don't need to be a seasoned pro.
I'm doing it myself: https://zena-lang.dev/
14 replies →
Claude can’t one-shot a C compiler.
None of these systems can. They need enormous training. They need alignment and reinforcement. They need harnesses. And most importantly they need a human that knows how to write and develop a C compiler.
The ISO specifications are not sufficient. Neither are the System V guidelines. Not even spec tests and compcert.
You're not going to one-shot it, but over the course of a couple of months of evenings you could come up with something usable/interesting if you manage the LLM well.
Nobody is one-shotting big projects. Even humans can't do that.
1 reply →
Yep, it's true, but the reality of what it would take to actually fork and commence with healthy use of AI alongside with the elusive soft/hard stance needed to lead technically and socially is probably a bigger challenge than anyone is willing to take on! Soo.. slow and (rather?) well it goes for Zig.
Slow and steady wins the race
Anthropic levels of slowness (pacing)?
Parables aren't reality.
in tech? really?
Language design and implementation is much more in the realm of actual computer science than it is "tech"
In programming languages, usually. Building software on top of a rapidly-changing language is not often a recipe for success.
1 reply →
Yes, the Lindy effect applies in tech too.
And apparently ignores legitimate compiler bugs on the way that someone happened to ask an LLM to double check their diagnosis