Comment by ninkendo
5 days ago
Yesterday I updated our dependency on the sqlx crate and put up a PR, and it failed in the CI build in a way I couldn’t reproduce locally.
I asked codex to take a look, and it:
- Grabbed the CI logs on its own to figure out what the CI error was
- Looked at my local setup
- Looked at the changes in sqlx from 0.8 to 0.9
And figured out that sqlx depends on an updated version of the “whoami” crate but doesn’t specify default features, which causes it to fall back on a stub implementation that makes the default user “anonymous”, which was failing to authenticate to the UNIX socket we use in our CI Postgres server. It patched the environment variable for our docker container to explicitly specify a username and the issue was fixed.
It would’ve taken me probably several hours to figure this out on my own. It took codex maybe 5 minutes.
Tell me again how LLM’s “don’t work”?
I agree with your point in the broad sense, but the example might be bad. If sqlx is an important crate, and not stable yet, upgrading it without reading the changelog is honestly a flaw in your team process. Using the AI to fix organisational issues is typically one of the reasons I'm very skeptical of AI improving productivity in the long run.
I'm not taking a shot, to be clear, we had a similar issue a few years ago and we made sure this wouldn't happen again, that's absolutely not a shot, nor do I think it's a character flaw to use AI, au contraire, this is a very good use. I'm just worried that because AI is so good at fixing minor issues caused by governance/organisation flaws, we will be stuck using it to fix those and be trapped in mediocrity (that's not an issue for me, mediocrity is where I work best, but I'm a bit sad for the great Devs I've worked with.)
> If sqlx is an important crate, and not stable yet, upgrading it without reading the changelog is honestly a flaw in your team process
It’s not in the changelog though, this is an update of a transitive dependency that inadvertently changed the default behavior. sqlx didn’t document this because they didn’t even know it changed.
Even if it was a documented change, our process caught it because it was caught by CI. The issue itself was only a result of how our CI was configured (we had a database url with a domain socket path that didn’t explicitly specify a username, and we inadvertently relied on the default of “the current user”, which the whoami crate now defaults to “anonymous”.) I don’t see an issue in our “team process” (whatever that means) at all.
You used it in a way where the result was simple and you could verify its correctness. You used it as a super-search tool, it's good at that. It's a different use case than having it generate a lot of code from scratch.
Exactly. If people understood that this is super-search and super-autocomplete, we'd maybe find a real net-positive use for the tech. But I think the conversational tone will keep fooling us, especially since the LLM providers have heavily invested in that direction.
It sounds like the real problem was that the programmer was not familiar with the tools they were using and decided to dig themselves out of a hole of their own making by turning to AI instead of learning to use their tools better.
What the fuck are you talking about?
A bug in a dependency means I’m not familiar with the tools I’m using? What exactly should I have had a priori knowledge of that would have solved this? That the whoami crate, which I don’t use, but was a transitive dependency, changed a default which affected the sqlx crate in a way that they didn’t know about either, in a way that produced an error that only happened in the CI environment and not locally? And I should have known this how exactly?
I hate what AI is doing to our industry and I hate the constant deluge of slop I have to review every day. I’m not an AI lover by any means. But Jesus Christ why do people bend over in any direction imaginable to pretend that reality isn’t reality (LLM’s must be useless because I personally don’t like them) and blame someone 3 levels away from a problem just to make themselves feel better.
But sure, no, it must be me “not knowing my tools enough” because I’m not aware of bugs in other projects that their maintainers themselves are not aware of. Because the alternative is that AI might actually be useful in some way, and we can’t admit that, can we?
Welcome to how programming used to be...
Not being aware of the limitations of the tools you're using is a hallmark of junior-level programmers. That's why experience matters; if it was just about raw coding talent nobody would care about experience. But making mistakes like this is supposed to be how you learn.
Unless of course you turn over the learning to the AI, as you did here. Otherwise you would have known that the defaults you're complaining about are clearly identified in the README files you didn't read.
1 reply →