← Back to context

Comment by lifthrasiir

4 days ago

Yeah, I recently wrote a moderate amount of C code [1] entirely with Gemini and while it was much better than what I initially expected I needed a constant steering to avoid inefficient or less safe code. It needed an extensive fuzzing to get the minimal amount of confidence, which caught at least two serious problems---seriously, it's much better than most C programmers, but still.

[1] https://github.com/lifthrasiir/wah/blob/main/wah.h

I've been doing this the better part of a lifetime and I still need to be careful so don't feel bad about it. Just like rust has an 'unsafe' keyword I realize all of my code is potentially unsafe. Guarding against UB, use-after-free, array overruns and so on is a lot of extra work and you only need to slip up once to have a bug, and if you're unlucky something exploitable. You get better at this over the years. But if I know something needs to be bullet proof the C compiler would not be my first tool of choice.

One good defense is to reduce your scope continuously. The smaller you make your scope the smaller the chances of something escaping your attention. Stay away from globals and global data structures. Make it impossible to inspect the contents of a box without going through a well defined interface. Use assertions liberally. Avoid fault propagation, abort immediately when something is out of the expected range.

  • I strategy that helps me is just not use open-coded pointer arithmetic or string manipulation but encapsulate those behind safe bounds-checked interfaces. Then essentially only life-time issues remain and for those I usually do have a simple policy and clearly document any exception. I also use signed integers and the sanitizer in trapping mode, which turns any such issue I may have missed into a run-time trap.

    • This is why I love C. You can build these guard rails at exactly the right level for you. You can build them all the way up to CPython and do garbage collection and constant bounds checking. Or keep them at just raw pointer math. And everywhere in between. I like your approach. The downside being that there are probably 100,000+ bespoke implementations of similar guard rails where python users for example all get them for free.

      2 replies →

    • > I strategy that helps me [...]

      In another comment recently I opined that C projects, initiated in 2025, are likely to be much more secure than the same project written in Python/PHP (etc).

      This is because the only people choosing C in 2025 are those who have been using it already for decades, have internalised the handful of footguns via actual experience and have a set of strategies for minimising those footguns, all shaped with decades of experience working around that tiny handful of footguns.[1]

      Sadly, this project has rendered my opinion wrong - it's a project initiated in 2025, in C, that was obviously done by an LLM, and thus is filled with footguns and over-engineering.

      ============

      [1] I also have a set of strategies for dealing with the footguns; I would gues if we sat down together and compared notes our strategies would have more in common than they would differ.

      1 reply →

This is exactly my problem with LLM C code, lack of confidence. On the other hand, when my projects get big enough to the point where I cannot keep the code base generally loaded into my brains cache they eventually get to the point where my confidence comes from extensive testing regardless. So maybe it's not such a bad approach.

I do think that LLM C code if made with great testing tooling in concert has great promise.

> It needed an extensive fuzzing to get the minimal amount of confidence, which caught at least two serious problems---seriously, it's much better than most C programmers, but still.

How are you doing your fuzzing? You need either valgrind (or compiler sanitiser flags) in the loop for a decent level of confidence.

  • The "minimal" amount of confidence, not a decent level of confidence. You are completely right that I need much more to establish anything higher than that.