Comment by saguntum

Three moments:

1. When ChatGPT came to the masses, it trivially solved my standard phone interview problem for new SWE hires. It's not particularly complicated, but it screened out a lot of candidates pre-AI and was a good filter.

2. At one point, there was a bug in some client software we ship that was erroneously displaying a protocol-level disconnect message to clients when the server shut the connection unexpectedly.

In very few turns, ChatGPT gave me working code to intercept the error at the client level via a pseudo-server that intercepted the requests and implemented the proper teardown procedure. It essentially implemented a micro-server for the protocol we were working in as a bug workaround.

3. I'm working on a major rearchitecture (6+ month project of very senior engineers) of an internal system for scalability and maintainability with tight latency bounds where correctness with the old system is necessary.

I came back from an international vacation quite jet lagged and was having trouble doing actual coding work for a bit since my focus was shot. Taking additional days off wouldn't have helped recover from the sleep issue more quickly due to childcare responsibilities.

I tried instead building a pipeline where I'd run an integration test suite, throw a list of failing tests at Claude, and make a PR if it made more tests pass. I automated as much of the AI "loop" as possible in my state at the time.

Using this technique, a coworker and I fixed over 200 failing integration tests for parity with the old code in 1 week. We subdivided failing tests between engineers and re-sub-divided with each success.

I re-reviewed the code before we launched when I was more well-rested, but this was an extremely effective technique and makes me think that AI-enhanced test driven development (TDD) is the future.