Comment by thaumasiotes
3 days ago
By the time you've done that, why not repeat it yourself?
I once needed to define a large table of trigrams in C (for the purpose of distinguishing "English" from "not English"). Setting it up looked like this:
1. Download a book from Project Gutenberg.
2. Write something that went through the book character-by-character, and for every character, remembered the trigram ending there.
3. Make the form of "remembering" be "append a line of code to this file that says `array[char1][char2][char3] = 1`".
4. Now I have a really long C file that compiles into the dictionary I want.
It sounds to me kind of like you want to replace step 3 with an LLM. If that's right... what value is the LLM adding?
Concretely, I was creating a cli application. I had implemented a few commands end-to-end and established some solid patterns. I used Codex (i.e. the PR creating flavor) to provide instructions and get it to review the existing patterns before continuing as asked it to rigorously follow them. I had to do about ~10 more things and it worked really well. It was easy for me to review and understand because I already knew the pattern and it seemed easy for it to get right.
It worked so well that I am always trying to look for opportunities like this but honestly, it isn't that common. Many times you aren't creating a pattern and repeating - you are creating a new pattern. AI is good to chat with to get ideas and come up with an approach in these situations seems to be more effective to me.
> It was easy for me to review and understand because I already knew the pattern and it seemed easy for it to get right.
I suggest that you already knowing the pattern actually makes it harder for you to review code that you expect to contain the pattern. You're likely to perceive it as being there whether it is or not. This strikes me as a way of using LLMs that is more dangerous than average.
Relatedly, proofreading your own work is much more error-prone than proofreading someone else's work, precisely because you have a mental model of your own work (created when you produced it) and you're likely to consult the mental model rather than the work.