Comment by winwang
9 hours ago
I had an agent (SOTA models, xhigh reasoning, etc.) write and optimize some CUDA code recently. They've gotten a lot better at this stuff. And yet, they couldn't (self-)realize the obvious issue with their code: it was written as if for a typical multicore CPU. What the heck was a (serial) queue doing there? "Work efficiency" as a tradeoff for stalling 20k threads. Its later proposed optimizations were all about "can we get the queue faster" rather than "maybe we should actually parallelize the work for our very-parallel processor".
Funny stuff, as if it were hell-bent on writing a paper rather than actual software.
Sometimes a session just goes bad and is best abandoned or rolled back to before it went off the rails. I've also been using agents to write and optimize CUDA (and the C++ app code) recently and didn't run into anything so braindead. But, I had one session recently where the LLM started making duplicated tool calls and just couldn't stop doing it. Had to generate a handoff and start again in another session and haven't run into that specific problem again. I don't remember specifics at this point but once and I while I have to rollback a session a half dozen turns when its clear the agent had gone completely off the rails.