I would argue that the better you can “close the loop” the better. One of my favourite agent activities is giving it a firmware bug and JTAG access + a serial port. That setup means that flashing, debugging (inspecting variables and whatnot, setting breakpoints, etc), and logging can all happen completely autonomously.
Works especially well for transient/hard to repro bugs. It’ll either set breakpoints or write a monitoring script, go to sleep, get woken up by the harness when the bug trips, inspect state, put together a new build based on a hypothesis built from the debugging, flash it, and loop. I’ve had sessions run completely autonomously for over a day and come with a beautiful one-line solution to a gnarly race condition.
In that regard it’s like a human: you can read code all day if you want and you might get somewhere, but being able to inspect what happened, build a new theory, test it, and iterate is way more powerful 99% of the time.
I was trying to upload HDR photos to Instagram and couldn't figure out how to make it preserve the HDR-ness of my photos, and was going back and forth with an agent - step by step we'd slowly iterate on a photo, I'd upload, then I'd download the jpg and give it to the agent and it'd think, change something in the format, and we'd try again.
I got impatient and just said fuck it and gave it chrome-devtools and a blank instagram account and a test photo, and it figured the whole thing out in about a half hour. I built a little browser-based tool from what it discovered, and I use it to run my HDR photos through to format them correctly for instagram and it works a treat.
> I built a little browser-based tool from what it discovered, and I use it to run my HDR photos through to format them correctly for instagram and it works a treat.
I was kinda expecting: "And when my agent built a perfect clone of Instagram and social-engineered all my contact list to join"
I would argue that the better you can “close the loop” the better. One of my favourite agent activities is giving it a firmware bug and JTAG access + a serial port. That setup means that flashing, debugging (inspecting variables and whatnot, setting breakpoints, etc), and logging can all happen completely autonomously.
Works especially well for transient/hard to repro bugs. It’ll either set breakpoints or write a monitoring script, go to sleep, get woken up by the harness when the bug trips, inspect state, put together a new build based on a hypothesis built from the debugging, flash it, and loop. I’ve had sessions run completely autonomously for over a day and come with a beautiful one-line solution to a gnarly race condition.
In that regard it’s like a human: you can read code all day if you want and you might get somewhere, but being able to inspect what happened, build a new theory, test it, and iterate is way more powerful 99% of the time.
This kind of approach works so well.
I was trying to upload HDR photos to Instagram and couldn't figure out how to make it preserve the HDR-ness of my photos, and was going back and forth with an agent - step by step we'd slowly iterate on a photo, I'd upload, then I'd download the jpg and give it to the agent and it'd think, change something in the format, and we'd try again.
I got impatient and just said fuck it and gave it chrome-devtools and a blank instagram account and a test photo, and it figured the whole thing out in about a half hour. I built a little browser-based tool from what it discovered, and I use it to run my HDR photos through to format them correctly for instagram and it works a treat.
> I built a little browser-based tool from what it discovered, and I use it to run my HDR photos through to format them correctly for instagram and it works a treat.
I was kinda expecting: "And when my agent built a perfect clone of Instagram and social-engineered all my contact list to join"