Comment by postalcoder
18 hours ago
Github had, by far, the most easily game-able agent usage policy. People would force the agent to run a script before the end of turns that consisted entirely of `input("prompt: ")` so that you could essentially talk endlessly to an agent for the price of a turn. I see this less about the future of this industry and more about fighting the costs incurred by bad actors.
I never played any games like that, but simply giving the agent a clear exit criteria and instructions to check the exit criteria every time it thinks it's done on a complex task was often enough to keep it chugging away for most of a day on a single prompt in my experience. Per-prompt pricing just isn't sustainable period, even if everyone is acting in good faith.
Charging by prompt was always wild to me.
I once asked it to do a comprehensive security review of our code. It churned for nearly an hour (and then produced 90% false positives). Insane that that usage was charged the same amount as me just saying "Hello".