Comment by cadamsdotcom

Biggest takeaway: the project succeeded because its acceptance criteria were clear and deterministic.

The human driving the LLM gave it a way to know when it was done and a way to move toward that goal. They used code to generate tests and let the agent evaluate its implementation in a deterministic way.

This is the value of an engineer: you understand when to introduce determinism to let the LLM do the bit it does best - while keeping it on the rails.