Comment by t0mpr1c3

For me, identifying a problem of "taste" doesn't exactly break it down. The problem is underspecified in some way. Can we be more specific?

The sense I get from this discussion is that the models/harnesses do not elicit feedback well. Where there is ambiguity, they tend to pick a solution and call it good.

A planning step aims to make these choices explicit. An iterative process is necessary to capture the detail.

It is natural to look to teams of agents to satisfy that process, but do they know where the decision points are?