Comment by andai
14 hours ago
Results seem mostly noise to me. One eval per model, in a large problem space (i.e. a problem which requires many attempts to solve well).
14 hours ago
Results seem mostly noise to me. One eval per model, in a large problem space (i.e. a problem which requires many attempts to solve well).
Yes I agree, but I actually did a lot more runs, with different prompts, different times ect... And each time /goal had a small or insignificant impact