Comment by reachableceo
8 days ago
Did you feed back the results of the tests / benchmark to the model?
I’m presuming you have a very robust test framework / benchmark setup etc?
I’m presuming you fed the model the baseline results of that setup as a starting point ?
Everything is contained in unit tests, so it is really easy to check a small change and get the results for the same dataset.
I have had put the agents in a small feedback loop - but limited to ~5 iterations, since I do not want to burn through my credits. Honestly, I think putting it in a loop for a case like this does not show the "intelligence" part of AI, it makes it more of a more-sophisticated random walk.