Comment by antichronology

That would be really cool. Navigating SRA and mining out reasonable $ relevant tasks is a huge bottleneck.

I find it takes a large amount of effort to parse what the authors are doing, whether the data is high quality, and how to pre-process it in a way that makes sense for the task at hand.

Would love to chat more about how you're thinking of evaluating quality of these agents.