Comment by 0x20cowboy
19 hours ago
I find this as well for models that make it to production.
On a more fun note, I also find just trying some hair brained idea is now extremely cheap - so why not? I used to have to weigh up spending a week of effort on something likely not to work and now I can now get spit out test in 20 minutes.
The incredible thing to me is I can say "I have a theory that X is a problem."
The agent then works for 20 minutes and comes back with, "Yes this is a problem. Here is the evidence. Here is a golden data set. Finally I built a new evaluation metric and benchmark where we can back test or use for new model experiments to see if we are closing the gap."
I converted an LSD experience into a 4k demo.
I always wanted to build a specific event/sequence in one LSD trip into a small game to give that experience some kind of playability.
Then I saw some 4k demo, wanted to see how good it is in doing this.
Its really fun yeah :)
Can you share it? Sounds fun!