Comment by red75prime
19 hours ago
At this point, it’s better to ponder a question: Are there fundamental reasons why a search trajectory during reinforcement learning can’t leave the envelope? And: what is the envelope, anyway? A metaphorical surface in a metaphorical space of ideas with no defined procedures to make it into an actual metric space?
Reinforcement learning can definitely (in principle) leave this envelope, but it depends crucially on whether the feedback generates new information from reality, rather than just repackaging what humans have already discovered in the form of RL environments and densified rewards. So the question is not about RL per se, but what is the information source, what is the bandwidth of information it supplies, and then only finally the effectiveness of RL in distilling information that into models.
Great point. Perhaps when we have AI robots interacting in the real world doing experiments we'll have that.
That is right now. [1]
[1] https://www.reuters.com/world/anthropic-quietly-sets-up-biol...