Comment by jrflo
1 day ago
I think that's a bit reductive, there are decades of sci-fi novels about how transformative artificial intelligence would be, so I feel like I don't need to explain where the trajectory could go. The applications and "collision with the real world" aren't there yet, but you can't pretend that a machine that's smarter than any living human is on the same level as Amazon drone deliveries.
I agree with you, but to play devil’s advocate: what if it is on the same level as Amazon drone deliveries? Is intelligence really a bottleneck in society? People in MENSA with high IQs don’t often accomplish anything in proportion to their relative intelligence. The most intelligent people are rarely even in charge of making decisions. There are a lot of intelligent people with intelligent ideas that nobody listens to. It seems like the real bottleneck in society is building trust, consensus and constructive incentives between groups of people to do the things we already know are smart, not a lack of intelligence.
Interesting idea, but I'd have to disagree. MENSA is basically a group of benchmaxxed people, they are super good at a set of testing but that doesn't mean they have the ambition or other social skills to make a difference in the world. When you align all those factors, you find the greatest contributors to science and engineering. What happens if you completely eliminate the need for someone to be inherently intelligent? Suddenly, the number of important difference makers in the world increases by an order of magnitude.
I agree with the sentiment though, intelligence is not all you need. We're not going to "solve science" through RLVR, people are still needed in the loop until we get superhuman construction robots and superhuman communicators.
Good points. Maybe a more general version, removing constraints on one resource is usually not hugely impactful because you simply hit the next constraint.
Concrete example, "coding is free" (TBC, but let's assume), and now we're code review constrained. Even if code review becomes free we probably just become product discovery constrained, or something else. Making all software free will just cause us to hit on hardware constraints, and so on. It's still progress, and more progress will bring more benefits, but there's a limit to how much better the world as a whole can get from any one change like this.
MENSA selects for puzzle solving abilities.
Reinforcement learning trains task completion abilities.
MENSA also selects for people who want someone to certify their intelligence. High iq people who have done a lot have much less desire for a MENSA membership. We should be looking at the non-MENSA members with high iqs and seeing what they're doing.
MENSA selects for puzzle solving on the basis that those are correlated with "being smart".
Reinforcement learning selects for models that appear to complete tasks, on the basis that appearing to complete a task is correlated with completing a task.
Both have problems. The latter is probably a better correlation, but by no means flawless. If we just take at face value that RL is training models to complete tasks, we're not engaging deeply enough with the process enough to see where it falls short.
I don't disagree with where AI could go, but I don't think we've actually proven that it's there. My point is that the folks claiming we are there already if only the rest of the world would just recognise it, are repeating the same thing from every previous tech cycle. In many cases it's the same folks.