Comment by xmprt

6 hours ago

I see a lot of researchers working on newer ideas so I wouldn't be surprised if we get a breakthrough in 5-10 years. After all, the gap between AlexNet and Attention is All You Need was only 6 years. And then Scaling Laws was about 3-4 years after that. It might seem like not much progress is being made but I think that's in part because AI labs are extremely secretive now when ideas are worth billions (and in the right hands, potentially more).

Of course 5-10 years is a long time to bang our heads against the wall with untenable costs but I don't know if we can solve our way out of that problem.

2 comments

xmprt

Reply
ghywertelling  4 hours ago

I think we will see models becoming small reasoning core which don't remember tonnes of facts but can reason with data fed to it or they can search.