Comment by mountainriver

Take it with a grain of salt, this is one man’s opinion, even though he is a very smart man.

People have been screaming about an AI winter since 2010 and it never happened, it certainly won’t happen now that we are close to AGI which is a necessity for national defense.

I prefer Dario’s perspective here, which is that we’ve seen this story before in deep learning. We hit walls and then found ways around them with better activation functions, regularization and initialization.

This stuff is always a progression in which we hit roadblocks and find ways around them. The chart of improvement is still linearly up and to the right. Those gains are the cumulation of small improvements adding up.