Comment by cma
21 hours ago
"I like the idea of a meta-mechanism that learns to update an associative memory based on how surprising the data is."
Just pointing out that that idea was in some of Schmidhuber's earlier work.
> Schmidhuber (or in this case you, on his behalf!) effectively saying "I worked on that problem, years ago" is irrelevant.
Ok. People do read his work and get ideas from it even if this didn't necessarily. He had a lot of good stuff.
> but the question is how do you combine and realize these functionalities into an actual cognitive architecture?
I believe Schmidhuber gave one at the time?
Does it work out-of-the-box today?
Execution is what matters. We can smoke a blunt and have some nice sounding ideas, but building something that works on data at scale is what actually counts.
I think it's widely agreed a lot of useful stuff came out of Schmidhubers lab. The example I gave was one of the first things that scaled in lots of ways especially in depth, and it shares some characteristics with this. I doubt it outperforms this Titan architecture or is equivalent. That's not the same as him just putting out random ideas while high.