Comment by mike_hearn

Google just published a paper on a new neural architecture that does exactly that, called Titans.