They never took off because we discovered, to our surprise to some extent, that gradient descent through back propagation works better than expected if you give it the right learning media and the right input and output encodings. It took a ton of fiddling ("graduate student descent") to figure those out.
Back then everyone thought it was doomed to get stuck at local minima, but it turns out that has a lower probability of happening if the search space has enough dimensions. It works well enough to make the sand talk back to us and now that particular design has sucked all the air out of the room.
Nobody has tried EC at anywhere near the scale of GPTs/LLMs because that amount of compute is expensive and at this point we know those will at least work.
I still think EC is fascinating and would love to play with it some more at some point, maybe trying it combined with back propagation in novel ways. Compute only gets cheaper.
They never took off because we discovered, to our surprise to some extent, that gradient descent through back propagation works better than expected if you give it the right learning media and the right input and output encodings. It took a ton of fiddling ("graduate student descent") to figure those out.
Back then everyone thought it was doomed to get stuck at local minima, but it turns out that has a lower probability of happening if the search space has enough dimensions. It works well enough to make the sand talk back to us and now that particular design has sucked all the air out of the room.
Nobody has tried EC at anywhere near the scale of GPTs/LLMs because that amount of compute is expensive and at this point we know those will at least work.
I still think EC is fascinating and would love to play with it some more at some point, maybe trying it combined with back propagation in novel ways. Compute only gets cheaper.