Comment by andischo
8 years ago
"Five individual logic cells were functionally disconnected from the rest— with no pathways that would allow them to influence the output— yet when the researcher disabled any one of them the chip lost its ability to discriminate the tones. Furthermore, the final program did not work reliably when it was loaded onto other FPGAs of the same type.
It seems that evolution had not merely selected the best code for the task, it had also advocated those programs which took advantage of the electromagnetic quirks of that specific microchip environment. The five separate logic cells were clearly crucial to the chip’s operation, but they were interacting with the main circuitry through some unorthodox method— most likely via the subtle magnetic fields that are created when electrons flow through circuitry, an effect known as magnetic flux."
This is absolutely incredible. Makes you wonder how much potential the real world has compared to the simulated environment usually used to test theoretical solutions.
It's an example of overspecialization and finding weird local minimum trough loophole in the way the problem is encoded.
Genetic algorithms have the ability to capture the imagination of public and computer science students because they can find very messy and random solutions if you run them long enough.
In the general context of search and optimization algorithms they are not impressive. When you can't use anting better, like Mote Carlo or simulated annealing, evolutionary algorithms are often the last hope before brute forcing it. GA can be very impressive when you can restrict the search space and find good representation for the problem.
I get the advantages of generic algorithms. But sometimes overfitting can be very useful. Imagine wind or water turbines where this method could be used to increase their efficiency based on the individual hardware.
Its also likely the solution it found would not work across the operating temperature range of the device. To do it properly would take a lot longer and need a lot more test cases.
Assuming proper instrumentation, GAs provide much better performance vs. brute force in locating software defects. E.g. http://lcamtuf.coredump.cx/afl/
Yes they do. I didn't mean to imply that they are useless.
Evolutionary algorithms and swarm optimization algorithms are the next step above brute force and random search. In the space of optimization algorithms are below everything else.
Usually they are the last option. Using them to locate software defects means that you have no special insight into software defects but you hope to generate something better than random.
I don't think it's that incredible - it demonstrates that even evolutionary algorithms are susceptible to overfitting. In this case, the algorithm found a solution which would work - however, only in the specific environment of the tests and not in the general set of environments that the researches expected the solutions to work.
Has anyone done an actual analysis of how the evolved circuit worked? The suggestion that the unconnected cells interacted through "magnetic flux" seems very hand-wavy. I'd be much more included to blame capacitive coupling between signals, noise spikes from switching, voltage fluctuations, leakage currents or even thermal effects.
We still don't fully understand all of the various quantum mechanical hacks that biology uses: https://en.wikipedia.org/wiki/Quantum_biology
You are right. This is incredible. Ignore the naysayers saying that this is just extreme over fitting. This is more than that. Literally it is using rules outside the rules of how to program an fpga to find a solution.
Additionally it is the one algorithm that has produced the human brain and the one algorithm that has produced biological nanomachines that move and think. No other algorithm has done such a feat. It is naive to dismiss such an algorithm as "overfitting" when the incredible results are all around you.