Neuroevolution: evolving a neural network
Neuroevolution is the use of evolutionary search to produce neural networks: their weights, and sometimes their structure. Instead of computing how each weight should change, you keep a population of networks, let the ones that do better leave more varied copies, and repeat.
How it works, and how it differs from training
Standard network training uses backpropagation: run an input through, compare the output with the answer you wanted, and push every weight a little in the direction that would have reduced the error. It is efficient, and it requires two things: a differentiable path from weights to error, and a known correct answer to compare against.
Neuroevolution requires neither. It needs only a way to say that one network did better than another. That makes it applicable where backpropagation is awkward: control problems with no labelled answers, rewards that arrive long after the decisions that earned them, and networks with components that are not differentiable at all.
The cost is sample efficiency. Gradient methods extract a lot of information from each example. Evolutionary search extracts one number from an entire lifetime of behaviour, so it usually needs far more evaluations. It parallelises very well, which partly compensates.
Some methods evolve only the weights of a network whose shape is fixed in advance. Others, notably NEAT (NeuroEvolution of Augmenting Topologies, Stanley and Miikkulainen, 2002), evolve the structure too, starting minimal and adding nodes and connections over time. Evolving structure raises the hard problem of comparing two networks that are not built the same way, which NEAT addressed with historical markings and by protecting new structures in their own niche long enough to be tuned rather than immediately outcompeted.
Where it is used
Neuroevolution has a long record in game playing and in robot control, and it reappears whenever the reward signal is sparse or deceptive. It is also common in artificial life, for a reason that is more principled than convenience: an evolutionary process is the thing being studied, so using an evolutionary process to produce the controller keeps the mechanism and the subject the same.
Neuroevolution in Emergent Life
Every organism carries its own network, and the weights are inherited from its parent with mutation. There is no shared model, no training set, no gradient and no reward signal supplied from outside. A network that leads its owner to run out of energy before reproducing is not corrected; it simply is not copied.
The architecture is a leaky recurrent network, fixed in shape across all organisms and documented with its real dimensions in the evolving neural controller. Structure is not evolved here, which is a deliberate limitation: a fixed shape makes two organisms directly comparable, and comparability is what most of this project's measurements depend on.
Sitting alongside the inherited weights there is a separate within-life mechanism that can adjust a network during an individual's own lifetime, and is discarded at death rather than inherited. The two are kept strictly apart, which is the subject of evolution versus learning.
References and further reading
Primary sources for the claims on this page. Where this project's own results are referred to, they link to the page carrying the evidence instead.
- Stanley, K. O., & Miikkulainen, R. (2002). Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation, 10(2), 99-127. doi:10.1162/106365602320169811
- Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. (2019). Designing neural networks through neuroevolution. Nature Machine Intelligence, 1, 24-35. doi:10.1038/s42256-018-0006-z
- Floreano, D., Durr, P., & Mattiussi, C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1, 47-62. doi:10.1007/s12065-007-0002-4