Abstract
AbstractWe investigate the optimisation capabilities of an algorithm inspired by the Evolutionary Transitions in Individuality. In these transitions, the natural evolutionary process is repeatedly rescaled through successive levels of biological organisation. Each transition creates new higher-level evolutionary units that combine multiple units from the level below. We call the algorithm Deep Optimisation (DO) to recognise both its use of deep learning methods and the multi-level rescaling of biological evolutionary processes. The evolutionary model used in DO is a simple hill-climber, but, as higher-level representations are learned, the hill-climbing process is repeatedly rescaled to operate in successively higher-level representations. The transition process is based on a deep learning neural network (NN), specifically a deep auto-encoder. Our experiments with DO start with a study using the NP-hard problem, multiple knapsack (MKP). Comparing with state-of-the-art model-building optimisation algorithms (MBOAs), we show that DO finds better solutions to MKP instances and does so without using a problem-specific repair operator. A second, much more in-depth investigation uses a class of configurable problems to understand more precisely the distinct problem characteristics that DO can solve that other MBOAs cannot. Specifically, we observe a polynomial vs exponential scaling distinction where DO is the only algorithm to show polynomial scaling for all problems. We also demonstrate that some problem characteristics need a deep network in DO. In sum, our findings suggest that the use of deep learning principles have significant untapped potential in combinatorial optimisation. Moreover, we argue that natural evolution could be implementing something like DO, and the evolutionary transitions in individuality are the observable result.
Funder
engineering and physical sciences research council
Publisher
Springer Science and Business Media LLC
Reference59 articles.
1. Aickelin U, Burke EK, Li J. An estimation of distribution algorithm with intelligent local search for rule-based nurse rostering. J Oper Res Soc. 2007;58(12):1574–85.
2. Bello I, Pham H, Le QV, Norouzi M, Bengio S. Neural combinatorial optimization with reinforcement learning. 2016. arXiv:1611.09940
3. Bosman PA, Thierens D. Linkage information processing in distribution estimation algorithms, vol. 1999. Information and Computing Sciences: Utrecht University; 1999.
4. Boyan J, Moore AW. Learning evaluation functions to improve optimization by local search. J Mach Learn Res. 2000;1(Nov):77–112.
5. Caldwell J, Knowles J, Thies C, Kubacki F, Watson R. Deep optimisation: multi-scale evolution by inducing and searching in deep representations. In: Castillo PA, Jiménez Laredo JL, editors. Applications of evolutionary computation. Cham: Springer International Publishing; 2021. p. 506–21.