A swarm optimizer with attention-based particle sampling and learning for large scale optimization

Abstract

AbstractAttention mechanism, which is a cognitive process of selectively concentrating on certain information while ignoring others, has been successfully employed in deep learning. In this paper, we introduce the attention mechanism into a particle swarm optimizer and propose an attention-based particle swarm optimizer (APSO) for large scale optimization. In the proposed method, the attention mechanism is introduced such that activating different particles to participate in evolution at different stages of evolution. Further, an attention-based particle learning is devised to randomly select three particles from a predominant sub-swarm, which is activated by the attention mechanism, to guide the learning of particles. The cooperation of these two strategies could be employed to achieve a balanced evolution search, thus appropriately searching the space of large-scale optimization problems. Extensive experiments have been carried out on CEC’2010 and CEC’2013 large scale optimization benchmark functions to evaluate the performance of proposed method and to compare with related methods. The results show the superiority of proposed method.