Intrinsic Decay Property of Ti/TiOx/Pt Memristor for Reinforcement Learning

Abstract

A memristor‐based reinforcement learning (RL) system has shown outstanding performance in achieving efficient autonomous decision‐making and edge computing. Sarsa (λ) is a classical multistep RL algorithm that records state with λ decay and guides policy updates, significantly improving the algorithm convergence speed. However, λ decay implementation of traditional computing hardware is confined by the extensive computation of power exponential decay. Herein, the value update equation for Sarsa (λ) is implemented by using the topological structure of the memristor array, without complex circuits. Where, most importantly, the critical λ decay function is realized by a TiOx‐based memristor with conductance decay property. The energy required for floating‐point operations can be significantly reduced while accelerating the convergence speed. Then, a path planning task is demonstrated based on intrinsic conductance decay property and shows outstanding performance. Finally, the information of rounds used for the task is obtained, which is based on the intrinsic decay property of the TiOx‐based memristor, maps into a 32 × 32 memristor array in parallel to calculate the value of each round. The results indicate that the experimental data have similar results to the simulations. Herein, thus, it provides a hardware‐enabled scheme for the memristor‐based RL algorithm implementation.