Affiliation:
1. Thomas Lord Department of Mechanical Engineering and Materials Science Duke University Durham North Carolina USA
Abstract
AbstractImitation learning enables synthesis of controllers for systems with complex objectives and uncertain plant models. However, ensuring an imitation learned controller is stable requires copious amounts of data and/or a known plant model. In this paper, we explore an input–output (IO) stability approach to imitation learning, which achieves stability with sparse data sets while only requiring coarse knowledge of the energy characteristics of the plant. A constrained optimization problem is developed, in which the controller learns to mimic expert data while maintaining stabilizing energy characteristics induced by the plant. While the learning objective is nonconvex, iterative convex overbounding (ICO) and projected gradient descent (PGD) are explored as methods to learn the controller. In numerical examples, it is shown that with little knowledge of the plant model and a small data set, the dissipativity constrained learned controller achieves closed loop stability and successfully mimics the behavior of the expert controller, while other methods often fail to maintain stability and achieve good performance.
Funder
NSF
ONR
Alfred P. Sloan Foundation
Reference47 articles.
1. End‐to‐end driving via conditional imitation learning;Codevilla F;IEEE Int Conf Robot Automat,2018
2. Lyapunov density models: Constraining distribution shift in learning‐based control;Kang K;Int Conf Mach Learn,2022
3. Imitation Learning With Stability and Safety Guarantees
4. Fitting a linear control policy to demonstrations with a Kalman constraint;Palan M;Proc 2nd Conf Learn Dyn Control (L4DC),2020