Reliability evaluation of reinforcement learning methods for mechanical systems with increasing complexity
-
Published:2023-12-22
Issue:
Volume:
Page:
-
ISSN:1384-5640
-
Container-title:Multibody System Dynamics
-
language:en
-
Short-container-title:Multibody Syst Dyn
Author:
Manzl Peter,Rogov Oleg,Gerstmayr Johannes,Mikkola Aki,Orzechowski Grzegorz
Abstract
AbstractReinforcement learning (RL) is one of the emerging fields of artificial intelligence (AI) intended for designing agents that take actions in the physical environment. RL has many vital applications, including robotics and autonomous vehicles. The key characteristic of RL is its ability to learn from experience without requiring direct programming or supervision. To learn, an agent interacts with an environment by acting and observing the resulting states and rewards. In most practical applications, an environment is implemented as a virtual system due to cost, time, and safety concerns. Simultaneously, multibody system dynamics (MSD) is a framework for efficiently and systematically developing virtual systems of arbitrary complexity. MSD is commonly used to create virtual models of robots, vehicles, machinery, and humans. The features of RL and MSD make them perfect companions in building sophisticated, automated, and autonomous mechatronic systems. The research demonstrates the use of RL in controlling multibody systems. While AI methods are used to solve some of the most challenging tasks in engineering, their proper understanding and implementation are demanding. Therefore, we introduce and detail three commonly used RL algorithms to control the inverted N-pendulum on the cart. Single-, double-, and triple-pendulum configurations are investigated, showing the capability of RL methods to handle increasingly complex dynamical systems. We show 2D state space zones where the agent succeeds or fails the stabilization. Despite passing randomized tests during training, blind spots may occur where the agent’s policy fails. Results confirm that RL is a versatile, although complex, control engineering approach.
Funder
Business Finland University of Innsbruck and Medical University of Innsbruck
Publisher
Springer Science and Business Media LLC
Subject
Control and Optimization,Computer Science Applications,Mechanical Engineering,Aerospace Engineering,Modeling and Simulation
Reference51 articles.
1. Shabana, A.A.: Dynamics of Multibody Systems, 4th edn. Cambridge University Press, New York (2013) 2. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. A Bradford Book, Cambridge (2018) 3. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing Atari with deep reinforcement learning. CoRR (2013). arXiv:1312.5602 4. Hashemi, A., Orzechowski, G., Mikkola, A., McPhee, J.: Multibody dynamics and control using machine learning. Multibody Syst. Dyn. (2023). https://doi.org/10.1007/s11044-023-09884-x 5. Benatti, S., Young, A., Elmquist, A., Taves, J., Serban, R., Mangoni, D., Tasora, A., Negrut, D.: PyChrono and gym-chrono: a deep reinforcement learning framework leveraging multibody dynamics to control autonomous vehicles and robots. In: Advances in Nonlinear Dynamics, pp. 573–584 (2022). https://doi.org/10.1007/978-3-030-81166-2_50
|
|