Reinforcement Learning as a Path to Autonomous Intelligent Cyber-Defense Agents in Vehicle Platforms
-
Published:2023-10-24
Issue:21
Volume:13
Page:11621
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Raio Stephen1, Corder Kevin2, Parker Travis W.3, Shearer Gregory G.3, Edwards Joshua S.3, Thogaripally Manik R.3, Park Song J.1, Nelson Frederica F.1
Affiliation:
1. U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory, Aberdeen Proving Ground, Aberdeen, MD 21005, USA 2. Parsons, Aberdeen Proving Ground, Aberdeen, MD 21005, USA 3. ICF International, Columbia, MD 21046, USA
Abstract
Technological advancement of vehicle platforms exposes opportunities for new attack paths and vulnerabilities. Static cyber defenses can help mitigate certain attacks, but those attacks must generally be known ahead of time, and the cyber defenses must be hand-crafted by experts. This research explores reinforcement learning (RL) as a path to achieve autonomous, intelligent cyber defense of vehicle control networks—namely, the controller area network (CAN) bus. We train an RL agent for the CAN bus using Toyota’s Portable Automotive Security Testbed with Adaptability (PASTA). We then apply the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory’s methodology for quantitative measurement of cyber resilience to assess the agent’s effect on the vehicle testbed in a contested cyberspace environment. Despite all defenses having similar traditional performance measures, our RL agent averaged a 90% cyber resilience measurement during drive cycles executed on hardware versus 41% for a naïve static timing defense and 98% for the bespoke timing-based defense. Our results also show that an RL-based agent can detect and block injection attacks on a vehicle CAN bus in a laboratory environment with greater cyber resilience than prior learning approaches (1% for convolutional networks and 0% for recurrent networks). With further research, we believe there is potential for using RL in the autonomous intelligent cyber defense agent concept.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference39 articles.
1. Russell, W.W. (2023, July 31). Weapon Systems Cybersecurity: Guidance Would Help DOD Programs Better Communicate Requirements to Contractors, Available online: https://www.gao.gov/products/gao-21-179. 2. Smith, S. (2023, January 9–10). Towards a scientific definition of cyber resilience. Proceedings of the International Conference on Cyber Warfare and Security, Towson, MD, USA. 3. Theron, P., Kott, A., Drašar, M., Rzadca, K., LeBlanc, B., Pihelgas, M., Mancini, L., and Panico, A. (2018, January 22–23). Towards an active, autonomous and intelligent cyber defense of military systems: The NATO AICA reference architecture. Proceedings of the 2018 International Conference on Military Communications and Information Systems (ICMCIS), Warsaw, Poland. 4. Kott, A., Théron, P., Drašar, M., Dushku, E., LeBlanc, B., Losiewicz, P., Guarino, A., Mancini, L., Panico, A., and Pihelgas, M. (2018). Autonomous intelligent cyber-defense agent (AICA) reference architecture. Release 2.0. arXiv. 5. Kott, A., Weisman, M.J., and Vandekerckhove, J. (December, January 28). Mathematical modeling of cyber resilience. Proceedings of the MILCOM 2022-2022 IEEE Military Communications Conference (MILCOM), Rockville, MD, USA.
|
|