Learning About the Effects of Alert Uncertainty in Attack and Defend Decisions via Cognitive Modeling

Abstract

Objective We aim to learn about the cognitive mechanisms governing the decisions of attackers and defenders in cybersecurity involving intrusion detection systems (IDSs). Background Prior research has experimentally studied the role of the presence and accuracy of IDS alerts on attacker’s and defender’s decisions using a game-theoretic approach. However, little is known about the cognitive mechanisms that govern these decisions. Method To investigate the cognitive mechanisms governing the attacker’s and defender’s decisions in the presence of IDSs of different accuracies, instance-based learning (IBL) models were developed. One model (NIDS) disregarded the IDS alerts and one model (IDS) considered them in the instance structure. Both the IDS and NIDS models were trained in an existing dataset where IDSs were either absent or present and they possessed different accuracies. The calibrated IDS model was tested in a newly collected test dataset where IDSs were present 50% of the time and they possessed different accuracies. Results Both the IDS and NIDS models were able to account for human decisions in the training dataset, where IDS was absent or present and it possessed different accuracies. However, the IDS model could accurately predict the decision-making in only one of the several IDS accuracy conditions in the test dataset. Conclusions Cognitive models like IBL may provide some insights regarding the cognitive mechanisms governing the decisions of attackers and defenders in conditions not involving IDSs or IDSs of different accuracies. Application IBL models may be helpful for penetration testing exercises in scenarios involving IDSs of different accuracies.