The Effect of Incorrect Reliability Information on Expectations, Perceptions, and Use of Automation

Abstract

Objective: We examined how providing artificially high or low statements about automation reliability affected expectations, perceptions, and use of automation over time. Background: One common method of introducing automation is providing explicit statements about the automation’s capabilities. Research is needed to understand how expectations from such introductions affect perceptions and use of automation. Method: Explicit-statement introductions were manipulated to set higher-than (90%), same-as (75%), or lower-than (60%) levels of expectations in a dual-task scenario with 75% reliable automation. Two experiments were conducted to assess expectations, perceptions, compliance, reliance, and task performance over (a) 2 days and (b) 4 days. Results: The baseline assessments showed initial expectations of automation reliability matched introduced levels of expectation. For the duration of each experiment, the lower-than groups’ perceptions were lower than the actual automation reliability. However, the higher-than groups’ perceptions were no different from actual automation reliability after Day 1 in either study. There were few differences between groups for automation use, which generally stayed the same or increased with experience using the system. Conclusion: Introductory statements describing artificially low automation reliability have a long-lasting impact on perceptions about automation performance. Statements including incorrect automation reliability do not appear to affect use of automation. Application: Introductions should be designed according to desired outcomes for expectations, perceptions, and use of the automation. Low expectations have long-lasting effects.