Creating meaningful work in the age of AI: explainable AI, explainability, and why it matters to organizational designers

Abstract

AbstractIn this paper, we contribute to research on enterprise artificial intelligence (AI), specifically to organizations improving the customer experiences and their internal processes through using the type of AI called machine learning (ML). Many organizations are struggling to get enough value from their AI efforts, and part of this is related to the area of explainability. The need for explainability is especially high in what is called black-box ML models, where decisions are made without anyone understanding how an AI reached a particular decision. This opaqueness creates a user need for explanations. Therefore, researchers and designers create different versions of so-called eXplainable AI (XAI). However, the demands for XAI can reduce the accuracy of the predictions the AI makes, which can reduce the perceived usefulness of the AI solution, which, in turn, reduces the interest in designing the organizational task structure to benefit from the AI solution. Therefore, it is important to ensure that the need for XAI is as low as possible. In this paper, we demonstrate how to achieve this by optimizing the task structure according to sociotechnical systems design principles. Our theoretical contribution is to the underexplored field of the intersection of AI design and organizational design. We find that explainability goals can be divided into two groups, pattern goals and experience goals, and that this division is helpful when defining the design process and the task structure that the AI solution will be used in. Our practical contribution is for AI designers who include organizational designers in their teams, and for organizational designers who answer that challenge.