Solving the XAI conundrum – but how?

Explainable AI (XAI) has emerged as a promising solution for addressing the implementation challenges of AI/ML in healthcare. However, little is known about how developers and clinicians interpret XAI and what conflicting goals and requirements they may have. This talk presents the findings of a longitudinal multi-method study involving 112 developers and clinicians who co-developed XAI solutions for a clinical decision support system.

‍

Our study identifies three key differences between developer and user mental models of XAI and proposes five design solutions that could help address the XAI conundrum in healthcare, including the use of causal inferences, counterfactual queries, interactive visualizations, personalized explanations, and contextual information. The importance of considering these possibilities in the design of XAI systems as well as practical recommendations for improving the effectiveness and usability of XAI in healthcare (and other contexts) will be discussed with the audience.