Intelligent agents are showing increasing promise for clinical decision-making in a variety of healthcare settings. While a substantial body of work has contributed to the best strategies to convey these agents’ decisions to clinicians, few have considered the impact of personalizing and customizing these communications on the clinicians’ performance and receptiveness. This raises the question of how intelligent agents should adapt their tone in accordance with their target audience. We designed two approaches to communicate the decisions of an intelligent agent for breast cancer diagnosis with different tones: a suggestive (non-assertive) tone and an imposing (assertive) one. We used an intelligent agent to inform about: (1) number of detected findings; (2) cancer severity on each breast and per medical imaging modality; (3) visual scale representing severity estimates; (4) the sensitivity and specificity of the agent; and (5) clinical arguments of the patient, such as pathological co-variables. Our results demonstrate that assertiveness plays an important role in how this communication is perceived and its benefits. We show that personalizing assertiveness according to the professional experience of each clinician can reduce medical errors and increase satisfaction, bringing a novel perspective to the design of adaptive communication between intelligent agents and clinicians.
https://doi.org/10.1145/3544548.3580682
Artificial intelligence (AI) supported clinical decision support (CDS) technologies can parse vast quantities of patient data into meaningful insights for healthcare providers. Much work is underway to determine the technical feasibility and the accuracy of AI-driven insights. Much less is known about what insights are considered useful and actionable by healthcare providers, their trust in the insights, and clinical workflow integration challenges. Our research team used a conceptual prototype based on AI-generated treatment insights for type 2 diabetes medications to elicit feedback from 41 U.S.-based clinicians, including primary care and internal medicine physicians, endocrinologists, nurse practitioners, physician assistants, and pharmacists. We contribute to the human-computer interaction (HCI) community by describing decision optimization and design objective tensions between population-level and personalized insights, and patterns of use and trust of AI systems. We also contribute a set of 6 design principles for AI-supported CDS.
https://doi.org/10.1145/3544548.3581251
Recent large language models (LLMs) have advanced the quality of open-ended conversations with chatbots. Although LLM-driven chatbots have the potential to support public health interventions by monitoring populations at scale through empathetic interactions, their use in real-world settings is underexplored. We thus examine the case of CareCall, an open-domain chatbot that aims to support socially isolated individuals via check-up phone calls and monitoring by teleoperators. Through focus group observations and interviews with 34 people from three stakeholder groups, including the users, the teleoperators, and the developers, we found CareCall offered a holistic understanding of each individual while offloading the public health workload and helped mitigate loneliness and emotional burdens. However, our findings highlight that traits of LLM-driven chatbots led to challenges in supporting public and personal health needs. We discuss considerations of designing and deploying LLM-driven chatbots for public health intervention, including tensions among stakeholders around system expectations.
https://doi.org/10.1145/3544548.3581503
Clinical needs and technological advances have resulted in increased use of Artificial Intelligence (AI) in clinical decision support. However, such support can introduce new and amplify existing cognitive biases. Through contextual inquiry and interviews, we set out to understand the use of an existing AI support system by ophthalmologists. We identified concerns regarding anchoring bias and a misunderstanding of the AI's capabilities. Following, we evaluated clinicians' perceptions of three bias mitigation strategies as integrated into their existing decision support system. While clinicians recognised the danger of anchoring bias, we identified a concern around the impact of bias mitigation on procedure time. Our participants were divided in their expectations of any positive impact on diagnostic accuracy, stemming from varying reliance on the decision support. Our results provide insights into the challenges of integrating bias mitigation into AI decision support.
https://doi.org/10.1145/3544548.3581513
Significant and rapid advancements in cancer research have been attributed to Artificial Intelligence (AI). However, AI's role and impact on the clinical side has been limited. This discrepancy manifests due to the overlooked, yet profound, differences in the clinical and research practices in oncology. Our contribution seeks to scrutinize physicians' engagement with AI by interviewing 7 medical-imaging experts and disentangle its future alignment across the clinical and research workflows, diverging from the existing "one-size-fits-all" paradigm within Human-Centered AI discourses. Our analysis revealed that physicians' trust in AI is less dependent on their general acceptance of AI, but more on their contestable experiences with AI. Contestability, in clinical workflows, underpins the need for personal supervision of AI outcomes and processes, i.e., clinician-in-the-loop. Finally, we discuss tensions in the desired attributes of AI, such as explainability and control, contextualizing them within the divergent intentionality and scope of clinical and research workflows.
https://doi.org/10.1145/3544548.3581506
Clinical decision support tools (DSTs), powered by Artificial Intelligence (AI), promise to improve clinicians' diagnostic and treatment decision-making. However, no AI model is always correct. DSTs must enable clinicians to validate each AI suggestion, convincing them to take the correct suggestions while rejecting its errors. While prior work often tried to do so by explaining AI's inner workings or performance, we chose a different approach: We investigated how clinicians validated each other's suggestions in practice (often by referencing scientific literature) and designed a new DST that embraces these naturalistic interactions. This design uses GPT-3 to draw literature evidence that shows the AI suggestions' robustness and applicability (or the lack thereof). A prototyping study with clinicians from three disease areas proved this approach promising. Clinicians' interactions with the prototype also revealed new design and research opportunities around (1) harnessing the complementary strengths of literature-based and predictive decision supports; (2) mitigating risks of de-skilling clinicians; and (3) offering low-data decision support with literature.