By Gabrielle Mostello
A novel framework for assessing the real-world fitness of AI models has shown that the technology falls short when communicating with patients, according to a study published on Thursday in Nature Medicine.
To assess the performance of AI in settings that closely mimic patient interactions, researchers from Harvard Medical School crafted the Conversational Reasoning Assessment Framework for Testing in Medicine (CRAFT-MD) and applied it to four large language models – GPT-4, GPT-3.5, Mistral and LLaMA-2-7b. The models were evaluated on 2000 clinical scenarios involving conditions that commonly present in the primary care setting and across 12 medical specialties.
In contrast with current evaluations that judge AI models based on their answers to medical exam questions, the CRAFT-MD framework focuses on natural dialogues. The former approach “assumes that all relevant information is presented clearly and concisely, often with medical terminology or buzzwords that simplify the diagnostic process, but in the real world, this process is far messier,” said first study author Shreya Johri.
The findings showed that the AI models had trouble conducting clinical conversations and making decisions based on information provided by patients, which affected their ability to take a patient’s history and make accurate diagnoses. Diagnostic accuracy declined when AI models were presented with open-ended versus multiple-choice answers and when engaging in back-and-forth exchanges instead of summarised conversations.
Using the information gathered from their study, the research team compiled a list of recommendations for AI developers and regulators who evaluate large language models to improve these tools. These include emphasising realistic doctor-patient conversations, taking a comprehensive patient history and using open-ended questioning as well as a combination of automated and expert evaluations.
The researchers shared that they would be updating the CRAFT-MD framework as improvements are made to patient AI models.
Although adjustments should be made before these AI models are routinely used in clinical practice, senior study author Roxana Daneshjou said that “CRAFT-MD creates a framework that more closely mirrors real-world interactions and thus it helps move the field forward when it comes to testing AI model performance in healthcare.”
