I recently read an article by Yale’s F. Perry Wilson, MD, MSCE about medicine’s Cassandra problem with artificial intelligence (AI). I was intrigued even before I read the first sentence because … who was Cassandra again?
For those of you who have forgotten your essential Greek mythology, allow me to be of assistance. “[Cassandra] caught the eye of the god Apollo, who was accustomed to getting what he wanted. He was amazed and displeased when she refused his romantic advances, and he became vengeful. He cursed Cassandra with a gift of prophecy with an especially cruel twist: he guaranteed that while she would always be right, no one would ever believe her predictions. Cassandra foretold the fall of Troy and other disastrous happenings, though she was ignored.”
Dr. Perry sees a connection between Cassandra’s problem and AI. He references a recent article from scientists at Vanderbilt University Medical Center who sought to assess if “an automated prognostic model embedded in the electronic medical record [could] help prevent hospital-acquired venous thromboembolism (HA-VTE) among hospitalized children and adolescents.” The answer is: yes. And no. Let me explain.
The researchers leveraged an algorithm that used discrete information available in the electronic health record (EHR) to identify patients at elevated risk of developing a clinically -significant blood clot. This algorithm had previously been shown to be more accurate than physicians’ own judgment regarding which patients are at higher risk and should be anti-coagulated. When the tool reached a given level of certainty, a hematologist not involved in the patient’s care reviewed the chart to ensure the patient was eligible for prophylaxis (preventative treatment). If there were no contraindications, the hematologist spoke with the primary clinical team caring for that patient and recommended interventions to prevent clots. The hematologist then documented the suggestion in the EHR.
I need to call out how unusual this is. Typically, we want to automate these sorts of alerts as part of routine clinical decision support. Hence, I would expect that the physicians caring for these patients would simply see a pop-up on their screen at (hopefully) appropriate parts of the workflow to inform them of their patient’s increased risk with recommendations for prevention. Research has shown that doctors often ignore these advisories due to alert fatigue. To avoid any confusing findings, the authors eschewed an alert and picked up the phone or just met with the primary clinicians on the hospital floors. In my experience (and confirmed by a recent JAMA Network Open article), a physician-to-physician conversation has become much less common since the proliferation of EHRs. Therefore, the findings from the Vanderbilt team were even more surprising to me than one might expect.
What are these surprising findings to which I refer? Recommendations to initiate thromboprophylaxis – made in person or over the phone by a specialist physician – were followed only 25.8% of the time. Wow! Even though the algorithm was shown to be more accurate than physicians’ own judgment, and even though the algorithm’s recommendations were reviewed by a hematologist who personally communicated them, the primary clinical team only followed the advice a quarter of the time. Cassandra indeed!
Why did the physicians caring for these patients not accept the advice? The researchers note that “[m]ultiple clinical teams cited their concern that initiating pharmacologic prophylaxis would increase their patient’s risk of bleeding; however, prior literature and the safety work performed during this trial demonstrate that it is unlikely that appropriately recommended prophylactic anticoagulation will increase an individual’s risk for significant bleeding [emphasis added]. Further work is planned to identify potential barriers as well as to optimize the implementation of incorporating the model into clinical practice, with the hope of improving acceptance of the hematology recommendations during future trials.”
As Dr. Perry writes: “A prediction is useless if it is wrong … But it's also useless if you don't tell anyone about it. It's useless if you tell someone but they can't do anything about it. And it's useless if they could do something about it but choose not to.” I couldn’t agree more!
As we collect more real-world evidence and make increasingly accurate predictions about our patients’ needs and outcomes, I fear we will continue to run into this Cassandra problem. Informaticians and other clinical leaders must continue to keep their eyes on the prize and insist that effective clinical decision support tools be judged by outcomes, not simply accuracy.