Bien et al. (2018) used deep learning, a subset of machine learning, to model the complex relationships between images and their interpretations. 6 The model was designed to detect general abnormalities and two specific diagnoses (anterior cruciate ligament tears and meniscal tears) on knee magnetic resonance imaging. For general abnormalities, there was no difference between the performance of the model and the general radiologists. For ACL tear detection, the model was highly specific but not significantly different from the specificity achieved by the radiologists. The authors also found that providing the radiologists with the predictions from the model improved their quality of interpretation of the MRI studies. Li et al. (2018) developed an AI tool to detect nasopharyngeal malignancies under endoscopic evaluation by oncologists. 7 Results indicate that the tool was significantly better in its performance compared with oncological experts; the overall accuracy was 88% vs. 80.5%. ECG Interpretation In the evaluation of cardiac health, 12-lead ECGs are accompanied by computer interpretations to assist the clinician with diagnoses. These interpretations have been shown to often be inaccurate, primarily because of noisy background signals that interfere with automated pattern recognition by the machine algorithms. However, four studies evaluated ECG interpretations by automated systems, and all found that the systems were no better or worse than human performance alone. Use in Pathology Two studies evaluated the use of AI to aid in the diagnostic work of pathologists. Vandenberghe et al. (2017) developed and evaluated the use of deep learning, an AI method, to identify specific cancer cell types. 8 For 71 breast tumor samples, they found that the use of this computer-aided diagnosis tool had a concordance rate of 83% with pathologist review. The pathologist re-reviewed the 12 samples that had discordance between the diagnoses of the pathologist and the computer- aided diagnosis tool, prompting modifications to 8 of the original diagnoses. Xiong et al. (2018), also using deep learning, developed and tested an AI-assisted method for the automatic detection of mycobacterium tuberculosis. 9 Results showed high sensitivity (97.9%) and moderate specificity (83.6%), with 2 false negatives and 17 false positive cases due to contaminants. Potential Benefits and Barriers In general, CDS tools have an added benefit of improving access to specialized care by providing theclinician with assistance in diagnosing conditions that would typically fall in the realm of a specialist. Several CDS tools, in addition to improving diagnostic accuracy, would also allow prioritization of work, creating greater efficiencies and improving workflow once implemented in clinical settings.
These systems flagged studies or diagnoses that required follow-up, allowing the clinicians to prioritize their work. For the CDS tools that generate DDX, some have raised the concern that presenting the clinician with a long list of diagnostic possibilities could be distracting or lead to unnecessary testing and procedures. The information generated by CDS for use in diagnosis is only as good as the information that is put into the system. For example, if the clinician interprets the physical exam incorrectly (e.g., saying that a physical sign is absent when it is present) and inputs that incorrect information into the tool, that error may negatively affect any diagnosis that is partially based on the presence of that sign. Accurate diagnosis can be achieved only if the clinician’s assessment of the patients’ signs and symptoms is correct, because the automated system will process only data that humans introduce. In the case of ECG interpretation, accurate ECG recording depends on many variables, including lead placement, weight, movement, coexisting electrolyte abnormalities, and symptoms. If the placement is wrong (e.g., leads are placed in wrong location), the interpretation may be wrong. Leveraging the “CDS Five Rights” Approach A useful framework for achieving success in CDS design, development, and implementation is the “CDS Five Rights” approach. 10 This model states that CDS-supported improvements in desired healthcare outcomes can be achieved if clinicians communicate: 1. The right information: evidence-based, suitable to guide action, pertinent to the circumstance 2. To the right person: considering all members of the care team, including clinicians, patients, and their caretakers 3. In the right CDS intervention format, such as an alert, order set, or reference information to answer a clinical question 4. Through the right channel: for example, a clinical information system such as the EHR, a personal health record, or a more general channel such as the Internet or a mobile device 5. At the right time in workflow, for example, at the time of decision/action/need. CDS has not reached its full potential in driving care transformation, in part because opportunities to optimize each of the five rights have not been fully explored and cultivated. Providing the Right Information to the End User: The process of integrating real-time analytics into clinical workflow represents a shift towards more agile and collaborative infrastructure building, expected to be a key feature of future health information technology strategies. As interoperability and big data analytics capabilities become increasingly central to crafting the healthcare information systems of the future, the need to address issues that ease the flow of health information and communication becomes even more important.
Without tools that select, aggregate, and visualize relevant information among the vast display of information competing for visual processing, clinicians must rely on cues by “hunting and gathering” in the EHR. Alerts that embody “right information” should provide just enough data to drive end user action, but not so much as to cause overload. Overload can create alert fatigue and lead to desensitization to the alerts, resulting in the failure to respond to warnings, both important and less important. Experience from the use of CDS in the medication ordering process has demonstrated this paradoxical increase in risk of harm due to alerts that were intended to improve safety. Providing Information in the Right Format: Lack of knowledge regarding how to present CDS to providers has impeded alert optimization, specifically the most effective ways to differentiate alerts, highlighting important pieces of information without adding noise, to create a universal standard. Thepotential solution that CDS represents is limited by problems associated with improper design, implementation, and local customization. In the absence of evidence-based guidelines specific to EHR alerting, effective alert design can be informed by several guidelines for design, implementation, and reengineering that help providers take the correct action at the correct time in response to recognitionof the patient’s condition. Right Workflow: A well-thought-out user- centered design or equivalent process during the implementation phase includes critical elements of leadership buy-in, dissemination plans, and outcome measurements. Knowledge needs to be gained about how to implement the CDS and how to create an interface between the system and the clinician that takes into consideration the cognitive and clinical workflow. The optimal approach to CDS should not be focused primarily—or even secondarily—on technology. Implementation is about people, processes, and technology. Systems engineering approaches, including consideration of user experience and improvements in user interface, can greatly improve the ability of CDS tools to reach their potential to improve quality of care and patient outcomes. The application of human factors engineering in determining the right workflow includes but is not limited to ethnographic research including workflow analysis and usability testing. Measurement Successful CDS deployment requires evaluating not only whether the intended clinicians are using the tool at the point of care, but also whether CDS use translates into improvements in clinical outcomes, workflows, and provider and patient satisfaction. However, success measures are often not clearly enunciated at the outset when developing or implementing CDS tools. As a result, it is often difficult to quantify the extent to which CDS has been effectively deployed, as well as whether it is effective at managing the original diagnostic problem it was designed to address.
76
Powered by FlippingBook