Over the past decade, we have witnessed unprecedented interest in patient safety. Reporting medical incidents is one of the leading initiatives to enhance patient safety.1 It was estimated in 1999 that 44,000 to 98,000 people died each year because of mistakes in U.S. hospitals. A recent study indicated a much higher figure – 210,000 to 440,000 deaths each year in the U.S. due to preventable medical errors.2-4
“More than 1,000 preventable deaths a day is too many.”5 There has been a pressing need to improve patient safety. Several techniques have been explored to collect, archive and analyze medical errors, for example: chart review, administrative data analysis and malpractice claims analysis.6-8 Traditional error analysis methods, such as paper-based chart review, are too labor-intensive and time-consuming for systematic and large-scale research. Utilitzing administrative or claims data is relatively low-cost, but it largely relies on inaccurate and inconsistent coding data across and within systems, and may generate high proportion of false alarms.9 Auto-extraction from electronic medical data is an efficient approach, however, the types of errors that can be detected have been limited to nosocomial infections and adverse drug events thus far.6-8
Based on the successful safety improvements in aviation and nuclear plant industries,10 web-based voluntary medical incident reporting systems, i.e. e-reporting systems, are considered as an effective mechanism for learning from and preventing errors.7 Such systems could offer a source of adverse event information, a reminder of hazards, and a means of monitoring potential problems as they recur. The data stored in the systems could be shared and used for comparisons within or between institutions. Ultimately, the systems would help researchers seek common solutions and translate reporting data into actionable knowledge.
Unfortunately, the current e-reporting systems in healthcare are mostly used as data repository due to the lack of structured data, uncertainty, ambiguity and incompleteness.1 The quality of voluntary reports is just as significant as the number of submissions.11 Current e-reporting systems show the issues of underreporting and low-quality reporting.11 It was estimated that 50% to 96% of adverse events failed to be reported each year.12 A significant percentage of submitted reports were incomplete or inaccurate, and thus cannot be thoroughly analyzed to understand the causes of medical errors.13,14 Underreporting and low-quality reporting of medical incidents can be attributed to organization and technology barriers.
Barriers to Reporting
The culture of blame and resistance to sharing have been identified as barriers to medical incident reporting at the organization level.10 Likewise, the management policy on mandatory and non-confidential reporting of medical incidents in fact discourages front-line clinicians from reporting internally.15 Last, but not least, the psychological stress that healthcare staff experience while discussing mistakes with their supervising managers, such as fear of embarrassment and loss of reputation or job,10 should not be ignored.
At the technology level, current medical incident reporting systems were not built on the basis of a consensus on the conceptual framework. Features that analyze medical errors collectively and facilitate learning have not been explored in current systems. One challenge of implementing such features is the inconsistency of data structure due to the difference in the conceptual framework, especially for home grown systems.16
In some cases, underreporting can occur just as a result of reporters unable to identify a proper classification or definition.17 Selecting “other” or “miscellaneous” as an incident category is a common problem for computerized analysis.17,18 Classification and definition used in reporting systems play a key role in assuring the quality of reports and may even determine whether an event is recognized or ignored.19-20 Several medical incident taxonomies or conceptual frameworks are available for the development of medical reporting system. Unfortunately, in practice, so many taxonomies that lack of consistency may impede the interoperability among different medical incident systems at a larger scope. The Common Formats (CFs) for medical incident reporting are recommended by AHRQ as the national effort towards an aggregated pool of medical errors.21,22
Furthermore, existing e-reporting systems are mainly template-based which is a combination of open-ended and structured questions. The template is aimed at maximizing the consistency and minimizing the variation of the level of details. Inevitably, it may have the unintended effect of homogenizing incident descriptions with a loss of detail.23 As a result, most e-reporting systems cannot synthesize incident data to generate actionable knowledge.16,24,25 In our systematic review of academic publications and publicly accessible webpages, none of the 53 medical incident reporting systems have any features that facilitate learning from mistakes or provide actionable feedback to reporters.26 Despite a large amount of studies that suggest instituting a “just culture” that encourages learning and non-punishment, few studies have investigated system difficulty and inefficiency regarding ease of use, ease of understanding and their relations with the level of details in reporting27-30 and it is rare to find research investigating data-driven learning features in medical incident reporting.25
In order to achieve the goal of preventing and reducing medical errors, medical incident reporting systems should be secure, easy to use and effective31 – that is, confidential or anonymous, with excellent user acceptance, and used in a meaningful way. Being able to facilitate learning from past mistakes is critical to such systems to eventually decrease recurring incidents.
Building a secure, easy to use and effective reporting system is not just a technical challenge, but also requires making a cultural change in hospitals to promote open discussion of errors and learning from failures.32 Earlier studies indicate that understanding barriers and incentives to reporting is essential for transforming the culture of blame and resistance to sharing and learning and increasing safety.10
Clinicians often feel that current reporting systems are difficult to use, especially considering their time-critical and multi-tasking work conditions.33,34 Discussion with senior managers about past mistakes and problems may be conducted in neither a timely manner due to heavy workload or schedule conflicts, nor a comfortable means due to a face-to-face consultation or confrontation. User-centered design (UCD) is an approach to information system design with user engagement at the early stage in order to make the system intuitive and easy to use. This proves to be successful in ensuring user acceptance in many areas.35,36 Anonymous or confidential reporting through computers can alleviate the psychological stress involved in discussions about mistakes. To fit into the time-critical and highly interruptive work environment in clinical settings, reporting systems should be designed with the goal of minimizing clinicians’ input effort. Therefore, a user-centered voluntary reporting system that emphasizes learning from errors and improving systems of care can help transform the culture by taking the organizational barriers into design consideration and thus serves as the foundation of an informed and safe culture.37
Currently there is a lack of UCD framework for incident reporting systems to effectively collect, catalog and analyze the reports. Specifically, the learning features enabled by structured and interoperable data are rarely reported, which in turn may reduce the usefulness of such systems.
The rationale behind error reporting is that with knowledge comes the power to detect problems, identify causes, and to effect changes.1 Features that facilitate learning from errors can effectively contribute to error prevention7 by educating front-line clinicians about similar incidents, providing actionable knowledge at the point of care, and increasing reporter-perceived usefulness of quality reporting. A wide array of healthcare professionals from hospitals (60%) and academic institutions (16%) expressed interests in learning from incidents beyond their own organization.38
The fundamental challenge of sharing, learning and maximizing the utility of incident data from various organizations is the consistency of the data structure across reporting systems. The recently released Common Formats have defined the terms based on the exact terms or conceptually similar terms used in other patient safety reporting systems, with a preference given to the WHO ICPS.22 The unified definitions theoretically enable the interoperability among different reporting systems at different levels. The forms serve as a framework for implementing interoperable online reporting systems. However, they do not support the computational transition as does an ontology.
In summary, to design, develop and utilize e-reporting systems, our goals are effectively gathering information from previous lessons and timely informing the subsequent action.39
Yang Gong is an associate professor at the School of Biomedical Informatics, University of Texas Health Science Center at Houston. He received his medical training in China and his PhD of health informatics from the University of Texas Health Science Center at Houston. He has a disciplinary background and core interest in human factors, human-centered computing, patient safety information system, clinical communication and clinical decision support. He has published or presented at national/international conferences, including AMIA, Medinfo, AHIMA, HIMSSasia, HIMSS, HCI International etc.
See Page Two for References
1. Holden RJ, Karsh B-T. A Review of Medical Error Reporting System Design Considerations and a Proposed Cross-Level Systems Research Framework. Human Factors: The Journal of the Human Factors and Ergonomics Society. April 1, 2007 2007;49(2):257-276.
18. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: Development of a computer-based monitor and comparison with chart review and stimulated voluntary report. Journal of American Medical Informatics Association. May 1 1998;5(3):305-314.
21. QuIC. Report to the President on medical errors. Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. Vol 2006: Quality Interagency Coordination Task Force; 2000: http://www.quic.gov/report/errors6.pdf.
32. Wu JH, Shen WS, Lin LM, Greenes RA, Bates DW. Testing the technology acceptance model for evaluating healthcare professionals’ intention to use an adverse event reporting system. International Journal for Quality in Health Care. Apr 2008;20(2):123-129.
33. Holden RJ, Karsh B. Applying a theoretical framework to the research and design of medical error reporting systems. In Proceedings of the International Conference on Healthcare Systems Ergonomics and Patient Safety 2005.
34. Holden RJ, Scanlon MC, Patel NR, et al. A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life. BMJ Qual Saf. Jan 2011;20(1):15-24.
35. Holden RJ, Brown RL, Scanlon MC, Karsh BT. Pharmacy workers’ perceptions and acceptance of bar-coded medication technology in a pediatric hospital. Research in social & administrative pharmacy : RSAP. Nov-Dec 2012;8(6):509-522.
38. Pace WD, Fernald DH, Harris DM, et al. Developing a taxonomy for coding ambulatory medical errors: A report from the asips collaborative. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation (Volume 2: Concepts and Methodology). Rockville (MD)2005.