Tracking Medical Error and Misdiagnosis

No one disputes that physicians and other medical professionals make mistakes. They are, after all, human beings — as fallible as any of us. Training, well-designed protocols and discipline can reduce the number of medical errors, but they will never eliminate them. Yet, how often are mistakes made, and how many people die because of them? That topic is the source of great dispute and debate, with several high-profile studies generating plenty of headlines, while divulging relatively little in the way of hard numbers.

For physicians charged with running a medical practice or hospital, trying to measure the risks posed to both patients and the fiscal solvency of the business is difficult — especially without the kind of numbers readily available for, say, healthcare-associated infections (HAIs) or falls. Indeed, practices, hospitals, clinics and other healthcare facilities have access to the statistics on HAIs and falls because both are regularly reported to and tracked by local and state health agencies and the Centers for Disease Control and Prevention (CDC). But, this is not so with deaths due to medical error and misdiagnosis. CDC does not have a mechanism in place for reporting these deaths, which makes gauging the severity of the issue little more than a series of educated guesses. It’s an actuary’s nightmare.

Tracing the Numbers

The issue of mortality from human error in the healthcare profession first came to the attention of the general public in the spring of 2016, when Johns Hopkins researchers published a meta-analysis that claimed more than a quarter million Americans die due to medical misdiagnosis or error every year.¹ The mainstream media quickly picked up on that startling claim, with headlines and newscasters proclaiming that going to the doctor was among the riskiest things one could do. In fact, the numbers suggested by that study would make medical error and misdiagnosis the third-leading cause of death in this country, behind only cancer and cardiovascular disease. If HAIs and falls in a healthcare setting were added to the Johns Hopkins claim, that would make an eye-catching number of roughly 400,000 Americans every year dying while under medical care.

Another group of Johns Hopkins researchers followed up that meta-analysis in December 2022 with a study focused specifically on diagnostic errors during emergency room visits. That study concluded that 250,000 deaths a year are attributable to medical errors in the emergency room alone.²

While the 2016 Johns Hopkins meta-analysis was the first time most laypeople had heard about medical error, it’s important to note that the authors relied on a survey of previous studies that had suggested medical error was underreported, and their numbers were extrapolated from those studies. Indeed, one of those earlier studies is still cited: a 1999 study from the Institute of Medicine (IOM) (since renamed the National Academy of Medicine) that estimated as many as 98,000 Americans die each year from preventable mistakes.³ And that study was based on data from a 1984 study of patients treated in New York hospitals and another from Colorado at about the same time.⁴

Raising Concerns About Those Numbers

The Johns Hopkins study didn’t raise nearly the same level of enthusiasm among the mainstream media about the concerns skeptics immediately raised about the 2016 Johns Hopkins meta-analysis. A few weeks after the meta-analysis was released, Medscape published an article reporting on it and then ran an analysis of the feedback it had received. Many physicians were among the 500 Medscape readers who posted criticisms about the methodology of the meta-analysis, with comments such as:⁵

• Correlation was confused with causation.
• Unavoidable complications were classified as errors.
• Gravely ill patients who were facing an imminent death regardless of treatment were included in the studies.
• Some high-risk conditions offer no easy nor universally agreed-upon treatment.

Just a few months after the publication of the meta-analysis, the same journal that had published it ran a rebuttal, pointing out that, scientifically speaking, the Johns Hopkins claims weren’t even contained in an actual study: “Though the paper by Makary and Daniel was widely cited as ‘a study,’ it presented no new data nor did it use formal methods to synthesize the data it used from previous studies. The authors simply took the arithmetic average of four estimates since the publication of the IOM report, including one from HealthGrades, a for-profit company that markets quality and safety ratings, a report from the U.S. Office of the Inspector General and two peer-reviewed articles. The paper did not apply any established methodology for quantitative synthesis, nor did it include a discussion either of the intrinsic limitations of the studies used or of the errors associated with the extrapolation process.”⁶

In the years since the 2016 meta-analysis was released, other researchers have combed through it and brought forth even more pointed criticism of both its methods and conclusions. An analysis in 2019 referred to the Johns Hopkins estimates of deaths from medical error as “outlandish.”⁷ Another analysis in 2021 by a Canadian researcher at McGill University observed, “The idea that medical error is the third-leading cause of death in the U.S. is indeed a fiction, an overestimation that has negative consequences.”⁸

Similar to the pushback to the 2016 meta-analysis, the 2022 Johns Hopkins study also has physicians questioning its methodology, as well as the scale of its claims. In the Dec. 30, 2022, edition of the Wall Street Journal, Kristen Panthagani, MD, PhD, an emergency physician at Yale New Haven Hospital, argued that the statistical analysis underlying the study was based on too small a sample size for any kind of meaningful extrapolation.⁹ It should be noted that two of the study authors responded to Dr. Panthagani’s opinion piece with a letter to the editor in the Jan. 5, 2023, edition of the Wall Street Journal, rebutting her arguments, and countering that she misrepresented their methodology.¹⁰

And going back to the beginning, the original 1999 IOM study also had its methods and conclusions questioned. The medical school at Indiana University identified several significant flaws in the study, including the fact that all the patients in the source study were classified as “very sick,” as well as the lack of a control group to provide comparison.¹¹ A month later, another analysis of the IOM study found that the estimated number of deaths seemed subjective, and that there was little supporting evidence for the claim that up to half of those deaths were preventable.¹² Most damaging, in 2005, the author of one of the studies cited by the IOM study weighed in, arguing that the IOM authors had inflated the risks in their extrapolations from his work, and had underplayed the increasing patient safety in hospitals.¹³

In Search of Hard Numbers

It goes without saying that an actual medical error has the potential to cause great human and financial harm. We’ve all heard or read anecdotal stories about patients who had the wrong leg removed during surgery, who received the wrong dosage of a medication or who was otherwise a victim of what can only be described as malpractice.

But anecdotes do not make data. And without hard numbers, doing something — anything — about the problem is impractical: If we don’t know what the numbers are to begin with, how on earth can we measure the efficacy of any solution? Further, if we don’t have solid data, we can’t even prioritize our response to the problem because we don’t know how serious it is, what the costs are or how much of our limited resources should be put into addressing the problem.

Fortunately, while CDC still does not list medical error on its cause-of-death reporting forms, another agency has been attempting to quantify the scope of the issue. Shortly after the original IOM study was published, Congress designated the Agency for Healthcare Research and Quality (AHRQ) (under the Department of Health and Human Services) to issue annual reports on patient safety.¹⁴ AHRQ commissioned a project in 2008 to explore alternative methods of documenting real cases of medical error. Its resultant report recommended allowing patients to report incidents they felt represented a failure of their medical providers that resulted in an adverse reaction.¹⁵

Still, in the decade since that report spelled out the parameters of how a parallel reporting system could be established to track deaths from medical error that the CDC does not capture, it does not seem the AHRQ has issued any reports on actual numbers of deaths from medical error.

Addressing Errors

While hard numbers remain scarce, that doesn’t mean the issue of medical error can’t be addressed. One organization working to reduce medical error is the The Joint Commission, a nonprofit agency that accredits hospitals and other medical facilities. The Joint Commission’s “National Patient Safety Goals” were established in 2002 to assist in providing guidance and goals for member facilities, and they are updated each year.¹⁶ The goals are fine-tuned for different kinds of facilities: acute-care hospitals (see 2023 National Patient Safety Goals for Hospitals), outpatient facilities and assisted living facilities. The Joint Commission then provides a formal protocol that any member can adopt as a baseline for its own program — covering basic issues that need to be addressed in a disciplined manner such as ensuring the right patient is brought in for surgery, ensuring a patient’s prescriptions are accurate and assigned to the right patient, setting response time for fall-prevention alarms, etc.

Many state licensing boards have instituted similar protocols in the wake of the IOM study and the 2016 Johns Hopkins meta-analysis.

Looking Ahead

Given the widespread mainstream media reporting on the Johns Hopkins’ meta-analysis and study and their claims of deaths from medical error, physicians will be dealing with patients who accept these numbers as uncritically as the media reports on them. And in the age of social media, “medical error as meme” is likely here to stay for many years to come — whether or not those numbers are scientifically founded or statistically valid.¹⁷

Until the governing agencies are able and willing to track medical error and misdiagnosis similar to what they do with HAIs and falls, physicians will continue to face all kinds of conjecture without having recourse to the kinds of hard data that can refute it. Committing to procedures and protocols that minimize the possibility of medical error remains the best way to reassure patients and secure the financial security of practices and facilities.