At the peak of the COVID-19 pandemic, the healthcare system was desperate to quickly diagnose COVID-induced pneumonia and separate it from other causes of pneumonia. The stakes were high, as early and correct treatment meant fewer deaths and faster recovery.  

Researchers were tasked with training an artificial intelligence (AI) model to distinguish COVID-19 pneumonia from bacterial pneumonia using chest X-rays. Many images were promptly collected. To maximize the amount of data for AI training, images were acquired from two hospitals: a community hospital and a university-based tertiary care center.  

The AI system was developed and deployed into clinical practice, but it failed to improve clinical outcomes. On further investigation, the problem stemmed from how the X-rays were collected.  

Patients with COVID-19 pneumonia were sicker on average and, therefore, were triaged to a tertiary care center where more advanced care was delivered. The AI was a powerful learner and quickly recognized the association between the final diagnosis and the hospital where the individual was treated.  

How was the connection made? It was discovered that the AI model determined whether pneumonia was caused by COVID-19 not based on clinical evaluation, but on the font used to label the X-rays.  

This example illustrates one of the most fundamental laws of machine learning. Data scientists coined the phrase “garbage in, garbage out,” suggesting that a system's output depends entirely on the quality of its input. If training data is biased (as in the example above), the model's performance will likely be biased as well, sometimes resulting in outcomes that are meaningless and even harmful.  

Garbage or Treasure?  

As well-intentioned practitioners of the medical AI arts, we have many questions. How do you spot “garbage?” What is the perfect dataset for AI training? Is there a way to measure data quality? What is the right balance between quality and quantity of data?  

Researchers try to answer these and other questions in the complicated field of medical data waste management.  One tool that helps get some answers is the METRIC framework, which includes five data quality clusters.  

1. The measurement process cluster addresses data errors. “To err is human,” and this type of garbage is produced in abundance by healthcare workers. Incorrect diagnoses, important omissions, implicit and explicit biases, typos and grammatical errors—we do it all. Some studies show that almost all progress notes in electronic health charts contain at least some factual errors.  To be fair, not all quality issues in the measurement domain are human-related. Device errors in accuracy and precision are also common.  
    

2. Being up-to-date is important.  Do you want to ask an AI model trained on 3-year-old data for medical advice? Those 3 years have led to medical science advances that the AI has no knowledge of. Obsolete AI equals obsolete answers.  
    

3. The ideal dataset for AI training is representative. That means it has data for all use cases across all population groups, both sexes, pediatric and adult, tall and short, etc. The ideal representative dataset for AI training will encompass all medical data available worldwide. And, yes, an AI cancer diagnosis tool not trained to recognize a rare cancer subtype (because the data was scarce) has exactly zero chance of diagnosing that cancer correctly.  
    

4. It's nice to train AI on rich, informative data. In medical machine learning, we love using International Classification of Diseases (ICD) codes, a consistent, structured system for accounting diagnoses. Not all billing codes are equally informative or commonly used.  You may not have enough data to train medical AI to recognize “a pedestrian on foot who was injured in a collision with a roller-skater, subsequent encounter” (ICD-10-CM code V00.01XD). Do you intend to distinguish medullary from papillary thyroid cancer? It’s not going to happen, at least by using ICD codes, because all thyroid cancer forms and types are labeled with just one ICD-10-CM code C73.  Nowadays, it is common to integrate various types of data, such as imaging, free-text notes, and procedure codes, to improve AI outputs. 
    

5. The last data quality cluster is consistency. A physical examination that finds normal pedal pulses in a patient with bilateral leg amputations does look like “garbage,” doesn’t it? Distribution drift, which describes gradual changes in medical data over time, is insidious and a major cause of AI performance degradation.  

Looking Out for Garbage

Perfect medical AI-training data doesn’t exist. Some errors in AI data are inevitable and acceptable. However, when reading AI medical literature, look out for these red flags that may suggest excessive problems:  

• Details are scarce on how the AI training data was collected.  

• Data quality is neither considered nor discussed.  

• Training data does not match the task (for example, a large language model trained solely on text data is asked to interpret a medical image).  

• Training data is small and unrepresentative (for example, medical records from a single small hospital are used to develop an AI system intended to work nationwide).  

• The bias in the data is obvious (for example, images of patients with the disease and healthy controls were collected from different sources).  

AI’s ability to integrate existing knowledge is extremely powerful, yet it is also its weakness, as its performance depends on the quality and quantity of data that it has access to.  

As warm-blooded healthcare providers, we benefit from the clinical judgment and intuition that help us navigate circumstances in which the data, clinical assessments, or medical literature do not add up. AI models that “know” vastly more than even the brightest physician do not necessarily have such flexibility. This is one of the reasons why AI, a powerful ally, is not yet ready to replace the common sense of traditional medicine delivered to humans by humans.  

Nikita Pozdeyev, MD, is an assistant professor of biomedical informatics and endocrinology at the University of Colorado School of Medicine in Aurora. Dr. Pozdeyev’s laboratory employs large-scale genetic and imaging datasets, analyzed using statistical genetics and AI methods to improve the diagnosis and treatment of thyroid cancer and other endocrine diseases.