Bulk of clinical AI systems' data comes from just 3 states

The lack of geographic diversity within the data that makes up medical artificial intelligence systems could mean the technology is unduly applying a one-size-fits-all approach to patient care, according to research published Sept. 22 in JAMA Network Open.

Stanford (Calif.) University researchers found that since many clinical AI researchers are affiliated with prestigious coastal academic medical centers and have access to these institutions' training data, the records used to inform clinical AI systems come mostly from patients in the three states: California, New York and Massachusetts.

"It became apparent that all the datasets just seemed to be coming from the same sorts of places: the Stanfords and UCSFs and Mass Generals," a member of the research team, Amit Kaushal, MD, PhD, told STAT.

The researchers argue the lack of geographic diversity in clinical AI training data could be problematic because many medical factors differ greatly by region, such as the prevalence of certain diseases, clinical treatment patterns and access to care. Dr. Kaushal told STAT these factors "end up getting baked into the dataset and become implicit assumptions in the data, which may not be valid assumptions nationwide."

The Stanford study comes a couple years after a study published in PLOS Medicine trained machine learning models to detect pneumonia in chest X-rays from the Washington, D.C.-based National Institutes of Health Clinical Center, New York City-based Mount Sinai Hospital and Indianapolis-based Indiana University Network for Patient Care. In three of the five experiments performed, the AI system performed significantly worse when applied to patient data from locations different from the one on which it had been trained.

https://www.beckershospitalreview.com/artificial-intelligence/bulk-of-clinical-ai-systems-data-comes-from-just-3-states-study-finds.html