Healthcare AI tools that flopped

 Reno, Nev.-based Renown Health spent months and significant IT resources putting AI-enabled cameras in patient rooms to prevent falls, but the number of falls on those units did not change at all.

“I don’t think that technology is quite there yet,” said Jason Knight, MD, chief medical officer of Renown Regional Medical Center in Reno. 

Four leaders across three health systems — Renown Health, Charlotte, N.C.-based Advocate Health and Cleveland Clinic — told Becker’s about AI-powered tools that failed. A few patterns emerged.

Tools that can see but cannot act. At Renown, the fall camera could flag a fall beginning, but it could not put hands on the patient. Virtual sitters have the same limit, Dr. Knight said: a virtual sitter “can’t immediately respond and prevent somebody from falling out of bed” — the remote employee can only call someone who can, “and sometimes that delay is all it takes for a patient to fall out of bed and hit the floor.”

Tools that add work instead of removing it. Remote patient monitoring is the clearest case, said Dan Davis, MD, vice president and physician lead for continuing health at Advocate Health.

“The technology has been advertised as the solution rather than a tool,” he said. Without clinical packaging, the data “ends up being noise, and sometimes can actually create more problems than it solves.”

Even where AI eventually worked, it rarely worked out of the box. When a cyclospora outbreak hit dozens of states, Cleveland Clinic’s AI stain-reading tool could not reliably report the parasite it was suddenly needed for, said Anisha Misra, PhD, medical director of mycology and parasitology. The team reverted to manual microscopy and ran more than 600 specimens to revalidate the tool before trusting it again.

Leaders converged on the same discipline. 

Ask “what problem are you trying to solve, and can it help you solve the problem?” before the demo, said Meredith Foxx, DNP, senior vice president and CNO at Cleveland Clinic. And weigh the hidden cost: big systems can end up with “500, 700, even 1,000 different software packages” bolted onto the EHR, Dr. Knight said — complexity that, in an era of cybersecurity risk, “doesn’t always make things easier and better.”

https://www.beckershospitalreview.com/quality/4-clinical-leaders-on-ai-tools-that-flopped/