Dexcom: Outage that kept diabetics from tracking blood sugar ‘complete surprise’

• Dexcom experienced it’s largest ever technical glitch over the weekend.
• The company’s technology chief acknowledged that Dexcom needs to do better.
• Dexcom is analyzing its tech infrastructure and working to bolster its communications systems.

Kevin Sayer, CEO, Dexcom

Scott Mlyn | CNBC

Dexcom’s technology chief admits that the maker of continuous blood sugar monitoring systems was caught flat-footed by a technology outage over the Thanksgiving holiday weekend, and was ill-equipped to properly inform users of the problem.

Because of a server glitch, a large number of Dexcom’s customers, who rely on the company’s technology to manage their diabetes treatment, weren’t alerted to potentially dangerous changes in their blood glucose levels. The issue was of particular concern to parents who use Dexcom’s system to monitor their kids’ health.

“It was a complete surprise,” said Jake Leach, Dexcom’s chief technology officer, in an interview late Monday, after CNBC published a story on the ongoing problem. Leach said the company didn’t have any scheduled updates over the weekend. “This is a real learning opportunity to look at our system architecture.”

Dexcom acknowledged that the bug affected a “large portion” of its users in the U.S. who rely on its Follow feature, which lets users share their glucose readings with caregivers or family members. It began very early Saturday morning, with some parents saying they figured out something was wrong after their kids went to bed Friday night.

The company says it has a system for monitoring technology problems 24 hours a day and recognized what was happening right away though it didn’t know that the outage was so widespread and that it was expanding. Leach said that over the weekend, Dexcom pulled in its internal engineers to work on restoring the system and recruited help from its technology infrastructure partners.

Dexcom relies on Google’s cloud service and also uses some Microsoft technology. Almost immediately, Leach’s team started reaching out to Google to see if it could troubleshoot the problem. Microsoft also offered to help fix what had rapidly become Dexcom’s largest ever glitch.

Dexcom is still experiencing intermittent technical issues.

Erin Black, CNBC

The company even evaluated potential influence from foreign hackers, but didn’t find evidence of a data breach.

As of Monday, the outage was mostly resolved but some intermittent issues persisted. Leach said Dexcom has been reassessing its entire back-end system to make sure this doesn’t happen again.

One major fix the company is working on is improving its ability to message customers in real time. Currently, the messaging feature isn’t set up to inform users that the alert system isn’t working, Leach said, so Dexcom has been limited to updating people through its Facebook page.

Users who were unaware of Dexcom’s Facebook updates were stuck. Some started deleting and reinstalling the smartphone app, only to find that they could no longer log in, because that feature was also affected.

As Melinda Wedding, a mother of a teenager with diabetes told CNBC, “I get more information, more quickly when there’s an outage on my Dropbox than I do for my child’s blood sugar monitor.”

Leach said the company is “always evaluating” upgrades to its customer service, especially in light of criticisms from users who say they didn’t receive a call back or email response quickly enough.

“Ultimately we need to be able to communicate faster, and in a broader way,” he said. “Clearly, there’s an opportunity there.“

https://www.cnbc.com/2019/12/03/dexcom-cto-our-biggest-ever-glitch-was-a-big-surprise.html