Eric Poon, MD, MPH, admits that during appointments with his patients he’s “a bit of a chatterbox.” As a result, he says, “I always run late.”
Over the past several months, however, Poon has been finishing his clinical schedule on time “for the first time in my life,” and the quality of his conversations with patients has improved. The main reason, he says, is an artificial intelligence (AI) tool that records the conversations and produces written notes — freeing Poon from the distraction of taking notes while talking to patients, which allows him to listen and respond to them more thoughtfully.
Duke Primary Care in North Carolina, where Poon practices within the Duke University Health System, is among a growing number of health care systems that are testing tools that essentially listen to doctor-patient conversations, then produce not a transcript but a shorter and organized text summary. Early feedback from participating physicians is that the tools improve the quality of doctor-patient visits, reduce the time doctors spend on documentation, and reduce feelings of physician burnout.
“I am finalizing my notes much more quickly,” says Poon, who edits text notes produced by the tool (called DAX) before approving them for inclusion in a patient’s electronic health record (EHR). “It’s not perfect, but it beats having to type while I talk to patients or recreate the whole [conversation] from my chicken scratch.”
The tool is not perfect because the technology is new and needs to evolve. The concerns include transcription errors; notes that leave out important medical details or include too much trivial chit chat; a risk that the time savings will be wiped out by pressure to squeeze in more work; and questions about how the recorded conversations will be stored and used.
“We’re not at a place where I would say you should absolutely give this to every doctor,” says Patricia Garcia, MD, associate chief medical information officer for ambulatory health care at Stanford Health Care (SHC) in California, which is trying out DAX. Her feelings about its potential? “If I had to describe it in one word, I would say, ‘Hopeful.’”
How it works
When Matthew Anderson, MD, walks into an exam room to see a patient at Atrium Health — which operates at 1,400 locations in the southeastern United States — he asks their permission to document the conversation using software on his cell phone so that he doesn’t have to take notes during the visit. So far, the patients have always said OK.
“I hit start, we have our visit, I walk out. Fifteen seconds later, I’ve got a draft,” says Anderson of the notes provided by DAX.
That’s basically how the tools function, with some variation. Called ambient intelligence tools (that is, they listen and respond to people in their environment), the programs analyze the conversation, look for the medically relevant information, and produce an organized clinical summary. For example: If a specific medical issue was discussed three different times during a visit, the tool is designed to group the summary of those discussions together. The physician makes corrections, additions, and deletions, perhaps reorganizes some text, then approves the notes for inclusion in the EHR.
Various ambient tools are being tested by health care systems on several EHR platforms, including: Epic, using DAX, which was developed by Nuance Communications; Oracle, through its Oracle Digital Assistant; Meditech, which partnered with Google for an emergency room app; and eClinicalWorks, which partners with sunoh.ai to summarize doctor-patient visits. (The last three companies were not able to provide information about their results so far.)
One way of envisioning the potential of the tools is to think of them as “augmented intelligence,” explains Kenneth Harper, MS, vice president of Nuance, which is owned by Microsoft. “It can’t replace a clinician, but it can help them do their jobs more efficiently and reduce the burden they face.”
Feedback from doctors
Here are some of the benefits and limitations of the ambient tools:
Conversations with patients: “It’s difficult for clinicians to juggle so many aspects of the patient encounter at once — trying to document [the conversation] at the same time that they’re listening to the patient and thinking about the patient’s problem,” says S. David McSwain, MD, MPH, chief medical informatics officer at UNC Health Care in North Carolina, where about three dozen physicians have been piloting DAX since September.
“Our clinicians feel that it helps them to declutter their mind during the visit, to stay focused on the patient rather than focusing on the computer,” McSwain says. “And I don’t think we can understate the value of that.”
The change was eye-opening for Poon at Duke.
“I realized how much of my brain I had devoted to being a transcriptionist,” he says. “I was slowing myself down, getting in my own way. When my fingers were not as quick as the conversation, I had to tell [patients] to stop talking so I could finish [typing].”
Poon says his conversations are more efficient now that “I’m not a stenographer.”
Accuracy: Overall, physicians report that the notes are factually accurate, but as with most early AI applications, errors occur either by misunderstanding words or omitting important facts. (More on the latter below).
“The technology is not at a point where it can function independently,” McSwain notes. “You have to have that engagement of the clinician in ensuring the accuracy of the documentation.”
One risk going forward stems from how human behavior typically changes as a technology improves and becomes ubiquitous. Will doctors slip into relying on the technology to get things right and grow lax on reviewing what it produces?
“It’s not something where we can check out mentally and not read the notes,” says Anderson at Atrium. “It’s all of our responsibility to make sure that what is documented is accurate.”
Comprehensiveness: One of the biggest challenges for the tools is what to include and exclude from the doctor-patient conversation. “Nobody wants a full transcript,” Garcia says.
But Harper says that while some physicians would like the notes to include just about every detail about the patient, others want to capture fewer specifics from each visit. At some health systems, doctors have reported that the tools include so much information that they have to spend a lot of time editing down the notes. At UNC Health, McSwain says, staff joke about a DAX note that said about a patient, “He eats rice.” When SHC asked for feedback from physicians, they offered lots of praise but also some frustration, such as, “it writes a very long HPI [history of present illness].”
On the other hand, some physicians say the tools omit important information, such as medical details and terminology that might be useful for other clinicians, and lifestyle habits that might affect health. McSwain notes that a patient’s snack preferences and activities on a recent vacation, recounted during small talk at the start of a visit, might seem trivial at first but prove medically relevant later.
Medical complexity is an issue as well. At SHC, Garcia reports, one doctor wrote that the tool “works for quick easy visits, but not for visits with multiple medical issues.”
Garcia explains the challenge: “If the only thing you’re saying is, ‘I have a cough,’ and the doctor says, ‘Let’s get you a COVID test,’ that’s no problem. But if there are a lot of things going on [medically] it sometimes struggles to encapsulate those in a concise way and capture all the relevant details.”
Saving time, reducing burnout: Many doctors report that editing the notes provided by the tools takes far less time than revising their own notes at the end of the day. Nuance reports that doctors using the tool are spending 2-7 minutes less on each patient visit and 50% less time on documentation, and are feeling less fatigued and burned out.
What is the gain if doctors use the time savings to see more patients? Nuance says that doctors have added, on average, five more appointments on each of their clinical days. That’s not a bad thing at a time when patients sometimes wait weeks or months for a medical appointment, but seeing more patients could offset the reductions in physician workload and burnout.
“We have to focus on burden reduction for our clinicians,” McSwain says. “If we only use this technology to improve revenue” by pressuring doctors to see more patients, “that would be an extremely negative outcome of this innovation.”
Atrium Health leaves it up to doctors to decide how to use their extra time.
“We have [doctors] who have said, ‘I have freed up more time to provide better access for patients,’” Anderson says. “And we’ve had people who said, ‘Now I’m able to cook dinner, spend more time with my family.’”
Privacy: Patients know that their doctors put notes from their visits into their medical records. But having someone record a conversation with a doctor — among the most sensitive conversations someone can have with a non-family member — raises concerns about who can get access to that recording and how they use it.
For starters, clinicians should always get consent from a patient to record the conversation — which doctors using the tools say they do but which might become a less common practice if the process becomes routine — explains Vardit Ravitsky, PhD, president of the Hastings Center, which addresses social and ethical issues in health, science, and technology.
Another issue is where the recording ends up. As a patient, “having a voice recording of that conversation attached to my health record, where maybe my family history and even DNA information are, is a much greater risk for me,” Ravitsky says.
In a world where it seems that any computer system can be hacked — including EHRs — how can patients and doctors be sure that their recorded conversations will never be exposed? Will someone’s voice be used against them to link them to health risks, and thus perhaps to deny insurance coverage? Can the physician’s conversation be used to investigate that doctor’s medical assessments?
Nuance says the hospital systems do not have access to the recordings and the recordings cannot be linked to an EHR. Other tool providers did not respond to questions about the storage of recordings.
The Future
Doctors see several areas for improvement. Harper says improvements will occur as the tools continue to learn from the recordings about how people talk. That should increase accuracy.
Several doctors say they would like easier ways to edit and customize the notes, perhaps through verbal instructions to convert text about symptoms to a bulleted list and to add medical terminology (“replace stomach scope with endoscopy,” Garcia says by way of example). Some would also like the tools to help with other tasks, such as ordering prescriptions.
As for the conundrum of what to include and exclude from the conversations, McSwain thinks the tools will have to allow for customization by doctors. “That will add more personalization, capability, and flexibility,” he says.
Garcia believes those kinds of adjustments will make these “go-to tools” for many doctors. That echoes the view of one of the doctors who provided feedback at SHC: “It still has a few kinks to work out, but it makes my days (and evenings) SO much better.”
