We wanted an A.I. revolution, instead we got dinner reservations

One of the coolest machine learning stories in recent memory involved the Google Duplex, a voice A.I. capable of calling a restaurant and making your dinner reservation while pretending to be human:

Simulating a personal assistant is cool, but it would be cooler to simulate a highly trained physician and potentially save lives. And given the massive hype surrounding artificial intelligence, I think it is fair to be ambitious.

“We wanted flying cars, instead we got 140 characters.”

— Peter Thiel

It was with this in mind that I took a break from my standard routine of “absolutely never leaving the hospital” to play around with a new coding project, focused on applications of natural language processing to healthcare using automated phone calls. Also, since I get approximately 40 calls per day from robotic telemarketers, I wanted to harness this power for good and build something cool. I implemented the project using Google Dialogflow, which was updated recently to include a Phone Gateway integration that makes it easy to create conversational telephone agents (à la the awesome Google Duplex).

For my first project, I made this simple but useful surgical concierge:

A summary, in case you hate fun videos: a patient with an upcoming surgery receives a call from a human-like surgery concierge. The concierge asks a series of pre-operative questions, learning about the patient’s allergies, medical history, and ability to get a ride home after surgery. Once this information is collected, the patient can freely ask questions to learn more about their upcoming procedure.

I’ll illustrate the value of this application with a story. Mr. Jones is a 75 year old male with liver cancer, and he desperately needs microwave ablation, a procedure to cure his cancer by burning it to a crisp with microwave energy. He was, however, never told to fast before the procedure and spent the morning eating breakfast and drinking coffee. This means he can’t safely get anesthesia (you need an empty stomach) and his procedure is cancelled. His treatment is thus delayed — will his cancer spread?— and the hospital loses money as an operative slot goes unfilled. Although the surgery concierge seems simple, preventing these minor communication issues allows us to avoid catastrophe.

Using natural language processing to automate phone calls offers significant value in healthcare. A few key points:

• Many of our elderly patients struggle with new technology, but nearly everyone gets the telephone. A similar tool could be created as a text-based chatbot, but could you grandmother use that chatbot? Maybe or maybe not, but it is very likely that she could take a phone call.
• Modern text-to-voice technology has surpassed a certain threshold of usability to become pretty damn good. Although imperfect, interacting with the telephone agent feels natural and similar to human-to-human communication. If you doubt this point, just ask all of the restauranteurs offering tables to robots from Google. There is elegance and power in this natural conversational interface.
• The optimal approach to many clinical problems will integrate participation from both humans and virtual agents. Although humans do a decent job of preparing patients for surgery, they sometimes neglect to mention a certain point (e.g. eating), forget to call a particular patient, or get confused and provide erroneous information. Bots offer a powerful back-up that can be carefully controlled by experts with a clear understanding of the latest clinical guidelines. This second layer of patient interaction has the potential to decrease communication error and prevent major problems, from cancelled surgery to death.

Pondering this new platform makes my mind race with promising medical use cases, from diabetes coaching to finding last minute MRI availability to reminding elderly people to take their medications. Man, I really should get back to work…

P.S. Please enjoy this bonus photo of my dog being glorious and majestic and enjoying our new home, San Francisco:

Captain posing with Karl the Fog

If you have an idea for a project or want a physician’s perspective on anything in tech, feel free to reach out on Twitter or LinkedIn.

Also, please hit the “clap” button and share if you enjoyed the article…it helps a lot! Thanks for reading. — Kevin