This Robot Fights Back Against Robocalls

How I used Twilio and a few lines of Python to stop all unwanted callers permanently.

I, like most people, am annoyed by telemarketers. But recently the problem has gotten so much worse due to the automation of scam calls. This was covered in detail by NPR’s Planet Money and investigated extensively by Reply All in an amazing 2 part podcast (available below). The Planet Money piece shows that these robocalls are unlikely to end anytime soon, since the FTC is moving very slowly on the issue. Then, right in the middle of the piece, they played an ad for Google Cloud Platform and I got an idea. I could make a bot that would screen for unwanted calls and only let quality calls through. Like a digital personal assistant.

Putting software between your phone and the world is a lot harder than it should be. Twilio offers a great interface for creating programatic phone systems, but porting a phone number to Twilio is a six step process that can take up to four weeks.

A better option, in my opinion (if you have an iPhone), involves creating a new number for Twilio, which you will use publicly, and then restrict your personal phone line to only accept calls from numbers in your contact book. You can restrict unknown numbers from making your phone ring by setting Do Not Disturb mode to allow calls from your contacts and silence all other numbers. The key here is to set the schedule from 10:00PM to 9:59PM so Do Not Disturb mode will stay on permanently.

This, in and of itself, is a good start and will help clean up unwanted calls. But it’s a bit of overkill, so setting up a new number that you can use publicly to forward calls to you based on criteria you set is a good next step.

Step 2: Check Contacts via Google API

If you sync your iPhone contacts with Google, accessing your contact listing via API is relatively simple. You just need to get the people resource from the Google People API. I find it best to crawl this for updates daily and save all the “known numbers” to a JSON file for easy access (it’s just a list of phone numbers that will be passed through to your real number).

When a new call comes in to the Twilio number you have setup (and given out freely), Python opens the JSON file with all the stored numbers and checks if the caller is in the list. If so, the call is automatically forwarded and the caller is unaware of this process, if not though, the caller will be prompted to explain who they are and why they are calling.

Step 3: Prompt caller for explanation via Twilio Speech Recognition

Twilio offers great out-of-the-box speech recognition that allows you to prompt a caller with a message and then automatically record and transcribe their response.

from twilio.twiml.voice_response import Gather, VoiceResponse, Sayspeech_result = VoiceResponse()gather = Gather(input='speech', action='/completed')gather.say('

This will return a transcribed SpeechResult object for further handling. There is the opportunity to inject some machine learning into this step to understand why someone is calling, maybe correlate that with my calendar or recent emails and handle the call accordingly, but that is out of scope of this post.

Step 4: Text me the transcription via Programmable SMS

Sending this result to me via text is trivial with Twilio. The python code looks like this:

client = Client(account_sid, auth_token)

client.messages.create(to="YOUR PHONE NUMBER HERE",from_="YOUR TWILIO NUMBER HERE",body="Call {}. Message: {}".format(caller_number, speech_result))

Then, based on the number and transcribed message, I can text back “yes” to be connected if it looks like a call worth taking.

Step 5: Forward the call to me if I approve.

Once a positive response is received the call needs to be patched through to my real number. Since I have my Twilio number saved in my contact book, the call will go through and ring on my personal line. Twilio Labs hosts a small app that allows for simple call forwarding. The URL structure is simple, I just substituted my own phone number and recorded a voicemail in case the call fails.

from twilio.twiml.voice_response import Dial, Number, VoiceResponseresponse = VoiceResponse()dial = Dial()dial.number('

Fortunately, very little customization is needed for this step.

Step 6: Ask to take a message if I don’t want to talk.

If I’m busy and don’t respond to the text, Twilio then prompts the caller to leave a message. I think 60 seconds is the longest amount of time you want to put an unknown caller on hold, so the Twilio script waits 60 seconds for a response before asking to take a message.

Step 7: Transcribe the message and send to my email.

Just like before, the voice message is transcribed using Twilio’s speech recognition service and then emailed to me using Python’s simple smtplib. Sending email with Python is well documented, so I won’t go into detail about how to do that here.

Next Steps:

I would love to turn this into a self-service app in the future and share it with other people who are interested, but there’s a bit too much on my plate at the moment and I’m sure companies will start adding these features to phone systems in the near future.

If you want a more basic version of this, you can use Google Voice to screen calls and get some of the same functionality, although it’s far less customizable. The Google Voice call screening solution still causes your phone to actually ring (vs. a much less invasive text message).

I can definitely see how some people might consider this overkill, but I think it’s appropriate for our current societal communication patterns. Increasingly, the legacy phone system is being abused for spam and most people have already shifted to texting or a variety of social platforms as the dominant form of communication. I would love for the FTC to roll out a great solution for this problem, but I just don’t see it happening fast enough. Until then, I’ll probably keep hacking my phone until I have the least amount of tolerable distraction in my life.

Thanks for reading! Please hit the clap button if you found this interesting!

You can connect with me on Twitter: www.twitter.com/johncoogan