Build your first Facebook Messenger bot in Ruby with Sinatra (Part 3/3)

January 21st 2017 1,969 reads

@progapanda
Andy Barnov

Going live

In Part 1 of this tutorial we set up our page on Facebook and hooked it to a Sinatra server running on our machine through ngrok. We introduced the facebook-messenger gem from Hyperoslo and we hooked our bot to Google Geocoding API. In Part 2 we added new features to our bot and saw the complexity of our code increase, so we had to refactor. We added two different sets of menus to make interaction with our bot easier.

This time we’ll teach our bot to use our current location. We will also admit some design mistakes we made previously, refactor yet again, and finally set our bot free from the tethers of your computer by deploying it to Heroku. You will be also guided through Facebook verification process, so your own bot can one day go live.

If you did not follow steps from last two parts, you can pick up the code to begin with here.

Here’s what we are building today:

Our bot by the end of this tutorial

Keeping secrets

Remember how we had to type in our environment variables like ACCESS_TOKEN or VERIFY_TOKEN by hand each time we opened a new terminal window to launch our bot on localhost? Well, some readers pointed out there is a gem for that and it will work with any Ruby project (doesn’t have to be Rails, as is the case with the popular figaro gem). Meet dotenv. First, install it on your system:

gem install dotenv

Then require it on top of both app.rb and bot.rb files

require 'dotenv/load'

Create a file named .env in your project folder and put exactly the same environment variables declaration that you used to set in your terminal. (Reminder: both tokens can be found in your Facebook developer console, see Part 1 for details)

VERIFY_TOKEN=here_comes_YOUR_verify_tokenACCESS_TOKEN=here_comes_YOUR_access_token

Now add another file to your project folder, called .gitignore. Put .env inside (yes, just the name of .env file) so that your tokens are not pushed to Github for the whole world to see. I assume you track your project with Git, that would be essential for our deployment to Heroku.

Great, now you every time your launch your local server with a rackup command your environment variables will be set automatically, no need to worry about closing and reopening your terminal window.

Where am I?

You have to admit, the last iteration of our bot was not very useful: you had to manually type in an address in order to lookup GPS coordinates. Nice to learn random things like the latitude of an Eiffel Tower, but not really helpful when you need to answer the question, old as humanity itself, “Where am I?”.

The question everyone asks oneself sometimes: “Where the hell am I?”

Your phone has GPS, right? Well, Facebook Messenger API lets us use it. Here’s there own example of how to enable location sharing.

curl -X POST -H "Content-Type: application/json" -d '{"recipient":{"id":"USER_ID"},"message":{"text":"Please share your location:","quick_replies":[{"content_type":"location",}]}}' "https://graph.facebook.com/v2.6/me/messages?access_token=PAGE_ACCESS_TOKEN"

But we are not accessing Facebook JSON endpoints directly, all this work is done behind the scenes by the facebook-messenger gem that we use. However, we are able to reason deductively and what do we see by looking at this raw JSON? We see that the part enabling user to send his/her location is “content_type”:”location”in the set of quick replies. That means, we need to send any message to the user first and attach a location query to it.

You will also need to enable a location webhook under Webhooks tab in your Facebook developer console.

We already have a wrapper over the JSON request in a form of say helper that we introduced in a previous part. Here’s how it looked like:

As you can see, this method takes an optional quick_replies argument that is an array of replies. We used it construct our “menu” of quick replies to show basic commands to our user. Meaning, we can pass [{ content_type: ‘location’ }] (that’s right, Hash inside an Array) to this method and voilà, we’ll send exactly the same JSON we saw above to Facebook API. That’s code reuse!

Also, remember that we can examine any messages our bot sends us with message.inspect. Try adding this line to the top of your wait_for_any_input method inside bot.rb, right before sender_id = message.sender[‘id’] line:

puts "Received '#{message.inspect}' from #{message.sender}"

Now, if you open Messenger on your mobile phone, connect to your bot and send your location instead of text, then by examining your rackup logs you can see how the location message looks like to our facebook-messenger Santa’s little helper. You should see something like this:

Received '#<Facebook::Messenger::Incoming::Message:0x007fad3890be00 @messaging={"sender"=>{"id"=>"1295486190524658"}, "recipient"=>{"id"=>"1614495692193246"}, "timestamp"=>1484597226465, "message"=>{"mid"=>"mid.1484597226465:b335687852", "seq"=>59617, "attachments"=>[{"title"=>"Andy's Location", "url"=>"https://www.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3D55.772721087%252C%2B37.683347132677%26FORM%3DFBKPL1%26mkt%3Den-US&h=ATNEWzpLzUVPzSBK4oIF9rpOfG1IDaYzKtbNpDNcLAG9729MvTMvXETHmOTMdhnrQEw5Af2Pg8Ct2uTwXnQoTRI4DbmXWMH-p1ajjeAXhcvMLmMjvsyhCFgMDMM9xkHrFPVTTeg&s=1&enc=AZOftxHdpeOTr32oSlm8KEimjwBgrzLEPiUtBmE5a0pwZZ2BTsgizwnV6p3b8D3HvtE7dmT-hQuRh7Vn8x-vWNPe", "type"=>"location", "payload"=>{"coordinates"=>{"lat"=>55.772721087, "long"=>37.683347132677}}}]}}>' from {"id"=>"1295486190524658"}

If we dig in deep enough, we can get coordinates, we don’t even have to make a call to Google Geocoding anymore! By trial and error we discover that “coordinates”key containing a hash with latitude and longitude values can be accessed from our Ruby program like so:

message.attachments.first['payload']['coordinates']

Great! Let’s write some new methods. Add this to bot.rb:

As you can see, we use say helper method to add attach ‘Send Location’ label to your message. In the interface it will look like so:

We will also have to make sure that whatever reply we get in lookup_location(sender_id) is indeed a location and not a plain old text in order to process it further.

Add this helper method to bot.rb:

Now let’s get down to business and put our main logic in another method:

Nice, now our bot is able to (in theory, we haven’t yet hooked this methods up to any bot conversation logic) extract your current GPS coordinates from the location you’ve sent. Let’s take it one step further. What if we could also guess full address based on location? Sure, we can! Google Geocoding API also lets us do something called reverse geocoding. From the sound of it — exactly what we’re looking for! In order to use it we need to send a GET request to a different URL, so let’s create another constant on top of our bot.rb, next to where we declared API_URL . I’ll call it REVERSE_API_URL for consistency, but I admit it’s not the best naming.

REVERSE_API_URL = 'https://maps.googleapis.com/maps/api/geocode/json?latlng='

Note: Remember we’re only using Geocoding API without an API key for development and testing. For production you should get it as instructed.

Now, let’s update our handle_user_location(message) method to reverse-geocode our coordinates. Note we are reusing get_parsed_response and extract_full_address methods that we created earlier.

Great, we have all we need for the feature, now let’s add it to the interface. The simplest place to add it is a persistent menu, defined in your thread settings on top of bot.rb. Update it:

Now add a postback to LOCATION payload that will call our new functionality. Update your Bot.on :postback block.

Also, add this constant somewhere on top of your file to avoid retyping [{ content_type: ‘location’ }] everywhere we need it:

TYPE_LOCATION = [{ content_type: 'location' }]

Now whenever a user picks ‘Location lookup’ from the persistent menu, we ask for a location and invite him/her to send it to us.

We were also interacting with the user through the set of quick replies. Update it to include a reference to location command:

We handle quick replies like any other user command from inside the wait_for_command method. Let’s account for location (and change previous naming while doing so):

Notice we start adding TYPE_LOCATION to all say method calls. That’s because we want to add ‘Send Location’ label to all of our bot’s functions. After all, for a postal address lookup user may want to type partial address in, or just send his/hers current location. Same goes for manual GPS lookup. Now we have to handle location inside show_coordinates and show_full_addres methods. We will have to revisit what we’ve already done.

Keeping it simple

An important principle in code (and in life)

OK. It’s time to admit sometimes you can over-refactor your code. As some of my more experienced readers pointed out to me, introduction of yield with argument inside the handle_api_request method in Part 2 was atrocious, because even if it helps us DRY-out 8 or so repeating lines, it makes our code very difficult to reason about. Also, if at some point we want our features to develop more separate functionality, we want to be able to modify them independent of each other. Let’s work on our mistakes: get rid of handle_api_request method and rewrite two features.

Take-away: Overthinking is a form of stupidity.

Here, we keep all our conditional logic inside our Bot.on :message block, we also add a separate method for each lookup in order to follow the SRP and reduce nesting.

That was a lot of code! Whenever you feel lost, go to this tutorial’s Github to see the latest good version of the bot.rb file.

Going international

You must have noticed the encode_ascii method that we haven’t yet introduced. As of Part 2, our bot would just take whatever a user texted and append it to API URL, assuming all is good. But, if you try using international characters in your address (“Красная площадь“ or 東京都), your bot will throw this:

ERROR URI::InvalidURIError: URI must be ascii only "https://maps.googleapis.com/maps/api/geocode/json?address=\u{424}\u{440}. \u{42d}\u{43d}\u{433}\u{435}\u{43b}\u{44c}\u{441}\u{430}, \u{414}. 23"

This error comes from Ruby standard library uri module that is used by HTTParty (the gem we use to make HTTP requests). We need a way to convert a string that comes from a user into the URL-friendly ASCII format.

For that, we will use the addressable gem. It has a method to perform percent-encoding on an URL, but you can also do it on any string. Add gem ‘addressable’ line to your Gemfile and require ‘addressable/uri’ in bot.rb. Then add ensode_ascii(s) method, it’s a one-liner.

Now a string “Красная площадь, д.1” will turn into “%D0%9A%D1%80%D0%B0%D1%81%D0%BD%D0%B0%D1%8F%20%D0%BF%D0%BB%D0%BE%D1%89%D0%B0%D0%B4%D1%8C,%20%D0%B4.1” . Google API is smart enough to decode it back once you make a request.

The alternative is to use unidecoder gem that will try to transliterate any string so “Красная площадь” will turn into “Krasnaia ploshchad’”, which is close enough, but “東京都” will turn into “Dong Jing Du“, which has nothing to do with Tokyo (for which that three Japanese characters stood in the first place). Actually, for the transliterated version of “東京都” Google API will return some address in Florida. That’s probably not what you want.

Robot abuse

You always have to assume that your users will do everything they can to break your bot, either because of sheer curiosity, or just by chance, because you did not account for some behaviour. Currently, if you ask user for an address and he/she sends you an image or a video, instead of text, as a reply — your bot is broken. To account for that, you’ll need to introduce a tiny new method:

Then perform this check in every action, like so:

Why humans have to be so cruel?

Spring cleaning

Okay, now our bot is well over 200 lines long. That makes our code difficult to maintain. It’s time to put some of common functionality into separate files. We can use classes or modules to break our program into pieces. We will use a simple class wrapper with a class method to invoke part of code (that’s debatable, and I’d like to hear more about the actual best practice in comments). The idea is to do something like this:

And then call Greetings.enable at the top of your bot.rb. I leave the details to you, in my project I created two files: greetings.rb and persistent_menu.rb to collect all Facebook::Messenger::Thread.set actions.

Commencing countdown, engines on

Our bot goes to Heroku cloud

Now after we tested all we could on localhost (yes, I know bot testing is kind of hell) we are ready to send our bot into the world. We will use Heroku for deployment. Go to Heroku’s website and sign up for free.

Then, install Heroku CLI if you haven’t done it before by following these instructions. Also, make sure your project folder is tracked with Git. If not, run:

git initgit add .git commit -m "First commit"

Then authenticate your machine with heroku login and create your heroku app by running this command:

heroku create YOUR_BOT_NAME

It will create a web address in the form of YOUR_BOT_NAME.herokuapp.com and add heroku as a remote Git repository. Check if all went well by running:

git remote -v

You should see that two new references (fetch and push) to your heroku remote.

Now, and this is important: comment out require ‘dotenv/load’ line both in bot.rb and app.rb. dotenv gem does not work with Heroku (there’s a fork that does, you can look it up, but I found it easier to set variables directly in the dashboard). Then in your Heroku dashboard pick your newly created app, go to its Settings and scroll to Config variables. Reveal them and type in your ACCESS_TOKEN and VERIFY_TOKEN like so:

Now, the moment of truth. If you followed all the steps then the contents of config.ru file should be enough to successfully deploy. Run:

git push heroku master

…and behold the magic happening as Heroku creates a functional web app from your code. All you have to do now is to go to Facebook developer’s console and change the address for a webhook from your ngrok URL to your new Heroku app, something like:

https://your-bot-name.herokuapp.com/webhook

If the webhook verification goes well — you have successfully deployed your bot!

Hooray!

For now you can test your bot yourself or add other Facebook users as testers under Roles tab in your Facebook developer console. If you are truly ready to make your bot available to the whole world, you need to go through the verification process. Meaning, actual live people at Facebook HQ will look at your bot (though, briefly) and check if it doesn’t do anything nasty. Go to App Review tab and make your bot “live”. Then go to Messenger tab and under App Review for Messenger add pages_messaging to submission. Follow the provided checklist.

You just need to review your pages_messaging in our case

You will need a privacy policy for your app that can be easily generated on numerous websites. I found this one to be most straightforward. Also, you need to supply an icon for your app and it must be exactly 1024x1024 in resolution.

Now cross fingers and wait.

In my case, Facebook reviewers cleared my bot in less than 15 minutes (I did it on Friday morning, Eastern time). Sometimes reviewers may ask you to record a screencast of your bot, which is done easily with QuickTime.

Ahoy, Captain!

Of course, our Coordinator bot does not provide the best user experience and some people may find it’s interface confusing. The conversation flow is rather rigid and if you make a mistake, you have to start over. However, I feel that some features, like a postal address lookup or finding the name of the street you’re on just by a single message can prove useful (at least, I find myself using them from time to time).

You can play with the latest version of this tutorial’s bot by following this link.

This project started with a contrived example to play around with the facebook-messenger gem as a personal exercise in Ruby. Later I realised it may help beginners like me and started working on this tutorial.

My job was, more than anything, to stimulate your imagination and inspire you to build your own things.

So I was happy to learn that one of my readers used this very tutorial to create a bot that helps navigate Paris subway.

Captain Metro bot, inspired by this very tutorial

If you live in Paris, you can test Captain Metro here.

About me: I’m a beginner programmer and ex senior international TV correspondent. I am looking for my first proper job as a developer. Currently, I am having hard time getting employed in Russia, where engineers are extremely good, and it seems no one wants to hire beginners. If you want a motivated coder with well-developed human language skills for your team, I’ll be glad to join, even remotely. You can also invite me for an internship.

I recently graduated top of the class from an excellent Le Wagon coding bootcamp in Paris and co-authored HALFWAY.ninja as a graduate project there. It’s an airfare comparator for couples. Go check it out! You can find me on Github, Twitter and Instagram. I speak English, Russian, French, Ruby, JavaScript, Python and Swift and I am currently based in Moscow, Russian Federation. Feel free to write me directly with any offers.