The Programmer’s Guide to Booking a Plane

About two months ago, I wanted to go on a vacation. I had the hotel more or less picked out, but the transportation was still up in the air.

I began by scouring the web for cheap plane tickets, like normal travel folks seem to do. I scoured all of the fancy airlines, but all of their fares were too high for my liking.

I decided to look into Southwest, the de facto airline for low fares (not really). I had heard from a few internet-goers that ticket prices would drop on Tuesday nights, so I set out to test that claim. I started checking fare prices periodically throughout the day.

Days passed.

Reload.

Reload.

“Would you like to resubmit this form?”

Yes.

Reload.

Nothing.

I started to get annoyed with the repetition (surprise, surprise!), so I set out to automate things as best I could. What if I wrote a script to do these annoying tasks for me? Yeah. I liked that idea.

Crawling the Southwest website

A quick look at Southwest’s site revealed no API. (No surprise there.)

I figured that straight up scraping their website would be an effective approach. I could write a script which would, you know, scrape their website for the data I wanted.

I chose Node because I figured they would have great tooling for web scraping, and I was more or less correct on that one.

As I began experimenting with the algorithm… err, script, I realized that it was a lot harder than I thought it would be to fake a form submission on Southwest’s website so that I could get to the results page.

I tried cURL.

I tried “copy as cURL.”

I tried faking the user agent.

I tried all the things.

The solution was to use osmosis to literally fill out and submit the search form, and then scrape the results page.

So, I built a quick and dirty command line script that scraped their website every so often. I then let it do its thing every night while I slept. It wasn’t fancy, but it did what I told it to do.

Analyzing the Fares

I wrote a Node script … wait I already said that. Okay. So, I have a script running. It’s printing out fare prices, but I’m not really sure what to do with all of this data. It’s not like I want to sit there and watch it all night.

So what did I do? I hooked up Twilio to send me a text message whenever a fare hit a certain threshold. Twilio offers a free account as long as you verify the phone number that you’re texting to, so that’s pretty rad. So I verified my number and let the texts come flowing in.

And since I didn’t think it through, I also chose to send the text message at a low interval so that it would keep pinging me until I stopped the script, even if I was out on a date with my lovely wife.

It wasn’t annoying at all. /s

But it worked quite well. I could see that fares did indeed drop, especially late at night. The internet-goers didn’t lie to me this time.

Plotting the Data

After a few days, I got kind of annoyed (what’s new?) with looking at a stream of text and thought of complex ways to make it more appealing to look at for the few seconds that I looked at it every day.

So what did I do? I hooked up blessed to plot me some graphs and make things look pretty. I had seen blessed be put to use in the lovely webpack-dashboard, so I thought “why ever not?”.

It was awesome. Thanks, Ken Wheeler.

Booking the Flight

Days later, on a normal Tuesday night, it happened: the prices dropped dramatically, something that did not happen every Tuesday, but rather quite randomly. I snagged our tickets for under $100. (I, of course, booked them manually because my script is not that fancy.)

Sweet, Sweet Victory

A month later, I went on vacation. And that’s about it. You can check out the project on GitHub if you’d also like to go on a nice vacation.

Note: as of July, 2017, the project is no longer available on GitHub due to a takedown request by Southwest.