Zeke Gabrielse

@ezekgabrielse

The Programmer’s Guide to Booking a Plane

About two months ago, I wanted to go on a vacation. I had the hotel more or less picked out, but the transportation was still up in the air.

I began by scouring the web for cheap plane tickets, like normal travel folks seem to do. I scoured all of the fancy airlines, but all of their fares were too high for my liking.

I decided to look into Southwest, the de facto airline for low fares (not really). I had heard from a few internet-goers that ticket prices would drop on Tuesday nights, so I set out to test that claim. I started checking fare prices periodically throughout the day.

Days passed.

Reload.

Reload.

“Would you like to resubmit this form?”

Yes.

Reload.

Nothing.

I started to get annoyed with the repetition (surprise, surprise!), so I set out to automate things as best I could. What if I wrote a script to do these annoying tasks for me? Yeah. I liked that idea.

Crawling the Southwest website

A quick look at Southwest’s site revealed no API. (No surprise there.)

I figured that straight up scraping their website would be an effective approach. I could write a script which would, you know, scrape their website for the data I wanted.

I chose Node because I figured they would have great tooling for web scraping, and I was more or less correct on that one.

As I began experimenting with the algorithm… err, script, I realized that it was a lot harder than I thought it would be to fake a form submission on Southwest’s website so that I could get to the results page.

I tried cURL.

I tried “copy as cURL.”

I tried faking the user agent.

I tried all the things.

The solution was to use osmosis to literally fill out and submit the search form, and then scrape the results page.

So, I built a quick and dirty command line script that scraped their website every so often. I then let it do its thing every night while I slept. It wasn’t fancy, but it did what I told it to do.

Analyzing the Fares

I wrote a Node script … wait I already said that. Okay. So, I have a script running. It’s printing out fare prices, but I’m not really sure what to do with all of this data. It’s not like I want to sit there and watch it all night.

So what did I do? I hooked up Twilio to send me a text message whenever a fare hit a certain threshold. Twilio offers a free account as long as you verify the phone number that you’re texting to, so that’s pretty rad. So I verified my number and let the texts come flowing in.

And since I didn’t think it through, I also chose to send the text message at a low interval so that it would keep pinging me until I stopped the script, even if I was out on a date with my lovely wife.

It wasn’t annoying at all. /s

But it worked quite well. I could see that fares did indeed drop, especially late at night. The internet-goers didn’t lie to me this time.

Plotting the Data

After a few days, I got kind of annoyed (what’s new?) with looking at a stream of text and thought of complex ways to make it more appealing to look at for the few seconds that I looked at it every day.

So what did I do? I hooked up blessed to plot me some graphs and make things look pretty. I had seen blessed be put to use in the lovely webpack-dashboard, so I thought “why ever not?”.

It was awesome. Thanks, Ken Wheeler.

Booking the Flight

Days later, on a normal Tuesday night, it happened: the prices dropped dramatically, something that did not happen every Tuesday, but rather quite randomly. I snagged our tickets for under $100. (I, of course, booked them manually because my script is not that fancy.)

Sweet, Sweet Victory

A month later, I went on vacation. And that’s about it. You can check out the project on GitHub if you’d also like to go on a nice vacation.

Note: as of July, 2017, the project is no longer available on GitHub due to a takedown request by Southwest.
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMI family. We are now accepting submissions and happy to discuss advertising & sponsorship opportunities.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!

More by Zeke Gabrielse

Topics of interest

More Related Stories