This article is a personal experience shared by Benoit, Sqreen’s first engineer who worked on the Ruby agent and on the Python backend.
Back in 2008 when I was still in engineering school, two of my friends decided to start a company called Feedbooks. Their goal was to make high-quality ebooks more easily available for people to find, download and share; a digital library that would group together public domain masterpieces, self-published writing, and commercially distributed works.
The website that allowed them to achieve this goal needed to be fast, efficient and easy to maintain. At the time, having just watched the now famous “how to make a blog in 15 minutes” Ruby on Rails demo video, the founders decided to give it a go and started coding away. A few weeks in, to develop their product as fast as possible, they needed additional help and asked me.
Like many engineers of my generation, I had previously dabbled quite a bit in PHP & Mysql websites, I had even contributed to a small opensource CMS system. I was familiar with everything web-related but had never even heard of Ruby. So, I watched the video & also fell under the spell of Ruby on Rails. I was hooked on Ruby for the next 10 years and counting. Our group of like-minded friends lent a hand to the library website and also developed quite a few other web apps using the technology.
But why Ruby, what’s so great about it?
Ruby is a very expressive language. Matz, the main designer of the language, had a very simple goal when creating Ruby: optimize for programmer happiness. To me, this has clearly been achieved. Ruby is an object-oriented scripting language. Every single thing in Ruby is an `Object`. The Ruby virtual machine takes care of translating the very natural looking syntax to actual instruction. This VM is constantly improving and the generated code gets faster with each new version. Objects in Ruby collaborate by sending themselves messages containing a method name and a list of arguments. Ruby also being very easy to introspect, it’s easy to change these messages or create ad-hoc interfaces.
These make it a very good language to write Domain Specific Languages (DSL) into. That’s precisely what David Heinemeier Hansson (aka DHH) did when he created Rails. Ruby on Rails is, at its core, a collection of DSLs plus a framework to easily generate model-driven web applications. To this day, it’s particularly adapted and good at creating apps where the business logic & modelization will play an important part.
And so, 6 years later when I joined Sqreen as the first engineer, the logical choice would have been to use Ruby. And yet, when we started Sqreen backend we didn’t go with Ruby but used Python instead.
You mean you love Ruby but chose Python for your backend?
Why is that?
First of all the Sqreen backend is actually multiple services exposed (or not) to the world. The first part a new customer sees is our corporate website which is actually a static website (though we do generate it using Jekyll, a Ruby-powered website generator). Then comes our dashboard, conversely it’s an extremely dynamic application, built as a single page app, that is written, after a first foray into Meteor (that’s a blog post for another day!) as a React application. The dashboard communicates with a dedicated API, that we call “backend for frontend”. This isn’t our only API, we have a few others, but by far the most important is our dedicated “backend for agents”.
We decided early on, that we didn’t want to write APIs in multiple languages so that we could easily reuse business logic code & models across our different APIs. We also envisioned very early that at least the backend for agents should be able to scale quite easily to a high number of concurrent clients as our agents would often ping the backend back.
Rails being unfortunately quite slow was never really our first choice. Obviously, Rails is not the only way to use Ruby for applications & APIs, there are numerous micro frameworks available in Ruby, the most well known being Sinatra. Sinatra is great, it has a very small and easy to use API and we could have gone for it.
We chose Python for two reasons. First, the two co-founders JB & Pierre were more familiar with Python, having used it for some internal tooling in their previous lives at Apple. Second and foremost, we have always believed that data analysis would be essential to Sqreen. Ruby is unfortunately not a great help on that account while the Python scientific ecosystem is simply top-notch. The choice was easy, let’s use Python & its very well known micro-framework solution; Flask.
Starting out in Python
At that point in time, I had only ever dabbled in Python to write small scripts, either to convert & reshape some data or perform calculations. I had to get up to speed using it to create APIs. Fortunately, the web and its base components do not change if you change programming language. HTTP/HTML/JSON are still text-based protocols/languages; REST is still the same. Whatever I learned during my career was still valid, only the way to output it was a bit different. Making applications communicate with the web at a lower level is done in Ruby using a common specification called Rack. It’s a very simple spec mostly codifying that HTTP Requests & Responses should be expressed using `Array`. It turns out, that Ruby’s Rack is actually very much inspired from an earlier initiative in Python called WSGI. Good, one big potential hurdle was out of the way.
But what about the language? Ruby as previously described is an object-oriented scripting language, well python is also an object-oriented scripting language. However, Python is a bit less pure in its approach to the concept. It wasn’t created as an object-oriented language and also supports other paradigms (mainly functional). But, they are still different languages, and so the syntax is quite different. Let’s name a few. Ruby describes blocks using keywords (begin/def end) Python uses whitespaces. This did take a bit of getting used to (and I still personally prefer Ruby’s way). You catch exceptions in Ruby using begin/rescue/end in Python you use try/except. But to me, the biggest change was not being able to do any inline block the way Ruby enables you to pass to many methods (like with `find`, `map`, `inject`…). In modern Python, people can use an inline comprehension to often do the equivalent operation but the syntax gets very messy very fast. This actually forces developers to think more about the number of loops being used and ultimately leads to more efficient code. I also discovered a few new and very thoughtful concepts. Example: context managers in Python enable you to apply clean up behaviors to resources and the actual syntax, again, pushes people to write more efficient code.
So how did I personally get up to speed in Python syntax? First by practicing using the very nice Python Koans. This open source project enables people to learn Python by fixing a set of progressively harder unit tests. They tour the different features of the language and provide an in-action and quite deep tutorial. I then read the Python documentation. Where the Ruby documentation can be terse and sometimes a bit lacking, the Python doc is extremely complete. The language is divided into modules that each have a long explanation generally going into the details of the rationale of why the module exists, which problem it aims to solve, how it solves it and quite often when not to use it! And yes obviously, it also includes examples. It’s a must-read for anybody looking to get into Python. Wait does all of this mean that Sqreen doesn’t use Ruby and that I don’t do any of it anymore? I’m glad you asked, actually no! I love Ruby and we actually do quite a fair bit of it at Sqreen.
First and foremost we have a Ruby agent that helps protect Ruby apps. Developing and subsequently maintaining the agent, which I did on & off during the last 3 years, is actually much more complex (read, fun!) Ruby than I ever did before. Each of our agents does dynamic instrumentation. In other words, we are writing code that will change the control flow of a running piece of software based on a description — cryptographically signed — sent over the wire. We first experimented this in Ruby but we are now doing it in five (soon six!) technologies. This is not trivial and it is a privilege to work with a team of talented & passionate people on this kind of topic.
Secondly, when we created the agent, and before sending it to our first testers (now customers) we wanted to test it thoroughly in real life conditions. We needed a real-looking application on which we could also measure the behavior of the agent. True to our engineering culture, we decide to dogfood ourselves and created a small but quite useful Rails admin interface. It’s used every day internally to monitor the behavior of our agents and help our customers.
A little advice from someone who’s been there
Are you a Ruby dev thinking about starting in Python? You should do it! It’s sufficiently similar to Ruby that you’ll feel productive very soon. However, it’s also sufficiently different that you will learn some new tricks that could open new doors for you. Being a one-trick pony is never a good idea anyway! A good interactive tutorial and the documentation are a good place to start. But you should also try working on a real app. Many idiosyncrasies of the ecosystem do not appear when only doing tutorials (e.g package management, testing, profiling…). Want to try it, with people who’ve already made the switch? Join us at Sqreen, obviously, we are recruiting 😉
Originally published at blog.sqreen.com on January 30, 2019.
