Turns out those years mastering Jenkins weren't in waste

This article contains research findings from Bitrise's report Mobile CI after Jenkins - 6 lessons learned. If you're not interested in my stories about uncle Jenkins, manga and struggles with stacks of Macs, just skip ahead a few sections, and I'll share data on how the folks that used to be responsible for Jenkins feel their jobs changed. PS. I work at Bitrise, the folks that put together this report

Uncle Jenkins

Did you know that in Japan, devs will often refer to Jenkins as 'Jenkins ojisan', or 'Uncle Jenkins'? 🇯🇵 When I first started working at Bitrise and Google Translated Japanese Tweets, that confused me tremendously. Fast forward a couple of years, and we actually hired a manga artist to tell the story of devs struggling with their Jenkins boxes:

But I digress.

Jenkins is free.... ? 🎉

We all know Jenkins. It's estimated to be what over 70% of devs use for their CI, and for good reason:

• It's free
• It's flexible
• It's open source

But that flexibility comes at a price. It's devilishly difficult to master for beginners, and - as detailed in the Nextdoor Engineering Blog recently - has its limitations when you try and use it at scale for mobile engineering:

• Maintenance is a constant struggle, with the necessary hardware to build for iOS as an (expensive) bonus
• There's a lack of isolation between builds, which means that - for example - the cache from an old build can cause a new build to fail (have fun troubleshooting that one 😓)Maintenance is a constant struggle, with the necessary hardware to build for iOS as an (expensive) bonus
• There's a limit to the speed at which you can spin up physical machines if your needs increase, and probably a finance department that wants to know why trucks filled with Macs keep pulling up

Oh, and on Jenkins being "free", Moataz Nabil (Senior Software Engineer at Delivery Hero) had the following to say:

"While Jenkins - technically - is free, the cost of hosting the server on which it will run can not be easily calculated. It's not always possible to estimate the kind of load the server needs to handle, or the amount of compute power you'll need from hardware stacked in the office. The total cost associated remains unpredictable."

TLDR: It's free, if you ignore the fact that you're spending a ton of time on it, and also need hardware to run it on. Unless that's fallen off of a truck by the docks, Jenkins is probably not 'free'

Let's migrate! But... What do we do with our Jenkins folks?

For these and a number of other reasons that could make up an entirely separate article, cloud-based alternatives like CircleCI, Travis and - yes - Bitrise have been edging out the traditional "Jenkins box" in most mobile engineering organisations.

That's usually great for those organisations (more on that later), but what about the folks that were responsible for maintaining Jenkins?

Are they out of a job? Demoted to getting coffee and bagels for the team? What does their day to day look like after we've said our goodbyes to uncle Jenkins?

We decided to find out, by asking.

The report

Over January and February of 2021, our UX research team, developer advocates, and a number of other folks surveyed and interviewed hundreds of ex-Jenkins maintainers who had switched to cloud CI for mobile.

Our goal was to find out exactly what had changed for these people, and - more then anything - figure out how they felt about the change.

What follows are some of the main takeaways:

Computer says... Engineers are happier!

(the computer actually said 'yes', but there's no GIF for that...)

73.9% of those interviewed answered that they were happier with their jobs after moving from Jenkins to a cloud CI alternative. Half of those even reported being much happier, a mental state that - according to Google - usually requires 7 minutes of exercise, more sleep, meditation and / or cake.

We found a lot of reasons for that increased happiness in answers throughout the survey and interviews, but found 3 big themes:

• The people responsible for Jenkins have usually worked (very) hard at gathering the necessary knowledge to support their teams, but feel like the constant firefighting leaves them with no time to spend on meaningful improvements to the team's CI / CD setup
• ex-Jenkins maintainers spend more time on building and improving workflows after switching to cloud CI, but see way more results in team performance for that time spent (a whopping 90% reports that their teams are faster and more productive after migrating)
• Turns out that - in most cases - those years 'learning Jenkins' won't go to waste:

Hiring: A Jenkins Expert to manage our Bitrise / CircleCI / Travis setup

Even though a little under 14% of those we talked to thought their Jenkins expertise made it more difficult to find their way around cloud CI, the majority disagreed. Almost 60% were able to use their Jenkins expertise to get started with cloud CI quicker. 34.1% even thought their time on Jenkins gave them a significant edge, and were able to see results sooner, and make a bigger difference in their team's performance.

Bonus fun fact:

Everyone saw that they were gaining a lot of time to spend on things other than fixing broken workflows on Jenkins. Only 8.7% of the folks we talked to spent that time on 'rest and relaxation', tho. The rest of them opted for things like creating new features, building and improving automation, and a range of - arguably more productive - efforts.

In conclusion

Jenkins is fine, the people that are responsible for Jenkins in their organisations are usually great, and when you give them cloud CI (like Bitrise) to play with, they're amazing.

Download the full report if you're interested in our other findings, and contributions from folks with impressive titles at cool companies. Stuff like WeTransfer, Nextdoor, Signify (the folks of the Hue lightbulbs!), and more. You get the picture 😉