Open Source Worse Practices: Serverless Inc. as an Example of What Can Go Wrong

What can go wrong when you don’t keep up with your developer community

We Germans have a charming expression which I love to break out from time to time: “stiefmütterlich behandelt”. Essentially it means “treated like an unloved stepchild”. It came to mind while I was poking around in the Serverless Inc’s GitHub repositories. I’m referring specifically to their repo for the Google Cloud Functions integration. It was heartbreaking.

“Is this provider still alive?”

This is the title of an issue filed on January 10th 2020, in which GitHub user “gretro” writes:

“I know Google Functions are not the most popular, but you built something very useful and I’m would like to contribute, but if the library is dead, please tell me right now so that I can either abandon the idea or fork the project and adopt a much faster rhythm of release.”

No answer came the stern reply.

But before I speculate on what’s going on with this repo, I need to back up and explain what lead me here in the first place.

It all started with a mail from Google

The subject line was “[Action Required] Please upgrade to Cloud Functions v1 API by April 15, 2020”. My sysadmins had forwarded it to me. “What’s this all about?” I thought.

“We are writing to remind you that your projects have recently made calls to the Cloud Functions v1beta2 API. Please upgrade to Cloud Functions v1 API by April 15, 2020. Google Cloud Functions v1 API has been available since October 2017.”

The message was referring to a Slack integration that I had built. I’d used webhooks to trigger a Google Cloud Function which in turn sent a notification to Slack. And I’d used the Serverless Framework to deploy the whole thing. I checked the serverless-google-cloudfunctions repo to see if they were working on a fix.

Indeed, there was a PR to fix the very issue that Google was warning us about.

But it was a year old and still not merged. The PR contained approximately 163 changes across 4 files. OK, not a tiny change but also not a gargantuan effort to review either. Are we expecting too much?

To answer this question, it’s worth delving a little bit into the history of Serverless Inc. and their relationship to the open-source community.

A short history of the Serverless Framework

The Serverless Framework started life as an open-source project called “JAWS” (Javascript Amazon Web Services). It was first introduced at the Amazon re:Invent conference in 2015 as “JAWS: The Monstrously Scalable Serverless Framework”. At the end of 2015, they later adopted the more SEO-friendly moniker “Serverless” and moved to the domain serverless.com.

The following year, they managed to attract significant investment. According to Crunchbase, Serverless Inc has raised a total of $13M in funding over 2 rounds. They announced their first seed round of $3 million in October 2016. In an ebullient blog post, founder Austen Collins announced the funding and applauded the open-source community for their contributions to the framework.

“This fundraising wouldn’t have been possible without the power of open-source and the amazing community of developers who have submitted countless lines of code to make the Serverless Framework what it is. Therefore, our first order of business is simply to invest in more open source.”

You can probably guess where I’m going with this one. The open-source community is woven into the success story of the company. In the Serverless Culture Handbook, one of their stated values is “Open Source Forever”

“The open source community is an integral strategy to our success. We will never forget where we came from and pledge to always give back.”

But the open-source community has many factions and so does the FaaS landscape. If Google Cloud Functions is the unloved stepchild, Amazon Lambda is beloved biological firstborn. Indeed, the entire Serverless Framework evolved on the back of Amazon’s early adoption of serverless technology. Is it any surprise that their Google integration receives no love?

Not if you believe Austen Collins’ optimistic blog post from back in 2016.

“AWS Lambda is the dominant player in the world of serverless computing, and rightfully so considering its level of maturity. Our main goal is still to provide the greatest possible developer experience for AWS Lambda. However, our new secondary goal is to review the current options for serverless computing, determine which ones give developers the most value (and the best “serverless” experience), and support them within the Framework.”

In line with their secondary goal, Serverless Inc. eventually added support for Google Cloud Functions (as well as Microsoft Azure and IBM Cloud among others).

A timeline of the Google Cloud Functions support

Serverless Inc initially announced their support of Google Cloud Functions in May 2017, as part of their v1.14.0 release. Back then, the Google Cloud Functions offering was still in beta.

Then, the following year, Google announced the general availability of Google Cloud Functions at the Google Cloud Next conference. Serverless Inc. lauded this announcement with another blog article “Google Cloud Functions goes GA: what it means for Serverless”. In this article, you start to see a shift in the way that Serverless Inc. describes its offering:

“The Serverless Framework is cloud-agnostic development framework that makes it easy for developers to build serverless applications on any FaaS provider.”

Notice the term “cloud-agnostic”. All the kids are now going to be treated equally. And the Google integration is going to get some more love (or so we’re told).

“The Serverless Framework has supported Google Cloud Functions for over a year now, and we are already moving to release an update to work with Google’s new APIs.”

This perhaps alludes the fact that Google officially deprecated the Beta version of the Cloud Functions API after the general availability announcement. So Serverless Inc. knew there was work to be done. But it didn’t get done.

More investment and a new strategy

Fast-forward to May 2018 and another, much larger Series A round is announced — $10M with the help of Lightspeed Venture Partners and Trinity Ventures. This time focus is more on the beta release their new Serverless Platform which provides more advanced monitoring and CI/CD functionality. It has a free option but you can get more functionality if you’re willing to pay.

The closing statement of this announcement indicates the direction in the years to come:

“With our Series A funding from Lightspeed Venture Partners and Trinity Ventures, we plan to double down on our efforts to expand the features of the Serverless Platform and continue actively contributing to our Framework, Components and Event Gateway open source projects.”

So the Serverless Platform becomes the new favourite child. Which we get. Project A is a VC. We understand the pressure that ventures are under to increase their value. But to neglect a section of the community which helped you build your value proposition in the first place? If one of our portfolio companies behaved like this, we’d be concerned.

I wondered if there was any commercial pressure at all to invest more in their Google Cloud Functions integration. Were there any major players using it? I scanned the 12 case studies listed on the Serverless case studies page to find the answer. The only thing I found was the case study for the Shamrock Trading Corporation. Even then, they’re primarily relying on Amazon’s infrastructure. Shamrock uses Google Cloud Functions as a secondary service for image recognition but the details are scant. The author of the case study prefers to talk about what they’re doing with Amazon.

The obligations of an open-source maintainer

Let’s cut back to the PR that started this whole discussion. After it’s submitted, more GitHub users start to express their concern. A month has gone by and there’s still no word from Serverless.

The cries begin to get frantic as Google’s deadline for migrating to the V1 API gets closer and closer…

The screenshot was taken on the 31st of March.

Finally, on the 22nd of March, someone from Serverless responds and the PR is merged a few days later. Over a year after the PR was created.

Is there any scenario where this behaviour is acceptable? And why is this such a big deal?

First off, I know it’s not self-evident that a maintainer is going to merge every pull request. Ralf Gommers, a regular contributor to the Python modules NumPy and SciPy wrote a revealing article about the cost of an open-source contribution.

“I can easily find NumPy or SciPy issues that I’d rather not see a fix PR for. They’re valid bug reports, but the value of fixing them is minimal and the costs are high.”

But Mathew Rocklin, another major contributor to the Python’s numeric computing ecosystem, writes about the role of maintainers who are “paid to maintain the project some modest amount time” and outlines some best practices. First on the list? “Timely Response”

“When someone is raising an issue or contributing a patch, try to give them a response within 24 hours, even if it’s not a very helpful response.”

Let’s be clear on that — it doesn’t have to be a helpful response. You can just write “I’ve seen this issue”.

Of course, maintainers have no legal obligations here. In his post titled “Open Source Maintainers Owe You Nothing”, Homebrew maintainer Mike Mcquaid talks about the implications of open-source licenses (the main serverless repo and the serverless-google-cloudfunctions repo both have MIT licenses).

“The maintainers provide no assurances that the software will ever work for any user or use case (even documented ones).”

Basically, he says you can’t sue an open-source project maintainer if the software breaks and your production systems stop working. OK, fine. But we’re not talking about legal obligations here. We’re talking about keeping a healthy relationship with the open-source community.

Nadia Eghbal a writer at Substack (and former GitHub employee) reflects on the ways to measure the health of an open-source repo in her essay “Methodologies for measuring project health”. While many people look at the number of contributors or the date of the latest commit, she proposes an alternative metric.

“While there’s room for improvement, I directionally prefer using some version of commits and/or PRs merged to measure project health.”

As we’ve seen, if we use her metric to measure the health of the repo for the Cloud Functions integrations, then it’s definitely not in good health at all. And that’s not good for Serverless Inc, not just for their relationship to the open-source community but to the wider business community too.

Why would a business pay for professional services if Serverless Inc. can’t support all their integrations? What kind of message does this send about their available resources and the level of support that a paying customer could expect?

A few days ago, I put these questions to Serverless. When I started researching this story, I signed up for a free account on the Serverless platform. Soon after registering, I got an automated email from a solutions architect asking “You didn’t deploy, how come?”. I answered the email with some questions about the ill-fated PR which triggered this story.

No answer …. came the stern reply.

A Disclosure Note On The Author and Project A

Stephan is CTO at Project A — a venture capital investor focusing on early-stage startups. Project A provides operational support to its portfolio companies, including developer expertise.

Previously published at https://insights.project-a.com/open-source-worse-practices-d3ccba536ff