What the Log4j Incident Means for Open Source and the Entire Internet

Photo by Martin Sanchez on Unsplash

This article discusses the log4j incident, why people are worried about the open-source software (OSS) supply chain, and how to work towards fixing it.

The Spark: Log4Shell

Last week (Dec 9th) a major vulnerability was discovered in an open-source logging project for Java called log4j.

The vulnerability called Log4Shell would allow anyone to remotely run arbitrary code if they sent a message in the right format to the server. This is one of the worst attacks your system can be susceptible to and if you are interested in the technical details of the problem, here is an overview.

The attack surface of Log4Shell is staggering. Amazon, Apple, Google, and the Apache Server are affected; it can almost not get bigger than this.

We will see the real fallout of Log4Shell in the upcoming weeks and months as right now servers worldwide are being scanned and prodded for this vulnerability.

Since there have been many supplychain attacks recently, the whole conundrum sparked a debate in the OSS and infosec community: Many believe that the OSS ecosystem is broken, maintainers need to become more professional and make OSS maintainer a real job.

Some argued that in this case the problem was not that maintainers were unpaid, burnt out, and taken advantage of, but more how this particular feature was implemented in log4j (Note: Maintainer burnout is still a real and significant problem for security).

Others insisted that open source is not broken - society and capitalism are the real culprits and everyone involved in OSS knows what they are getting into.

Open source as a model of distribution, development, or business is not a model of either a dystopian nightmare or a utopian dream. Every project is different and there are no silver bullet solutions to sustainability.

Open Source Maintainer as a Real Job

It is a real problem that software engineers maintaining critical software infrastructure used by governments and corporations worth billions are not able to make a living off of it.

Maintainers often can only work on OSS in their free time. This is fine for a pet project, but critical infrastructure projects, such as Log4j, should be more resilient.

There should be some type of collective fund set up. Enough donations to be able to work on their projects full-time are likely a tiny fraction of all open source maintainers.

In a perfect world, everyone who is maintaining such an important piece of code can do it full time and with adequate compensation. But this is not a perfect world. The best we can do is work on securing each link in the chain.

Sponsorships

GitHub sponsorships and Open Collective are a good start, but not enough to sustain infrastructure development. For example, the Ory ecosystem (most notably Ory Hydra) - used by billion-dollar companies and securing >30 billion requests per month - has received $22k on Open Collective over the last six years.

That is not a small amount compared to what most other OSS projects receive.

Still, if split between the two original core maintainers(@aeneasr and @zepatrik) it would amount to about 150$/month over the years, which is an absurd amount for a full-time maintainer that requires a deep level of expertise in security, cryptography and web infrastructure - not counting the additional maintainers that have been added to the project since its inception.

Towards Sustainable Open-Source Maintainership

Making a living off open source software and being able to work full time on it is a dream for many maintainers.

At Ory, we are working hard to make this dream come true. All our open source packages are now led by maintainers paid full time for their work.

What About Dependencies?

Dependencies play a major role in the saga of the Log4j vulnerability and security complications in general.

It is mind-boggling how big ‘dependency trees’ can get, in many cases, people had no idea they were even running Log4j between the thousands of dependencies in their stack.

Ory depends on many software packages so it is also in our and our users’ best interest to ensure a secure and hardened OSS supply chain.

Conclusion

“Open source isn’t broken. It’s working exactly as intended, and it’s by far the most powerful force in the technology world, and it will outlive any of the corporations so many people bend over backward to please today.” (source)

Fund Open Source Software

• How to pay for free software.
• Back your stack analyzes the codebase and shows OSS dependencies and where to fund them.
• GitHub Sponsors, the same principle as above on GitHub.
• How to talk to your company about sponsoring an open source project.