The trouble with Software Innovation: from past to present
Once upon a time, a bottleneck on software development innovation was access to computers.
Over time, as access to computing became prevalent and software industry became a big part of the economy, the bottleneck on software development innovation switched from access to compute (since computers became ubiquitous) to the new type of development: in pursuit of profits, corporations started to restrict open access to software.
To counteract this new development, a very important movement was started, led by people like Richard Stallman.
For those who need a refresher: Richard Stallman leads the Free Software Movement, which shows how the usual non-free software subjects users to the unjust power of its developers, plus their spying and manipulation, and campaigns to replace it with free (freedom-respecting) software.
In his essay on “Why software should be free”, Richard Stallman argues that software *development* should be viewed independently from software *distribution* or *modification*. He presents several reasons why preventing software distribution or modification is harmful to society: fewer people can use it, none of the users can adapt or fix the software, and other developers cannot learn from the software or create new work on top of it.
What Richard Stallman emphasizes in his writings is that the value transfer between consumer and producer should be identified, quantified and made transparent so resources can be allocated accordingly.
For example, Facebook makes $fn(x) per user based on their data footprint, and the user benefits $fn(y) from Facebook for having available and reliable services for social interactions, according to the amount of usage by the user. If Richard Stallman were to design such a value transfer, Facebook should be paying the user some marginal amount for each interaction where they glean user data and the user should be paying Facebook some marginal amount for each interaction.
The vision was good, but the reality, unfortunately, didn’t exactly pan out the way Richard Stallman envisioned it.
Reality: not only code is closed, but the state is also closed
Unfortunately, the open source movement didn’t exactly win (for now). Even if open source as an industry is bigger than ever, most of the code is still closed due to the superior returns the software creator can extract from the ecosystem by prohibiting the free use of their software.
Importantly, as the software industry evolves, something else becomes even more important than the software itself: the state of the system. “State” is information about application users and the web service itself that gets constantly updated as users use the web service.
As Denis Nazarov mentioned in his essay on “What Comes after Open Source?”, the transition from personal software tools pre-Internet (i.e. Excel, PhotoShop) to web services (i.e. Spotify, Netflix, Uber, Instagram) led to one crucial difference: instead of users maintaining their own state, a web service maintains state on the users’ behalf. This, in turn, led to a consolidation of control in the hands of services that accumulated state data (i.e. databases holding all of the information about users using the service).
Over time, the code that a web service is built on becomes less important than the state of the service. As more users produce more state, it leads to a more useful service that attracts further users and, as a consequence, even larger state. It becomes a virtuous (for the provider of service) cycle, resulting in more control in the hands of web services.
For example, Spotify serves better music recommendations as its algorithm gets more music listening data, leading to even better recommendations and thus better user experience. As a result, more users join Spotify, which in turn generates even more data for Spotify, which then further improves the product and the user experience.
It’s a flywheel: the more of the state data you have as a web service, the more value will accrue to you in long term. If you want to understand why Facebook-Amazon-Netflix-Google companies are so powerful, that’s the answer.
2019: Is Innovation Dead?
This leads us to today, where both consumer web services and cloud services are in fact pretty close to oligopolies. It’s bad for the end consumer and for market innovation.
When it comes to the consumer web, as Ben Thompson wrote his in aggregation theory:
“The internet has fundamentally changed the plane of competition: no longer do distributors compete based on exclusive supplier relationships, with consumers/users an afterthought. Instead, suppliers can be commoditized leaving consumers/users as a first-order priority. By extension, this means that the most important factor determining success is the user experience: the best distributors/aggregators/market-makers win by providing the best experience, which earns them the most consumers/users, which attracts the most suppliers, which enhances the user experience in a virtuous cycle.”
What it also means is that data saved in closed state systems leads to better user experience, further helping the dominant service grow their user base and thus grow their data. At the time of this writing, I have 11 Google apps, 7 Amazon apps and 4 Facebook apps on my phone.
When it comes to the consumer web, there is a clear centralization of power going on.
Not only is the consumer web consolidated in hands of few large players, but the same situation can also be observed in the enterprise software space.
Ben Thompson recently observed that AWS, Azure and other cloud infrastructure services will eat open source companies alive. MongoDB is an example, but you can imagine that there will be a lot more casualties in the future. AWS and Azure will eat up the infrastructure market, while Salesforce, Workday, and ServiceNow will eat up the enterprise applications market.
What’s important here is distribution: while these vendors benefit every day from using open source software, they are consolidating power due to their superior distribution. They win using the same formula as their consumer web oligopolist peers are using: the more customer data they have, the more products and services they can up-sell and cross-sell in the future.
Where does it leave us?
We have a consumer web controlled by companies that I can count with the fingers of a single hand, and we have an infrastructure cloud increasingly controlled by roughly the same amount of vendors.
Given this consolidation, innovation outside of these companies is hard to come by.
If you are an aspiring entrepreneur in 2019, it will be very difficult for you to get users, regardless of whether you’re working in the consumer web or enterprise software space.
The Way Out
There is a new technology that will unlock the bottleneck on software innovation.
What if we take Richard Stallman’s ideas about open access to software and make them a reality? Not only this: what if we also make state data open and available for each web service?
There is a combination of incentives and technology that can enable such a world.
Imagine a new kind of computing.
Per Vitalik Buterin’s definition, this type of computing is both architecturally and politically decentralized, but logically centralized.
It is architecturally decentralized because this new computing is made of many computers talking to each other. It is politically decentralized because many individuals or organizations control the computers that the system is made up of. It is logically centralized because there is one commonly agreed state and the system behaves like a single computer.
What are the advantages of having such a decentralized computer?
- It would allow users to see the source code that’s being executed on the back end
- All public data would be openly available to everyone
- Most importantly, it would allow for the biggest wave of innovation that the software industry has ever seen because all of the open services would be talking to each other and exchanging data
Let’s call this last property “composability of services”.
Composability
If we take the analogy of Richard Stallman’s approach to open source software and apply it to open services, the world will be vastly different from what it is today.
Per Stallman’s writings, it is important for developers to cooperate and reuse the code openly.
If we apply this thinking to web services, it would mean allowing different services to use each others’ state, in turn enabling developers to reuse the web services, make them talk to each other and improve on each other’s work.
Let’s see how it might work.
Example #1: Open Services for Censorship Resistance
Imagine a driver driving for 100 trips on Uber and gaining a stellar reputation. Imagine a scenario in which Uber censors the driver due to a bogus report, completely deleting his reputation data in the process.
If OpenUber exists and is built with open data so the service talks to other similar open services, this driver now can carry his or her reputation to OpenLyft, OpenInstacart, OpenDoorDash and other web services available without any additional registration, KYC, or, more importantly, without any risk of losing the reputation. This allows all of the services to be interoperable and reusable for other interfaces and businesses, allowing the reputation of users to be also portable. This also saves users from subjective censorship.
Example #2: Open Data for Better User Experience
Imagine a world in which the end user owns its data. The user can provide access to his or her data in exchange for customized experience in return, thus allowing for a mutually beneficial relationship.
For example, I provide my shopping data from OpenAmazon to OpenNike.com, thus allowing OpenNike.com to customize my shoe shopping experience based on the previous shoes purchasing history available through OpenAmazon.
In such a world, I own the data, not the corporation I am working with. I provide my data access in exchange for differentiated user experience. It’s up to me, not up to the corporation.
Conclusion
That sounds like a future I want to be a part of.
In order to bring this future to the present and make it a reality, we must create a new computing paradigm that would allow building scalable, decentralized and open services within minutes with intuitive development experience.
As I mentioned earlier, the new type of computing, which is decentralized architecturally and politically, and yet is centralized logically, is something that can enable such a future. This future will unblock the current bottleneck in software development innovation.
There are several projects focused on working towards the future outlined in this article. Solid led by Tim Berners Lee from MIT is building a lot of fundamentals. Another example is NEAR that is building an infrastructure for open state applications, and researches ways to make such applications easy to develop and use, as well as have meaningful business models.
We hope that this article will spin up more projects in the area.
