How to build lean software, create more value through engineering and change your company culture forever
A pandemic hit us hard a few years back. At that time, every startup was building the Airbnb for this or the Uber of that. The technology industry was convinced that startups could only be worthwhile if they looked like a Platform; if they aspired to be the next Facebook.
Off the back of the successes of juggernauts such as Airbnb, Uber, Amazon, Facebook, and eBay, our industry was shoehorning every business case into a Platform business model. It became the default. Unfortunately, this neurosis is still ingrained in our culture.
If the description of your startup has the word “platform” in it, there’s a better description that doesn’t — Paul Graham
I used to hate it when people described their business as a Platform. A platform, by its very nature, is uninspiring in a business sense. Platforms are inert. They are static. And they do little more than just exist.
Even an oil-drilling platform, while impressive at first glance, is immovable, cumbersome and almost impossible to upgrade. What’s more, they can blow up at any time! Essentially they are doomed to be obsolete after sucking all of the value out of their market.
Not an inspiring vision for business.
The Platform Fallacy
The word platform doesn’t mean much anymore. We’ve been guilty of using the term loosely to describe any of the following: products featuring Application Programming Interfaces (APIs), Software-as-a-service (Saas), Aggregators and sometimes real Platforms.
The truth is, our startup business probably isn’t a Platform. Platforms derive much of their value from the communities they serve. If we’re just starting out and haven’t grown a community yet, then we’re not a Platform.
That’s a crock of shit. This isn’t a platform. A platform is when the economic value of everybody that uses it, exceeds the value of the company that creates it. Then it’s a platform — Bill Gates
But the harm is not in the mislabelling. It’s in obsessing over what has worked for others. It’s in getting sidetracked from our mission, which is ultimately: to provide the best solution we can afford to address our market’s pain points.
It’s time to be less platform-obsessed and more customer-obsessed.
Watching from the outside
The success stories of Airbnb, Uber, Amazon, eBay, Etsy, etc. have sadly skewed our understanding of what works and what doesn’t. We are observing Platforms from across the Internet, presumptuously drawing conclusions and speculating on uneducated recipes for success.
We’re watching from the outside. We have virtually no idea of the numerous growth systems that have been running in the background and we can only see the surface.
And what’s tragic about keeping a bird’s-eye view on Internet success stories is the eventual belief in one giant system — in One Big Software Thing. All of a sudden, our own moonshot is to create One Big Software Thing in order to be successful.
Alas, once we start believing we need One Big Thing, we stop aiming for that deep understanding of our tribe. We stop putting customer service first. We become political over our work, and we divorce ourselves from our mission, which we must always remind ourselves is: to provide the best solution we can afford to address our market’s pain points.
We cannot solve our problems with the same thinking we used when we created them — Albert Einstein
Before it’s too late, we must quit observing from the outside, avoid imitating the Big Things to imitate and start building from the inside-out.
Feeding the Monster
The worst part about the One Big Thing mindset is that we’re fighting an uphill battle. We’re stuck with a centralized architecture. We add onto it over time: dashboards, a convoluted back-office and — in a last-ditch attempt to handle everything — a Content Management System (CMS). What a nightmare!
This is a hazardous move for a startup. Why? Well, one can only design One Big Piece of Software after predicting all of the types of support it will need to provide in the foreseeable future! Without a crystal ball, that’s near impossible, and any forecast error will be at the expense of our agility.
When it’s too late, all we can do is coerce the market into our One Big Thing. We force-fit customers into it. We go from being market-driven to marketing-driven.
We must refrain from designing one-size-fits-all Big Things until there’s no other choice. Remember, we want our mission statements to be so ambitious that they will never possibly be fully served by one website, one application or One Thing. Our Company is the ultimate Product and it can’t be obliterated by One Software Thing.
If you want to make change, begin by making culture. […] Culture beats strategy — so much that culture is strategy — Seth Godin
A calm and profitable future — made of Pipes
Let’s go back to basics for a second. We’re in the business of customer state management. We take them from state A to state B, from stranger to happy customer. We add value by doing this and we charge accordingly. What we sell is the customer’s ability to change.
That change is progressive; it won’t happen if the customer is stagnating on a platform. Our customer goes on a journey, a journey we can picture as a Pipe. If the customer is a person, our Chief Marketing Officer would call our Pipes funnels. Marketers’ funnels end with the close of a sale, whereas our Pipes end with the delivery of the service to the customer and a complete change of state.
In a marketing or sales funnel, every lead is in a known state, anywhere from cold to closed. For a Platform-obsessed person, there are only two polarised states: signed up or not, user or visitor. A Pipe-obsessed person, on the other hand, has transcended that binary mindset and sees the myriad of micro-state changes while leads and customers flow down our Pipes: strangers in, happy customers out, and all the micro-states in between.
Pipes have been around us for as long as we’ve had industry. They’ve been the dominant model of business. Firms create stuff, push them out and sell them to customers. Value is produced upstream and consumed downstream. There is a linear flow, much like water flowing through a pipe — Sangeet Paul Choudary
Pipes are a much more natural way of thinking about our businesses, and yet, most of us still aim for One Big Thing. Great business owners, on the contrary, think and describe their companies in terms of “when a customer does this, then we do that”. They have a plan — and a system — for every required customer state change, to the point that it is all procedures and systems, across the company and the product. This delights the customer (and the prospect, but that’s for another discussion) and brings scalability to the business since it doesn’t laboriously serve its customers and can, therefore, focus on growth.
Our mission — as business-builders — is to define all of these when-then statements and lead our teams into shipping software, content, and systems — i.e. Pipes — that relentlessly embody these statements and move customers, one state at a time, down our overall service Pipe.
This must be more of a priority than creating a one-size-fits-all Platform or Thing. This is the top priority.
Of course, it’s not the Platform model that is broken. It’s when we try to make companies fit the Platform mold.
The Five Engineering Revolutions
Hang on, isn’t it harder to build Pipes (i.e. Many Things) rather than One Big Thing? Frankly, it used to be easier to think about technology as One Big Thing, as too many moving parts were cumbersome to deploy and operate.
That may have been the case in the past, but it’s no longer true today. Five major software engineering revolutions have changed the game.
1 & 2. The Cloud and Micro-services
The first revolution — the Cloud — is no news to us. We lease our servers from Amazon Web Services (AWS), Google Cloud Engine or Microsoft Azure as required.
Before the Cloud, we had to be very careful about how we treated our servers. We waited for weeks for the rare thrill of unboxing a brand-new Dell rack server. We memorized their IP addresses, as if to send them Christmas cards. We gave them names like pets. We would keep them for years.
It’s no surprise we wouldn’t let anyone touch our beloved “pet” servers. Developers weren’t allowed new deployments or to modify their configurations, nor to add to our infrastructure! A senior sysadmin (or team of) would act as a gatekeeper — or rather bottleneck — and our product release cycle depended on them.
With the Cloud, we can retire or hire servers at will. We treat our servers like cattle. We only pay for computing resources by the second. And as such, we let our engineers order new resources as they need them.
At the same time, engineers started architecting their backends in micro-services. Micro-services are small, autonomous services that work together. Along with many other benefits, they reduce the average codebase size of each deployable service, and therefore the risk in updating them.
Micro-services are a software development technique that structures an application as a collection of loosely coupled services
In short, deploying Many Things is easier on the Cloud.
3. API-first Products & Headless Software
What a cardinal change API-first products and headless software have given us.
Historically, platforms such as Apple, Microsoft, WordPress, Magento, etc., have been designed for us to build our entire business on them. These deep platforms want to lock us into their ecosystem. They want us to build complex Big Things onto them, Big Applications users will need to download, install and sign up to.
But in the last few years, more technology companies, not just platforms, opened up for programmatic integration to their products via APIs. API-enabled products — or pseudo-platforms — disrupt how we build technology in two ways_._ Internally, we can automate and run more of our systems on top of pseudo-platforms. Externally, we can deliver value to our customers on pseudo-platforms they already use: it is less demanding, it respects their existing habits and we meet them where they spend most of their time. Examples are API-enabled messaging apps like Messenger, Messages, and Telegram for B2C, and a plethora of API-enabled services for B2B. While we can’t build all of our businesses on these, with a healthy obsession for decoupling, we can now build parts of our businesses there.
We can have one Pipe on Messenger and one Pipe on Typeform. One Pipe on TransferWise and one Pipe feeding on our Xero data. A few pipes running out of our Customer Relationship Management (CRM) and a few other ones out of our customer service software. One Pipe on our company Slack account and one that supplies our Tableau and AirTable dashboards.
With Pipes, we can build the most appropriate parts of our business on the most appropriate pseudo-platforms. We’re more in control, we’re faster and we’re not putting all of our eggs into One Big Application.
But the biggest leverage from building Pipes on pseudo-platforms is delegating all of the heavy lifting to them: User Interface (UI), User Authentication (UA) and Security.
UI — including back-offices— and UA both make for expensive software. They need a strict definition of requirements, boring front-end development, careful data input validation, complex yet secure collaboration, and access control capabilities. The alternative, building Pipes on pseudo-platforms, offers state-of-the-art UI, User Authentication and Security layers out of the box, on our desktops, tablets, and phones, sometimes for free.
More than ever before, we can build our business logic layers as headless software and mandate the mundane parts to the pseudo-platforms. And that too makes building Pipes a winner.
Headless software is software capable of working without a graphical user interface. Headless CMS and headless e-commerce services are increasingly popular.
4. Serverless
Let’s consider what serverless means for us.
At the very beginning, in the Cloud, was the Virtual Machine (VM). We were allocated computing power out of a fixed pool of hardware, within a few minutes.
Years later, the Cloud brought us managed services. Most people were ordering VMs to install and operate the same piece of software, say a popular database engine. As such, the Cloud providers started releasing higher level services for the most popular use cases (for example database-as-a-service), with not only abstraction of the underlying hardware but also of the installation, maintenance, and scaling of the software (database engine in our example) running on the VMs.
More recently, the Cloud created serverless services. It now orchestrates the scaling of our workload on a software layer we don’t own, on a pool of resources we don’t manage. All we have to do is provide our code to be executed and pay per usage. Popular serverless services are Storage- and Function-as-a-service (respectively S3 and Lambda on AWS), but they also exist for databases, media transcoding, machine learning models, analytics etc.
Since we pay per use rather than for a number of servers, decoupled systems aren’t more expensive to host than mutualized systems anymore. A custom micro-service backed by a non-relational database costs pennies a month on AWS. Serverless makes it economically viable to think in Pipes.
5. Infrastructure-as-code
The last revolution is a total game changer for Pipes.
Public Clouds, being API-enabled themselves, have opened up new ways to manage our infrastructure, programmatically.
This astounding technology — called infrastructure-as-code (IAC)— is using the Cloud APIs to describe our infrastructures (servers, databases, firewalls, storage disks etc) with code (or any markup or configuration language). With IAC, it is easier than ever to manage Many Things. More importantly, it is now possible to clone them.
Popular IAC tools include Hashicorp Terraform, Serverless Framework and AWS CloudFormation
This fundamentally changes how we build software services, especially in enterprise software. Instead of building One Big Thing cluttered with complex code to support multi-tenancy or a CMS, we clone our solutions — or Pipes — for each new customer and fine-tune their deployment right there in the reproduced version, without touching our other customers’ deployments.
This is white labelling on steroids, and it’s achieved in seconds. It’s liberating not to write code for multi-tenancy, not to have to predict all the business cases our One Big Thing should later support. What a feat — to be able to simplify our code to that level. And we share this relief with our customers through affordability, resilience, and quality.
IAC makes Pipe-thinking a no-brainer.
Now the question is… When will our engineering teams master these groundbreaking paradigms?
The El Dorado: a future-proof engineering culture
No matter how it looks at first, it’s always a people problem — Gerald M. Weinberg
In the future, every company will be a tech company. And they will only build the right sort of technology — technology which is uncomplicated, Big Thing-free, and customer-obsessed.
We need a new standard, a culture of creating lean systems — or Pipes — as we go. And this is how we must train our people:
- First of all, we must get our engineers up to speed with no-brainer practices for managing code, releases, feature requests, bugs, and production infrastructure. We must ignite an automation mindset and enforce simple battle-tested procedures for everything from feature request to production deployment. I call this Minimum Viable DevOps
- We must get our engineers up to speed with the Cloud, and how to see clearly through its hundreds of services on offer. We must make heavy use of IAC. I call this Cloud Simplicity
- We must shift our leaders’ mindset away from the toxic One Big Thing culture, which makes it slow and risky to add features and will kill our products over time. We must nurture a healthy obsession for decoupling services and running on pseudo-platforms. I call this Lean Architecture
- Finally, we must master more advanced architecture practices for performance, availability, resilience, and security of our services. We must know the best practices for logging, monitoring, debugging and customer support. We need a full-stack engineering culture that includes customer service and sales. We must charge our engineers with ownership of not only the software quality, but also the service quality. I call this Operational Excellence
With such standards, we’ll create a leaner and safer culture of building future-proof technology businesses.
And with such standards, we will eradicate some of the challenges of entrepreneurship, our work will have maximum impact and we will only build technology that matters.
Lionel is Chief Technology Officer of London-based startup Wi5 and author of the Future-Proof Engineering Culture course. You can reach out to him on https://getlionel.com