I just came across a really interesting article on medium discussing how a change in Firebase pricing resulted in a 7,000% cost increase for a HomeAutomation startup. As a co-founder of deepstreamHub, a realtime backend as a service, this landed very close to home. Therefore, I would love to provide a bit of insight into the challenges we faced and how we tried to overcome them when developing and running a Platform as a Service (“PaaS”) offering.
Let’s start with a bit of background as to why PaaS is such a booming business.
The dream of running any competitive business is offering lower costs. This is usually obtained by providing a multi-tenant system that allows your systems’ resources to be shared across multiple users. Pretty much the same premise as a holiday home timeshare or AWS infrastructure as a service. You think you have an entire machine, but you truly only have a few minutes an hour. The benefit to the user? You get to run your entire stack on less than $50 and you never need to worry about infrastructure issues like scaling, database migrations, security issues, ssl issues, and the list goes on… For the platform itself, we have the benefit of being able to distribute load effectively (meaning we can actually make a profit), specialise in the hard aspects, and concentrate on making the experience better with every single release.
Okay, so that’s coming from a guy at a PaaS company, so I can see how that might be viewed with scepticism. The general comments from developers indicate that maybe a better way to avoid vendor lock-in would be to host everything yourself. Makes sense right? You can run your own docker images using container orchestration services like Kubernetes and Docker Swarm, or reduce your buy-in with all the benefits of cloud offerings like AWS’ Elastic Container Service or Google Container Engine. You end up having no reliance on anything specific to a particular vendor, and switch to any other infrastructure based company the minute pricing or terms change.
All that is great, conceptually. The thing that people tend to sometimes forget to leave in the small print is that while you’re developing your application using a specific vendor, you may have gone into the thousands, your recurring costs on the platform itself, remains at less than $50 a month. That’s a pretty sweet deal.
As a guy who loves devops, let us break it down to how you can actually deploy the same services to power a realtime application. In full disclosure, I will be basing this on deepstream.io, the open source core that powers deepstreamHub. The requirements would be:
- Constant uptime
- A centralised cache to provide low latency
- A persistent database
- SSL, because it’s 2017 people
This means hosting deepstream.io on AWS would require a minimum of:
- Two machines in two separate availability zones
- A message bus to allow them to communicate with each other
- A cache layer to allow quick reads and writes to allow latencies around 16ms
- A database for persistent data
So that would look like this (as of today’s posting date):
That’s almost $50 with none of the hidden costs attached, such as the cost of throughput (that really does add up!), ALB units, the costs of setting up and maintaining it, etc.
Also keep in mind this is the smallest stack possible! A nano machine provides 512mb memory with 1 minute CPU bursts! That means your load needs to be really, really small for it to work.
Full disclosure, we use that stack in our development environment — brilliantly cheap for non production use cases!
Okay, I hope I still have you all with me! And I hope you all kind of agree that self hosting certain systems is just not as cheap as a PaaS might make it seem.
Which leaves us to the main and most important part of this post.
Is using a platform as a service a curse, or a blessing?
My personal opinion, obviously slightly biased, is that can be either! Being able to use something that is both open source and hosted for you should definitely be considered a blessing. You have people in this space like:
- deepstreamHub (based on deepstream.io)
- graph.cool (based on graphql protocols)
- baasil.io (based on socketcluster)
- meteor galaxy (based on meteor)
- AWS Lambda (I’ll get to that in a second!)
The benefit of this is, if something happens like a huge price surge, unwelcome new terms, or even a missing feature set you really want that found itself in a fork, there tends to be enough people self hosting in production to make moving to a self hosted environment almost as seamless as exporting your data from one place and importing it into another.
The curse is if it isn’t open-source your vendor lock-in can bite you in the rear. Some platforms like parse made that curse more bearable by open sourcing their core in advance before officially going offline, but as long as you can’t make that guarantee, there is always an associated risk.
Finally, as a few comments on the amazingly insightful blog post today:
Alex R. Young suggests: “Wouldn’t relying on AWS Lambda potentially cause the same problem? Why rely on any specific vendored service when you can run services in containers or VMs that can easily be redeployed to any of the big 3’s architectures?”
I have recently attended an AWS summit which heavily recommended building your application logic into a separate dependency. The entry points would be generic enough to run either from lambda, from a http server running on your own machine instance, a docker service, or in our case, even called from different protocols (deepstream.io RPCs).
keep in mind that the services you use can change at any moment and put you in a situation where you don’t have options if you aren’t careful.
Whenever possible, rely on your own infrastructure. The SaaS model seems attractive to both Startups and Service Providers… but in the end, it’s the Startup that gets bitten by it and the providers that make the real money.
Always heavily consider open source alternatives (something which didn’t exist for Firebase at the time…) but now alternatives like Horizon and Backendless exist, for example.
I hope I covered both of these well enough in this post, and would love to hear your thoughts on it!