Get started fast!
The vast array of choices when sketching out your backend architecture can be bewildering. The decision fatigue is real.
With every tool you pick, you have to wonder about community support, edge cases it doesn’t take care of, whether there are established best practices for it and more.
Most early-stage, web-based startups can get away with a fairly similar architecture for the first year or two. You will have an application layer, an asynchronous tasks layer, caching, a database service (start with Relational DBMSes unless you really know what you’re doing), a text-search service and S3 as an object store.
At TapChief, we’ve spent a lot of thought into getting these right. We tried Solr for search and realised it didn’t work out well for us. An earlier Redis service provider was finicky and expensive.
I’m listing out a set of tools that we use in-house at TapChief that have stood the test of time for us. Hopefully, it can be a good way for someone to jump-start their backend architecture.
Heroku
Heroku takes away a lot of the headache of deployment. Deploying to it is as simple as writing “git push heroku”, irrespective of how many instances (dynos) you have running for a given service.
It also has great add-ons for services like Redis, Postgres, and AMQP. There is hardly any lock-in to the platform and you should be able to move out easily as your codebase is not aware of Heroku at all.
Heroku is built on top of AWS so if you really need to, you could always keep some of your more expensive services running on AWS and have Heroku running other services in the same AWS region.
We deploy our services via Nginx and gunicorn with gevent enabled for extra-concurrency.
web: bin/start-nginx gunicorn -c config/gunicorn.py tapchief.wsgi --worker-class gevent --log-file -
Django
A boring stable framework with a ridiculously useful admin panel feature. If you are working at a start-up with back-office operations, you’ll really appreciate this.
Django operates as a shared-nothing framework which means scaling it is as simple as running more instances of it in parallel. And it’s old and popular enough that it has an ample number of well-tested libraries. StackOverflow is teeming with answers to help you out in case you get stuck.
You could use Rails or the Play Framework for your application too. Both follow a similar shared-nothing MVC type of structure. But the Django Admin panel seems to be a unique and useful feature.
Types
I miss Java here, I’ll admit. We’re a start-up, we don’t really have too much test coverage. Seeing runtime exceptions because we thought an object was a string when it was actually a list is painful.
What we have done is write types for all arguments in the docstring of most functions we use. We let PyCharm yell at us whenever it thinks we’re doing something wonky.
PyCharm
Use a real IDE. Something which lets you refactor effortlessly, tells you when you aren’t using coding conventions appropriately and highlights potential bugs. I’ve heard Visual Studio has gotten quite good and is much less of a memory hog than PyCharm/IntelliJ.
Celery
You’re going to need asynchronicity. Your application service should not be doing long-running tasks. Instead, it should offload it on to a queue and worker processes should pick up and work on these tasks in the background.
This is a critical part of how to achieve scale with your app.
Celery is also a great way to set periodic tasks. Want to send automated daily analytics reports on how many users have signed up? Celery has you covered.
CloudAMQP is a reliable provider on Heroku for the message broker (queue) part of this system.
Redis
Caching is another key part of the strategy to build a scalable and low-latency solution. Common read queries which do not change frequently can be cached to prevent them from causing database hits.
Keep in mind that your database is usually going to be the hardest thing to scale. Especially if you go for a relational store. So use your cache to “protect” your database from heavy reads.
RedisGreen. There are loads of add-on providers for Redis on Heroku. But this is probably the cheapest and with the most comprehensive metrics dashboard.
Postgres
Heroku Postgres. This is an add-on officially supported by Heroku. And it’s excellent. For $50 a month, you get daily automated back-ups, excellent metrics, roll-back ability, and a quick way to snapshot and download your database at any time.
Django plays particularly well with Postgres.
Django Rest Framework
Abstracting out your UI layer as Rest APIs is a better idea than using Django’s templating system. You can support multiple different clients this way. Django’s Rest Framework is a great choice here.
S3
Don’t serve static content from your backend. Application servers like Django are especially not designed for that. Nginx is still a reasonable option for this.
CloudFlare
Use a CDN like CloudFlare to cache and deliver your static files.
Angular? React? Vue?
We’ve used Angular here at TapChief. And we absolutely love it!
We like structured opinionated frameworks taking away the cognitive load of choosing which packages to use together. That batteries-included approach is why we like Django over Flask. And we’ve found exceptional success with Angular, Universal and NativeScript.
But somehow, the rest of the community seems to be rallying around React and more recently, VueJS. *shrug*
I’m really not sure what to pick here. (Pick Angular!)
LogEntries
This is a one-click easy and free solution to get logs of the past 7 days. Very easy to install and comes as a Heroku add-on for any service you host there. There should be paid plans with more extensive logging options but we haven’t needed to explore them all that much.
ElasticSearch
Search is one of the first things you should offload from your application server and primary database. And ElasticSearch provides a really good DSL for Python.
Bonsai ElasticSearch on Heroku has been an easy way to scale and reliable option for us.
Scalability
This is often conflated with concurrency. A (horizontally) scalable system is one in which adding more machines should lead to a linear increase in the workload the system can handle.
Django is excellent here since it is stateless. One Django instance is not enough for all your requests? Well, just add a couple more behind your load balancer. (Pssst.. this is super easy with Heroku).
In terms of concurrency (requests per system), a Go-based REST API Service will probably do better. But you can get pretty far with Nginx buffering your requests and using gunicorn with gevents enabled.
Of course, the real bottleneck here ends up becoming the relational datastore you use. These don’t scale out as well as their NoSQL counterparts can do. It is entirely possible to use Django with NoSQL stores to tackle this.
But make sure your system load is heavy enough to warrant the additional complexity that NoSQL systems can add.
Abstractions
Have you heard of the fundamental theorem of software engineering?
Indirection. Most issues in software occur since one layer makes certain assumptions about another layer.
Okay, fine, it’s not a real theorem, but it is remarkable how many problems in Computer Science can be solved with an additional layer of indirection.
Your goal as an architect is to keep the system flexible.
The classic Django MTV architecture assumes a relational store accessed via an ORM. Fat models, i.e. business logic coupled with objects generated via the ORM are commonplace.
What if you need to use NoSQL for some of your heavy transactional objects later? It would be a pain to move away all that business logic.
Instead, we can keep a Service Layer between the Views and the ORM. And our business entities could be plain Python objects. (POJOs as they are called in the Java world).
You access them via an interface and it doesn’t matter if they are originally from the ORM or another adapter you built out to work with NoSQL stores.
Is this overdoing it? It really depends on your use-case. This works if you’re sure you’ll need to handle that sort of work-load extremely quickly.
If you can keep going with Postgres and Redis for a year or two, you probably shouldn’t bother with this. You can just slowly abstract out the entities you want to move over to a NoSQL store. It’s unlikely that you will want to move your entire database to NoSQL anyways.
You’ll notice that this has been a very monolithic architecture. A service-oriented architecture is probably where you’ll eventually be headed but starting with that up-front usually slows down development at a time where you should be aggressively trying to find product-market fit.
Be mindful of the fact that you may have to migrate to an SOA, and keep different concepts as separate as possible. This way, you’ll be able to peel out different entities into their own services later on.
Read Hackernews
Google “hackernews golang vs java”. You’ll see quality discussions between folks who’ve used different technologies in the wild.
Hackernews has built a great community of people who build things. People share their experiences with new languages, frameworks, and technology. Think of it as a better behaved Reddit for technology.
An interesting discussion on Hackernews about picking the “right” framework.
I know this isn’t really a part of the architecture, but the discussions help shape your understanding of the trade-off of various tools.
It can be mind-boggling trying to fit all the pieces together to build a scalable architecture. But that’s the really fun part of backend engineering. The tradeoffs of shipping quickly v/s the “right” architecture. What and where to cache. When to move over to NoSQL stores. How much technical debt to accumulate in order to get product clarity.
Running your own start-up? Tell us what stack/tools/architecture you use. It’s always comforting hearing that someone else is using the same tools as us!
Want a less tool-oriented guide on designing your technical architecture? You should check out Clean Architecture by Robert C. Martin.
Or for a quicker overview of backend architecture in general: https://github.com/donnemartin/system-design-primer