Scaling Web Applications Is Like Walking a Tight Rope in a Storm

Scaling web applications is a skill. It’s like walking a tight rope, in the midst of a storm. There are multiple factors acting on you,

Speed
Reliability
Availability
Scalability
Time to market
Resources

Each will pull you in the opposite direction, you have to balance between them. You will need to decide what is the equilibrium point for your application based on your use case and resources available to you.

Currently there are lot of options on cloud that provide tools to help in deploying, managing, and scaling web applications. The tools can be broadly classified as (Note: Below analogy is just for easy understanding).

Saas (AWS Lambda, Google Cloud Functions, Google App Engines)

These tools build your application, and also set up, configure and run your application. All you need to do is provide the source. You do not need to worry about creating docker files, creating deployment yaml etc. This is used for quickly deploying your application. You forgo control in favor of lesser complexity.

Paas (Google Cloud Run, AWS Fargate, Heroku, Pivotal Cloud)

These tools are serverless container platforms, where you are responsible for the built artifacts (docker images). The deployment of your container is managed for you. It also provides auto scale abilities.

Iaas (AKS, EKS, GKE, Digital Ocean Kubernetes)

These tools provide Kubernetes as a platform, and you can run your containerized workloads on them. You will need to create and configure deployment yaml the instruct Kubernetes where to deploy your workloads. This is a good balance between control and complexity.

VM/Physical Servers (EC2, Google Compute Engine, Bare Metal Servers)

These only give you server/vms on which you need to install, configure, and manage Kubernetes. Here you have complete control, but also you need dedicated devops teams to maintain your infrastructure.

You need to select your appropriate platform/stack based on below below metrics.

Control
Granularity
Cost
Complexity

You can use a variety of higher level tools to manage your deployments in a CI/CD pipeline. These tools can be used with Paas and Iaas categories as defined above.

Containers: While Docker files are de facto for creating containers, you can use buildpacks to build reuseable, transparent docker images. Refer to Modularizing docker images.
Managing deployments: Helm is the de facto for easy packaging and deploying workloads on Kubernetes. kustomize can also be used. It provides manifests that can be used to template your deployment and reuse a base deployment and provide overlays (deltas) for each landscape
Managing clusters: To manage your cluster as a whole, you can look into Argo. This provides an automated way to update your cluster by monitoring your git versioned deployment files. Also while Kubernetes ensures that specified number of pods are running, if a deployment is decommissioned using kubectl, or any changes are made to the deployment, Kubernetes is not aware of these changes. So Argo monitors whether your running cluster matches the desired state.

As you can see, a lot goes into making sure your web service can scale up. So if you are starting out it makes sense to start with Saas category products. If this does not handle your workload, you will need to progressively hop to the more granular categories.