Building our own Containers, but WHY ? So, you've tried to get to grips with building your own containers but you are in tutorial hell and nothing makes sense. It is finally time to ship your app and you need to understand and own the process. Building your own containers empowers you to ship to your own cloud or someone else's that needs your apps to be containerised. Or perhaps you’re one of the folks that say "I'm done with docker" and rely heavily on other people's SaaS, BaaS to package up your apps for you, however, the bills are coming in. And they are unexpected ! I hope to break down my approach to building containers and to pass on my experiences so you don't have to learn the hard way. Where I'm coming from Back in the 2010's containers were referred to as 'Docker containers'. They still are. As in internet search, we say ‘Google’ to mean search. Likewise, containers were given the company name that was dominant in the day. I was fascinated and eager to adopt this then new technology. Containers are not a new thing, Linux Containers, LXC for example, and other approaches being around well before this. Docker containers promised reliability, repeatability an the end of 'it works on my laptop' arguments. I learnt and used Unix from the 90's and then Linux has become almost a replacement for Unix in peoples minds. I think that having at least more than a passing interest in Unix and how to install and run it must be a foundation learning for building containers. So if you don't have this interest, likely working with containers is not for you. The good news is that you don't need to have mastered Linux from Scratch, Gentoo, Arch or NixOS to understand containers. In time, these things can help. Trying to learn everything all at once however, can lead to overload and burn out. So I'd encourage taking it easy to started. Just focus on core concepts. I will focus for now on 3 container topics: entry point
process management
base image entry point process management base image Within base images, we’ll consider 3 main commands RUN
COPY
USER RUN COPY USER For all of the code in this article, you can see, download and run it all from a development branch on Codeberg. I'm coding in public a lightweight, headless Content Management System (CMS) download and run it all from a development branch on Codeberg coding in public a lightweight, headless Content Management System If you’re unable to find my latest Dockerfile check for a development branch. Likely this will be a part of the current dev branch if not main as the project reaches a stable state. Dockerfile dev main 1. The Container Entry Point. 1. The Container Entry Point. I think Entry Point is key to understanding containers. ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/docker-entrypoint.sh"]
CMD ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"] ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/docker-entrypoint.sh"]
CMD ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"] I've sneaked in CMD after ENTRYPOINT so as to clarify what confused me in the past. CMD ENTRYPOINT The entry point is like a kind of hook. It is called when the container starts up and it is often used, as it is here, to run the starting process for the container. In Unix, there is a single process that is started and from which all other processes run. You could think if it as being 'the one process to rule them all'. Its responsibility is to be the first, marshalling process, by which the entire operating system boots. In this example, we use tini which is a light weight process used in containers typically to do this as efficiently as possible. tini Tini runs an entrypoint.sh script in which we can include procedures and commands that must take place on application start. In this use case, should this step be missed, there would need to be manual intervention to: Tini entrypoint.sh run migrations
clear cache
apply configurations run migrations clear cache apply configurations CMD provides arguments for ENTRYPOINT to run. CMD ENTRYPOINT If we did not specify an entry point, we would have to at least specify CMD . CMD would then become the 'command to run' on container start up. So it would become the 'default' entry point. CMD CMD As you look around different containers, sometimes you see just a CMD command and nothing else and that is fine. In other cases you will see both. I found this confusing as many tutorials only talked about CMD ,  so I share this slightly more complicated pattern so you don’t hit the same issue. CMD CMD There is quite a lot going on then with this example of just 2 lines. The CMD line also calls supervisord and a configuration file. This is another layer of configuration and it is fair to say, complexity. CMD supervisord Supervisor is a tool that has been around for a while and is used to keep processes running in a sort of 'stack' or 'configuration'. The configuration file for this 'stack' looks something like this : Supervisor [supervisord]
nodaemon=true
loglevel=info

[program:php-fpm]
command=/usr/local/sbin/php-fpm -F
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:nginx]
command=/usr/sbin/nginx -g "daemon off;"
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:queue-worker]
command=/usr/local/bin/php /var/www/artisan queue:work --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 [supervisord]
nodaemon=true
loglevel=info

[program:php-fpm]
command=/usr/local/sbin/php-fpm -F
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:nginx]
command=/usr/sbin/nginx -g "daemon off;"
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:queue-worker]
command=/usr/local/bin/php /var/www/artisan queue:work --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
startretries=3
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0 The above configuration is running 3 programs for us as 'services'. You could think of supervisord as a kind of 'service manager'. It manages each program to auto start and will restart any of these programs should they fail. supervisord Logs are echo'ed to standard out. This is a convention in container land and accepted practice - this is so that other systems can easily gather and collate logs from all of your containers in the one place. 2. Process Management: The Elephant in the Room Ah, I hear some veterans of the dark arts of containers say, you have committed the mortal sin of not having 1 container per service. This should have been broken up into at least 3 containers ! And you would be right to say this. So long as you are Google or of similar size but I am not. As a solo programmer / entrepreneur I am trying to keep things stupid simple. If I can run one container for my simple app that needs 2 things aside of Laravel to survive, I can reduce my operating costs by having a single, multi function container. Within a single container, each process has access to other running processes. The web server for example, has full access to the PHP Laravel applications, their relative file systems, environment variables and even their process ids. If each program were separated into 3 containers, each would need to be given extra cloud resource, Shared file systems, secrets and likely queue mechanisms to mention a few. Yet more can easily become requirements the more we split out and ‘scale’ our containers. All these additional options make sense when it is time to scale up the app and after you have your first 50 customers. All of these extra tools and patterns can easily be applied later. For now at least, I am happy with a 'fat container'. When it is time to, I can easily split each program into separate containers, container sets, deployments and so on, as we scale to the size of Google or Netflix. Meantime, I can save on some cash, keep my team down to 1 or 2 engineers to run a smaller stack plus pass on lower operating costs to clients. 3. The Base Image Container base images come in many different shapes and sizes. It is possible to create containers from 'bare images' with little more than a file system. It is not uncommon however to use more mainstream images based on typical Linux distributions such as Debian or RedHat. The following FROM statement at the head of a Dockerfile uses an Alpine OS image: FROM Dockerfile FROM php:8.4-fpm-alpine3.23 FROM php:8.4-fpm-alpine3.23 This PHP alpine image has tooling for PHP applications. I used Alpine as this tends to use less storage space than other operating systems such as Ubuntu, Debian, Rocky or Fedora. Alpine has a different package management system called apk. This differs to the more commonly known apt or dnf used by Debian or Fedora but nothing to fear as you cant resolve syntax differences in a few minutes with a competent AI. apk apt dnf Building applications with an image that is as close to your application requirements as possible means less work for you. I'm building a Laravel app, so with PHP-fpm I have all of the main tooling that I need. Having a base image, we can tweak what packages are added, perform simple shell commands and select a user to run the container as. The default with many containers can be the ‘root’ user which is not good for security if we can avoid it. Lets take a look then at 3 common commands that are used on base images to fine tune and extend the container to host and deliver our application: RUN
COPY
USER RUN COPY USER RUN RUN apk update && apk add --no-cache \
    zip \
    vim \
    unzip \
    && rm -rf /var/cache/apk/* RUN apk update && apk add --no-cache \
    zip \
    vim \
    unzip \
    && rm -rf /var/cache/apk/* This RUN command is formatted over several lines for readability using \ to escape newlines for readability. It installs 3 packages after getting a list of the latest available packages and after installing, clears its cache to save on space. RUN \ You can get much of the work done in a Dockerfile using justRun commands. Dockerfile Run As you can see, it is possible to compress many commands into a single RUN block. As each code block in a Dockerfile results in a ‘layer’ being created in the final image, you can reduce the amount of layers by combining blocks where possible. RUN Dockerfile COPY COPY cmd/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh COPY cmd/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh The COPY command is perhaps the most important as we use it to copy files from the build environment into the container. COPY This one takes the entry point script we eluded to earlier and places it into its final resting place in the container. It is common to copy entire directories with something like COPY . . COPY . . which would copy the entire working directory  where the Dockerfile is located into the container. It is common to combine this with a .dckerignore file, similar in form to that of a .gitignre file when using git source control. Dockerfile .dckerignore .gitignre USER USER www USER www Finally, the 3rd of our main commands USER switches the container to run as a user different to that of root. In our Dockerfile I have taken care to create files and directories over which this user has sufficient access to both read and where necessary, write to. USER Dockerfile See the working code for more and also, the build script cmd/build_prod_container.sh which does what the name suggests. working code cmd/build_prod_container.sh Summing Up At the time of writing, the Dockerfile used for my Laravel application is built to be 'fat' - having multiple programs running in the one container and manage by a supervisor. Dockerfile This can, and will be changed as time and use changes. I hope you can see how a single container can do a lot and deploying a typical web application can be relatively simple and repeatable. If using supervisord, a Python application, other alternatives could be used to reduce image size and increase efficiency. A supervisord Golang port could be used in its place. Another alternative could be s6 overlay, a suite of c applications that are optimised for containers. supervisord Golang port s6 overlay The image created by the Dockerfile in my share-lt application is quite big - currently over 900MB and I have been able to reduce this in similar projects by: Optimising layers within the Dockerfile ( by rationalising RUN statements
Replacing python applications like superviosrd with compiled options
Using multi-stage builds in the Dockerfile - to reduce the amount of build tools in the final image Optimising layers within the Dockerfile ( by rationalising RUN statements Replacing python applications like superviosrd with compiled options Using multi-stage builds in the Dockerfile - to reduce the amount of build tools in the final image And you will likely see these and other changes depending on when you read this article and check out the repo. Already the share-lt application is in a more advanced state than only a few weeks ago and I continue to add features and tune as I go along. As with my previous article, this one also is written using the same application and is now running in a container deployment that we discussed above. Speaking of deployments, this container can be deployed using docker, podman, swarm but ideally kubernetes and of course, I am using my own, minimal viable kubernetesand will write more about my experiences in this regard in future posts. my own, minimal viable kubernetes Let Me Know So what about you ? Have you started your path to to container nirvana or are you undecided ? Are you already a master of all things container and have a few tips and tricks you can share ? Let me know - contact me on Linkedin or my other socials. I hope this helps you get on the road to container based deployments that save you time, effort and money in the process. Linkedin

This story contains new, firsthand information uncovered by the writer.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Why I Started Building My Own Containers (and What I Wish Tutorials Told Me)

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Coding in Public With Filament: Building a Minimal CMS on the TALL Stack

The Noonification: How to Implement a Merkle Tree in Solidity (11/19/2023)

101 Stories To Learn About Cloud Infrastructure

10 Things to Think About When Choosing a Hosting Provider

4 IaC Services For Your ML Infrastructure All MLOps Leaders Should Know

4 iPaaS Use Cases for 2023

Coding in Public With Filament: Building a Minimal CMS on the TALL Stack

The Noonification: How to Implement a Merkle Tree in Solidity (11/19/2023)

101 Stories To Learn About Cloud Infrastructure

10 Things to Think About When Choosing a Hosting Provider

4 IaC Services For Your ML Infrastructure All MLOps Leaders Should Know

4 iPaaS Use Cases for 2023

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps