PHP on Docker from Scratch in 2022

... with webservers, queues and databases Set up a docker development environment for PHP application from scratch. This article appeared first on at https://www.pascallandau.com/ Docker from scratch for PHP 8.1 Applications in 2022 [Tutorial Part 4.1] https://www.youtube.com/watch?v=NuSWKx9FSso in my .You find the branch for this tutorial at All code samples are publicly available Docker PHP Tutorial repository on Github part-4-1-docker-from-scratch-for-php-applications-in-2022 are collected under a dedicated page at . The previous part was and the following one is . All published parts of the Docker PHP Tutorial Docker PHP Tutorial Structuring the Docker setup for PHP Projects PhpStorm, Docker and Xdebug 3 on PHP 8.1 in 2022 Table of contents Introduction Local docker setup Docker docker-compose file and required ENV variables .docker/.env Images PHP images ENV vs ARG Image naming convention Environments and build targets Makefile includes .make/*.mk Shared variables: .make/.env Manual modifications Enforce required parameters Make + Docker = <3 Ensuring the build order Run commands in the docker containers Solving permission issues PHP POC Wrapping up Introduction If you have read the previous tutorial you might encounter some significant changes. The tutorial was published over 2 years ago, Docker has evolved and I have learned more about it. Plus, I gathered practical experience (good and bad) with the previous setup. I would now consider most of the points under as either "not required" or simply "overengineered / too complex". To be concrete: Structuring the Docker setup for PHP Projects Fundamentals on building the containers Setting the timezone not required if the default is already UTC (which is almost always the case) Synchronizing file and folder ownership on shared volumes this is only an issue if files need to be by containers and the host system - which is only really relevant for the PHP containers modified in addition, I would recommend adding a completely new user (e.g. ) instead of re-using an existing one like - this simplifies the whole user setup application www-data a lot from now on we will be using as the user name ( ) and the user id ( ; following the best practice to ) application APP_USER_NAME 10000 APP_USER_ID not use a UID below 10,000 Modifying configuration files just use - no need for a dedicated script sed Installing php extensions see - will now be done via PHP images apk add Installing common software see - since there is only one base image there is no need for a dedicated script PHP images Cleaning up didn't really make sense because the "cleaned up files" were already part of a previous layer we might "bring it back" later when we optimize the image size to speed up the pushing/pulling of the images to/from the registry Providing host.docker.internal for linux host systems can now be done via the magic reference host-gateway services: myservice: extra_hosts: - host.docker.internal:host-gateway thus, no custom entrypoint is required any longer Local docker setup The goal of this part is the introduction of a working local setup . In other words: We want the bare minimum to have something running locally. without development tools The main components are: the setup in the and in the directory make Makefile .make/ the docker setup in the directory .docker/ some PHP files that act as a POC for the end2end functionality of the docker setup Check out the code via git checkout part_4_section_1_docker_from_scratch_for_php_applications_in_2022_code_structure initialize it via make make-init make docker-build and run it via make docker-up Now you can access the web interface via . The following diagram shows how the containers are connected http://127.0.0.1 See also the for a full test of the setup. PHP POC Docker The docker setup consists of an nginx container as a webserver a MySQL database container a Redis container that acts as a queue a php base image that is used by a php worker container that spawns multiple PHP worker processes via supervisor a php-fpm container as a backend for the nginx container an application container that we use to run commands We keep the directory from the previous tutorial , though it will be split into and like so: .docker/ docker-compose/ images/ . └── .docker/ ├── docker-compose/ | ├── docker-compose.yml | └── ├── images/ | ├── nginx/ | | ├── Dockerfile | | └── | └── ├── .env └── .env.example docker-compose All images are via because the file(s) provide a nice abstraction layer for the build configuration. In addition, we can also use it to the containers, i.e. control volumes, port mappings, networking, etc. - as well as start and stop them via and . build docker-compose docker-compose.yml orchestrate docker-compose up docker-compose down FYI: Even though it is to use for both things, I found it also to make the setup more complex than it needs to be when running things later in production (when we are using any longer). I believe the problem here is that some modifications are ONLY required for building while others are ONLY required for running - and combining both in the same file yields a certain amount of noise. But: It is what it is. convenient docker-compose not docker-compose We use three separate files: docker-compose.yml docker-compose.yml contains all information valid for all environments docker-compose.local.yml contains information specific to the environment, see local Environments and build targets docker-compose-php-base.yml contains information for building the php base image, see PHP images file and required ENV variables .docker/.env In our docker setup we basically have 3 different types of variables: variables that of an individual developer, e.g. the on the host machine (because the default one might already be in use) depend on the local setup NGINX_HOST_HTTP_PORT variables that , e.g. the location of the codebase within a container's file system are used in multiple images variables that , e.g. the exact version of a base image hold information that is "likely to change" Since - again - we strive to retain a single source of truth, we extract the information as variables and put them in a file. In a perfect world, I would like to separate these different types in different files - but only allows a single file, see e.g. . If the file does not exist, it is copied from . .docker/.env docker-compose .env this comment .docker/.env.example The variables are then used in the file(s). I found it to be "the least painful" to always use the modifier on variables so that . docker-compose.yml ? docker-compose fails immediately if the variable is missing Note: Some variables are expected to be passed via environment variables when is invoked (i.e. they are required but not defined in the file; see also docker-compose .env Shared variables: .make/.env Images For and we do not use custom-built images but instead and configure them through environment variables when starting the containers. In production, we won't use docker anyway for these services but instead rely on the managed versions, e.g. MySQL redis use the official ones directly redis => or Memorystore for Redis (GCP) ElastiCache für Redis (AWS) mysql => or Cloud SQL for MySQL (GCP) RDS for MySQL (AWS) The remaining containers are defined in their respective subdirectories in the directory, e.g. the image for the container is build via the located in . .docker/images/ nginx Dockerfile .docker/images/nginx/Dockerfile PHP images We need 3 different PHP images (fpm, workers, application) and use a slightly different approach than in Structuring the Docker setup for PHP Projects: Instead of using the (i.e. cli or fpm), we use a "plain" alpine base image and install PHP and the required extensions manually in it. This allows us to build a common base image for all PHP images. Benefits: official PHP base images a central place for shared tools and configuration (no more need for a directory) .shared/ reduced image size when pushing the individual images (the base image is recognized as a layer and thus "already exists") installing extensions via is faster than via apk add a lot docker-php-ext-install This new approach has two major downsides: we depend on the alpine release cycle of PHP (and PHP extensions) the image build process is more complex, because we must build the base image first before we can build the final images Fortunately, both issues can be solved rather easily: maintains an repository with the latest PHP versions for alpine codecasts/php-alpine apk we use a dedicated target to build the images instead of invoking directly - this enables us to define a "build order" (base first, rest after) while still having to run only a single command as a developer (see ) make docker-compose Ensuring the build order ENV vs ARG I've noticed that some build arguments are required in multiple PHP containers, e.g. the name of the application user defined in the ENV variable. The username is needed APP_USER_NAME in the base image to create the user in the fpm image to define the user that runs the fpm processes (see ) php-fpm.d/www.conf in the worker image to define the user that runs the worker processes ( see ) supervisor/supervisord.conf Instead of passing the name to all images via build argument, i.e. define it explicitly under in the file services.*.build.args docker-compose.yml "retrieve" it in the Dockerfile via ARG APP_USER_NAME I've opted to make the username available as an variable in the base image via ENV ARG APP_USER_NAME ENV APP_USER_NAME=${APP_USER_NAME} and thus be able to access it in the child images directly, I can now write RUN echo ${APP_USER_NAME} instead of ARG APP_USER_NAME RUN echo ${APP_USER_NAME} I'm not 100% certain that I like this approach as I'm more or less "abusing" ENV variables in ways that they are likely not intended ("Why would the username need to be stored as an ENV variable?") - but I also don't see any other practical downside yet. Image naming convention Defining a will make it much easier to reference the images later, e.g. when pushing them to the registry. fully qualified name for images The naming convention for the images is , e.g. $(DOCKER_REGISTRY)/$(DOCKER_NAMESPACE)/$(DOCKER_SERVICE_NAME)-$(ENV) docker.io/dofroscra/nginx-local $(DOCKER_REGISTRY)---^ ^ ^ ^ docker.io $(DOCKER_NAMESPACE)-------------^ ^ ^ dofroscra $(DOCKER_SERVICE_NAME)-------------------^ ^ nginx $(ENV)-----------------------------------------^ local and it is used as value for , e.g. for services.*.image nginx services: nginx: image: ${DOCKER_REGISTRY?}/${DOCKER_NAMESPACE?}/nginx-${ENV?}:${TAG?} In case you are wondering: stems from cker m tch dofroscra Do Fro Scra Environments and build targets Our final goal is a setup that we can use for local development in a CI/CD pipeline in production and even though we strive to for a , there will be differences due to fundamentally different requirements. E.g. parity between those different environments on I want a container production including the sourcecode without any test dependencies on I want a container CI including the sourcecode WITH test dependencies on I want a container local that mounts the sourcecode from my host (including dependencies) This is reflected through the environment variable. We use it in two places: ENV as part of the image name as a suffix of the service name (see ) Image naming convention to specify the target build stage See the file for example: docker-compose-php-base.yml services: php-base: image: ${DOCKER_REGISTRY?}/${DOCKER_NAMESPACE?}/php-base-${ENV?}:${TAG?} build: dockerfile: images/php/base/Dockerfile target: ${ENV?} Using enables us to keep a but also include . See the Dockerfile of the image for example multiple targets in the same Dockerfile common base environment specific instructions php-base ARG ALPINE_VERSION FROM composer:${COMPOSER_VERSION} as composer FROM alpine:${ALPINE_VERSION} as base RUN apk add --update --no-cache \ bash WORKDIR $APP_CODE_PATH FROM base as local RUN apk add --no-cache --update \ mysql-client \ it first defines a stage that includes software required in all environments base and then defines a stage that adds additionally a that helps us to debug connectivity issues local mysql-client After the build for is finished, we end up with an image called that used the build stage as target build stage. local php-base-local local Makefile In the following section I will , e.g. for building and running containers. And to be honest, I find it kinda challenging to keep them in mind without having to look up the exact options and arguments. I would or an alias in my local file in a situation like that - but that wouldn't be available to other members of the team then and it would be very specific to this one project. introduce a couple of commands usually create a helper function .bashrc Instead we'll use a that acts as the central entrypoint in the application. Since Makefiles tend to grow over time, I've adopted some strategies to keep them "sane" via includes, shared variables and better error handling. self-documenting Makefile includes .make/*.mk Over time the setup will grow substantially, thus we split it into multiple files in the directory. The individual files are prefixed with a number to ensure their order when we include them in the main via make .mk .make/ Makefile include .make/*.mk . └── .make/ ├── 01-00-application-setup.mk ├── 01-01-application-commands.mk └── 02-00-docker.mk Shared variables: .make/.env We try to make available here, because we can then pass them on to individual commands as a prefix, e.g. shared variables .PHONY: some-target some-target: ## Run some target ENV_FOO=BAR some_command --baz This will make the available as environment variable to . ENV_FOO some_command Shared variables are used by different components, and we always try to maintain only a . An example would be the variable that we need to define the in the files but also when pushing/pulling/deploying images via make targets later. In this case, the variable is required by as well as . single source of truth DOCKER_REGISTRY image names of our docker images docker-compose.yml make docker-compose To have a clear separation between variables and "code", we use a file located at . It can be initialized via .env . make/.env make make-init by copying the to . .make/.env.example .make/.env . └── .make/ ├── .make/.env.example └── .make/.env The file is included in the main via Makefile -include .make/.env The prefix ensures that make doesn't fail if the file does not exist (yet), see . - GNU make: Including Other Makefiles Manual modifications You can always file manually if required. This might be the case when you run on Linux and need to match the of your host system with the of the docker container. It is common that your local user and group have the id . In this case you would add the entries manually to the file. modify the .make/.env docker user id user id 1000 .make/.env APP_USER_ID=1000 APP_GROUP_ID=1000 See also section . Solving permission issues Enforce required parameters We kinda "abuse" make for executing arbitrary commands (instead of building artifacts) and some of those commands require parameters that can be in the form passed as command arguments make some-target FOO=bar There is no way to "define" those parameters as we would in a method signature - but we can still ensure to fail as early as possible if a parameter is missing via @$(if $(FOO),,$(error FOO is empty or undefined)) See also SO: How to abort makefile if variable not set? We use this technique for example to ensure that all required variables are defined when we execute docker targets via the precondition target: validate-docker-variables .PHONY: validate-docker-variables validate-docker-variables: @$(if $(TAG),,$(error TAG is undefined)) @$(if $(ENV),,$(error ENV is undefined)) @$(if $(DOCKER_REGISTRY),,$(error DOCKER_REGISTRY is undefined - Did you run make-init?)) @$(if $(DOCKER_NAMESPACE),,$(error DOCKER_NAMESPACE is undefined - Did you run make-init?)) @$(if $(APP_USER_NAME),,$(error APP_USER_NAME is undefined - Did you run make-init?)) @$(if $(APP_GROUP_NAME),,$(error APP_GROUP_NAME is undefined - Did you run make-init?)) .PHONY:docker-build-image docker-build-image: validate-docker-variables $(DOCKER_COMPOSE) build $(DOCKER_SERVICE_NAME) Make + Docker = <3 We already introduced quite some complexity into our setup: "global" variables (shared between and ) make docker multiple files docker-compose.yml build dependencies Bringing it all together "manually" is quite an effort and prone to errors. But we can nicely tuck the complexity away in by defining the two variables and .make/02-00-docker.mk DOCKER_COMPOSE DOCKER_COMPOSE_PHP_BASE DOCKER_DIR:=./.docker DOCKER_ENV_FILE:=$(DOCKER_DIR)/.env DOCKER_COMPOSE_DIR:=$(DOCKER_DIR)/docker-compose DOCKER_COMPOSE_FILE:=$(DOCKER_COMPOSE_DIR)/docker-compose.yml DOCKER_COMPOSE_FILE_LOCAL:=$(DOCKER_COMPOSE_DIR)/docker-compose.local.yml DOCKER_COMPOSE_FILE_PHP_BASE:=$(DOCKER_COMPOSE_DIR)/docker-compose-php-base.yml DOCKER_COMPOSE_PROJECT_NAME:=dofroscra_$(ENV) DOCKER_COMPOSE_COMMAND:=ENV=$(ENV) \ TAG=$(TAG) \ DOCKER_REGISTRY=$(DOCKER_REGISTRY) \ DOCKER_NAMESPACE=$(DOCKER_NAMESPACE) \ APP_USER_NAME=$(APP_USER_NAME) \ APP_GROUP_NAME=$(APP_GROUP_NAME) \ docker-compose -p $(DOCKER_COMPOSE_PROJECT_NAME) --env-file $(DOCKER_ENV_FILE) DOCKER_COMPOSE:=$(DOCKER_COMPOSE_COMMAND) -f $(DOCKER_COMPOSE_FILE) -f $(DOCKER_COMPOSE_FILE_LOCAL) DOCKER_COMPOSE_PHP_BASE:=$(DOCKER_COMPOSE_COMMAND) -f $(DOCKER_COMPOSE_FILE_PHP_BASE) uses and extends it with DOCKER_COMPOSE docker-compose.yml docker-compose.local.yml uses only DOCKER_COMPOSE_PHP_BASE docker-compose-php-base.yml The variables can then be used later in make recipes. Ensuring the build order As mentioned under , we and use the following make targets: PHP images need to build images in a certain order .PHONY: docker-build-image docker-build-image: ## Build all docker images OR a specific image by providing the service name via: make docker-build DOCKER_SERVICE_NAME= $(DOCKER_COMPOSE) build $(DOCKER_SERVICE_NAME) .PHONY: docker-build-php docker-build-php: ## Build the php base image $(DOCKER_COMPOSE_PHP_BASE) build $(DOCKER_SERVICE_NAME_PHP_BASE) .PHONY: docker-build docker-build: docker-build-php docker-build-image ## Build the php image and then all other docker images As a developer, I can now simply run - which will first build the image via and then build all the remaining images via (by not specifying the variable, will build services listed in the files). make docker-build php-base docker-build-php docker-build-image DOCKER_SERVICE_NAME docker-compose all docker-compose.yml I would argue that the and easy to understand but when we run them with the option to only "Print the recipe that would be executed, but not execute it", we get a feeling for the complexity: make recipes themselves are quite readable -n $ make docker-build -n ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose-php-base.yml build php-base ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml build Run commands in the docker containers Tooling is an important part in the development workflow. This includes things like linters, static analyzers and testing tools but also "custom" tools geared towards your specific workflow. Those tools usually . For now, we only have a single "tool" defined in the file . It ensures that a table called is created. require a PHP runtime setup.php jobs To run this tool, we must first start the docker setup via and then execute the script in the container. The corresponding target is defined in : make docker-up application .make/01-00-application-setup.mk .PHONY: setup-db setup-db: ## Setup the DB tables $(EXECUTE_IN_APPLICATION_CONTAINER) php setup.php $(ARGS); which essentially translates to docker-compose exec -T --user application application php setup.php if we are outside of a container and to php setup.php if we are inside a container. That's quite handy, because we can without having to log into a container. run the tooling directly from the host system The "magic" happens in the variable that is defined in as EXECUTE_IN_APPLICATION_CONTAINER .make/02-00-docker.mk EXECUTE_IN_WORKER_CONTAINER?= EXECUTE_IN_APPLICATION_CONTAINER?= EXECUTE_IN_CONTAINER?= ifndef EXECUTE_IN_CONTAINER # check if 'make' is executed in a docker container, # see https://stackoverflow.com/a/25518538/413531 # `wildcard $file` checks if $file exists, # see https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html # i.e. if the result is "empty" then $file does NOT exist # => we are NOT in a container ifeq ("$(wildcard /.dockerenv)","") EXECUTE_IN_CONTAINER=true endif endif ifeq ($(EXECUTE_IN_CONTAINER),true) EXECUTE_IN_APPLICATION_CONTAINER:=$(DOCKER_COMPOSE) exec -T --user $(APP_USER_NAME) $(DOCKER_SERVICE_NAME_APPLICATION) EXECUTE_IN_WORKER_CONTAINER:=$(DOCKER_COMPOSE) exec -T --user $(APP_USER_NAME) $(DOCKER_SERVICE_NAME_PHP_WORKER) endif We can take a look via again to see the resolved recipe on the host system -n pascal.landau:/c/_codebase/dofroscra# make setup-db ARGS=--drop -n ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application application php setup.php --drop Within a container it looks like this: root:/var/www/app# make setup-db ARGS=--drop -n php setup.php --drop; Solving permission issues If you are using Linux, you that are shared between the host system and the docker containers as explained in section Synchronizing file and folder ownership on shared volumes of the previous tutorial. might run into permission issues when modifying files when the user id is not the same In this case, you need to manually and add the and variables according to your local setup. This to ensure that the correct is used in the images. modify the .make/.env APP_USER_ID APP_GROUP_ID must be done building the images before user id In very rare cases it can lead to problems, because . I've personally never run into this problem, but you can read about it in more detail at . The author via . your local ids will in the docker containers already exist Docker and the host filesystem owner matching problem even proposes a general solution the Github project "FooBarWidget/matchhostfsowner" PHP POC To ensure that everything works as expected, the repository contains a minimal PHP proof of concept. By default, port 80 from the host ist forwarded to port 80 of the container. nginx FYI: I would also recommend to add the following entry in the hosts file on the host machine 127.0.0.1 app.local so that we can access the application via instead of . http://app.local http://127.0.0.1 The files of the POC essentially ensure that the container connections outlined in work as expected: Local docker setup dependencies.php returns configured and objects to talk to the queue and the database Redis PDO setup.php => can talk to ensures that application mysql public/index.php is the web root file that can be accessed via http://app.local => and are working ensures that nginx php-fpm contains 3 different "routes": http://app.local?dispatch=some-job-id dispatches a new "job" with the id on the queue to be picked up by a worker some-job-id => can talk to ensures that php-fpm redis http://app.local?queue shows the content of the queue http://app.local?db shows the content of the database => can talk to ensures that php-fpm mysql worker.php is started as daemon process in the container php-worker checks the redis datasbase for the key every second 0 "queue" if a value is found it is stored in the table of the database jobs => can talk to and ensures that php-worker redis mysql A full test scenario is defined in and looks like this: test.sh $ bash test.sh Building the docker setup //... Starting the docker setup //... Clearing DB ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application application php setup.php --drop; Dropping table 'jobs' Done Creating table 'jobs' Done Stopping workers ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application php-worker supervisorctl stop worker:*; worker:worker_00: stopped worker:worker_01: stopped worker:worker_02: stopped worker:worker_03: stopped Ensuring that queue and db are empty Items in queue array(0) { } Items in db array(0) { } Dispatching a job 'foo' Adding item 'foo' to queue Asserting the job 'foo' is on the queue Items in queue array(1) { [0]=> string(3) "foo" } Starting the workers ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application php-worker supervisorctl start worker:*; worker:worker_00: started worker:worker_01: started worker:worker_02: started worker:worker_03: started Asserting the queue is now empty Items in queue array(0) { } Asserting the db now contains the job 'foo' Items in db array(1) { [0]=> string(3) "foo" } Wrapping up Congratulations, you made it! If some things are not completely clear by now, don't hesitate to leave a comment. Apart from that, you should now have a running docker setup and the means to "control" it conveniently via . In the next part of this tutorial, we will configure PhpStorm as our IDE to use the docker setup. make Also Published Here