I love . Also, I was really proud of the newsletter post title and the reference to the Dua Lipa song. docker buildx bake https://www.youtube.com/watch?v=b5hjqSnliKk&t=20s?embedable=true I like experimenting with a lot of different ideas, and for the last several years, I've been trying to figure out how to maximize my ability to try new ideas with as little work as possible. In this post, I want to go over what I've settled on these days, which is to use a monolithic git repository, and to leverage Docker's bake functionality. We'll talk about how I structure my repository, how I build the containers, etc. If you'd just like to look through the repo, check it out at . abatilo/newsletter-bake-monorepo For the backend services, I write web services with Go. I define a global go module at the root of my repository. Go workspaces are a thing as of Go 1.18 but their value is largely when you want to make it easy to work on module changes across repos. For frontend applications, I use react applications. I'm not much of a frontend developer, and even though I know there are benefits to some of the newer frameworks, this one is comfortable to me which lets me focus on building. With the react applications, I do leverage npm workspaces. I serve the static assets directly from my Go application, which means that there's only one application container that I need to worry about. That also means that I can serve the frontend and backend on the same domain, which means I get to avoid any kind of CORS configuration. I serve the applications behind CloudFront as my CDN, and so there's still static asset caching, and backend applications get API acceleration, DDoS protection, and more thanks to the CDN. I organize building the containers using a docker-bake file. Bake is a high level build command available to people using docker buildx. In it's simplest definition, you can write a file in the Hashicorp terraform format, and running bake will build all of the targets, and will share common image stages when it can, and will traverse the dependency tree in parallel as much as possible. You can easily define several docker containers, and with a single command, build all of them, very comparable to having a Makefile. create-react-app With this high level overview, let's break it down. One Dockerfile is all it takes, falling in love with bake How’s this work? It’s a where we build the static assets in one stage, build the go binaries in another stage, and put everything together in the last stage. Here’s a link to the whole Dockerfile. multi-stage build The frontend stages: FROM node:18.10.0-alpine as node-modules

WORKDIR /app/web
COPY ./web/package.json ./web/package-lock.json ./
COPY ./web/app1/package.json ./app1/
COPY ./web/app2/package.json ./app2/
RUN npm ci --no-audit

FROM node-modules as node-builder
COPY ./web ./

ARG app
RUN npm run build --workspace ${app} So we start out with a stage that’s exclusively for installing dependencies. With , if you want deterministic builds, you want to use for your build environments. And with , you have to copy in the for each workspace so that the can make sure to install everything needed. There are ways to install the dependencies for a single workspace, but since we’ll end up configuring our docker cache for the entirety of this stage, downloading the cache all at once actually ends up being significantly faster than downloading the dependencies for each workspace in parallel. Especially since there’s a lot of shared dependencies between each website. We keep the itself as an independent stage so that when we cache it, it doesn’t need to worry about the for the app that we want to actually build. That’ll make more sense when we get to looking at the contents of the npm npm ci npm workspaces package.json npm ci node-modules ARG docker-bake.hcl The backend stages: FROM golang:1.19.0 as go-modules

# Install dependencies
WORKDIR /app
COPY ./go.mod ./go.sum ./
RUN go mod download

FROM go-modules as go-builder

COPY ./internal ./internal
COPY ./cmd ./cmd
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go install -ldflags="-w -s" ./cmd/... Same as the frontend stages, we have a stage that’s for go dependencies, and we download them all at once and we’ll cache the entire stage. And as opposed to the frontend build, we actually just compile the entirety of the repo. The Go compiler is extremely fast, and in a future newsletter post that I have planned, we’ll talk about how to leverage the Go compiler build cache to make this even faster. Consider subscribing to find out how I do that in a Dockerfile! The final stage: FROM gcr.io/distroless/static-debian11:nonroot
ENV STATIC_ASSETS_PATH /static

ARG app
COPY --from=node-builder /app/web/${app}/build /static
COPY --from=go-builder /go/bin/${app} /usr/local/bin/entrypoint
ENTRYPOINT ["entrypoint"] Just like I mentioned, we take the static assets for a specific app and the statically linked binary and put those into the same final container. Writing your docker-bake.hcl The common config: variable "GITHUB_SHA" {
  default = "latest"
}

variable "REGISTRY" {
  default = "ghcr.io/abatilo/newsletter-bake-monorepo"
}

group "default" {
  targets = [
    "app1",
    "app2",
  ]
} I use the GitHub SHA for my container tags but you can use whatever you want. Writing a block in the bake file will automatically be populated by any environment variables that are available when the command is invoked. In GitHub Actions, is always populated. Setting a default means that I get to build the containers locally without having to remember to set . variable docker buildx bake GITHUB_SHA GITHUB_SHA The next block is for the container registry prefix to push to, without the image name. This one I don’t actually override anywhere but there isn’t a way to set local variables like in Terraform with blocks. So I define the registry in one place and we’ll use it for each application’s container target. variable locals is the default target to build if all you do is run without any additional arguments. I define all of the images in my project here. group “default” docker buildx bake Defining the cache targets: target "node-modules" {
  target = "node-modules"
  cache-from = ["type=gha,scope=node-modules"]
  cache-to = ["type=gha,mode=max,scope=node-modules"]
}

target "go-modules" {
  target = "go-modules"
  cache-from = ["type=gha,scope=go-modules"]
  cache-to = ["type=gha,mode=max,scope=go-modules"]
} Bake will build a dependency graph for any shared docker stages that get used by later images, and will walk the dependency graph to build the different targets. This is awesome for the monorepo and how I structured the Dockerfile. It means that I can define the dependency stages as their own images with the cache config, and when I run , these two targets will be built first and since bake knows that the different apps depend on these targets, we build the dependencies first before them getting used in parallel for each application build. Buildkit has a backend for specifically using the GitHub Actions cache API which is incredibly easy to use. You can read more about . docker buildx bake this cache backend here target "app1" {
  contexts = {
    node-modules = "target:node-modules",
    go-modules = "target:go-modules"
  }
  args = {
    app = "app1"
  }
  tags = [
    "${REGISTRY}/app1:${GITHUB_SHA}",
  ]
  cache-from = ["type=gha,scope=app1"]
  cache-to = ["type=gha,mode=max,scope=app1"]
}

target "app2" {
  contexts = {
    node-modules = "target:node-modules",
    go-modules = "target:go-modules"
  }
  args = {
    app = "app2"
  }
  tags = [
    "${REGISTRY}/app2:${GITHUB_SHA}",
  ]
  cache-from = ["type=gha,scope=app2"]
  cache-to = ["type=gha,mode=max,scope=app2"]
} Now, we have examples of the actual definitions for a single application’s docker container. Notice that we set the key here which references the dependency targets we just defined. You can basically think of this like the block if you’ve used Terraform before. The key let’s us populate the variables in the . This is what is ultimately different between each of the containers. , the buildx team might support using loops like in Terraform but for now each block will have a bit of duplication. contexts depends_on args ARG Dockerfile In the future for_each What does CI look like? There’s an amazing which makes it insanely easy to build all of your containers in the most optimal way. Since we’ve set the block in the , config is very minimal. One step in your GitHub Action workflow file will build all of your images and will push all of your cache layers, tag all of your containers, and push all your final images. You’ll still have to do things like checkout the code and don’t forget that you’ll want to use the since bake is a buildx feature. There’s one quick gotcha for the actual . We don’t want to push PR builds and we don’t want to pollute the cache with PR builds. docker/bake-action group “default” docker-bake.hcl docker/setup-buildx-action docker/bake-action - if: github.event_name == 'push'
        uses: docker/bake-action@v2.3.0
        with:
          push: true

      - if: github.event_name == 'pull_request'
        uses: docker/bake-action@v2.3.0
        with:
          push: false
          # Explicitly remove pushing to cache to not pollute the cache with PR
          # specific layers.
          set: |
            *.cache-to= The way we set to an empty string is inspired by . The wildcard sets the property for all targets. cache-to int128/docker-build-cache-config-action cache-to Wrapping up Reminder that a full blown example repo is available at . This project structure makes it so insanely easy to add new projects to CI and makes it easy for me to continue to try new ideas. Maintenance cost is very low since it’s a single Dockerfile and the Dockerfile itself is very minimal. These docker builds are optimized for small docker images, parallel builds, and let’s me try new ideas without much effort. abatilo/newsletter-bake-monorepo I hope you give docker bake a try. I think it’s a great way to maintain many images in the same repo and keep builds optimal and simple. Also published . here

Whooosh: A Comprehensive Guide for the Fastest Possible Docker Builds in Human Existence

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

One Dockerfile is All it Takes: How I Fell in Love With Bake

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

ChatGPT Answered 50,000 Trivia Questions - Here's How It Did

Cloud-Native Buildpacks, BuildKit, and Container Images: Qu'est-ce Que C'est?

104 Stories To Learn About Go

105 Stories To Learn About Functional Programming

100+ Free Pluralsight Courses to learn Python, Java, and Spring Boot

ChatGPT Answered 50,000 Trivia Questions - Here's How It Did

Cloud-Native Buildpacks, BuildKit, and Container Images: Qu'est-ce Que C'est?

104 Stories To Learn About Go

105 Stories To Learn About Functional Programming

100+ Free Pluralsight Courses to learn Python, Java, and Spring Boot

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps