Hard lessons from real-world deployments, and how to avoid the most expensive mistakes. Hard lessons from real-world deployments, and how to avoid the most expensive mistakes. Docker is amazing — until it’s not. You write a Dockerfile, run it locally, and it works like magic. But the moment you try to scale it, secure it, or run it reliably in production, reality hits hard. From bloated images to silent security risks, there’s a lot that can go wrong. Dockerfile This article collects the hard-won lessons I’ve learned from building and running Dockerized platforms in production: what broke, what slowed us down, and how we fixed it. hard-won lessons 1. Your Docker Image is Probably Too Big The problem: Most production Docker images are bloated. I’ve seen images over 1GB that do nothing more than serve a simple Node.js or Python app. These slow down CI/CD, eat up bandwidth, and kill cold start performance. 1GB The fix: Use multi-stage builds and start from minimal base images. multi-stage builds minimal base images Example: # dev stage
FROM node:18 as builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# production stage
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/server.js"] # dev stage
FROM node:18 as builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# production stage
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/server.js"] With this change, we reduced image size from 900MB to 160MB. Faster builds, faster deploys, happier ops. 900MB to 160MB Also: Use .dockerignore


Don’t copy entire project blindly Use .dockerignore Use .dockerignore .dockerignore Don’t copy entire project blindly Don’t copy entire project blindly 2. Stop Running Containers as Root The problem: By default, Docker containers run as root. If someone breaks into the container, they have root privileges in the container, which can be escalated depending on your host setup. root The fix: Add a non-root user inside your image: RUN addgroup app && adduser -S -G app app
USER app RUN addgroup app && adduser -S -G app app
USER app And don’t forget to use --user flag in docker run if needed. --user docker run Also: Don’t mount the Docker socket unless absolutely required


Use tools like dockle, trivy, hadolint for security checks Don’t mount the Docker socket unless absolutely required Don’t mount the Docker socket unless absolutely required Use tools like dockle, trivy, hadolint for security checks Use tools like dockle, trivy, hadolint for security checks dockle trivy hadolint 3. Your Build Context is Slowing Everything Down The problem: If your .dockerignore is missing or incomplete, Docker might be copying your entire repo, including .git, node_modules, .env, etc. into the image. .dockerignore .git node_modules .env The fix: Use .dockerignore like your .gitignore: .dockerignore .gitignore .git
node_modules
*.log
.env
.DS_Store
tests/ .git
node_modules
*.log
.env
.DS_Store
tests/ This cuts down build context size dramatically, making builds faster and less prone to cache busting. 4. Volume Mounts Work Differently in Production The problem: In development, you rely on volume mounts for hot-reloading and persistence. In production, improper volume usage can: Break statelessness
Create permission issues
Introduce state inconsistency between replicas Break statelessness Create permission issues Introduce state inconsistency between replicas The fix: Only mount named volumes with a clear purpose


Make volumes read-only where possible


Never write to container’s internal FS expecting it to persist Only mount named volumes with a clear purpose Only mount named volumes with a clear purpose named volumes Make volumes read-only where possible Make volumes read-only where possible read-only Never write to container’s internal FS expecting it to persist Never write to container’s internal FS expecting it to persist 5. Logging Can Wreck Your Node The problem: Docker’s default logging driver (json-file) stores logs on disk. If you don’t set limits, logs can consume all space. json-file on disk The fix: Set logging limits in docker-compose.yml: docker-compose.yml logging:
  driver: json-file
  options:
    max-size: "10m"
    max-file: "3" logging:
  driver: json-file
  options:
    max-size: "10m"
    max-file: "3" Or use centralized logging: Fluentd, Logstash, or a logging SaaS. Golden rule: log to stdout/stderr, not to files. log to stdout/stderr 6. Use Docker's Built-in Security Features The problem: Containers share the host kernel. They’re isolated by namespaces, but not foolproof. The fix: Drop unnecessary Linux capabilities:
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE ...



Run with --read-only FS where possible


Apply a seccomp profile or AppArmor


Don’t run privileged containers unless absolutely necessary Drop unnecessary Linux capabilities:
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE ... Drop unnecessary Linux capabilities: docker run --cap-drop ALL --cap-add NET_BIND_SERVICE ... docker run --cap-drop ALL --cap-add NET_BIND_SERVICE ... Run with --read-only FS where possible Run with --read-only FS where possible --read-only Apply a seccomp profile or AppArmor Apply a seccomp profile or AppArmor Don’t run privileged containers unless absolutely necessary Don’t run privileged containers unless absolutely necessary 7. CI/CD Pipelines Need Container-Aware Design The problem: Many CI pipelines rebuild full images unnecessarily or fail to cache layers properly. The fix: Use layer caching: GitHub Actions cache, GitLab’s built-in Docker cache


Split COPY in Dockerfile to minimize cache invalidation:
COPY package.json .
RUN npm ci
COPY . .



Avoid docker:dind unless necessary Use layer caching: GitHub Actions cache, GitLab’s built-in Docker cache Use layer caching: GitHub Actions cache, GitLab’s built-in Docker cache cache Split COPY in Dockerfile to minimize cache invalidation:
COPY package.json .
RUN npm ci
COPY . . Split COPY in Dockerfile to minimize cache invalidation: COPY Dockerfile COPY package.json .
RUN npm ci
COPY . . COPY package.json .
RUN npm ci
COPY . . Avoid docker:dind unless necessary Avoid docker:dind unless necessary docker:dind Conclusion: Docker is Not Magic Docker is a brilliant abstraction, but it doesn’t mean "write once, scale forever." Treat containers as production infrastructure, not dev toys. Optimize them. Secure them. Monitor them. And most importantly: test everything in conditions as close to production as possible. Bonus: Tooling I Recommend Dive — Inspect Docker layers and image size


Trivy — Vulnerability scanner


Dockle — Best practices and security analyzer


Hadolint — Linter for Dockerfiles


Docker Scout — Docker's built-in SBOM and vuln check Dive — Inspect Docker layers and image size Dive — Inspect Docker layers and image size Dive Trivy — Vulnerability scanner Trivy — Vulnerability scanner Trivy Dockle — Best practices and security analyzer Dockle — Best practices and security analyzer Dockle Hadolint — Linter for Dockerfiles Hadolint — Linter for Dockerfiles Hadolint Docker Scout — Docker's built-in SBOM and vuln check Docker Scout — Docker's built-in SBOM and vuln check Docker Scout Stay safe and ship fast. Questions or feedback? I’d love to hear what lessons you learned from Docker in production. you [Author] - Oleksii Bondar

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

GitHub

These 5 Unexpected Use Cases Prove WebRTC Has Outgrown Its Roots as "Just" a Video-calling Tool

What No One Tells You About Running Docker in Production

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Scaling Real-Time Video on AWS: How We Keep WebRTC Latency Below 150ms with Kubernetes Autoscaling

Unveiling GitHub's Secret Blueprint—How to Handle Millions of Transactions Daily

These 5 Unexpected Use Cases Prove WebRTC Has Outgrown Its Roots as "Just" a Video-calling Tool

This Web IDE Runs Your Code in the Cloud—Without Melting Your Laptop

19 Apps and Websites All Student Developers Should Check Out

Scaling Real-Time Video on AWS: How We Keep WebRTC Latency Below 150ms with Kubernetes Autoscaling

Unveiling GitHub's Secret Blueprint—How to Handle Millions of Transactions Daily

These 5 Unexpected Use Cases Prove WebRTC Has Outgrown Its Roots as "Just" a Video-calling Tool

This Web IDE Runs Your Code in the Cloud—Without Melting Your Laptop

19 Apps and Websites All Student Developers Should Check Out

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps