Cure the disease of “heavy” Docker development environment: the secret to synchronizing CI and local
After I hit `docker-compose up`, I made a cup of coffee, checked SNS, and then came back and it was still running. If you're justifying such "waiting time", you should change your mind now.
The reason why the development environment is slow to build is not just a "specification". It's "technical debt" that will kill your development experience (DX). If you reduce a 1-minute build to 5 seconds, you'll get 19 minutes of "thinking time" just by building 20 times a day.
Today, I will explain the optimal solution for "speeding up" the Docker development environment and "CI coordination" that I arrived at after a long process of trial and error.
1. “Multi-stage build” is a survival strategy, not a religion
The idea that ``it is justice to separate Dockerfiles for development and production'' is old. In this day and age, the golden rule is to have one Dockerfile and optimize it in stages.
Heavy development environments are often caused by "unnecessary dependencies" or "huge build contexts". Please take a look at the configuration below.
# syntax=docker/dockerfile:1
FROM node:18-slim AS base
WORKDIR /app
COPY package*.json package-lock.json ./
# Development stage: Include tools necessary for hot reloading
FROM base AS dev
RUN npm ci
COPY.
CMD ["npm", "run", "dev"]
# Production stage: Extract only build artifacts
FROM base AS prod
RUN npm ci --only=production
COPY.
RUN npm run build
CMD ["node", "dist/main.js"]
This way, you only need to specify the `target` and use it accordingly. For `docker-compose`, just write `build: target: dev`. This way, no test libraries are mixed into the production image, and the development environment will start up very quickly.
2. Architecture Trap: Don't fix "linux/amd64" on M1/M2 Mac
I'm sure many people have been taught in the past, ``Specify `platform: linux/amd64` to match the local and CI environments.'' Delete it now.
If you do this on Apple Silicon (M1/M2/M3), Docker will force it to emulate x86_64 via QEMU. The build speed will be slow, and above all, the operation will become unstable.
The correct answer is "build natively and optimize on the CI side."
When building in a CI environment (GitHub Actions), use `docker/build-push-action` to take advantage of GitHub's cache. This is now the de facto term for "modern CI acceleration."
`
# .github/workflows/ci.yml
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: false
tags: my-app:latest
cache-from: type=gha
cache-to: type=gha,mode=max
This will free you from the nightmare of "it works locally but crashes in CI".
3. Does `.dockerignore` really work?
Build context bloat is Docker's biggest enemy. Sending the `.git` directory to the container as is is suicide.
# .dockerignore
.git
.github
node_modules
dist
.env
*.log
.DS_Store
Of particular note is `.git`. The larger the repository, the more you end up transferring several GB of history each time you `COPY . .`. This alone can change the build time by tens of seconds. When dust accumulates, it becomes a mountain, so go on the ground.
4. Why does "node_modules" cache fail?
Have you ever tried to cache `node_modules` in GitHub Actions and got stuck due to path inconsistencies or OS differences?
Here's the conclusion I came to:
"Trust Docker's layer caching. Leave the node_modules caching to Docker"
Rather than forcibly saving `node_modules` in `actions/cache`, it is far more robust to optimize the layer order of the Dockerfile and increase the cache hit rate when `npm ci` runs.
1. Copy `package.json`
2. Run `npm ci`
3. Then copy the source code (`COPY . .`)
Just by following this order, installation of heavy dependencies will be skipped unless changes are made to `package.json`. This is the "strongest justice" with the lowest maintenance costs.
Conclusion: Don't be swayed by tools, understand them.
In the end, if you think Docker is a magic box, you'll end up complaining about how slow it is and how it doesn't work.
- Prevent bloat with Multi-stage build.
- Platform specification believes native.
- Thoroughly stripped down the build context.
- Make the most of layer caching.
Your development environment is a mirror of your technical ability.