Our pipeline took 24 minutes. Nobody waited for it. People pushed, switched to another task, and came back an hour later — by which time they had lost the context of what they were doing, and half the time the failure was a flaky test that needed a re-run, adding another 24 minutes.
We got it to just under six. The code did not change. What changed was four things, none of them clever, and the effect on how the team worked was much larger than the twenty minutes suggests.
Ten Minutes Is the Threshold
There is a real cliff in how people behave, and it is somewhere around ten minutes.
Under ten, developers wait. They watch the run, and when it fails they fix it immediately with everything still in their head. Over ten, they context-switch — and now a failure means reloading the problem an hour later, which costs far more than the twenty minutes of machine time.
So this is not really about compute cost. It is about whether your feedback loop is short enough for people to stay inside it. That is why the payoff is bigger than the numbers imply.
Measure Before Optimising
Most people guess that tests are the slow part. Sometimes. Our breakdown before touching anything:
checkout 0:20
setup node 0:35
npm ci 4:10 ← no cache at all
lint 1:05
typecheck 2:15
unit tests 3:30
integration tests 6:40 ← running serially, fresh DB per file
build 3:20
docker build + push 2:30 ← rebuilding every layer every time
-----
24:25
Dependency installation and Docker build together were nearly seven minutes of pure repetition — the same work, from scratch, on every single run. That is where I would look first on almost any pipeline.
The Four Changes
1. Cache dependencies properly
The most common miss, and usually the biggest single win. Most setup actions have caching built in and it is off by default:
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm # keyed on package-lock.json automatically
- run: npm ci --prefer-offline --no-audit --fund=false
Four minutes became about forty seconds. --no-audit matters more than it looks — the audit call is a network round trip you do not need on every build, and you should be running dependency scanning as its own job anyway.
2. Run independent jobs in parallel
Lint, typecheck and tests do not depend on each other. Running them sequentially is a habit, not a requirement.
jobs:
quality:
strategy:
fail-fast: true # one failure cancels the rest — stop burning minutes
matrix:
task: [lint, typecheck, test:unit]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20, cache: npm }
- run: npm ci --prefer-offline
- run: npm run ${{ matrix.task }}
Wall-clock time becomes the slowest job rather than the sum. fail-fast is worth setting deliberately: if lint fails in ten seconds, there is no point paying for six more minutes of tests.
3. Fix the integration tests
These were 6:40 because each test file spun up a fresh database and re-ran migrations. Two changes:
One database, transactions per test. Start the container once, run migrations once, and wrap each test in a transaction that rolls back. Isolation without the setup cost, and tests can then run in parallel safely.
Use a service container, not a script that installs Postgres. It is already there and already warm.
services:
postgres:
image: postgres:16-alpine
env: { POSTGRES_PASSWORD: test }
options: >-
--health-cmd pg_isready --health-interval 5s --health-retries 5
6:40 down to about 1:50.
4. Cache the Docker build
Docker was rebuilding every layer because CI starts with an empty layer cache. Buildx with a registry cache fixes it, and ordering the Dockerfile so dependencies come before source means a code change only rebuilds the last layers:
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
push: true
tags: registry.example.com/app:${{ github.sha }}
cache-from: type=registry,ref=registry.example.com/app:buildcache
cache-to: type=registry,ref=registry.example.com/app:buildcache,mode=max
Only Build What Changed
The next tier of improvement, and where the largest wins are in a monorepo. If a pull request only touches the docs, there is no reason to run the backend test suite.
Path filters handle the simple case:
on:
pull_request:
paths: ["apps/api/**", "packages/shared/**", "package-lock.json"]
For anything more complex, a build tool with a dependency graph — Turborepo, Nx, Bazel — will skip tasks whose inputs have not changed and reuse cached outputs across machines. That is a bigger investment and it pays off once you have several packages.
Related and often overlooked: cancel superseded runs. Pushing three times in five minutes should not run three full pipelines.
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
Move the Slow Things Out of the PR
Not everything has to block a merge. Ask of each step: would a failure here stop us shipping?
On the pull request: lint, typecheck, unit tests, integration tests, a secret scan, and a build. Fast, and every one of them can block.
After merge: the full end-to-end browser suite, deeper security scanning, performance tests, multi-version compatibility runs.
On a schedule: re-scanning deployed images for newly disclosed vulnerabilities, dependency freshness, load tests.
This split alone takes several minutes off most pipelines and loses very little, because the things you moved rarely fail in a way a merge would have prevented.
Flaky Tests Are a Pipeline Problem
Worth saying plainly: a suite with three flaky tests effectively doubles your pipeline time, because people re-run it. And it does something worse — it teaches the team that a red build might not mean anything, which is the point at which CI stops working as a signal.
Track which tests fail and then pass on a re-run. Quarantine them out of the blocking suite, and fix or delete them within a week. Leaving them is the most expensive decision on this page.
What It Was Worth
24 minutes to 5:50, in about a day of work, with dependency caching and integration test setup accounting for most of it.
The measurable effect was not the machine time. It was that people started waiting for the pipeline again — which meant failures got fixed in minutes instead of hours, and pull requests started merging on the same day they were opened.
