Speed up slow CI pipelines and optimize expensive jobs

Symptoms

How to tell if slow pipelines are costing you money

Open your repository's Actions tab and look at the run durations. If you see these patterns, you have a pipeline speed problem:

• High p90 pipeline duration. Your median run might be 20 minutes, but the p90 is 35+ minutes. That tail represents 10% of your runs consuming disproportionate compute. When developers experience the 35-minute runs, they lose trust in CI speed and start re-triggering or context-switching.

• A few jobs dominate the bill. GitHub Actions job cost follows a power law. The top 5 jobs by billable minutes typically account for 60–80% of total spend. If one integration test job runs 3–5x longer than others in the same workflow, that single job is where most of your money goes. You optimized lint and saved 30 seconds. The real waste is the 14-minute job you haven't touched.

• The same build runs in multiple jobs. Your test job runs npm run build, your lint job runs npm run build, and your deploy job runs npm run build. Each job compiles identical source into identical output, independently. Three 8-minute builds means 24 minutes of build time for a single push, and only one of those builds was necessary.

• No correlation between change size and run duration. A one-line config change takes the same 30 minutes as a 50-file refactor. This means your pipeline has no incremental behavior and does full builds, full test suites, and full installs on every run regardless of what changed. This is a sign you could benefit from path filtering to skip CI for non-code changes.

Metrics

Quantify the cost of slow pipelines

The cost formula is billable minutes per run × runs/day × rate. Here's a typical scenario for a team running 30 CI runs per day on Linux, where the pipeline averages 22 billable minutes per run:

That's one workflow on Linux. On macOS runners at $0.062/min, which is 10x the rate, the same 22→13 minute reduction saves $370/mo per workflow. And this doesn't account for the productivity cost: a 2025 case study found that cutting pipeline time from 20 minutes to 9 minutes reduced runner hours by 35% and dropped flaky failures from 15% to 4%.

Step 1

Find your hotspots: identify the top-cost jobs

Before optimizing anything, measure where the money actually goes. Most pipelines have 2–3 jobs that account for 60–80% of total billable minutes. Fixing the wrong job wastes effort. A 30% improvement on a 15-minute job saves more than a 90% improvement on a 1-minute job.

GitHub's billing page shows total minutes but not per-job breakdowns. Use the workflow jobs API to pull job-level timing and rank by total cost. This script does it over the last 30 days:

#!/bin/bash
# Rank jobs by total cost over the last 30 days
# Requires: gh CLI, jq

OWNER="your-org"
REPO="your-repo"
SINCE=$(date -v-30d +%Y-%m-%dT00:00:00Z)

# Get completed run IDs from the last 30 days
gh api --paginate \
  "/repos/$OWNER/$REPO/actions/runs?created=>=$SINCE&status=completed" \
  --jq '.workflow_runs[].id' \
  | while read -r run_id; do
    # Get job name and duration for each run
    gh api "/repos/$OWNER/$REPO/actions/runs/$run_id/jobs" \
      --jq '.jobs[] |
        select(.conclusion != null) |
        [.name,
         ((.completed_at | fromdateiso8601) -
          (.started_at | fromdateiso8601))] | @tsv'
  done | awk -F'\t' '{
    seconds[$1] += $2; count[$1]++
  } END {
    for (j in seconds) {
      mins = seconds[j] / 60
      # Adjust rate for your runner type
      cost = mins * 0.006
      printf "%s\t%.0f mins\t%d runs\t$%.2f\n",
        j, mins, count[j], cost
    }
  }' | sort -t$'\t' -k4 -rn | head -10

A typical output shows a steep drop-off after the top 3–5 jobs:

The top 3 jobs (test-integration, build, test-e2e) account for 74% of total cost. A targeted 30% reduction in these 3 jobs alone saves more than a 10% reduction spread across all jobs combined. Once you know your hotspots, apply the fixes below in order.

Step 2

Add targeted caching to the expensive jobs

The most common reason a pipeline is slow is that it redoes work every run: reinstalling dependencies, recompiling code, rebuilding Docker images. GitHub's actions/cache can store and restore these outputs between runs, but only if the cache key is configured correctly. A cache keyed on github.sha misses every run. Key on your lockfile hash instead.

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm          # Built-in dep caching
      - run: npm ci

      # Cache build output so unchanged source skips the build
      - uses: actions/cache@v4
        id: build-cache
        with:
          path: dist/
          key: build-${{ runner.os }}-${{ hashFiles('src/**', 'package-lock.json') }}
          restore-keys: |
            build-${{ runner.os }}-

      - if: steps.build-cache.outputs.cache-hit != 'true'
        run: npm run build

      - run: npm test

The restore-keys fallback is critical for incremental caching. When the exact key misses (because source changed), the prefix match restores the closest recent cache. Build tools with incremental compilation, such as Next.js, Webpack 5, Gradle, and Turborepo, can then rebuild only changed modules instead of starting from scratch. A full build that takes 8 minutes can drop to under 2 minutes with a warm incremental cache.

One caveat: cache transfer time matters. If your cached directory exceeds 5 GB, the restore step alone can take over a minute. At that point, the cache may cost more than it saves. Split large caches by purpose (dependencies vs. build output) and measure whether the cache step itself has become a bottleneck. GitHub provides 10 GB of cache storage per repository; if you're seeing low hit rates, check cache size and prune stale branch caches.

Step 3

Speed up build steps

Build steps (compile, bundle, transpile) are often the single largest line item in a CI workflow. When multiple jobs each run the same build from scratch, the cost multiplies. An 8-minute build running in 3 jobs costs 24 minutes per run. The fix has two parts: build once, and enable framework-level caching so even that single build runs faster.

Build once and share output via artifacts

The most impactful fix is structural: build once in a dedicated job, then share the output with downstream jobs using actions/upload-artifact and actions/download-artifact. Test, lint, and deploy jobs download the pre-built output instead of rebuilding it. This turns 3 builds into 1, saving 16 minutes per run immediately.

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci

      # Restore cached build output if source hasn't changed
      - uses: actions/cache@v4
        id: build-cache
        with:
          path: dist/
          key: build-${{ runner.os }}-${{ hashFiles('src/**', 'package-lock.json') }}
          restore-keys: |
            build-${{ runner.os }}-

      # Only build if cache missed
      - if: steps.build-cache.outputs.cache-hit != 'true'
        run: npm run build

      # Share output with downstream jobs
      - uses: actions/upload-artifact@v4
        with:
          name: build-output
          path: dist/
          retention-days: 1

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          name: build-output
          path: dist/
      - run: npm test

Set retention-days: 1 on ephemeral build artifacts. You only need them for the duration of the workflow run. The default retention is 90 days for public repos, and keeping build artifacts that long wastes storage quota. The deploy job can also use download-artifact to deploy the exact artifact that was tested, never a fresh build.

Enable framework-level build caching

Most modern build tools support incremental compilation caches. When the GitHub Actions cache restores the tool's internal cache directory, the build tool can skip recompiling unchanged modules. This reduces even the non-cached builds from full recompilation to incremental updates.

For Next.js, cache .next/cache with a restore-keys fallback. On partial match, Next.js does an incremental rebuild, recompiling only changed modules. A full 5-minute build can drop to under 1 minute. For Gradle, use gradle/actions/setup-gradle@v4 with cache-read-only: true on non-main branches to prevent PR builds from polluting the cache.

Step 4

Split long test suites (but watch billable minutes)

If your test job is the bottleneck, say 20 minutes of a 25-minute pipeline, splitting it across parallel runners can cut wall-clock time significantly. But parallelism has a cost trap: GitHub bills each job independently, rounded up to the nearest minute. Splitting a 20-minute suite across 5 runners doesn't cost 20 minutes. It costs 5 × (4 min + setup overhead), and setup overhead (checkout, install, cache restore) can easily add 2–3 minutes per shard.

The key insight: parallelism reduces wall-clock time but can increase total billable minutes. Only add parallelism when it reduces total minutes, not just wall time. For a deeper look at the cost traps of excessive splitting, see over-parallelized test suites. Here's how to split tests with a matrix while controlling cost:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3]
      fail-fast: false
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci

      # Split by test file groups, not random assignment
      - run: npx jest --shard=${{ matrix.shard }}/3

If you need parallelism for developer experience but want to control cost, limit shards on PRs (e.g., 2–3) and run the full split on main where correctness matters more than speed.

Step 5

Reorder jobs so cheap checks run first

A common pattern in slow pipelines is running expensive jobs (build, E2E tests, security scans) before cheap ones (lint, typecheck, unit tests). If the linter would catch an error in 30 seconds, but it doesn't run until after a 15-minute build, you've wasted 15 minutes on a run that was always going to fail. Our guide on running expensive jobs before cheap ones covers this pattern in detail.

Use the needs keyword to create a dependency graph where fast-fail jobs gate expensive work:

jobs:
  build:    # 8 min
  test:     # 12 min
  lint:     # 1 min
  e2e:      # 15 min
# Lint fails at 1 min, but build
# and e2e already burned 8+15 min

jobs:
  lint:     # 1 min, runs first
  build:
    needs: lint  # 8 min
  test:
    needs: build # 12 min
  e2e:
    needs: build # 15 min
# Lint fails at 1 min → nothing
# else runs. Saved 35 minutes.

The tradeoff: chaining jobs increases wall-clock time on success (lint + build + test runs sequentially instead of in parallel). But it dramatically cuts billable minutes on failure, which is where most of the waste occurs. If your lint/typecheck failure rate is even 5%, the savings on failed runs outweigh the added wall time on passing runs.

Step 6

Move heavy jobs off PR triggers

Not every test needs to run on every PR. If your test-e2e job takes 6 minutes and has a 2% failure rate on PRs, 98% of those runs pass without catching anything. Running it 400 times a month costs 2,400 minutes × $0.006 = $14.40. Move the full E2E suite to main or a merge queue, and run a focused smoke test on PRs instead.

jobs:
  # Runs on every PR (fast, cheap)
  test-unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm test

  # Full E2E only on main and merge queue
  test-e2e:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run test:e2e

One caveat: if test-e2e is a required status check, skipping it on PRs will block merging. Either remove it from required checks and rely on the merge queue run, or replace it with a lightweight smoke test that satisfies the check. See our E2E tests running too often guide for patterns.

Reference

Optimization strategies compared: cost vs. speed

Always prefer strategies that reduce both wall time and billable minutes (caching, artifact reuse, build-once patterns). Use parallelism selectively, and only when the wall-time improvement justifies the cost increase after you've exhausted the cheaper optimizations first.

Reference

When larger runners make sense

For CPU-bound jobs (compilation, bundling, test execution), a larger runner can reduce wall-clock time and total cost, provided the speedup is proportional to the rate increase. But I/O-bound jobs like integration tests and database-heavy suites rarely benefit from more CPU cores.

Team / Enterprise Larger runners (4-core and above) are available on GitHub Free, Pro, and Team plans but are not included in free-tier minutes. You pay per-minute at the larger runner rate.

The "break-even time" column shows how fast the build must complete on the larger runner to cost the same as 20 minutes on a 2-core. If an 8-core runner finishes the build in 6 minutes, the per-run cost is $0.132, which is 10% more but 3.3x faster. If it finishes in 5 minutes, you break even on cost while getting 4x the speed. Always benchmark your actual build before committing to a larger runner.