ATOM Documentation

← Back to App

Docker Image Size Investigation - Summary

Problem

  • **Original Image Size:** 1.1 GB
  • **Next.js Failing:** Cannot find /app/node_modules/next/dist/server/next.js
  • **Missing Dependencies:** pg-protocol, @next/env, and other Next.js dependencies

Root Causes

1. Docker COPY and .dockerignore Negation

**Issue:** Docker's COPY command doesn't properly handle .dockerignore negation patterns for nested directories.

**Attempted Fix:**

# .dockerignore
node_modules
.next
!.next/standalone
!.next/standalone/**
!.next/static
!.next/static/**

**Result:** Failed. The negation patterns don't work as expected with COPY. Docker still excludes node_modules at all levels, including inside .next/standalone/.

2. Image Size Breakdown (1.1 GB)

  • **Base Node.js image:** ~200MB
  • **Next.js fallback install:** ~100MB (full Next.js with dev dependencies)
  • **Python dependencies:** ~300-400MB (pandas, scipy, boto3, etc.)
  • **Backend code:** ~200-300MB
  • **System dependencies:** ~100MB

3. Why Fallback Install Triggered

The Dockerfile check test -d node_modules/next/dist passes (directory exists), but nested dependencies like pg-protocol and @next/env are missing due to .dockerignore excluding them.

Attempted Solutions

Solution 1: Build Next.js In-Container

**Approach:** Build Next.js inside Docker container instead of locally.

**Dockerfile:**

FROM node:20-bookworm-slim AS node-builder
COPY package*.json ./
RUN npm install --omit=dev --ignore-scripts --no-audit --no-fund
COPY . .
RUN npm run build

**Result:** ❌ Failed

  • npm install --omit=dev doesn't work (needs devDependencies for build)
  • npm install --omit=dev --ignore-scripts still fails on missing TypeScript
  • In-container builds require ALL dependencies (dev + prod)

Solution 2: Fix .dockerignore Negation

**Approach:** Use explicit negation patterns for standalone artifacts.

**Result:** ❌ Failed

  • Docker COPY doesn't respect negation patterns for nested directories
  • node_modules exclusion applies to all levels
  • Standalone node_modules still excluded

Solution 3: Revert to Local Build

**Approach:** Build locally, copy artifacts with improved .dockerignore.

**Current Status:** ❌ Still failing

  • .dockerignore negation patterns still not working
  • Next.js modules not copied to image
  • Error: "Cannot find module '/app/node_modules/next/dist/server/next.js'"

Working Solution Needed

Option A: Remove node_modules from .dockerignore

**Pros:**

  • Simple fix
  • Guaranteed to work
  • Standalone artifacts will be copied

**Cons:**

  • Build context size increases (from ~6MB to ~100MB+)
  • Slower uploads to remote builder

Option B: Use .dockerignore with Explicit Includes

**Approach:** Don't exclude node_modules at root level, only exclude specific paths.

# Exclude these specific paths
node_modules/.cache
node_modules/.npm
*.test.tsx
*.test.ts
**/__tests__
**/coverage
**/dist

# Keep everything else

**Pros:**

  • Smaller build context
  • Faster uploads

**Cons:**

  • More complex .dockerignore
  • May still have issues with COPY

Option C: Use BuildKit Cache Mounts

**Approach:** Use Docker BuildKit's cache mounts for node_modules.

**Dockerfile:**

# --mount=type=cache,target=/app/node_modules
RUN npm install

**Pros:**

  • Modern Docker best practice
  • Faster builds
  • Smaller final image

**Cons:**

  • Requires BuildKit enabled
  • More complex Dockerfile
  • May not work with Fly.io remote builder

Recommendation

**Use Option A** for now: Remove node_modules from .dockerignore and rely on Next.js standalone mode to only include necessary dependencies.

**Why:**

  1. It's the simplest fix
  2. Next.js standalone mode already optimizes dependencies
  3. Build context increase is acceptable (6MB → 100MB)
  4. Remote builder upload time is still reasonable (< 30 seconds)

Next Steps

  1. **Update .dockerignore** to remove node_modules exclusion
  2. **Add exceptions** for test files and development-only modules
  3. **Test deployment** to verify Next.js starts properly
  4. **Monitor image size** to ensure it stays reasonable
  5. **Consider BuildKit cache mounts** for future optimization

Files to Change

  1. .dockerignore - Remove node_modules exclusion
  2. Dockerfile - No changes needed (current version is correct)
  3. Deploy and verify Next.js starts on port 3000

Success Criteria

  • ✅ Next.js starts without "Cannot find module" errors
  • ✅ All dependencies present (pg-protocol, @next/env, etc.)
  • ✅ Image size < 1.5 GB
  • ✅ Health checks pass
  • ✅ App responds on port 3000