From 0febe47d7b3e404fa6e494e713076c994f54fe1c Mon Sep 17 00:00:00 2001 From: Daniel Radl Date: Sat, 25 Apr 2026 00:35:11 +0200 Subject: [PATCH] refactor(assets): externalize assets from sites volume using symlink --- .../02-docker-immutability.md | 4 +- docs/06-migration/01-assets-volume-change.md | 40 ++++++++++++ ...d => 02-migrate-from-multi-image-setup.md} | 11 ---- ...igration.md => 04-traefik-v3-migration.md} | 0 .../08-reference/07-how-assets-are-handled.md | 61 +++++++++++++++++++ images/custom/Containerfile | 5 ++ images/layered/Containerfile | 5 ++ images/production/Containerfile | 5 ++ 8 files changed, 118 insertions(+), 13 deletions(-) create mode 100644 docs/06-migration/01-assets-volume-change.md rename docs/06-migration/{01-migrate-from-multi-image-setup.md => 02-migrate-from-multi-image-setup.md} (84%) rename docs/06-migration/{02-traefik-v3-migration.md => 04-traefik-v3-migration.md} (100%) create mode 100644 docs/08-reference/07-how-assets-are-handled.md diff --git a/docs/01-getting-started/02-docker-immutability.md b/docs/01-getting-started/02-docker-immutability.md index c42ba65d..259db0ba 100644 --- a/docs/01-getting-started/02-docker-immutability.md +++ b/docs/01-getting-started/02-docker-immutability.md @@ -35,11 +35,11 @@ This allows you to: Installing apps into a running container is **not supported**. -`bench get-app` is an examples of an common but unsupported action. +`bench get-app` and `bench build` are examples of common but unsupported actions. ### Why? -- Apps are part of the **Docker image** +- Apps and assets are part of the **Docker image** - Runtime changes are lost on container recreation - This ensures reproducibility and stability diff --git a/docs/06-migration/01-assets-volume-change.md b/docs/06-migration/01-assets-volume-change.md new file mode 100644 index 00000000..f7c4ae65 --- /dev/null +++ b/docs/06-migration/01-assets-volume-change.md @@ -0,0 +1,40 @@ +--- +title: Assets Volume Change +--- + +# Migration Guide: Assets Volume Change + +## Background + +The way `sites/assets` is handled has changed. Previously, assets were stored inside the a volume and persisted across container recreations. This could caused stale or mismatched assets after image updates. See [Assets Reference](../08-reference/07-how-assets-are-handeled.md) for details on the new approach. + +## Who needs to migrate? + +Anyone running an existing setup where the `sites` volume was created with **`frappe_docker` version `v3.1.0` or lower**. + +**New setups are unaffected.** + +## Migration Steps + +1. **Pull an updated Image** + +2. **Recreate all containers (`docker compose up --force-recreate`)** + +3. **Enter the backend container** + + ```bash + docker compose -p frappe exec -it backend bash + ``` + +4. **Run commands in container** + ```bash + rm -rf /home/frappe/frappe-bench/sites/assets && \ + ln -s /home/frappe/frappe-bench/assets /home/frappe/frappe-bench/sites/assets && \ + exit + ``` + +## What this does + +Replaces `sites/assets` directory with a symlink pointing to `/home/frappe/frappe-bench/assets`, which lives in the image layer. This ensures assets always match the running image version. + +After this manual migration is made once no further steps are needed on further deployments. diff --git a/docs/06-migration/01-migrate-from-multi-image-setup.md b/docs/06-migration/02-migrate-from-multi-image-setup.md similarity index 84% rename from docs/06-migration/01-migrate-from-multi-image-setup.md rename to docs/06-migration/02-migrate-from-multi-image-setup.md index 04d8a2c1..0fd2e344 100644 --- a/docs/06-migration/01-migrate-from-multi-image-setup.md +++ b/docs/06-migration/02-migrate-from-multi-image-setup.md @@ -114,14 +114,3 @@ create-site: # ... removed for brevity ``` - -## Upgrading from images with a nested sites/assets volume - -Previous images declared `VOLUME /home/frappe/frappe-bench/sites/assets` separately. This created an implicit nested mountpoint inside the `sites` volume, which could cause Docker to attach different anonymous volumes per container in multi-container setups. -That declaration has been removed. `sites` is now the single shared mount, consistent with the compose setup and docs. - -**After pulling the updated image:** - -- Recreate all containers (`docker compose up --force-recreate`). Without this, Docker may keep the old anonymous `sites/assets` volume - attached from before the change. -- No `bench build` is needed — this only fixes mount consistency, not the asset workflow. diff --git a/docs/06-migration/02-traefik-v3-migration.md b/docs/06-migration/04-traefik-v3-migration.md similarity index 100% rename from docs/06-migration/02-traefik-v3-migration.md rename to docs/06-migration/04-traefik-v3-migration.md diff --git a/docs/08-reference/07-how-assets-are-handled.md b/docs/08-reference/07-how-assets-are-handled.md new file mode 100644 index 00000000..96f2a99d --- /dev/null +++ b/docs/08-reference/07-how-assets-are-handled.md @@ -0,0 +1,61 @@ +--- +title: How Assets are handled +--- + +# Assets Reference + +## Problem + +The `sites` directory contains both persistent data (site config, uploaded files, etc.) and build-time artifacts (`sites/assets`). Mounting the entire `sites` directory as a Docker volume causes assets to be persisted alongside config, which leads to: + +- Stale assets surviving image updates +- Asset/manifest mismatches after rebuilds +- Assets being tied to the volume lifecycle rather than the image lifecycle + +## Solution + +Assets are moved out of the `sites` volume during the Docker build and replaced with a **symlink**. This means assets always come from the image layer, while the rest of `sites` remains persistent. + +### How it works + +During the image build (`Containerfile`), the following is done: + +```dockerfile +RUN cp -r /home/frappe/frappe-bench/sites/assets /home/frappe/frappe-bench/assets && \ + rm -rf /home/frappe/frappe-bench/sites/assets && \ + ln -s /home/frappe/frappe-bench/assets /home/frappe/frappe-bench/sites/assets +``` + +This runs **before** the `VOLUME` declaration, so the symlink is baked into +the image layer. + +At runtime: + +``` +/home/frappe/frappe-bench/ +├── assets/ ← image layer (ephemeral, always matches the image) +├── sites/ +│ ├── assets -> /home/frappe/frappe-bench/assets ← symlink +│ ├── common_site_config.json ← persisted in volume +│ └── / ← persisted in volume +└── logs/ ← persisted in volume +``` + +### Volume behavior + +| Path | Persistent | Source | +| -------------------------- | ----------------------- | ---------------------- | +| `sites/` (except assets) | ✅ Yes | Named volume | +| `sites/assets` (symlink) | ✅ Yes (symlink itself) | Named volume (`sites`) | +| `assets/` (symlink target) | ❌ No | Image layer | +| `logs/` | ✅ Yes | Unnamed volume | + +The symlink itself is persisted in the volume, but it always points to +`assets/` which lives in the image layer and is discarded on container +recreation. + +## Important: `bench build` at runtime + +Running `bench build` inside a running container will write new assets and eventually cause a mismatch between `assets.json` and the actual assets, breaking the UI. This can be recovered by recreating the containers + +> Note: restarting the containers is not sufficient — they need to be recreated to discard the writable layer. diff --git a/images/custom/Containerfile b/images/custom/Containerfile index febc72b9..368e7bac 100644 --- a/images/custom/Containerfile +++ b/images/custom/Containerfile @@ -143,6 +143,11 @@ COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe WORKDIR /home/frappe/frappe-bench +# Move assets out of the sites volume, replace with symlink +RUN cp -r /home/frappe/frappe-bench/sites/assets /home/frappe/frappe-bench/assets && \ + rm -rf /home/frappe/frappe-bench/sites/assets && \ + ln -s /home/frappe/frappe-bench/assets /home/frappe/frappe-bench/sites/assets + VOLUME [ \ "/home/frappe/frappe-bench/sites", \ "/home/frappe/frappe-bench/logs" \ diff --git a/images/layered/Containerfile b/images/layered/Containerfile index 808f898f..3cdfe33c 100644 --- a/images/layered/Containerfile +++ b/images/layered/Containerfile @@ -33,6 +33,11 @@ COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe WORKDIR /home/frappe/frappe-bench +# Move assets out of the sites volume, replace with symlink +RUN cp -r /home/frappe/frappe-bench/sites/assets /home/frappe/frappe-bench/assets && \ + rm -rf /home/frappe/frappe-bench/sites/assets && \ + ln -s /home/frappe/frappe-bench/assets /home/frappe/frappe-bench/sites/assets + VOLUME [ \ "/home/frappe/frappe-bench/sites", \ "/home/frappe/frappe-bench/logs" \ diff --git a/images/production/Containerfile b/images/production/Containerfile index 48df0d9d..dadc7814 100644 --- a/images/production/Containerfile +++ b/images/production/Containerfile @@ -136,6 +136,11 @@ COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe WORKDIR /home/frappe/frappe-bench +# Move assets out of the sites volume, replace with symlink +RUN cp -r /home/frappe/frappe-bench/sites/assets /home/frappe/frappe-bench/assets && \ + rm -rf /home/frappe/frappe-bench/sites/assets && \ + ln -s /home/frappe/frappe-bench/assets /home/frappe/frappe-bench/sites/assets + VOLUME [ \ "/home/frappe/frappe-bench/sites", \ "/home/frappe/frappe-bench/logs" \