From d2308438d0799bc7527d0cf8a342ba1f1d94ae83 Mon Sep 17 00:00:00 2001 From: jslocomotor <210083531+jslocomotor@users.noreply.github.com> Date: Fri, 24 Apr 2026 17:11:06 +0200 Subject: [PATCH 1/4] fix(build): add apps.json hash for cache invalidation --- docs/02-setup/02-build-setup.md | 78 +++++++++++++++++++++------------ images/custom/Containerfile | 2 + images/layered/Containerfile | 2 + 3 files changed, 55 insertions(+), 27 deletions(-) diff --git a/docs/02-setup/02-build-setup.md b/docs/02-setup/02-build-setup.md index d0e6907b..9d14e596 100644 --- a/docs/02-setup/02-build-setup.md +++ b/docs/02-setup/02-build-setup.md @@ -7,7 +7,7 @@ This guide walks you through building Frappe images from the repository resource # Prerequisites - git -- docker (Engine **v23.0+**) or podman +- docker (Engine **v23.0+** with buildx) or podman - docker compose v2 or podman compose > Install containerization software according to the official maintainer documentation. Avoid package managers when not recommended, as they frequently cause compatibility issues. @@ -23,7 +23,7 @@ cd frappe_docker # Define custom apps -If you dont want to install specific apps to the image skip this section. +If you don't want to include custom apps in the image, skip this section. To include custom apps in your image, create an `apps.json` file in the repository root: @@ -31,11 +31,11 @@ To include custom apps in your image, create an `apps.json` file in the reposito [ { "url": "https://github.com/frappe/erpnext", - "branch": "version-15" + "branch": "version-16" }, { "url": "https://github.com/frappe/hrms", - "branch": "version-15" + "branch": "version-16" }, { "url": "https://github.com/frappe/helpdesk", @@ -44,7 +44,9 @@ To include custom apps in your image, create an `apps.json` file in the reposito ] ``` -# Build the image +# Build custom images + +## Manually Choose the appropriate build command based on your container runtime and desired image type. This example builds the `layered` image with the custom `apps.json` you created. @@ -55,9 +57,10 @@ Choose the appropriate build command based on your container runtime and desired ```bash docker build \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-15 \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ --secret=id=apps_json,src=apps.json \ - --tag=custom:15 \ + --tag=custom:16 \ --file=images/layered/Containerfile . ``` @@ -66,30 +69,51 @@ docker build \ ```bash podman build \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-15 \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ --secret=id=apps_json,src=apps.json \ - --tag=custom:15 \ + --tag=custom:16 \ --file=images/layered/Containerfile . ``` -## Build args +## CI/CD pipelines -| Arg | Purpose | -| -------------------- | ----------------------------------------------------------------------------------------------- | -| **Frappe Framework** | | -| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | -| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-15 | -| **Custom Apps** | | -| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | -| **Dependencies** | | -| PYTHON_VERSION | Python version for the base image | -| NODE_VERSION | Node.js version | -| WKHTMLTOPDF_VERSION | wkhtmltopdf version | -| **bench only** | | -| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | -| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | +Example: -# env file +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +## Build args, secrets and flags + +| Variable | Purpose | +| -------------------- | ------------------------------------------------------------------------------------------------ | +| **Frappe Framework** | | +| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | +| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | +| **Custom Apps** | | +| APPS_JSON_HASH | Hash of `apps.json`, used to invalidate the cached layer when `apps_json` is passed as a secret. | +| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | +| **Dependencies** | | +| PYTHON_VERSION | Python version for the base image | +| NODE_VERSION | Node.js version | +| WKHTMLTOPDF_VERSION | wkhtmltopdf version | +| **bench only** | | +| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | +| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | + +# Deploy the stack + +## env file The compose file requires several environment variables. You can either export them on your system or create a `.env` file. @@ -103,7 +127,7 @@ For this setup, make sure **at least** the following values are added to `custom ```txt CUSTOM_IMAGE=custom -CUSTOM_TAG=15 +CUSTOM_TAG=16 PULL_POLICY=missing ``` @@ -113,7 +137,7 @@ PULL_POLICY=missing **⚠️ This is not meant to be a complete `.env` configuration guide. These are only the minimal additions required for this example. Please have a look at [env-variables.md](04-env-variables.md) for a full description of all available variables and adjust them according to your needs.** -# Creating the final compose file +## Creating the final compose file Combine the base compose file with appropriate overrides for your use case. This example adds MariaDB, Redis, and exposes ports on `:8080`: diff --git a/images/custom/Containerfile b/images/custom/Containerfile index febc72b9..298f0381 100644 --- a/images/custom/Containerfile +++ b/images/custom/Containerfile @@ -118,7 +118,9 @@ USER frappe ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe +ARG APPS_JSON_HASH="" RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ + : "${APPS_JSON_HASH}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \ diff --git a/images/layered/Containerfile b/images/layered/Containerfile index 808f898f..fd1398a5 100644 --- a/images/layered/Containerfile +++ b/images/layered/Containerfile @@ -5,10 +5,12 @@ FROM ${FRAPPE_IMAGE_PREFIX}/build:${FRAPPE_BRANCH} AS builder ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe +ARG APPS_JSON_HASH="" USER frappe RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ + : "${APPS_JSON_HASH}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \ From a260d9a4314dcfa043d0dbfffbd9eded0f266746 Mon Sep 17 00:00:00 2001 From: jslocomotor <210083531+jslocomotor@users.noreply.github.com> Date: Sat, 25 Apr 2026 17:06:20 +0200 Subject: [PATCH 2/4] fix(build): add optional CACHE_BUST for custom image rebuilds --- docs/02-setup/02-build-setup.md | 74 ++++++++++++++++++++++----------- images/custom/Containerfile | 5 ++- images/layered/Containerfile | 4 +- 3 files changed, 55 insertions(+), 28 deletions(-) diff --git a/docs/02-setup/02-build-setup.md b/docs/02-setup/02-build-setup.md index 9d14e596..34a1b16d 100644 --- a/docs/02-setup/02-build-setup.md +++ b/docs/02-setup/02-build-setup.md @@ -56,9 +56,9 @@ Choose the appropriate build command based on your container runtime and desired ```bash docker build \ + --no-cache \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ --build-arg=FRAPPE_BRANCH=version-16 \ - --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ --secret=id=apps_json,src=apps.json \ --tag=custom:16 \ --file=images/layered/Containerfile . @@ -68,9 +68,9 @@ docker build \ ```bash podman build \ + --no-cache \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ --build-arg=FRAPPE_BRANCH=version-16 \ - --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ --secret=id=apps_json,src=apps.json \ --tag=custom:16 \ --file=images/layered/Containerfile . @@ -78,38 +78,64 @@ podman build \ ## CI/CD pipelines -Example: +When using automated builds, `CACHE_BUST` can be used to control cache invalidation of the frappe layer in order to rebuild it. + +Possible techniques: + +- No override: normal Docker layer caching is used +- Timestamp: force a rebuild on every pipeline run +- Pipeline run ID: rebuild once per CI run +- Commit SHA: rebuild once per commit +- apps.json hash: rebuild only when the custom app definition changes + +Examples: +Commit SHA from GitHub ```yaml - name: Build Docker image shell: sh run: | docker build \ - --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-16 \ - --build-arg=APPS_JSON_HASH="$(sha256sum apps.json | awk '{print $1}')" \ - --secret=id=apps_json,src=apps.json \ - --tag=custom:16 \ - --file=images/layered/Containerfile . + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$GITHUB_SHA" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +apps.json hash + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$(sha256sum apps.json | awk '{print $1}')" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . ``` ## Build args, secrets and flags -| Variable | Purpose | -| -------------------- | ------------------------------------------------------------------------------------------------ | -| **Frappe Framework** | | -| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | -| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | -| **Custom Apps** | | -| APPS_JSON_HASH | Hash of `apps.json`, used to invalidate the cached layer when `apps_json` is passed as a secret. | -| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | -| **Dependencies** | | -| PYTHON_VERSION | Python version for the base image | -| NODE_VERSION | Node.js version | -| WKHTMLTOPDF_VERSION | wkhtmltopdf version | -| **bench only** | | -| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | -| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | +| Variable | Purpose | +| -------------------- | ----------------------------------------------------------------------------------------------- | +| **Frappe Framework** | | +| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | +| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | +| **Custom Apps** | | +| CACHE_BUST | Can be used to invalidate the cached layer when `apps_json` is passed as a secret. | +| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | +| **Dependencies** | | +| PYTHON_VERSION | Python version for the base image | +| NODE_VERSION | Node.js version | +| WKHTMLTOPDF_VERSION | wkhtmltopdf version | +| **bench only** | | +| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | +| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | # Deploy the stack diff --git a/images/custom/Containerfile b/images/custom/Containerfile index 298f0381..c99ff880 100644 --- a/images/custom/Containerfile +++ b/images/custom/Containerfile @@ -118,9 +118,10 @@ USER frappe ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe -ARG APPS_JSON_HASH="" +ARG CACHE_BUST="" + RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ - : "${APPS_JSON_HASH}" && \ + : "${CACHE_BUST}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \ diff --git a/images/layered/Containerfile b/images/layered/Containerfile index fd1398a5..0048930f 100644 --- a/images/layered/Containerfile +++ b/images/layered/Containerfile @@ -5,12 +5,12 @@ FROM ${FRAPPE_IMAGE_PREFIX}/build:${FRAPPE_BRANCH} AS builder ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe -ARG APPS_JSON_HASH="" +ARG CACHE_BUST="" USER frappe RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ - : "${APPS_JSON_HASH}" && \ + : "${CACHE_BUST}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \ From 8428dfe9ba39e8ae96c5a683e2db678c8a82f3a8 Mon Sep 17 00:00:00 2001 From: jslocomotor <210083531+jslocomotor@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:37:16 +0200 Subject: [PATCH 3/4] docs(production): add automated builds and deployment guide --- docs/02-setup/02-build-setup.md | 43 +----- .../06-automated-builds-and-deployment.md | 128 ++++++++++++++++++ 2 files changed, 131 insertions(+), 40 deletions(-) create mode 100644 docs/03-production/06-automated-builds-and-deployment.md diff --git a/docs/02-setup/02-build-setup.md b/docs/02-setup/02-build-setup.md index 34a1b16d..0bf47810 100644 --- a/docs/02-setup/02-build-setup.md +++ b/docs/02-setup/02-build-setup.md @@ -76,48 +76,11 @@ podman build \ --file=images/layered/Containerfile . ``` -## CI/CD pipelines +## Automated -When using automated builds, `CACHE_BUST` can be used to control cache invalidation of the frappe layer in order to rebuild it. +This repository is fully suited for automated builds, i.e. using CI/CD pipelines. -Possible techniques: - -- No override: normal Docker layer caching is used -- Timestamp: force a rebuild on every pipeline run -- Pipeline run ID: rebuild once per CI run -- Commit SHA: rebuild once per commit -- apps.json hash: rebuild only when the custom app definition changes - -Examples: -Commit SHA from GitHub - -```yaml -- name: Build Docker image - shell: sh - run: | - docker build \ - --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-16 \ - --build-arg=CACHE_BUST="$GITHUB_SHA" \ - --secret=id=apps_json,src=apps.json \ - --tag=custom:16 \ - --file=images/layered/Containerfile . -``` - -apps.json hash - -```yaml -- name: Build Docker image - shell: sh - run: | - docker build \ - --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-16 \ - --build-arg=CACHE_BUST="$(sha256sum apps.json | awk '{print $1}')" \ - --secret=id=apps_json,src=apps.json \ - --tag=custom:16 \ - --file=images/layered/Containerfile . -``` +See [Automated Builds and Deployment](../03-production/06-automated-builds-and-deployment.md) for more information. ## Build args, secrets and flags diff --git a/docs/03-production/06-automated-builds-and-deployment.md b/docs/03-production/06-automated-builds-and-deployment.md new file mode 100644 index 00000000..85113f51 --- /dev/null +++ b/docs/03-production/06-automated-builds-and-deployment.md @@ -0,0 +1,128 @@ +--- +title: Automated Builds and Deployment +--- + +# Introduction + +This is a brief guide to automated builds and deployment for custom Frappe images. +Depending on your specific setup, environment and security rules, the information below may need to be adapted to your needs. + +# Requirements + +## Knowledge + +Basic knowledge of Docker and build pipelines is expected. + +Please refer to the Setup chapter first, especially [Build Setup](../02-setup/02-build-setup.md), for basic understanding. + +## Additional Files + +### Apps + +At build time an `apps.json` file can be provided. This specifies additional Frappe framework compatible apps to include in custom images. + +### Build + +A workflow file for your CI platform and environment is required. + +## Build Cache + +Unlike manual builds, automated build commands should generally not use `--no-cache`. + +Reusing cached layers can greatly reduce build times, disk usage, and bandwidth usage when pushing to image registries. + +Instead, `CACHE_BUST` can be used to control cache invalidation of the Frappe layer when rebuilding is desired. + +This is especially relevant because `apps.json` is provided as a secret. Secret contents are not part of Docker layer cache keys and therefore cannot trigger cache invalidation automatically. + +As a result, Docker may reuse an older cached layer even when the custom app definition has changed. + +Exception: Newer releases of the Frappe framework may still trigger rebuilding the layer. + +### Possible techniques for cache invalidation using `CACHE_BUST`: + +1. No override: normal Docker layer caching is used - not recommended in this use case +2. Timestamp: force a rebuild on every pipeline run - since the value will change every run +3. Pipeline run ID: rebuild once per CI run +4. Commit SHA: rebuild once per commit +5. apps.json hash: rebuild only when the custom app definition changes - additional requirements, see below example + +### Examples: + +#### 1. No override - not recommended + +This will reuse a previously build layer and won't check for app updates except Frappe framework + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 2. Timestamp + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$(date +%s)" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 3. Pipeline run ID from GitHub + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$GITHUB_RUN_ID" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 4. Commit SHA from GitHub + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$GITHUB_SHA" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 5. apps.json hash + +Note: When using branch references in `apps.json`, the hash only changes when the file content changes, not when an upstream app branch receives updates. This method works best when pinning specific commits or releases. + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$(sha256sum apps.json | awk '{print $1}')" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` From 0b835b0819c7a884da0be6df3564f0b26cfd14ed Mon Sep 17 00:00:00 2001 From: jslocomotor <210083531+jslocomotor@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:56:00 +0200 Subject: [PATCH 4/4] docs(production): add reference to CACHE_BUST explanation --- docs/02-setup/02-build-setup.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/02-setup/02-build-setup.md b/docs/02-setup/02-build-setup.md index 0bf47810..326de9bf 100644 --- a/docs/02-setup/02-build-setup.md +++ b/docs/02-setup/02-build-setup.md @@ -84,21 +84,21 @@ See [Automated Builds and Deployment](../03-production/06-automated-builds-and-d ## Build args, secrets and flags -| Variable | Purpose | -| -------------------- | ----------------------------------------------------------------------------------------------- | -| **Frappe Framework** | | -| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | -| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | -| **Custom Apps** | | -| CACHE_BUST | Can be used to invalidate the cached layer when `apps_json` is passed as a secret. | -| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | -| **Dependencies** | | -| PYTHON_VERSION | Python version for the base image | -| NODE_VERSION | Node.js version | -| WKHTMLTOPDF_VERSION | wkhtmltopdf version | -| **bench only** | | -| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | -| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | +| Variable | Purpose | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| **Frappe Framework** | | +| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | +| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | +| **Custom Apps** | | +| CACHE_BUST | Can be used to invalidate the cached layer. See [Build Cache](../03-production/06-automated-builds-and-deployment.md#build-cache) | +| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | +| **Dependencies** | | +| PYTHON_VERSION | Python version for the base image | +| NODE_VERSION | Node.js version | +| WKHTMLTOPDF_VERSION | wkhtmltopdf version | +| **bench only** | | +| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | +| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | # Deploy the stack