diff --git a/docs/02-setup/02-build-setup.md b/docs/02-setup/02-build-setup.md index d0e6907b..326de9bf 100644 --- a/docs/02-setup/02-build-setup.md +++ b/docs/02-setup/02-build-setup.md @@ -7,7 +7,7 @@ This guide walks you through building Frappe images from the repository resource # Prerequisites - git -- docker (Engine **v23.0+**) or podman +- docker (Engine **v23.0+** with buildx) or podman - docker compose v2 or podman compose > Install containerization software according to the official maintainer documentation. Avoid package managers when not recommended, as they frequently cause compatibility issues. @@ -23,7 +23,7 @@ cd frappe_docker # Define custom apps -If you dont want to install specific apps to the image skip this section. +If you don't want to include custom apps in the image, skip this section. To include custom apps in your image, create an `apps.json` file in the repository root: @@ -31,11 +31,11 @@ To include custom apps in your image, create an `apps.json` file in the reposito [ { "url": "https://github.com/frappe/erpnext", - "branch": "version-15" + "branch": "version-16" }, { "url": "https://github.com/frappe/hrms", - "branch": "version-15" + "branch": "version-16" }, { "url": "https://github.com/frappe/helpdesk", @@ -44,7 +44,9 @@ To include custom apps in your image, create an `apps.json` file in the reposito ] ``` -# Build the image +# Build custom images + +## Manually Choose the appropriate build command based on your container runtime and desired image type. This example builds the `layered` image with the custom `apps.json` you created. @@ -54,10 +56,11 @@ Choose the appropriate build command based on your container runtime and desired ```bash docker build \ + --no-cache \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-15 \ + --build-arg=FRAPPE_BRANCH=version-16 \ --secret=id=apps_json,src=apps.json \ - --tag=custom:15 \ + --tag=custom:16 \ --file=images/layered/Containerfile . ``` @@ -65,31 +68,41 @@ docker build \ ```bash podman build \ + --no-cache \ --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-15 \ + --build-arg=FRAPPE_BRANCH=version-16 \ --secret=id=apps_json,src=apps.json \ - --tag=custom:15 \ + --tag=custom:16 \ --file=images/layered/Containerfile . ``` -## Build args +## Automated -| Arg | Purpose | -| -------------------- | ----------------------------------------------------------------------------------------------- | -| **Frappe Framework** | | -| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | -| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-15 | -| **Custom Apps** | | -| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | -| **Dependencies** | | -| PYTHON_VERSION | Python version for the base image | -| NODE_VERSION | Node.js version | -| WKHTMLTOPDF_VERSION | wkhtmltopdf version | -| **bench only** | | -| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | -| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | +This repository is fully suited for automated builds, i.e. using CI/CD pipelines. -# env file +See [Automated Builds and Deployment](../03-production/06-automated-builds-and-deployment.md) for more information. + +## Build args, secrets and flags + +| Variable | Purpose | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| **Frappe Framework** | | +| FRAPPE_PATH | Repository URL for Frappe framework source code. Defaults to | +| FRAPPE_BRANCH | Branch to use for Frappe framework. Defaults to version-16 | +| **Custom Apps** | | +| CACHE_BUST | Can be used to invalidate the cached layer. See [Build Cache](../03-production/06-automated-builds-and-deployment.md#build-cache) | +| (secret) apps_json | Passed via `--secret=id=apps_json,src=apps.json`. Never use `--build-arg` for this file. | +| **Dependencies** | | +| PYTHON_VERSION | Python version for the base image | +| NODE_VERSION | Node.js version | +| WKHTMLTOPDF_VERSION | wkhtmltopdf version | +| **bench only** | | +| DEBIAN_BASE | Debian base version for the bench image, defaults to `bookworm` | +| WKHTMLTOPDF_DISTRO | use the specified distro for debian package. Default is `bookworm` | + +# Deploy the stack + +## env file The compose file requires several environment variables. You can either export them on your system or create a `.env` file. @@ -103,7 +116,7 @@ For this setup, make sure **at least** the following values are added to `custom ```txt CUSTOM_IMAGE=custom -CUSTOM_TAG=15 +CUSTOM_TAG=16 PULL_POLICY=missing ``` @@ -113,7 +126,7 @@ PULL_POLICY=missing **⚠️ This is not meant to be a complete `.env` configuration guide. These are only the minimal additions required for this example. Please have a look at [env-variables.md](04-env-variables.md) for a full description of all available variables and adjust them according to your needs.** -# Creating the final compose file +## Creating the final compose file Combine the base compose file with appropriate overrides for your use case. This example adds MariaDB, Redis, and exposes ports on `:8080`: diff --git a/docs/03-production/06-automated-builds-and-deployment.md b/docs/03-production/06-automated-builds-and-deployment.md new file mode 100644 index 00000000..85113f51 --- /dev/null +++ b/docs/03-production/06-automated-builds-and-deployment.md @@ -0,0 +1,128 @@ +--- +title: Automated Builds and Deployment +--- + +# Introduction + +This is a brief guide to automated builds and deployment for custom Frappe images. +Depending on your specific setup, environment and security rules, the information below may need to be adapted to your needs. + +# Requirements + +## Knowledge + +Basic knowledge of Docker and build pipelines is expected. + +Please refer to the Setup chapter first, especially [Build Setup](../02-setup/02-build-setup.md), for basic understanding. + +## Additional Files + +### Apps + +At build time an `apps.json` file can be provided. This specifies additional Frappe framework compatible apps to include in custom images. + +### Build + +A workflow file for your CI platform and environment is required. + +## Build Cache + +Unlike manual builds, automated build commands should generally not use `--no-cache`. + +Reusing cached layers can greatly reduce build times, disk usage, and bandwidth usage when pushing to image registries. + +Instead, `CACHE_BUST` can be used to control cache invalidation of the Frappe layer when rebuilding is desired. + +This is especially relevant because `apps.json` is provided as a secret. Secret contents are not part of Docker layer cache keys and therefore cannot trigger cache invalidation automatically. + +As a result, Docker may reuse an older cached layer even when the custom app definition has changed. + +Exception: Newer releases of the Frappe framework may still trigger rebuilding the layer. + +### Possible techniques for cache invalidation using `CACHE_BUST`: + +1. No override: normal Docker layer caching is used - not recommended in this use case +2. Timestamp: force a rebuild on every pipeline run - since the value will change every run +3. Pipeline run ID: rebuild once per CI run +4. Commit SHA: rebuild once per commit +5. apps.json hash: rebuild only when the custom app definition changes - additional requirements, see below example + +### Examples: + +#### 1. No override - not recommended + +This will reuse a previously build layer and won't check for app updates except Frappe framework + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 2. Timestamp + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$(date +%s)" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 3. Pipeline run ID from GitHub + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$GITHUB_RUN_ID" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 4. Commit SHA from GitHub + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$GITHUB_SHA" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` + +#### 5. apps.json hash + +Note: When using branch references in `apps.json`, the hash only changes when the file content changes, not when an upstream app branch receives updates. This method works best when pinning specific commits or releases. + +```yaml +- name: Build Docker image + shell: sh + run: | + docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=version-16 \ + --build-arg=CACHE_BUST="$(sha256sum apps.json | awk '{print $1}')" \ + --secret=id=apps_json,src=apps.json \ + --tag=custom:16 \ + --file=images/layered/Containerfile . +``` diff --git a/images/custom/Containerfile b/images/custom/Containerfile index febc72b9..c99ff880 100644 --- a/images/custom/Containerfile +++ b/images/custom/Containerfile @@ -118,7 +118,10 @@ USER frappe ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe +ARG CACHE_BUST="" + RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ + : "${CACHE_BUST}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \ diff --git a/images/layered/Containerfile b/images/layered/Containerfile index 808f898f..0048930f 100644 --- a/images/layered/Containerfile +++ b/images/layered/Containerfile @@ -5,10 +5,12 @@ FROM ${FRAPPE_IMAGE_PREFIX}/build:${FRAPPE_BRANCH} AS builder ARG FRAPPE_BRANCH=version-16 ARG FRAPPE_PATH=https://github.com/frappe/frappe +ARG CACHE_BUST="" USER frappe RUN --mount=type=secret,id=apps_json,target=/opt/frappe/apps.json,uid=1000,gid=1000 \ + : "${CACHE_BUST}" && \ export APP_INSTALL_ARGS="" && \ if [ -f /opt/frappe/apps.json ] && [ -s /opt/frappe/apps.json ]; then \ export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \