diff --git a/production/apps.json b/production/apps.json new file mode 100644 index 00000000..4ac0921f --- /dev/null +++ b/production/apps.json @@ -0,0 +1,10 @@ +[ + { + "url": "https://github.com/frappe/erpnext", + "branch": "v15.88.1" + }, + { + "url": "https://github.com/resilient-tech/india-compliance", + "branch": "v15.23.2" + } +] \ No newline at end of file diff --git a/production/README.md b/production/docs/README.md similarity index 80% rename from production/README.md rename to production/docs/README.md index 9ebfcb3f..d8fdb3db 100644 --- a/production/README.md +++ b/production/docs/README.md @@ -513,107 +513,230 @@ docker compose -f production/production.yaml ps ## Custom Apps & Third-Party Integrations -### Why ship custom logic as apps? +**Production Standard**: This guide covers deploying ERPNext with custom or third-party apps using **immutable Docker images** with pre-compiled assets. This approach provides reproducible deployments, instant rollbacks, and eliminates runtime build complexity. -- **Upstream-safe**: Apps keep your business logic outside the upstream fork, so rebasing on `frappe_docker` stays painless. -- **Repeatable**: Every site receives the exact same code (DocTypes, patches, API clients) whenever the container is rebuilt. -- **Supported**: This mirrors the official [frappe_docker custom app workflow](../docs/container-setup/02-build-setup.md#define-custom-apps). +> **πŸ“– Detailed Guide**: For comprehensive implementation details, troubleshooting, and CI/CD integration, see [`production/docs/custom-image-workflow.md`](docs/custom-image-workflow.md) -### 1. Describe the apps you need (`apps.json`) +> **πŸ“š Technical Deep-Dive**: For understanding different deployment patterns and asset management approaches, see [`production/docs/asset-management-frappe.md`](docs/asset-management-frappe.md) -Create a manifest in the repository root that lists every app you want baked into the imageβ€”first-party or third-party: +### Why Custom Images? + +- **True Immutability**: Apps frozen at specific versions (tags/commits) +- **Zero Runtime Builds**: Assets pre-compiled during image build +- **Reliable Rollbacks**: Switch image tags to revert instantly +- **Upstream-Safe**: Custom logic isolated from infrastructure updates +- **Audit Trail**: Image tag maps to exact deployed code + +### 1. Create apps.json with Pinned Versions + +Create `production/apps.json` listing **custom/third-party apps only** with pinned versions: ```json [ - { "url": "https://github.com/frappe/erpnext", "branch": "version-15" }, - { "url": "https://github.com/frappe/hrms", "branch": "version-15" }, - { "url": "https://github.com/acme/custom_integrations", "branch": "main" } + { + "url": "https://github.com/frappe/erpnext", + "branch": "v15.88.1" + }, + { + "url": "https://github.com/resilient-tech/india-compliance", + "branch": "v15.23.2" + }, + { + "url": "https://github.com/frappe/hrms", + "branch": "v15.12.0" + } ] ``` -Convert it to base64 once so the build context can read it without extra files: +**Important**: +- **Frappe Framework** is controlled via `FRAPPE_BRANCH` build arg (not in apps.json) +- Use specific tags (e.g., `v15.88.1`) for custom apps, NOT moving branches (e.g., `version-15`) +- This ensures reproducible buildsβ€”same apps.json = identical image +**Find available versions**: ```bash -export APPS_JSON_BASE64=$(base64 -w 0 apps.json) +# Check tags on GitHub +curl -s https://api.github.com/repos/frappe/erpnext/tags | grep '"name"' | head -5 +curl -s https://api.github.com/repos/resilient-tech/india-compliance/tags | grep '"name"' | head -5 + +# For Frappe Framework (use as FRAPPE_BRANCH build arg) +curl -s https://api.github.com/repos/frappe/frappe/tags | grep '"name"' | head -5 ``` -### 2. Build (and optionally push) a custom ERPNext image +### 2. Build Immutable Image -Use the official layered image as the base and inject your apps list: +Build a custom image with your apps and pre-compiled assets: ```bash +# Encode apps.json +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) + +# Generate traceable image tag (date + git commit) +BUILD_DATE=$(date +%Y%m%d) +GIT_SHA=$(git rev-parse --short HEAD) +IMAGE_TAG="ghcr.io/YOUR_USERNAME/erpnext-custom:${BUILD_DATE}-${GIT_SHA}" + +# Build image (includes bench build - assets compiled into image) docker build \ - --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ - --build-arg=FRAPPE_BRANCH=version-15 \ --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ - --tag=registry.example.com/erpnext-custom:15 \ - --file=images/layered/Containerfile . - -# optional -docker push registry.example.com/erpnext-custom:15 + --build-arg=PYTHON_VERSION=3.11.6 \ + --build-arg=NODE_VERSION=18.18.2 \ + --tag=$IMAGE_TAG \ + --tag=ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest \ + --file=images/layered/Containerfile \ + . ``` -> Prefer `docker buildx bake -f docker-bake.hcl --set erpnext.args.APPS_JSON_BASE64=$APPS_JSON_BASE64` if you already rely on Buildx/CI. +**What happens during build**: +1. Installs all apps from `apps.json` +2. Installs Python and Node.js dependencies +3. **Runs `bench build`** - compiles all JS/CSS assets +4. Creates immutable image with everything baked in -### 3. Point production to the new image +**Push to registry**: +```bash +# Push specific version (for production) +docker push ghcr.io/YOUR_USERNAME/erpnext-custom:${BUILD_DATE}-${GIT_SHA} -Edit `production/production.env` so compose uses your artifact everywhere: +# Push latest tag (convenience pointer) +docker push ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest +``` + +**Image tagging strategy**: +- `20251118-4c860c6` - Immutable tag for production (date + git commit) +- `production-latest` - Mutable pointer to newest build (for staging/testing) + +### 3. Update Production Configuration + +Edit `production/production.env` to use your custom image: ```env -CUSTOM_IMAGE=registry.example.com/erpnext-custom -CUSTOM_TAG=15 +CUSTOM_IMAGE=ghcr.io/YOUR_USERNAME/erpnext-custom +CUSTOM_TAG=20251118-4c860c6 # Use your BUILD_DATE-GIT_SHA PULL_POLICY=always ``` -Regenerate and redeploy so every service shares the same build: +**Important**: Use the specific date-commit tag in production, not `production-latest`. This ensures you can rollback by simply changing the tag. + +Regenerate configuration and deploy: ```bash -./scripts/deploy.sh --regenerate -./scripts/deploy.sh +./scripts/deploy.sh --regenerate # Updates production.yaml with new image +./scripts/deploy.sh # Pulls and deploys new image ``` -### 4. Install or update apps on sites +Verify all containers use the same image: +```bash +docker compose -f production/production.yaml images +``` -All apps listed in `apps.json` become available inside the bench. You still choose which sites receive them. +### 4. Install Apps on Sites -**New site** +Apps are in the image but need to be activated per site. +**New site with apps**: ```bash ./scripts/create-site.sh erp.example.com docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com install-app custom_integrations hrms + bench --site erp.example.com install-app india_compliance hrms ``` -**Existing site** - +**Existing site - add new app**: ```bash -# Install a newly added app +# Install the app on the site docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com install-app custom_integrations + bench --site erp.example.com install-app india_compliance -# Apply database patches after pulling latest code/image +# Run database migrations docker compose -f production/production.yaml exec backend \ bench --site erp.example.com migrate - -# Rebuild assets when the app ships JS/CSS -docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com build ``` -### 5. Wire in third-party services securely +**That's it!** No `bench build` or asset sync neededβ€”assets are already compiled and present in all containers from the image. -- Store API keys or secrets per site with `bench --site set-config SERVICE_API_KEY value --as-dict` so they land in `site_config.json` instead of the repo. -- Use background jobs (`frappe.enqueue`) inside your app for webhook callbacks, polling jobs, or queue workers that call external APIs. -- Mount extra certificates or client libraries via an override compose file if an integration needs system packages. -- Keep outbound allow-lists in Traefik/MariaDB untouchedβ€”integrations happen from the backend container, so no Traefik tweaks are required unless you expose a new inbound service. +Verify it works: +```bash +# Check installed apps +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com list-apps -### 6. Keep apps synchronized +# Test site access +curl -k -I https://erp.example.com/app/home +``` -- Version pin each entry in `apps.json` (tag, branch, or commit) so rebuilds are deterministic. -- When a third-party releases an update, bump the branch or tag, rebuild the image, redeploy, and run `bench migrate` on every existing site. -- Automate this via CI to ensure upstream merges (`git fetch upstream && git merge upstream/main`) and app bumps happen in the same pipeline. +### 5. Update Apps -Following this flow keeps the deployment upstream-compatible while giving you a repeatable way to include bespoke code, official marketplace apps, or deep third-party integrations without touching container internals manually. +When apps release new versions: + +```bash +# 1. Update apps.json with new versions (custom apps only) +nano production/apps.json +# Example: India Compliance: "branch": "v15.23.2" β†’ "branch": "v15.24.0" + +# 2. Update Frappe version if needed (via build arg) +# Check available versions: curl -s https://api.github.com/repos/frappe/frappe/tags | grep '"name"' + +# 3. Rebuild image with new tag +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) +NEW_TAG="ghcr.io/YOUR_USERNAME/erpnext-custom:$(date +%Y%m%d)-$(git rev-parse --short HEAD)" + +docker build \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=$NEW_TAG \ + --tag=ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest \ + --file=images/layered/Containerfile \ + . + +# 3. Push new image +docker push $NEW_TAG +docker push ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest + +# 4. Update production.env +nano production/production.env +# CUSTOM_TAG=20251119-xyz5678 # New date-commit tag + +# 5. Deploy +./scripts/deploy.sh --regenerate +./scripts/deploy.sh + +# 6. Migrate all sites +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com migrate +``` + +**Rollback if needed**: +```bash +# Just revert to previous tag +nano production/production.env +# CUSTOM_TAG=20251118-4c860c6 # Previous working version + +./scripts/deploy.sh +# Old image still exists in registry! +``` + +### 6. Uninstall Apps + +Remove an app from a site: + +```bash +# 1. Backup first (uninstall deletes DocTypes and data!) +./scripts/backup-site.sh erp.example.com --with-files --auto-copy + +# 2. Uninstall from site +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com uninstall-app india_compliance + +# 3. Clear cache and restart +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com clear-cache +docker compose -f production/production.yaml restart frontend +``` + +**Notes**: +- The app remains in the image's `/apps/` but is deactivated on the site +- To completely remove: rebuild image without it in `apps.json` +- Check dependencies before uninstalling +- Always backup firstβ€”uninstall deletes all app data --- @@ -717,16 +840,13 @@ docker compose -f production/production.yaml up -d # Create new site ./scripts/create-site.sh erp2.example.com -# Install custom apps that were baked into the image +# Install apps from the image docker compose -f production/production.yaml exec backend \ - bench --site erp2.example.com install-app custom_integrations hrms + bench --site erp2.example.com install-app india_compliance hrms -# Run migrations and build assets once +# Run migrations docker compose -f production/production.yaml exec backend \ bench --site erp2.example.com migrate - -docker compose -f production/production.yaml exec backend \ - bench --site erp2.example.com build ``` --- @@ -843,13 +963,12 @@ Because the site is created after the image rebuild, it automatically receives t **Existing sites that were updated** 1. Stop users (maintenance window) and take a backup: `./scripts/backup-site.sh erp.example.com --with-files`. -2. After redeploying containers, run: +2. After redeploying containers, run migrations: ```bash docker compose -f production/production.yaml exec backend \ bench --site erp.example.com migrate -docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com build + docker compose -f production/production.yaml exec backend \ bench --site erp.example.com clear-cache ``` @@ -860,31 +979,39 @@ docker compose -f production/production.yaml exec backend \ ### Update Custom Apps & Integrations -**What this updates**: Custom Frappe apps, DocTypes, webhook handlers, and any bundled third-party modules. +**What this updates**: Custom apps with new features or bug fixes. ```bash -# 1. Pull or merge the new code for each app, then refresh apps.json -git pull origin main # inside every custom app repo -vim apps.json # bump branch/tag references if needed +# 1. Update apps.json with new versions +nano production/apps.json +# Example: Update Frappe, ERPNext, or custom apps +# Frappe: "branch": "v15.88.1" β†’ "branch": "v15.89.0" +# India Compliance: "branch": "v15.23.2" β†’ "branch": "v15.24.0" + +# 2. Rebuild the image with new tag +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) +NEW_TAG="ghcr.io/YOUR_USERNAME/erpnext-custom:$(date +%Y%m%d)-$(git rev-parse --short HEAD)" -# 2. Rebuild the image with the refreshed manifest -export APPS_JSON_BASE64=$(base64 -w 0 apps.json) docker build \ - --build-arg=FRAPPE_BRANCH=version-15 \ --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ - --tag=registry.example.com/erpnext-custom:15 . -docker push registry.example.com/erpnext-custom:15 + --tag=$NEW_TAG \ + --tag=ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest \ + --file=images/layered/Containerfile . -# 3. Update production to pull the new tag -sed -i 's/CUSTOM_TAG=.*/CUSTOM_TAG=15/' production/production.env +docker push $NEW_TAG +docker push ghcr.io/YOUR_USERNAME/erpnext-custom:production-latest + +# 3. Update production.env with new tag +nano production/production.env +# CUSTOM_TAG=20251119-xyz5678 + +# 4. Deploy ./scripts/deploy.sh --regenerate ./scripts/deploy.sh -# 4. Apply database patches and rebuild assets per site +# 5. Migrate all sites docker compose -f production/production.yaml exec backend \ bench --site erp.example.com migrate -docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com build ``` ### Complete Update (All Layers) @@ -913,12 +1040,9 @@ docker compose -f production/production.yaml up -d docker compose -f production/production.yaml exec backend \ bench --site erp.example.com migrate -# 7. Rebuild assets and clear cache +# 7. Clear cache docker compose -f production/production.yaml exec backend \ bench --site erp.example.com clear-cache - -docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com build --force ``` ## Git Workflow @@ -948,19 +1072,23 @@ docker compose -f production/production.yaml exec backend \ - Keep each custom Frappe app in its own repository. - Tag or branch the app when you are ready to promote (`git tag v2.4.0`). -- Update `apps.json` with that tag/commit and keep the file sorted. This manifest is the **source of truth** for `APPS_JSON_BASE64`. -- Rebuild/push the custom image from the repo root: +- Update `production/apps.json` with that tag/commit. This manifest is the **source of truth** for `APPS_JSON_BASE64`. +- Rebuild/push the custom image: ```bash - export APPS_JSON_BASE64=$(base64 -w0 apps.json) - docker build -f images/custom/Containerfile \ - --build-arg FRAPPE_BRANCH=version-15 \ - --build-arg APPS_JSON_BASE64=$APPS_JSON_BASE64 \ - -t registry.example.com/erpnext-custom:v15-2024.09 . - docker push registry.example.com/erpnext-custom:v15-2024.09 + export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) + NEW_TAG="ghcr.io/YOUR_USERNAME/erpnext-custom:$(date +%Y%m%d)-$(git rev-parse --short HEAD)" + + docker build \ + --build-arg=FRAPPE_BRANCH=version-15 \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=$NEW_TAG \ + --file=images/layered/Containerfile . + + docker push $NEW_TAG ``` -- Update `production/production.env` (`CUSTOM_TAG=v15-2024.09`), regenerate, then run migrations for each site. +- Update `production/production.env` with the new `CUSTOM_TAG`, regenerate, then migrate each site. ### Helpful habits @@ -1114,11 +1242,9 @@ grep CUSTOM_TAG production/production.env **Re-apply the app to a site:** ```bash docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com install-app custom_integrations + bench --site erp.example.com install-app india_compliance docker compose -f production/production.yaml exec backend \ bench --site erp.example.com migrate -docker compose -f production/production.yaml exec backend \ - bench --site erp.example.com build ``` **Integration secrets not picked up?** @@ -1386,28 +1512,29 @@ traefik:v2.11 ``` **You maintain:** -- `production/` directory (deployment scripts, configs) -- `apps.json` (or CI secrets) describing the custom/third-party apps you ship +- `production/` directory (deployment scripts, configs, docs) +- `production/apps.json` - manifest for custom/third-party apps - `.gitignore` (excludes *.env files) **You track upstream:** - `compose.yaml` (base infrastructure) - `overrides/compose.*.yaml` (feature overlays) -- `images/*/Containerfile` (if you need custom builds) +- `images/layered/Containerfile` (for building custom images) ### Why This Approach? **Benefits:** -- βœ… Get official, tested ERPNext images +- βœ… Get official, tested ERPNext images (or build custom ones) - βœ… Receive infrastructure updates from frappe_docker -- βœ… Keep your custom apps and integrations isolated from infrastructure changes +- βœ… Keep custom apps isolated from infrastructure changes - βœ… Easy to merge upstream improvements -- βœ… Only rebuild images when you really need additional apps or dependencies +- βœ… Pre-compiled assets eliminate runtime build complexity -**When you'd build custom images:** -- Need to modify Python dependencies -- Add system packages to containers -- Install or update custom/third-party Frappe apps +**When to build custom images:** +- Adding custom or third-party Frappe apps +- Modifying Python/Node dependencies +- Adding system packages +- Need reproducible production deployments --- @@ -1421,6 +1548,6 @@ traefik:v2.11 --- -**Tested With**: ERPNext v15.82.1, Docker 24.0+, Ubuntu 22.04 LTS -**Script Optimization**: 35% reduction in code, 100% help coverage -**Last Updated**: October 2025 \ No newline at end of file +**Tested With**: ERPNext v15.88.1, Frappe v15.88.1, Docker 24.0+, Ubuntu 22.04 LTS +**Deployment Method**: Immutable images with pre-compiled assets +**Last Updated**: November 2025 \ No newline at end of file diff --git a/production/docs/custom-image-workflow.md b/production/docs/custom-image-workflow.md new file mode 100644 index 00000000..a4b81937 --- /dev/null +++ b/production/docs/custom-image-workflow.md @@ -0,0 +1,1052 @@ +# Building Custom ERPNext Images for Production + +**Production-Grade Workflow for Third-Party and Custom Apps** + +This guide covers the **Pattern 2 (Gold Standard)** approach: building immutable Docker images with custom apps and pre-compiled assets. This is the recommended method for production deployments. + +--- + +## Table of Contents + +1. [Overview](#overview) +2. [Prerequisites](#prerequisites) +3. [Quick Start](#quick-start) +4. [Step-by-Step Guide](#step-by-step-guide) +5. [Real-World Example: India Compliance](#real-world-example-india-compliance) +6. [Adding Custom Apps](#adding-custom-apps) +7. [Updating Apps](#updating-apps) +8. [Uninstall Apps](#uninstall-apps) +9. [Deployment Workflow](#deployment-workflow) +10. [Troubleshooting](#troubleshooting) +11. [Best Practices](#best-practices) + +--- + +## Overview + +### What This Achieves + +- βœ… **True Immutability**: Apps frozen at specific versions (tags/commits) +- βœ… **Zero Runtime Builds**: No `bench build` needed in production +- βœ… **No Asset Sync**: All containers have identical `/apps/` trees +- βœ… **Fast Deployments**: Pull image β†’ deploy β†’ activate on sites +- βœ… **Reliable Rollbacks**: Switch image tags instantly +- βœ… **Audit Trail**: Image tag = exact code deployed + +### When to Use This Method + +- βœ… Production environments +- βœ… Need reproducible deployments +- βœ… Regulatory compliance required +- βœ… Apps change weekly/monthly (not daily) +- βœ… Want reliable rollbacks + +### Key Difference from Runtime Install (Pattern 3) + +| Aspect | Pattern 3 (Runtime) | Pattern 2 (This Guide) | +|--------|---------------------|------------------------| +| Apps installed | At runtime with `bench get-app` | Baked into image at build time | +| Assets compiled | `bench build` in production | Pre-compiled during image build | +| Asset sync | Manual `tar` pipeline required | Not needed - assets in image | +| Immutability | Partial (apps can drift) | Complete (frozen versions) | +| Rollback | Complex | Change image tag | + +--- + +## Prerequisites + +### Required Tools + +```bash +# Verify Docker is installed +docker --version # Need 20.10+ + +# Verify git is available +git --version + +# Verify you have base64 +base64 --version +``` + +### GitHub Container Registry Access + +1. Create Personal Access Token: + - Go to: https://github.com/settings/tokens/new + - Select scope: `write:packages` + - Generate token and save it securely + +2. Login to GitHub Container Registry: + ```bash + export GITHUB_TOKEN=your_token_here + echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USERNAME --password-stdin + ``` + +3. Verify login: + ```bash + docker pull ghcr.io/YOUR_USERNAME/test || echo "Ready to push" + ``` + +--- + +## Quick Start + +**5-minute walkthrough** for experienced users: + +```bash +# 1. Define apps with pinned versions (custom apps only) +cat > production/apps.json < production/apps.json <> production/production.env + +# 6. Deploy +./scripts/deploy.sh --regenerate +./scripts/deploy.sh + +# 7. Activate on site +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost install-app india_compliance + +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost migrate + +# 8. Verify +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost list-apps +``` + +### What You Get + +- βœ… India Compliance v15.23.2 installed +- βœ… All GST, TDS, and compliance features available +- βœ… Assets pre-compiled (no 404 errors) +- βœ… Can rollback to previous image anytime +- βœ… Image SHA = exact deployed code + +--- + +## Adding Custom Apps + +### Scenario: Add Your Own Frappe App + +**1. Prepare your custom app**: +```bash +cd /path/to/your/custom_integrations + +# Tag a release +git tag v1.0.0 +git push origin v1.0.0 + +# Or note the commit hash +git log --oneline -1 +# abc1234 Add webhook integration +``` + +**2. Add to apps.json**: +```json +[ + { + "url": "https://github.com/frappe/erpnext", + "branch": "v15.88.1" + }, + { + "url": "https://github.com/resilient-tech/india-compliance", + "branch": "v15.23.2" + }, + { + "url": "https://github.com/YOUR_ORG/custom_integrations", + "branch": "v1.0.0" + } +] +``` + +**3. Rebuild image** (same process as above): +```bash +# New image tag reflects new date +BUILD_DATE=$(date +%Y%m%d) # 20251119 +GIT_SHA=$(git rev-parse --short HEAD) + +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) + +docker build \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=ghcr.io/duthink/erpnext-custom:${BUILD_DATE}-${GIT_SHA} \ + --tag=ghcr.io/duthink/erpnext-custom:production-latest \ + --file=images/layered/Containerfile . + +docker push ghcr.io/duthink/erpnext-custom:${BUILD_DATE}-${GIT_SHA} +docker push ghcr.io/duthink/erpnext-custom:production-latest +``` + +**4. Deploy new image**: +```bash +# Update production config +nano production/production.env +# CUSTOM_TAG=20251119-def5678 + +./scripts/deploy.sh --regenerate +./scripts/deploy.sh +``` + +**5. Install on sites**: +```bash +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost install-app custom_integrations + +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost migrate +``` + +### Private Repositories + +**For private GitHub repos**: + +```json +[ + { + "url": "https://YOUR_TOKEN@github.com/YOUR_ORG/private_app", + "branch": "v1.0.0" + } +] +``` + +**Security Note**: Never commit tokens to git! Use environment variables: + +```bash +# In CI/CD or local build +export GITHUB_TOKEN=your_token +export APPS_JSON_BASE64=$(cat production/apps.json | sed "s/YOUR_TOKEN/$GITHUB_TOKEN/g" | base64 -w0) +``` + +--- + +## Updating Apps + +### Scenario: India Compliance Releases v15.24.0 + +**1. Check for new version**: +```bash +curl -s https://api.github.com/repos/resilient-tech/india-compliance/tags | grep '"name"' | head -3 +# New output: "name": "v15.24.0" +``` + +**2. Update apps.json**: +```bash +nano production/apps.json + +# Change: +# "branch": "v15.23.2" β†’ "branch": "v15.24.0" +``` + +**3. Rebuild with new tag**: +```bash +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) +BUILD_DATE=$(date +%Y%m%d) +GIT_SHA=$(git rev-parse --short HEAD) + +docker build \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=ghcr.io/duthink/erpnext-custom:${BUILD_DATE}-${GIT_SHA} \ + --tag=ghcr.io/duthink/erpnext-custom:production-latest \ + --file=images/layered/Containerfile . + +docker push ghcr.io/duthink/erpnext-custom:${BUILD_DATE}-${GIT_SHA} +docker push ghcr.io/duthink/erpnext-custom:production-latest +``` + +**4. Test in staging first** (recommended): +```bash +# Use production-latest for staging +nano staging/staging.env +# CUSTOM_TAG=production-latest + +./scripts/deploy-staging.sh +# Test thoroughly... +``` + +**5. Deploy to production**: +```bash +nano production/production.env +# CUSTOM_TAG=20251125-xyz9999 # New specific tag + +./scripts/deploy.sh --regenerate +./scripts/deploy.sh +``` + +**6. Migrate sites**: +```bash +docker compose -f production/production.yaml exec backend \ + bench --site erp.localhost migrate +``` + +**7. Rollback if issues**: +```bash +# Just change to old tag +nano production/production.env +# CUSTOM_TAG=20251118-4c860c6 # Previous working version + +./scripts/deploy.sh +# Old image still exists in registry! +``` + +--- + +## Deployment Workflow + +### Standard Deployment Process + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ 1. Update apps.json with pinned versions β”‚ +β”‚ └─ Commit to git β”‚ +β”‚ β”‚ +β”‚ 2. Build image β”‚ +β”‚ └─ Tag with BUILD_DATE-GIT_SHA β”‚ +β”‚ └─ Also tag as production-latest β”‚ +β”‚ β”‚ +β”‚ 3. Push to registry β”‚ +β”‚ └─ Both tags pushed β”‚ +β”‚ β”‚ +β”‚ 4. Test (optional but recommended) β”‚ +β”‚ └─ Pull production-latest in staging β”‚ +β”‚ └─ Run smoke tests β”‚ +β”‚ β”‚ +β”‚ 5. Deploy to production β”‚ +β”‚ └─ Update CUSTOM_TAG with specific tag β”‚ +β”‚ └─ Regenerate production.yaml β”‚ +β”‚ └─ Deploy (pulls new image) β”‚ +β”‚ β”‚ +β”‚ 6. Migrate sites β”‚ +β”‚ └─ bench migrate on each site β”‚ +β”‚ β”‚ +β”‚ 7. Monitor β”‚ +β”‚ └─ Check logs β”‚ +β”‚ └─ Verify assets load β”‚ +β”‚ └─ Test critical features β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +### CI/CD Integration (GitHub Actions Example) + +```yaml +# .github/workflows/build-image.yml +name: Build ERPNext Custom Image + +on: + push: + paths: + - 'production/apps.json' + - 'images/**' + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Login to GitHub Container Registry + run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin + + - name: Build image + run: | + export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) + BUILD_DATE=$(date +%Y%m%d) + GIT_SHA=$(git rev-parse --short HEAD) + + docker build \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=ghcr.io/${{ github.repository_owner }}/erpnext-custom:${BUILD_DATE}-${GIT_SHA} \ + --tag=ghcr.io/${{ github.repository_owner }}/erpnext-custom:production-latest \ + --file=images/layered/Containerfile . + + - name: Push image + run: | + BUILD_DATE=$(date +%Y%m%d) + GIT_SHA=$(git rev-parse --short HEAD) + + docker push ghcr.io/${{ github.repository_owner }}/erpnext-custom:${BUILD_DATE}-${GIT_SHA} + docker push ghcr.io/${{ github.repository_owner }}/erpnext-custom:production-latest +``` + +--- + +## Uninstall Apps + +When you need to remove an app from a site: + +```bash +# 1. Backup first (uninstall deletes DocTypes and data!) +./scripts/backup-site.sh erp.example.com --with-files --auto-copy + +# 2. Uninstall from site +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com uninstall-app india_compliance + +# 3. Clear cache and restart +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com clear-cache +docker compose -f production/production.yaml restart frontend +``` + +**Important Notes:** +- The app remains in the image's `/apps/` directory but is deactivated on the site +- No `bench build` or asset sync neededβ€”assets are pre-compiled in the image +- Uninstall permanently deletes all app DocTypes and database records +- Always backup before uninstalling +- To completely remove an app from future deployments: rebuild image without it in `apps.json` + +**Complete Removal Workflow:** + +If you want to stop deploying an app entirely: + +```bash +# 1. Uninstall from all sites first +docker compose -f production/production.yaml exec backend \ + bench --site site1.example.com uninstall-app india_compliance +docker compose -f production/production.yaml exec backend \ + bench --site site2.example.com uninstall-app india_compliance + +# 2. Remove from apps.json +nano production/apps.json +# Delete the india_compliance entry + +# 3. Rebuild image without the app +export APPS_JSON_BASE64=$(base64 -w0 production/apps.json) +NEW_TAG="ghcr.io/YOUR_USERNAME/erpnext-custom:$(date +%Y%m%d)-$(git rev-parse --short HEAD)" + +docker build \ + --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \ + --build-arg=FRAPPE_BRANCH=v15.88.1 \ + --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \ + --tag=$NEW_TAG \ + --file=images/layered/Containerfile . + +docker push $NEW_TAG + +# 4. Update production.env and deploy +nano production/production.env +# CUSTOM_TAG=20251119-newsha + +./scripts/deploy.sh --regenerate +./scripts/deploy.sh +``` + +**Verify Clean State:** +```bash +# Check app is not in image +docker compose -f production/production.yaml exec backend bench list-apps + +# Check app is not active on sites +docker compose -f production/production.yaml exec backend \ + bench --site erp.example.com list-apps +``` + +--- + +## Troubleshooting + +### Issue: Build Fails with "App not found" + +**Symptom**: +``` +ERROR: Could not find app: india_compliance +``` + +**Causes**: +- Typo in repository URL +- Branch/tag doesn't exist +- Private repo without authentication + +**Solution**: +```bash +# Verify URL and branch exist +curl -I https://github.com/resilient-tech/india-compliance +curl -I https://github.com/resilient-tech/india-compliance/tree/v15.23.2 + +# Check apps.json syntax +cat production/apps.json | python3 -m json.tool +``` + +### Issue: Build Fails with "Node modules not found" + +**Symptom**: +``` +ERROR: Cannot find module 'xyz' +``` + +**Cause**: Upstream dependency issue in one of the apps + +**Solution**: +```bash +# Try building with specific Node version +docker build \ + --build-arg=NODE_VERSION=18.18.2 \ + ... + +# Or check app's package.json for required Node version +``` + +### Issue: Assets Return 404 After Deployment + +**Symptom**: CSS/JS files show 404 in browser + +**This should NOT happen with Pattern 2**, but if it does: + +**Diagnosis**: +```bash +# Verify all containers use same image +docker compose -f production/production.yaml images + +# Check if assets exist in image +docker compose -f production/production.yaml exec backend \ + ls /home/frappe/frappe-bench/apps/india_compliance/india_compliance/public/dist + +# Check frontend can access them +docker compose -f production/production.yaml exec frontend \ + ls /home/frappe/frappe-bench/apps/india_compliance/india_compliance/public/dist +``` + +**Solution**: Rebuild image, ensure `bench build` completed during build. + +### Issue: Image Size Too Large + +**Symptom**: Image is 3+ GB + +**Cause**: Includes development dependencies or build cache + +**Solution**: +```dockerfile +# Multi-stage builds help (already in images/layered/Containerfile) +# Ensure .dockerignore is present: +cat > .dockerignore <