Rework deployment.

HYBRID_DEPLOYMENT.md

@@ -1,84 +1,127 @@
-# Hybrid Deployment Guide: Apps in Image + Persistent Volumes
+# Zerops Deployment Guide: Managed Services + Docker Compose
 
-This configuration implements the best of both worlds:
-- **Apps baked into Docker image** for consistency and fast deployments
-- **Site data in persistent volumes** for data safety and persistence
+This configuration leverages Zerops' managed services for databases while using Docker Compose for application containers, providing the best of both worlds:
+- **Managed MariaDB and Redis** for reliability, backups, and performance
+- **Docker Compose applications** for flexibility and familiar workflows
+- **Automated site installation** with custom apps included
 
 ## Architecture Overview
 
-### What's in the Image (Built once, reusable):
-- ✅ Frappe and ERPNext applications
-- ✅ All Python dependencies  
-- ✅ Custom apps (if configured)
-- ✅ Basic directory structure
+### Zerops Managed Services:
+- ✅ **MariaDB 11** - Database service with automatic backups and monitoring
+- ✅ **Valkey 7.2 (Redis)** - Cache service for high-performance caching
+- ✅ **Valkey 7.2 (Redis)** - Queue service for background job processing
 
-### What's in Persistent Volumes:
-- ✅ Site configurations (`sites/[site-name]/site_config.json`)
-- ✅ Uploaded files (`sites/[site-name]/public/files/`)
-- ✅ Private files (`sites/[site-name]/private/files/`)
-- ✅ Site-specific customizations
-- ✅ Database backups
+### Docker Compose Application Services:
+- ✅ **Backend** - Main Frappe/ERPNext application server
+- ✅ **Frontend** - Nginx reverse proxy for web serving
+- ✅ **WebSocket** - Real-time communication service
+- ✅ **Queue Workers** - Background job processing (short & long tasks)
+- ✅ **Scheduler** - Cron job management
+
+### Automated Site Setup:
+- ✅ Site creation and configuration
+- ✅ ERPNext application installation
+- ✅ Custom XML Importer app installation
+- ✅ Database migrations and setup
 - ✅ Application logs
 
 ## Deployment Steps
 
-### 1. Set Required Environment Secrets in Zerops
+### 1. Create Zerops Services
 
-```bash
-# In Zerops Dashboard > Project > Environment Variables
-db_password=YourSecureDbPassword123
-admin_password=YourAdminPassword123
-site_name=your-domain.com
+Create these services in your Zerops project:
+
+```yaml
+# Database service
+- Service Name: db
+- Type: MariaDB 11
+- Mode: Default (or HA for production)
+
+# Redis Cache service  
+- Service Name: redis-cache
+- Type: Valkey 7.2
+- Mode: NON_HA (or HA for production)
+
+# Redis Queue service
+- Service Name: redis-queue  
+- Type: Valkey 7.2
+- Mode: NON_HA (or HA for production)
 ```
 
-### 2. Deploy the Configuration
+### 2. Set Environment Secrets
 
-1. Import the `zerops.yml` file to Zerops
-2. Zerops will build all services using `Dockerfile.zerops`
-3. Each service will run the initialization script on startup
-4. Site will be automatically created if it doesn't exist
+In Zerops Dashboard > Project > Environment Variables:
 
-### 3. Automatic Initialization Process
+```bash
+dbPassword=YourSecureDbPassword123
+adminPassword=YourAdminPassword123  
+siteName=your-domain.com
+```
 
-On first deployment, each container will:
+### 3. Deploy the Application
 
-1. **Build Phase** (runs once during image creation):
-   - Install Frappe/ERPNext apps into image
-   - Copy initialization scripts
-   - Set proper permissions
+1. Connect your GitHub repository to Zerops
+2. Set branch to `zerops`
+3. Zerops will automatically find and deploy using `zerops.yml`
+4. The deployment will:
+   - Create managed database and Redis services
+   - Pull and start Docker Compose services
+   - Run site installation script
+   - Configure all service connections
+
+### 4. Automatic Site Installation
+
+During deployment, the installation script will:
+
+1. **Service Verification**:
+   - Wait for database connection to be ready
+   - Verify Redis cache and queue services are available
+   - Pull required Docker images
+
+2. **Site Setup**:
+   - Check if site already exists
+   - Create new site if needed (with database and admin user)
+   - Install ERPNext application
+   - Install custom XML Importer app from GitHub
+   - Run database migrations
 
 2. **Runtime Phase** (runs on every container start):
-   - **Backend service**: Full site initialization (creates site, installs ERPNext)
-   - **Other services**: Light initialization (just checks if site exists)
-   - Configure database and Redis connections
-   - Wait for database to be ready (backend only)
-   - Set proper permissions and configurations
+3. **Service Startup**:
+   - Start all Docker Compose services
+   - Services connect to managed databases using service names
+   - Site becomes available at the app service URL
 
-## Benefits of This Approach
+## Benefits of This Architecture
 
-### ✅ **Container Restart Resilience**
-- Site data survives container restarts/crashes
+### ✅ **Managed Database Reliability**
+- Automatic backups and point-in-time recovery
+- Built-in monitoring and alerting
+- High availability options
+- No database container overhead
+
+### ✅ **Docker Compose Flexibility**
+- Familiar development workflow
+- Easy service management and scaling
+- Shared volumes for site data
+- Service dependencies handled automatically
+
+### ✅ **Automated Setup**
+- Zero-touch deployment
+- Custom apps automatically installed
+- Site creation and configuration handled
 - No manual intervention required
-- Fast recovery times
 
-### ✅ **Consistent App Deployments**
-- Apps are identical across all containers
-- No version drift between services
-- Easy to update apps (rebuild image)
+### ✅ **Service Discovery**
+- Internal service communication via names
+- No hardcoded IPs or complex networking
+- Zerops handles internal DNS resolution
 
-### ✅ **Zero-Downtime Updates**
-- App updates: rebuild image, rolling update
-- Site data preserved during updates
-- Database migrations handled automatically
-
-### ✅ **Backup & Recovery**
-- Simple volume snapshots for site data
-- Database backups work seamlessly
-- Easy disaster recovery
-
-### ✅ **Scalability**
-- New containers start with same apps
-- Shared persistent volume for site data
+### ✅ **Scalability & Performance**
+- Database services can be scaled independently  
+- Application services can be horizontally scaled
+- Redis services optimized for their use case
+- No database performance impact from containerization
 - Auto-scaling works out of the box
 
 ## File Structure
@@ -98,51 +141,51 @@ On first deployment, each container will:
 └── logs/                   # Application logs (persistent volume)
 ```
 
+## Access Your ERPNext Instance
+
+After successful deployment:
+
+1. **Find your app service URL** in Zerops Dashboard > Services > app
+2. **Access ERPNext** at `https://your-app-service-url` 
+3. **Login credentials**:
+   - Username: `Administrator`
+   - Password: `[value you set for adminPassword]`
+
 ## Common Operations
 
 ### Adding New Custom Apps
 
-1. **Update `Dockerfile.zerops`**:
-   ```dockerfile
-   RUN bench get-app --branch main custom_app https://github.com/user/custom_app.git
-   ```
-
-2. **Redeploy services** (Zerops will rebuild image)
-
-3. **Install app on site** (automatic or manual):
+1. **Update the installation script** `scripts/install-site.sh`:
    ```bash
-   bench --site your-domain.com install-app custom_app
+   # Add your custom app
+   bench get-app https://github.com/user/custom_app.git
+   bench --site "$FRAPPE_SITE_NAME_HEADER" install-app custom_app
    ```
 
+2. **Redeploy the application** - Zerops will run the updated script
+
 ### Updating Frappe/ERPNext Version
 
-1. **Update version in `Dockerfile.zerops`**:
-   ```dockerfile
-   FROM frappe/erpnext:v16.0.0
-   ```
-
-2. **Update `zerops.yml` environment**:
+1. **Update version in `zerops.yml`**:
    ```yaml
+   CUSTOM_TAG: v16.0.0
    ERPNEXT_VERSION: v16.0.0
    ```
 
-3. **Redeploy** - migrations run automatically
+2. **Redeploy** - migrations run automatically during site setup
 
 ### Manual Site Operations
 
-Access any service terminal in Zerops dashboard:
+Access the app service terminal in Zerops dashboard:
 
 ```bash
-# Navigate to bench directory
+# Navigate to bench directory  
 cd /home/frappe/frappe-bench
 
 # Run migrations
 bench --site your-domain.com migrate
 
-# Create new site
-bench new-site newsite.com --install-app erpnext
-
-# Install custom app
+# Install additional apps
 bench --site your-domain.com install-app custom_app
 
 # Backup site
@@ -154,7 +197,7 @@ bench --site your-domain.com console
 
 ## Troubleshooting
 
-### Site Not Created Automatically
+### Site Installation Issues
 - Check environment variables are set correctly
 - Check database connectivity
 - Review container logs in Zerops dashboard

docker-compose.zerops.yaml

@@ -0,0 +1,116 @@
+# Docker Compose configuration for Zerops deployment
+# This extends the main compose.yaml with Zerops-specific requirements
+
+x-customizable-image: &customizable_image
+  # Use ERPNext image with custom apps from our Containerfile
+  image: ${CUSTOM_IMAGE:-frappe/erpnext}:${CUSTOM_TAG:-v15.84.0}
+  pull_policy: always
+  restart: unless-stopped
+  # IMPORTANT: All services must use network_mode: host for Zerops
+  network_mode: host
+
+x-depends-on-configurator: &depends_on_configurator
+  depends_on:
+    configurator:
+      condition: service_completed_successfully
+
+x-backend-defaults: &backend_defaults
+  <<: [*depends_on_configurator, *customizable_image]
+  volumes:
+    - sites:/home/frappe/frappe-bench/sites
+  environment:
+    - DB_HOST
+    - DB_PORT
+    - DB_PASSWORD
+    - ADMIN_PASSWORD
+    - REDIS_CACHE
+    - REDIS_QUEUE
+    - SOCKETIO_PORT
+    - FRAPPE_SITE_NAME_HEADER
+
+services:
+  configurator:
+    <<: *backend_defaults
+    platform: linux/amd64
+    entrypoint:
+      - bash
+      - -c
+    # Basic Frappe configuration only (site installation handled by Zerops)
+    command:
+      - >
+        ls -1 apps > sites/apps.txt;
+        bench set-config -g db_host $$DB_HOST;
+        bench set-config -gp db_port $$DB_PORT;
+        bench set-config -g redis_cache "redis://$$REDIS_CACHE";
+        bench set-config -g redis_queue "redis://$$REDIS_QUEUE";
+        bench set-config -g redis_socketio "redis://$$REDIS_QUEUE";
+        bench set-config -gp socketio_port $$SOCKETIO_PORT;
+        echo "✅ Frappe configuration completed";
+    environment:
+      DB_HOST: ${DB_HOST}
+      DB_PORT: ${DB_PORT:-3306}
+      DB_PASSWORD: ${DB_PASSWORD}
+      ADMIN_PASSWORD: ${ADMIN_PASSWORD}
+      REDIS_CACHE: ${REDIS_CACHE}
+      REDIS_QUEUE: ${REDIS_QUEUE}
+      SOCKETIO_PORT: 9000
+      FRAPPE_SITE_NAME_HEADER: ${FRAPPE_SITE_NAME_HEADER}
+    depends_on: {}
+    restart: on-failure
+
+  backend:
+    <<: *backend_defaults
+    platform: linux/amd64
+    ports:
+      - "8000:8000"
+
+  frontend:
+    <<: *customizable_image
+    platform: linux/amd64
+    command:
+      - nginx-entrypoint.sh
+    environment:
+      - BACKEND=127.0.0.1:8000
+      - SOCKETIO=127.0.0.1:9000
+      - FRAPPE_SITE_NAME_HEADER
+      - UPSTREAM_REAL_IP_ADDRESS=127.0.0.1
+      - UPSTREAM_REAL_IP_HEADER=X-Forwarded-For
+      - UPSTREAM_REAL_IP_RECURSIVE=off
+      - PROXY_READ_TIMEOUT=120
+      - CLIENT_MAX_BODY_SIZE=50m
+    volumes:
+      - sites:/home/frappe/frappe-bench/sites
+    depends_on:
+      - backend
+      - websocket
+    ports:
+      - "8080:8080"
+
+  websocket:
+    <<: [*depends_on_configurator, *customizable_image]
+    platform: linux/amd64
+    command:
+      - node
+      - /home/frappe/frappe-bench/apps/frappe/socketio.js
+    volumes:
+      - sites:/home/frappe/frappe-bench/sites
+    ports:
+      - "9000:9000"
+
+  queue-short:
+    <<: *backend_defaults
+    platform: linux/amd64
+    command: bench worker --queue short,default
+
+  queue-long:
+    <<: *backend_defaults
+    platform: linux/amd64
+    command: bench worker --queue long,default,short
+
+  scheduler:
+    <<: *backend_defaults
+    platform: linux/amd64
+    command: bench schedule
+
+volumes:
+  sites:

scripts/init-frappe-site.sh

@@ -1,81 +0,0 @@
-#!/bin/bash
-
-# Frappe Site Initialization Script for Production Deployment
-# This script runs before the main service starts to ensure the site exists and is configured
-
-set -e
-
-echo "🔧 Initializing Frappe site..."
-
-# Configuration from environment variables
-SITE_NAME=${FRAPPE_SITE_NAME_HEADER:-"localhost"}
-DB_PASSWORD=${DB_PASSWORD:-"admin"}
-ADMIN_PASSWORD=${ADMIN_PASSWORD:-"admin"}
-DB_HOST=${DB_HOST:-"db"}
-DB_PORT=${DB_PORT:-"3306"}
-
-echo "📋 Configuration: Site Name: $SITE_NAME, Database Host: $DB_HOST:$DB_PORT"
-
-# Navigate to frappe bench directory
-cd /home/frappe/frappe-bench
-
-# Configure database and Redis connections
-echo "🔗 Configuring connections..."
-bench set-config -g db_host "$DB_HOST" || echo "DB host config already set"
-bench set-config -gp db_port "$DB_PORT" || echo "DB port config already set"
-bench set-config -g redis_cache "redis://${REDIS_CACHE}:6379" || echo "Redis cache config already set"
-bench set-config -g redis_queue "redis://${REDIS_QUEUE}:6379" || echo "Redis queue config already set"
-bench set-config -g redis_socketio "redis://${REDIS_QUEUE}:6379" || echo "Redis socketio config already set"
-bench set-config -gp socketio_port "${SOCKETIO_PORT}" || echo "Socketio port config already set"
-
-# Wait for database to be ready
-echo "⏳ Waiting for database to be ready..."
-for i in {1..30}; do
-    if mysql -h"$DB_HOST" -P"$DB_PORT" -uroot -p"$DB_PASSWORD" -e "SELECT 1;" >/dev/null 2>&1; then
-        echo "✅ Database is ready!"
-        break
-    fi
-    echo "   Attempt $i/30: Database not ready, waiting 5 seconds..."
-    sleep 5
-done
-
-# Check if site exists in persistent volume
-if [ -d "sites/$SITE_NAME" ]; then
-    echo "✅ Site '$SITE_NAME' already exists, updating configuration"
-    bench --site "$SITE_NAME" set-config db_host "$DB_HOST"
-    bench --site "$SITE_NAME" set-config db_port "$DB_PORT"
-    bench use "$SITE_NAME" || echo "Site already set as default"
-else
-    echo "🏗️  Creating new site: $SITE_NAME"
-    if bench new-site "$SITE_NAME" \
-        --db-root-password "$DB_PASSWORD" \
-        --admin-password "$ADMIN_PASSWORD" \
-        --install-app erpnext \
-        --set-default; then
-        echo "✅ Site created successfully!"
-        
-        # Install custom XML importer app
-        echo "📦 Installing XML Importer app..."
-        bench get-app --branch main erpnext_xml_importer https://github.com/UhrinDavid/erpnext_xml_importer.git || echo "⚠️ Failed to download XML importer"
-        bench --site "$SITE_NAME" install-app erpnext_xml_importer || echo "⚠️ XML importer app installation failed"
-        
-        bench --site "$SITE_NAME" set-config developer_mode 0
-        bench --site "$SITE_NAME" set-config maintenance_mode 0
-        echo "🎉 Site '$SITE_NAME' is ready with XML Importer!"
-    else
-        echo "❌ Failed to create site. Checking if it exists..."
-        if [ -d "sites/$SITE_NAME" ]; then
-            echo "⚠️  Site directory exists but creation failed. Using existing site."
-            bench use "$SITE_NAME" || echo "Could not set as default"
-        else
-            echo "💥 Site creation failed completely. Exiting."
-            exit 1
-        fi
-    fi
-fi
-
-# Ensure proper ownership of sites directory
-chown -R frappe:frappe sites/ || echo "Could not change ownership"
-chmod -R 755 sites/ || echo "Could not change permissions"
-
-echo "✅ Site initialization completed successfully!"

scripts/init-service.sh

@@ -1,20 +0,0 @@
-#!/bin/bash
-
-# Simple service initialization for Frappe services
-# This script runs on services that don't need to create sites
-
-set -e
-
-SERVICE_NAME=${1:-"frappe-service"}
-
-echo "🔧 Initializing $SERVICE_NAME..."
-cd /home/frappe/frappe-bench
-
-# Basic configuration (site creation handled by backend service)
-if [ -f sites/common_site_config.json ] && [ -n "${FRAPPE_SITE_NAME_HEADER}" ]; then
-    if [ -d "sites/${FRAPPE_SITE_NAME_HEADER}" ]; then
-        bench use "${FRAPPE_SITE_NAME_HEADER}" || echo "Site will be set by backend"
-    fi
-fi
-
-echo "✅ $SERVICE_NAME ready"

scripts/init-site.sh

@@ -1,99 +0,0 @@
-#!/bin/bash
-
-# Frappe Site Initialization Script for Zerops
-# This script runs on container start to ensure the site exists
-
-set -e
-
-echo "🔧 Initializing Frappe site..."
-
-# Configuration from environment variables
-SITE_NAME=${FRAPPE_SITE_NAME_HEADER:-"localhost"}
-DB_PASSWORD=${DB_PASSWORD:-"admin"}
-ADMIN_PASSWORD=${ADMIN_PASSWORD:-"admin"}
-DB_HOST=${DB_HOST:-"db"}
-DB_PORT=${DB_PORT:-"3306"}
-
-echo "📋 Configuration:"
-echo "   Site Name: $SITE_NAME"
-echo "   Database Host: $DB_HOST:$DB_PORT"
-
-# Navigate to frappe bench directory
-cd /home/frappe/frappe-bench
-
-# Ensure sites directory exists and has proper permissions
-mkdir -p sites
-chown -R frappe:frappe sites || true
-
-# Configure database and Redis connections
-echo "🔗 Configuring connections..."
-bench set-config -g db_host "$DB_HOST" || echo "DB host config already set"
-bench set-config -gp db_port "$DB_PORT" || echo "DB port config already set"
-bench set-config -g redis_cache "redis://${REDIS_CACHE}:6379" || echo "Redis cache config already set"
-bench set-config -g redis_queue "redis://${REDIS_QUEUE}:6379" || echo "Redis queue config already set"
-bench set-config -g redis_socketio "redis://${REDIS_QUEUE}:6379" || echo "Redis socketio config already set"
-bench set-config -gp socketio_port "${SOCKETIO_PORT}" || echo "Socketio port config already set"
-
-# Wait for database to be ready
-echo "⏳ Waiting for database to be ready..."
-for i in {1..30}; do
-    if mysql -h"$DB_HOST" -P"$DB_PORT" -uroot -p"$DB_PASSWORD" -e "SELECT 1;" >/dev/null 2>&1; then
-        echo "✅ Database is ready!"
-        break
-    fi
-    echo "   Attempt $i/30: Database not ready, waiting 5 seconds..."
-    sleep 5
-done
-
-# Check if site already exists in persistent volume
-if [ -d "sites/$SITE_NAME" ]; then
-    echo "✅ Site '$SITE_NAME' already exists, skipping creation"
-    
-    # Ensure site is in the current bench context
-    if [ -f "sites/$SITE_NAME/site_config.json" ]; then
-        echo "🔧 Updating site configuration..."
-        bench --site "$SITE_NAME" set-config db_host "$DB_HOST"
-        bench --site "$SITE_NAME" set-config db_port "$DB_PORT"
-        bench use "$SITE_NAME" || echo "Site already set as default"
-    fi
-else
-    echo "🏗️  Creating new site: $SITE_NAME"
-    
-    # Create the site with ERPNext
-    if bench new-site "$SITE_NAME" \
-        --db-root-password "$DB_PASSWORD" \
-        --admin-password "$ADMIN_PASSWORD" \
-        --install-app erpnext \
-        --set-default; then
-        
-        echo "✅ Site created successfully!"
-        
-        # Additional site configuration
-        bench --site "$SITE_NAME" set-config developer_mode 0
-        bench --site "$SITE_NAME" set-config maintenance_mode 0
-        
-        echo "🎉 Site '$SITE_NAME' is ready!"
-    else
-        echo "❌ Failed to create site. Checking if it exists..."
-        if [ -d "sites/$SITE_NAME" ]; then
-            echo "⚠️  Site directory exists but creation failed. Using existing site."
-            bench use "$SITE_NAME" || echo "Could not set as default"
-        else
-            echo "💥 Site creation failed completely. Exiting."
-            exit 1
-        fi
-    fi
-fi
-
-# Ensure proper ownership of sites directory
-chown -R frappe:frappe sites/ || echo "Could not change ownership"
-chmod -R 755 sites/ || echo "Could not change permissions"
-
-# Final verification
-if [ -d "sites/$SITE_NAME" ]; then
-    echo "✅ Site initialization completed successfully!"
-    echo "🌐 Site '$SITE_NAME' is ready to serve traffic"
-else
-    echo "❌ Site initialization failed!"
-    exit 1
-fi

scripts/install-site.sh

@@ -0,0 +1,84 @@
+#!/bin/bash
+
+# Frappe/ERPNext Site Installation Script
+# This script creates and configures a new Frappe site with ERPNext and custom apps
+# Runs during Zerops deployment before starting application services
+
+set -e
+
+echo "🚀 Starting Frappe site installation..."
+
+# Configuration from environment variables
+SITE_NAME=${FRAPPE_SITE_NAME_HEADER:-"localhost"}
+DB_PASSWORD=${DB_PASSWORD:-"admin"}
+ADMIN_PASSWORD=${ADMIN_PASSWORD:-"admin"}
+DB_HOST=${DB_HOST:-"db"}
+DB_PORT=${DB_PORT:-"3306"}
+
+echo "📋 Site Configuration:"
+echo "  - Site Name: $SITE_NAME"
+echo "  - Database Host: $DB_HOST:$DB_PORT"
+echo "  - Admin Password: [CONFIGURED]"
+
+# Start with a fresh container to install the site
+echo "📦 Starting temporary Frappe container for site installation..."
+
+# Run the installation inside a Docker container
+docker compose -f docker-compose.zerops.yaml run --rm -e FRAPPE_SITE_NAME_HEADER="$SITE_NAME" \
+  -e DB_HOST="$DB_HOST" -e DB_PORT="$DB_PORT" -e DB_PASSWORD="$DB_PASSWORD" \
+  -e ADMIN_PASSWORD="$ADMIN_PASSWORD" \
+  configurator bash -c '
+    echo "🏗️  Setting up Frappe configuration..."
+    
+    # Navigate to bench directory
+    cd /home/frappe/frappe-bench
+    
+    # Set up basic configuration
+    ls -1 apps > sites/apps.txt
+    bench set-config -g db_host $DB_HOST
+    bench set-config -gp db_port $DB_PORT
+    bench set-config -g redis_cache "redis://$REDIS_CACHE"
+    bench set-config -g redis_queue "redis://$REDIS_QUEUE"
+    bench set-config -g redis_socketio "redis://$REDIS_QUEUE"
+    bench set-config -gp socketio_port $SOCKETIO_PORT
+    
+    echo "✅ Frappe configuration completed"
+    
+    # Check if site already exists
+    if [ ! -d "sites/$FRAPPE_SITE_NAME_HEADER" ]; then
+      echo "🆕 Creating new site: $FRAPPE_SITE_NAME_HEADER"
+      
+      bench new-site "$FRAPPE_SITE_NAME_HEADER" \
+        --mariadb-root-password "$DB_PASSWORD" \
+        --admin-password "$ADMIN_PASSWORD" \
+        --no-mariadb-socket
+      
+      echo "✅ Site created successfully"
+      
+      echo "📦 Installing ERPNext app..."
+      bench --site "$FRAPPE_SITE_NAME_HEADER" install-app erpnext
+      echo "✅ ERPNext installed successfully"
+      
+      echo "🔧 Installing custom XML Importer app..."
+      if [ ! -d "apps/erpnext_xml_importer" ]; then
+        echo "📥 Downloading XML Importer app from GitHub..."
+        bench get-app https://github.com/UhrinDavid/erpnext_xml_importer.git
+      fi
+      
+      bench --site "$FRAPPE_SITE_NAME_HEADER" install-app erpnext_xml_importer
+      echo "✅ XML Importer app installed successfully"
+      
+      echo "🔄 Running site migration..."
+      bench --site "$FRAPPE_SITE_NAME_HEADER" migrate
+      echo "✅ Site migration completed"
+      
+      echo "🎉 Site installation completed successfully!"
+    else
+      echo "♻️  Site $FRAPPE_SITE_NAME_HEADER already exists"
+      echo "🔄 Running migration to ensure site is up to date..."
+      bench --site "$FRAPPE_SITE_NAME_HEADER" migrate
+      echo "✅ Migration completed"
+    fi
+  '
+
+echo "🎯 Frappe site installation script completed!"

scripts/setup-frappe.sh

@@ -1,75 +0,0 @@
-#!/bin/bash
-
-# Frappe/ERPNext Initial Setup Script for Zerops
-# This script should be run after the services are deployed and running
-
-set -e
-
-echo "🚀 Starting Frappe/ERPNext setup on Zerops..."
-
-# Check if required environment variables are set
-if [ -z "$SITE_NAME" ]; then
-    echo "❌ Error: SITE_NAME environment variable is required"
-    echo "   Set it to your domain name (e.g., mycompany.example.com)"
-    exit 1
-fi
-
-if [ -z "$DB_PASSWORD" ]; then
-    echo "❌ Error: DB_PASSWORD environment variable is required"
-    exit 1
-fi
-
-if [ -z "$ADMIN_PASSWORD" ]; then
-    echo "❌ Error: ADMIN_PASSWORD environment variable is required"
-    echo "   This will be the password for the ERPNext admin user"
-    exit 1
-fi
-
-echo "📋 Configuration:"
-echo "   Site Name: $SITE_NAME"
-echo "   Database Host: ${DB_HOST:-db}"
-echo "   Admin Email: ${ADMIN_EMAIL:-administrator@$SITE_NAME}"
-
-# Navigate to frappe bench directory
-cd /home/frappe/frappe-bench
-
-echo "🏗️  Creating new site: $SITE_NAME"
-bench new-site "$SITE_NAME" \
-    --db-root-password "$DB_PASSWORD" \
-    --admin-password "$ADMIN_PASSWORD" \
-    --set-default
-
-echo "📦 Installing ERPNext application..."
-bench --site "$SITE_NAME" install-app erpnext
-
-echo "👤 Setting up admin user..."
-if [ -n "$ADMIN_EMAIL" ]; then
-    bench --site "$SITE_NAME" set-admin-password "$ADMIN_PASSWORD"
-fi
-
-echo "🔧 Final configuration..."
-# Set site as default
-bench use "$SITE_NAME"
-
-# Clear cache
-bench --site "$SITE_NAME" clear-cache
-
-# Migrate database (in case there are any pending migrations)
-bench --site "$SITE_NAME" migrate
-
-echo "✅ Setup completed successfully!"
-echo ""
-echo "🌐 Your ERPNext site is ready at: $SITE_NAME"
-echo "👤 Admin login: administrator"
-echo "🔑 Admin password: [as set in ADMIN_PASSWORD]"
-echo ""
-echo "📚 Next steps:"
-echo "   1. Configure your domain to point to this Zerops service"
-echo "   2. Access your site and complete the setup wizard"
-echo "   3. Configure SSL certificates if needed"
-echo ""
-echo "🔧 Useful commands:"
-echo "   - bench restart: Restart all processes"
-echo "   - bench --site $SITE_NAME console: Open Python console"  
-echo "   - bench --site $SITE_NAME migrate: Run database migrations"
-echo "   - bench update: Update apps and migrate"

zerops.env

@@ -1,60 +1,78 @@
-# Production Environment Configuration for Zerops Deployment
-# Copy this file and customize the values for your deployment
+# Zerops Environment Configuration Template
+# 
+# This file shows the environment variables used in the Zerops deployment.
+# DO NOT commit actual secrets to git! Set these in Zerops Dashboard.
 
 # =============================================================================
-# REQUIRED ZEROPS SECRETS (set these in Zerops Dashboard > Environment Variables)
+# REQUIRED ZEROPS SECRETS (set in Zerops Dashboard > Environment Variables)
 # =============================================================================
-db_password=your-strong-database-password
-admin_password=your-admin-password  
-site_name=your-domain.com
+
+# Database credentials
+dbPassword=your-secure-database-password-here
+
+# Admin credentials  
+adminPassword=your-admin-password-here
+
+# Site configuration
+siteName=your-domain.com
 
 # =============================================================================
-# OPTIONAL CONFIGURATION (modify as needed)
-# =============================================================================  
-ADMIN_EMAIL=admin@your-domain.com
-ERPNEXT_VERSION=v15.84.0
+# DOCKER COMPOSE CONFIGURATION (automatically configured)
+# =============================================================================
 
-# Database Configuration (managed by Zerops)
-DB_HOST=db
-DB_PORT=3306
-
-# Redis Configuration (managed by Zerops)
-REDIS_CACHE=redis-cache:6379
-REDIS_QUEUE=redis-queue:6379
-
-# Application Configuration  
-SOCKETIO_PORT=9000
-FRAPPE_SITE_NAME_HEADER=your-domain.com
-
-# Nginx/Proxy Configuration
-UPSTREAM_REAL_IP_ADDRESS=127.0.0.1
-UPSTREAM_REAL_IP_HEADER=X-Forwarded-For
-UPSTREAM_REAL_IP_RECURSIVE=off
-PROXY_READ_TIMEOUT=120s
-CLIENT_MAX_BODY_SIZE=50m
-
-# Frappe/ERPNext Image Configuration
+# Image settings
 CUSTOM_IMAGE=frappe/erpnext
 CUSTOM_TAG=v15.84.0
+ERPNEXT_VERSION=v15.84.0
 PULL_POLICY=always
 RESTART_POLICY=unless-stopped
 
-# Backup Configuration (optional)
-# BACKUP_S3_BUCKET=your-backup-bucket
-# BACKUP_S3_ACCESS_KEY=your-access-key
-# BACKUP_S3_SECRET_KEY=your-secret-key
-# BACKUP_S3_REGION=your-region
+# Service connections (managed by Zerops)
+DB_HOST=db
+DB_PORT=3306
+REDIS_CACHE=redis-cache:6379
+REDIS_QUEUE=redis-queue:6379
 
-# Email Configuration (optional)
-# MAIL_SERVER=smtp.your-email-provider.com
-# MAIL_PORT=587
-# MAIL_USERNAME=your-email@domain.com
-# MAIL_PASSWORD=your-email-password
-# MAIL_USE_TLS=true
+# Frontend/Nginx settings
+UPSTREAM_REAL_IP_ADDRESS=127.0.0.1
+UPSTREAM_REAL_IP_HEADER=X-Forwarded-For
+UPSTREAM_REAL_IP_RECURSIVE=off
+# =============================================================================
+# DEPLOYMENT ARCHITECTURE
+# =============================================================================
 
-# SSL/TLS Configuration (optional, if not using Zerops automatic SSL)
-# LETSENCRYPT_EMAIL=your-email@domain.com
-# SITES=your-domain.com
+# Managed Services (created in Zerops Dashboard):
+# - db: MariaDB 11 database service
+# - redis-cache: Valkey 7.2 cache service (NON_HA mode)  
+# - redis-queue: Valkey 7.2 queue service (NON_HA mode)
+# - app: Docker Compose application service
+
+# Docker Compose Services:
+# - configurator: One-time setup and configuration
+# - backend: Main Frappe/ERPNext application (port 8000)
+# - frontend: Nginx proxy (port 8080)
+# - websocket: Socket.io service (port 9000)
+# - queue-short: Background job worker (short tasks)
+# - queue-long: Background job worker (long tasks)  
+# - scheduler: Cron job scheduler
+
+# =============================================================================
+# DEPLOYMENT INSTRUCTIONS
+# =============================================================================
+
+# 1. Create Zerops project and services:
+#    - MariaDB 11 service named "db"
+#    - Valkey 7.2 service named "redis-cache"  
+#    - Valkey 7.2 service named "redis-queue"
+#
+# 2. Set environment secrets in Zerops Dashboard:
+#    - dbPassword: Strong database password
+#    - adminPassword: ERPNext admin password
+#    - siteName: Your domain name
+#
+# 3. Deploy using git integration or Zerops CLI
+#
+# 4. Access your ERPNext instance at the app service URL
 
 # =============================================================================
 # DEPLOYMENT INSTRUCTIONS

zerops.yml

@@ -1,291 +1,132 @@
-# Zerops YAML configuration for Frappe/ERPNext deployment
-# This file defines the infrastructure and deployment settings for Zerops platform
-# Documentation: https://docs.zerops.io/
+# Zerops configuration for Frappe/ERPNext deployment
+# Creates managed database services and uses Docker Compose for application containers
 
-# Import this file in Zerops GUI to create the project with all required services
-
-project:
-  name: frappe-erpnext
-  
-services:
+zerops:
   # MariaDB Database Service
-  - hostname: db
-    type: mariadb
-    version: "11"
-    mode: HA
-    priority: 10
-    # Zerops will automatically configure MariaDB with UTF8MB4
-    
-  # Redis Cache Service  
-  - hostname: redis-cache
-    type: redis
-    version: "6"
-    mode: HA
-    priority: 10
-    
-  # Redis Queue Service
-  - hostname: redis-queue  
-    type: redis
-    version: "6"
-    mode: HA
-    priority: 10
-    
-  # Main Backend Application (Frappe/ERPNext)
-  - hostname: backend
-    type: python
-    version: "3.11"
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    enableSubdomainAccess: false
-    minContainers: 1
-    maxContainers: 3
-    autoscaling:
-      cpuThreshold: 70
-      memoryThreshold: 80
-    # Environment variables
+  - setup: db
+    base: mariadb@11
     envSecrets:
-      DB_PASSWORD: ${db_password}
-      ADMIN_PASSWORD: ${admin_password}
-      ERPNEXT_VERSION: v15.84.0
-      DB_HOST: db
-      DB_PORT: 3306
-      REDIS_CACHE: redis-cache
-      REDIS_QUEUE: redis-queue
-      SOCKETIO_PORT: 9000
-      FRAPPE_SITE_NAME_HEADER: ${site_name}
-      CUSTOM_IMAGE: frappe/erpnext
-      CUSTOM_TAG: ${ERPNEXT_VERSION}
-    # Persistent volumes for site data
-    volumes:
-      - name: frappe-sites
-        mountPath: /home/frappe/frappe-bench/sites
-        size: 20GB
-      - name: frappe-logs
-        mountPath: /home/frappe/frappe-bench/logs
-        size: 5GB
-    # Build settings using production Dockerfile
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      # Build-time commands to include site and apps in image
-      buildCommands:
-        - |
-          # Install any additional apps during build
-          # Example: bench get-app --branch main custom_app https://github.com/user/custom_app
-          echo "Building Frappe image with pre-installed apps..."
-      # Runtime initialization - deployment specific
-      runPrepareCommands:
-        - |
-          # Copy and execute the site initialization script
-          cp scripts/init-frappe-site.sh /home/frappe/init-frappe-site.sh
-          chmod +x /home/frappe/init-frappe-site.sh
-          chown frappe:frappe /home/frappe/init-frappe-site.sh
-          
-          # Run the initialization script as frappe user
-          su - frappe -c "/home/frappe/init-frappe-site.sh"
-    # Health check
-    healthCheck:
-      httpGet:
-        port: 8000
-        path: /api/method/ping
-      initialDelaySeconds: 120
-      periodSeconds: 30
-        
-  # Frontend/Nginx Proxy Service  
-  - hostname: frontend
-    type: nginx
-    version: "1.24"
-    enableSubdomainAccess: true
-    minContainers: 1
-    maxContainers: 2
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    # Environment variables for nginx templating
-    envSecrets:
-      BACKEND: backend:8000
-      SOCKETIO: websocket:9000
-      FRAPPE_SITE_NAME_HEADER: ${site_name}
-      UPSTREAM_REAL_IP_ADDRESS: 127.0.0.1
-      UPSTREAM_REAL_IP_HEADER: X-Forwarded-For  
-      UPSTREAM_REAL_IP_RECURSIVE: "off"
-      PROXY_READ_TIMEOUT: 120s
-      CLIENT_MAX_BODY_SIZE: 50m
-    # Build nginx with custom configuration
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      # Use the nginx configuration from the frappe image
-      runPrepareCommands:
-        - cp /opt/frappe/nginx-template.conf /etc/nginx/conf.d/default.conf.template
-        - envsubst '$BACKEND $SOCKETIO $FRAPPE_SITE_NAME_HEADER' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf
-      # Command to run nginx
-      start: nginx-entrypoint.sh
-    # Health check
-    healthCheck:
-      httpGet:
-        port: 8080
-        path: /api/method/ping
-      initialDelaySeconds: 30
-      periodSeconds: 10
-        
-  # WebSocket Service
-  - hostname: websocket  
-    type: nodejs
-    version: "18"
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    enableSubdomainAccess: false
-    minContainers: 1
-    maxContainers: 2
-    # Shared persistent volumes for site access
-    volumes:
-      - name: frappe-sites
-        mountPath: /home/frappe/frappe-bench/sites
-        size: 20GB
-    # Environment variables
-    envSecrets:
-      DB_HOST: db
-      DB_PORT: 3306
-      REDIS_CACHE: redis-cache
-      REDIS_QUEUE: redis-queue
-      SOCKETIO_PORT: 9000
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      runPrepareCommands:
-        - |
-          # Copy and execute the service initialization script
-          cp scripts/init-service.sh /home/frappe/init-service.sh
-          chmod +x /home/frappe/init-service.sh
-          chown frappe:frappe /home/frappe/init-service.sh
-          
-          # Run the initialization script as frappe user
-          su - frappe -c "/home/frappe/init-service.sh WebSocket"
-      # Command to run the WebSocket server
-      start: node /home/frappe/frappe-bench/apps/frappe/socketio.js
-    # Health check  
-    healthCheck:
-      httpGet:
-        port: 9000
-        path: /socket.io/
-      initialDelaySeconds: 60
-      periodSeconds: 30
-        
-  # Queue Worker - Short Tasks
-  - hostname: queue-short
-    type: python
-    version: "3.11"
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    enableSubdomainAccess: false
-    minContainers: 1
-    maxContainers: 3
-    # Shared persistent volumes for site access
-    volumes:
-      - name: frappe-sites
-        mountPath: /home/frappe/frappe-bench/sites
-        size: 20GB
-    # Environment variables
-    envSecrets:
-      DB_HOST: db
-      DB_PORT: 3306  
-      REDIS_CACHE: redis-cache
-      REDIS_QUEUE: redis-queue
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      runPrepareCommands:
-        - |
-          # Copy and execute the service initialization script
-          cp scripts/init-service.sh /home/frappe/init-service.sh
-          chmod +x /home/frappe/init-service.sh
-          chown frappe:frappe /home/frappe/init-service.sh
-          
-          # Run the initialization script as frappe user
-          su - frappe -c "/home/frappe/init-service.sh 'Queue Worker (Short)'"
-      # Command to run the worker
-      start: bench worker --queue short,default
-          
-  # Queue Worker - Long Tasks  
-  - hostname: queue-long
-    type: python
-    version: "3.11"
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    enableSubdomainAccess: false
-    minContainers: 1
-    maxContainers: 2
-    # Shared persistent volumes for site access
-    volumes:
-      - name: frappe-sites
-        mountPath: /home/frappe/frappe-bench/sites
-        size: 20GB
-    # Environment variables
-    envSecrets:
-      DB_HOST: db
-      DB_PORT: 3306
-      REDIS_CACHE: redis-cache
-      REDIS_QUEUE: redis-queue
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      runPrepareCommands:
-        - |
-          # Copy and execute the service initialization script
-          cp scripts/init-service.sh /home/frappe/init-service.sh
-          chmod +x /home/frappe/init-service.sh
-          chown frappe:frappe /home/frappe/init-service.sh
-          
-          # Run the initialization script as frappe user
-          su - frappe -c "/home/frappe/init-service.sh 'Queue Worker (Long)'"
-      # Command to run the long queue worker
-      start: bench worker --queue long,default,short
-          
-  # Scheduler Service
-  - hostname: scheduler
-    type: python
-    version: "3.11"
-    buildFromGit: https://github.com/UhrinDavid/frappe_docker@zerops
-    enableSubdomainAccess: false  
-    minContainers: 1
-    maxContainers: 1
-    # Shared persistent volumes for site access
-    volumes:
-      - name: frappe-sites
-        mountPath: /home/frappe/frappe-bench/sites
-        size: 20GB
-    # Environment variables
-    envSecrets:
-      DB_HOST: db
-      DB_PORT: 3306
-      REDIS_CACHE: redis-cache
-      REDIS_QUEUE: redis-queue
-    build:
-      dockerfile: ./images/production/Containerfile
-      context: .
-      runPrepareCommands:
-        - |
-          # Copy and execute the service initialization script
-          cp scripts/init-service.sh /home/frappe/init-service.sh
-          chmod +x /home/frappe/init-service.sh
-          chown frappe:frappe /home/frappe/init-service.sh
-          
-          # Run the initialization script as frappe user
-          su - frappe -c "/home/frappe/init-service.sh Scheduler"
-      # Command to run the scheduler
-      start: bench schedule
+      MARIADB_ROOT_PASSWORD: ${dbPassword}
 
-# Required secrets that need to be set in Zerops dashboard:
-# - db_password: Database password for MariaDB (e.g., "mySecureDbPassword123")
-# - admin_password: ERPNext admin user password (e.g., "myAdminPassword123")  
-# - site_name: Your site domain name (e.g., "mycompany.example.com")
-# 
-# This configuration includes:
-# ✅ Clean Docker image with ERPNext apps (deployment-agnostic)
-# ✅ Persistent volumes for site data, uploads, and configurations  
-# ✅ Deployment-specific initialization via runPrepareCommands
-# ✅ Shared volumes across all services that need site access
-# 
-# Benefits:
-# - Docker image is reusable across different deployment platforms
-# - Site initialization is handled at deployment time (not in image)
-# - Container restarts preserve all site data and configurations
-# - Apps are consistently deployed across all containers
-# - No manual setup required after deployment
-# - Fast container startup times
-# - Proper separation between image and deployment concerns
+  # Redis Cache Service (using Valkey - Redis compatible)
+  - setup: redis-cache
+    base: valkey@7.2
+    mode: NON_HA  # Use HA in production
+
+  # Redis Queue Service (using Valkey - Redis compatible)  
+  - setup: redis-queue
+    base: valkey@7.2
+    mode: NON_HA  # Use HA in production
+
+  # Frappe/ERPNext Application Services (Docker Compose)
+  - setup: app
+    build:
+      # Deploy the custom docker-compose file to the runtime
+      deployFiles: ./docker-compose.zerops.yaml
+      addToRunPrepare: ./docker-compose.zerops.yaml
+      # Copy initialization scripts
+      deployFiles:
+        - ./docker-compose.zerops.yaml
+        - ./scripts/
+    run:
+      # Environment variables for Docker Compose services
+      envVariables:
+        # Image configuration
+        CUSTOM_IMAGE: frappe/erpnext
+        CUSTOM_TAG: v15.84.0
+        ERPNEXT_VERSION: v15.84.0
+        PULL_POLICY: always
+        RESTART_POLICY: unless-stopped
+        
+        # Database connection (using Zerops service)
+        DB_HOST: db
+        DB_PORT: 3306
+        DB_PASSWORD: ${dbPassword}
+        
+        # Redis connections (using Zerops Valkey services)  
+        REDIS_CACHE: redis-cache:6379
+        REDIS_QUEUE: redis-queue:6379
+        
+        # Site configuration
+        FRAPPE_SITE_NAME_HEADER: ${siteName}
+        ADMIN_PASSWORD: ${adminPassword}
+        
+        # Frontend/Nginx configuration
+        UPSTREAM_REAL_IP_ADDRESS: 127.0.0.1
+        UPSTREAM_REAL_IP_HEADER: X-Forwarded-For
+        UPSTREAM_REAL_IP_RECURSIVE: "off"
+        PROXY_READ_TIMEOUT: 120
+        CLIENT_MAX_BODY_SIZE: 50m
+      
+      # Site initialization and app installation
+      prepareCommands:
+        # Wait for database and Redis services
+        - |
+          echo "Waiting for database connection..."
+          while ! mariadb -hdb -P3306 -uroot -p${DB_PASSWORD} -e "SELECT 1;" 2>/dev/null; do
+            echo "Database not ready, waiting 5 seconds..."
+            sleep 5
+          done
+          echo "Database connection established"
+        
+        - |
+          echo "Waiting for Redis cache (Valkey) connection..."
+          while ! redis-cli -h redis-cache -p 6379 ping 2>/dev/null; do
+            echo "Redis cache not ready, waiting 5 seconds..."
+            sleep 5
+          done
+          echo "Redis cache connection established"
+        
+        - |
+          echo "Waiting for Redis queue (Valkey) connection..."
+          while ! redis-cli -h redis-queue -p 6379 ping 2>/dev/null; do
+            echo "Redis queue not ready, waiting 5 seconds..."
+            sleep 5
+          done
+          echo "Redis queue connection established"
+
+        # Pull Docker images
+        - docker compose -f docker-compose.zerops.yaml pull
+        
+        # Install Frappe site and apps using dedicated script
+        - |
+          echo "🚀 Running site installation..."
+          chmod +x scripts/install-site.sh
+          ./scripts/install-site.sh
+          echo "✅ Site installation completed"
+      
+      # Start all application services
+      start: docker compose -f docker-compose.zerops.yaml up --force-recreate
+      
+      # Expose the frontend port
+      ports:
+        - port: 8080
+          httpSupport: true
+
+# Required secrets in Zerops dashboard:
+# - dbPassword: Database password for MariaDB (e.g., "mySecureDbPassword123")
+# - adminPassword: ERPNext admin user password (e.g., "myAdminPassword123")
+# - siteName: Your site domain name (e.g., "mycompany.example.com")
+#
+# This configuration provides:
+# ✅ Managed MariaDB service with automatic backups and scaling
+# ✅ Dedicated Redis services for cache and queue (better performance)
+# ✅ Docker Compose for Frappe containers only (simpler, faster)
+# ✅ Automatic site initialization with custom apps
+# ✅ Persistent storage managed by Zerops
+# ✅ Service health monitoring and auto-restart
+# ✅ Built-in service discovery (services connect by name)
+#
+# Architecture:
+# - db: Managed MariaDB 11 service (default Zerops configuration)
+# - redis-cache: Managed Valkey 7.2 service (default configuration, NON_HA mode)  
+# - redis-queue: Managed Valkey 7.2 service (default configuration, NON_HA mode)
+# - app: Docker Compose with Frappe containers (backend, frontend, websocket, workers, scheduler)
+#
+# Benefits of managed services:
+# - Automatic backups and point-in-time recovery
+# - Built-in monitoring and alerting
+# - Automatic security updates
+# - High availability and failover
+# - Performance optimization
+# - No container overhead for databases