- Add eval expansion of references when populating stack .env files
- Resolves PROJECTS_DIR=${STACKS_DIR} to actual path value
- Ensures Docker Compose receives fully resolved paths, not variable references
Changes:
- Docker-compose files: Skip validation entirely since we intentionally leave environment/volume variables as ${VAR}
- Config files: Changed from error+exit to warning only
- Prevents false positives where variables like ${PUID}, ${PGID}, ${TZ} in environment sections were flagged as errors
Now all missing variables will warn but not cause script to exit.
The Python script wasn't receiving the file path correctly because heredoc syntax doesn't support command-line arguments the way it was written. Changed to use environment variable COMPOSE_FILE_PATH instead.
This fixes the SyntaxError that occurred during deployment where Python was trying to parse YAML as Python code.
Major improvements to environment variable management:
1. Added .env.example files for ALL stacks
- Each stack now has its own .env.example with only required variables
- Variables include comments explaining their purpose
- Examples: core, dockge, infrastructure, dashboards, arcane, media, media-management, transcoders, monitoring, sablier, utilities, productivity, wikis, vpn, homeassistant, alternatives
2. Created .env.global generation
- Generates /opt/stacks/.env.global with all variables
- Strips comments and blank lines for clean output
- Available to all stacks for reference
3. Improved variable replacement strategy
- Variable replacement now ONLY targets labels and x-dockge sections in docker-compose files
- Configuration files (traefik, authelia) still get full variable replacement
- Uses Python script for precise section detection
- Preserves environment variables and volume mounts as-is
4. New deployment approach
- Each stack copies .env.example to .env
- Values populated from main ~/EZ-Homelab/.env file
- No more manual sed commands to remove unused variables
- Consistent approach across all deployment functions
5. Updated deployment functions
- deploy_dockge, deploy_core, deploy_infrastructure, deploy_dashboards, deploy_arcane, deploy_sablier_stack
- All now use process_stack_env() for clean .env handling
- All use updated localize_yml_file() for targeted variable replacement
Benefits:
- Clean, minimal .env files for each stack
- No unnecessary variables or comments in deployed .env files
- Variables in compose files preserved for Docker Compose to handle
- Easier to understand what each stack needs
- Uniform deployment approach across all stacks
- Add placeholder check for AUTHELIA_ADMIN_PASSWORD (like other secrets)
- Save AUTHELIA_ADMIN_PASSWORD to .env file after generation
- Now properly replaces 'generate-with-openssl-rand-hex-64' with DEFAULT_PASSWORD
- Ensures password is saved to both commented and uncommented versions in .env
- Removed backup logic from localize_yml_file function
- Added backup checks before file copy in all deploy functions
- Backups now only created when docker-compose.yml already exists
- Prevents unnecessary .backup files on first deployment
- Add placeholder string checks to AUTHELIA_JWT_SECRET, AUTHELIA_SESSION_SECRET, and AUTHELIA_STORAGE_ENCRYPTION_KEY
- Now generates secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Ensures consistent behavior across all auto-generated secrets
- Check if ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET contain placeholder string
- Generate new secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Previously secrets were not generated because .env file loaded placeholder as non-empty value
- Add commented/uncommented sed handling for ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET
- Use sudo for file copying in deploy_arcane to ensure proper permissions
- Ensures secrets are properly saved to .env file in repo folder
- Added ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET to .env.example
- Created deploy_arcane() function in ez-homelab.sh
- Auto-generate Arcane secrets after Authelia secrets
- Deploy Arcane in both Option 2 (Core Server) and Option 3 (Additional Server)
- Added Arcane docker-compose.yml configuration
The registration function creates '-server-routes.yml'
but the verification was checking for 'docker-provider-.yml'.
Changes:
- Updated file verification to check for correct filenames
- Added DOMAIN variable loading in add_remote_server_to_traefik()
- Replaced hard-coded domain with ${DOMAIN} variable in route rules
- Routes now use dynamic domain from .env instead of hard-coded value
This fixes the 'docker-provider file not created' error in step 9.
Instead of exiting immediately when SSH key setup fails, the script now:
- Offers retry option for transient issues
- Allows skipping SSH setup with manual configuration instructions
- Provides option to return to main menu
- Continues deployment flow more gracefully
This prevents frustrating exits when SSH configuration needs adjustment.
The prepare_deployment() function was defined but never called, causing
option 1 (Install Prerequisites) to not execute. Added the function call
after the menu selection loop to properly handle special deployment modes.
Issue: Option 3 deployment failed because traefik-network wasn't created,
but the cleanup function didn't properly strip network references from files.
Changes:
- scripts/ez-homelab.sh: Add traefik-network creation in Step 2
(Network is harmless if unused - prevents docker compose errors)
- Improve sed patterns in configure_remote_server_routing():
* Use anchored patterns (^ and $) to match exact lines
* Fix network removal regex to match indentation correctly
The traefik-network will exist but remain unused on additional servers.
Services are accessed via core Traefik's manual HTTP routes.
- Add ports 80/443 to remote Traefik template
- Enable API access (insecure=true) for debugging
- Update configure_remote_server_routing to use HTTP-only config
- Change entrypoint from websecure to web
- Remove TLS cert resolver references
- Remove authelia middleware references
- Replace Docker TLS provider with manual HTTP routes
- Core Traefik now uses file provider with direct URLs
- Fixes 404/gateway timeout errors on remote services
- Routes HTTPS from core to HTTP on remote servers
- Fix port in routes (dozzle uses 8085, not 8082)
Resolves issues with remote server service accessibility.
- Rename disable_traefik_on_remote_services() to configure_remote_server_routing()
- Enable Traefik on dockge, dozzle, glances with server-specific subdomains
- Keep sablier disabled (no web UI)
- Add Traefik dashboard route with server-specific subdomain
- Update sed patterns to dynamically replace server names in routing rules
This allows infrastructure/monitoring services to coexist on multiple
servers without conflicts, while shared services remain centralized.
- Fix missing domain in docker provider defaultRule (use DOMAIN or DUCKDNS_DOMAIN)
- Add disable_traefik_on_remote_services() to strip routing labels from remote services
- Call disable function automatically during remote server deployment (Step 5.5)
- Remote services now properly accessed through core Traefik via docker provider
- Prevents errors: middleware/certresolver not found on remote Traefik
Resolves Traefik errors on remote servers where services had labels for
SSL/auth that only exist on core server.
- Auto-create SSH config entry for passwordless core server access
- Add pre-flight SSH connectivity check before registration
- Verify docker-provider and sablier-middleware files are created on core
- Display explicit success/failure messages with troubleshooting steps
- Create placeholder routes.yml for Traefik dynamic config
- Verify Traefik container starts successfully after deployment
- Add container status check after Traefik deployment
Fixes issues where remote server registration silently failed and
Traefik deployment did not verify configuration files existed.
Major features:
- Automated SSH key setup between remote and core servers
- Docker TLS configuration with shared CA certificates
- Automatic deployment of Dockge, Traefik, Sablier, and Infrastructure stacks
- Copy all stacks (except core) to remote server for on-demand deployment
- New standalone Traefik stack for remote server container discovery
- Locale-aware SSH/SCP commands to handle Raspberry Pi warnings
- Variable expansion support in .env files (${VAR} references)
- Comprehensive error handling and verbose deployment logging
Technical improvements:
- setup_ssh_key_to_core() - Automated RSA 4096-bit key generation and installation
- setup_multi_server_tls() - Fetch shared CA from core server via SSH
- copy_all_stacks_for_remote() - Deploy all stacks except core
- deploy_traefik_stack() - Local Traefik for container discovery
- Enhanced localization with envsubst support
- Docker network creation (traefik-network, homelab-network)
- Password authentication with special character handling
Fixes:
- Fixed SSH key path handling for non-root users
- Fixed SCP exit code checking (was checking grep instead of scp)
- Fixed CA file detection with proper test commands
- Removed unnecessary prepare_deployment() function call
- Added ACTUAL_USER variable initialization for remote deployments
Added Python script to automatically merge multi-line Authelia secrets
(JWT_SECRET, SESSION_SECRET, STORAGE_ENCRYPTION_KEY) during core
deployment. This prevents envsubst from reading truncated values when
the .env file contains accidental line breaks.
Fixes Authelia startup errors:
- 'encryption key does not appear to be valid for this database'
- Invalid URL parsing due to incomplete variable expansion
- Move users_database.yml template from config/ to secrets/ folder
- Authelia configuration expects users file at /secrets/users_database.yml
- Replace hardcoded values in configuration.yml with variables:
- jwt_secret, session secret, storage encryption_key
- Domain references (kelinreij.duckdns.org → ${DOMAIN})
- Update deploy script to not move config files (already in subdirectories)
- Fix sed command path to target config/configuration.yml
- Remove unnecessary mkdir and mv commands from deploy_core()
This ensures Authelia uses the correctly generated file with user credentials.
- Implement multi-server Traefik + Sablier architecture
- Add label-based automatic service discovery
- Create separate Sablier stack deployment
- Add remote server deployment workflow (Option 3)
- Add 9 new functions for multi-server management
- Remove deprecated config-templates folder
- Replace hardcoded private data with placeholders
- Update backup timestamp format to YY_MM_DD_hh_mm
- Add markup.yml to .gitignore
Breaking changes:
- Removed Sablier from core docker-compose.yml (now separate stack)
- Config templates moved from config-templates/ to docker-compose/core/
- REQUIRED_VARS now dynamic based on deployment type
- Use awk to extract only the argon2 hash from Authelia command output
- Prevent 'Digest:' or 'digest:' prefixes from being included in password hashes
- Apply fix to both password generation locations in the script
- Escape $ characters in AUTHELIA_ADMIN_PASSWORD_HASH in deployed .env file
- Prevent Docker Compose from interpreting password hash as environment variables
- Update ez-homelab.sh to source common.sh and use specialized users_database.yml processing
- Update localize_users_database_file to properly resolve nested variables in AUTHELIA_ADMIN_EMAIL
- Fix template to use correct AUTHELIA_* variables instead of DEFAULT_* variables
- Update deploy-core.sh to only process files containing variables and fix .env path
- Fix file permissions for authelia config files
- Reorganize Authelia configuration files
- Add new dynamic routing files for Traefik
- Update various service docker-compose files
- Remove outdated templates and scripts
- Remove localize_deployment call from main flow to avoid modifying repo files
- Enhance localize_yml_file to recursively expand nested variables using envsubst
- Ensure config files contain actual values, not variable names
- Fix password hash copying to user_database.yml by removing premature env cleanup
- Update menu case logic to match display options (1-4)
- Add input validation loop for better user experience
- Handle special menu options (prerequisites, NVIDIA drivers) appropriately
- Add professional ASCII box styling to main menu
- Implement interactive variable configuration with validation
- Add icons to all prompts (🌐🌍🦆🔑👤🔒📧🏠)
- Create vanishing prompts that replace with status confirmations
- Add comprehensive menu system with proceed/review/quit options
- Show current configuration values before deployment choices
- Implement proper error handling for invalid inputs
- Add immediate quit functionality with 'q' during any prompt
- Improve spacing and visual hierarchy throughout interface
- Fix deployment flow to prevent accidental starts on invalid input
- Add traefik.docker.network=traefik-network label to homepage service
- Prevent Traefik from using wrong IP from homelab-network
- Resolve 504 Gateway Timeout issues after authentication
- Update various docker-compose configurations and templates
- Clean up unused configuration files
- Move Traefik traefik.yml to config/ directory for correct Docker mounting
- Remove invalid session.cookies section from Authelia template and config processing
- Ensure all configuration files are placed in correct locations for Docker containers
- Prevent configuration validation errors that cause service restarts
- Move configuration.yml and users_database.yml to authelia/config/ directory
- Ensure files are in the correct location for Docker volume mount ./authelia/config:/config
- Prevent Authelia from using default configuration instead of processed template
