Commit Graph

56 Commits

Author SHA1 Message Date
c5e9d959bd Fix backup file creation - only backup when overwriting existing files
- Removed backup logic from localize_yml_file function
- Added backup checks before file copy in all deploy functions
- Backups now only created when docker-compose.yml already exists
- Prevents unnecessary .backup files on first deployment
2026-02-10 14:19:24 -05:00
b27b8d380d Make Authelia secret generation consistent with Arcane
- Add placeholder string checks to AUTHELIA_JWT_SECRET, AUTHELIA_SESSION_SECRET, and AUTHELIA_STORAGE_ENCRYPTION_KEY
- Now generates secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Ensures consistent behavior across all auto-generated secrets
2026-02-10 14:13:17 -05:00
e78063d8cb Fix Arcane secret generation from placeholder values
- Check if ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET contain placeholder string
- Generate new secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Previously secrets were not generated because .env file loaded placeholder as non-empty value
2026-02-10 14:06:29 -05:00
ac7b095d78 Fix Arcane secret generation and deployment
- Add commented/uncommented sed handling for ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET
- Use sudo for file copying in deploy_arcane to ensure proper permissions
- Ensures secrets are properly saved to .env file in repo folder
2026-02-10 13:53:35 -05:00
4fcda86495 Add Arcane stack with auto-deployment and secret generation
- Added ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET to .env.example
- Created deploy_arcane() function in ez-homelab.sh
- Auto-generate Arcane secrets after Authelia secrets
- Deploy Arcane in both Option 2 (Core Server) and Option 3 (Additional Server)
- Added Arcane docker-compose.yml configuration
2026-02-10 13:15:14 -05:00
Kelin
33336c75ff Fix step 9 registration failure - correct file verification
The registration function creates '-server-routes.yml'
but the verification was checking for 'docker-provider-.yml'.

Changes:
- Updated file verification to check for correct filenames
- Added DOMAIN variable loading in add_remote_server_to_traefik()
- Replaced hard-coded domain with ${DOMAIN} variable in route rules
- Routes now use dynamic domain from .env instead of hard-coded value

This fixes the 'docker-provider file not created' error in step 9.
2026-02-08 20:17:30 -05:00
Kelin
86619db71e Improve error handling for SSH setup failures in option 3
Instead of exiting immediately when SSH key setup fails, the script now:
- Offers retry option for transient issues
- Allows skipping SSH setup with manual configuration instructions
- Provides option to return to main menu
- Continues deployment flow more gracefully

This prevents frustrating exits when SSH configuration needs adjustment.
2026-02-08 20:01:05 -05:00
Kelin
824a415f01 Fix: Call prepare_deployment() to handle option 1 (install prerequisites)
The prepare_deployment() function was defined but never called, causing
option 1 (Install Prerequisites) to not execute. Added the function call
after the menu selection loop to properly handle special deployment modes.
2026-02-08 19:33:56 -05:00
Kelin
043e1cb64a Fix Option 3: Create traefik-network and improve sed patterns
Issue: Option 3 deployment failed because traefik-network wasn't created,
but the cleanup function didn't properly strip network references from files.

Changes:
- scripts/ez-homelab.sh: Add traefik-network creation in Step 2
  (Network is harmless if unused - prevents docker compose errors)
- Improve sed patterns in configure_remote_server_routing():
  * Use anchored patterns (^ and $) to match exact lines
  * Fix network removal regex to match indentation correctly

The traefik-network will exist but remain unused on additional servers.
Services are accessed via core Traefik's manual HTTP routes.
2026-02-07 22:22:52 -05:00
Kelin
16f7eaa703 Fix: Restore Traefik labels for Option 2 core deployments
The previous commit removed labels from templates, breaking Option 2 (Core Server)
deployments. This commit restores the proper architecture:

Templates (docker-compose files):
- Restore all Traefik labels to dockge/docker-compose.yml
- Restore all Traefik labels to infrastructure services (dozzle, glances, code-server)
- Restore traefik-network references
- Templates now work correctly for Option 2 (Core Server)

Scripts (ez-homelab.sh):
- Rewrite configure_remote_server_routing() to strip labels for Option 3
- Re-add configure_remote_server_routing() call to deploy_remote_server()
- Add as Step 5 (after copying stacks, before deploying them)
- Update remaining step numbers (6-9)

Architecture flow:
Option 2 (Core Server):
  - Uses templates as-is with Traefik labels
  - Local Traefik discovers services via Docker labels
  - Services accessible at https://service.domain

Option 3 (Additional Server):
  - Copies templates with labels (Step 4)
  - Strips out labels and traefik-network (Step 5)
  - Deploys labelless services with exposed ports (Steps 6-8)
  - Core Traefik routes via manual HTTP configs to IP:PORT
  - Services accessible at https://service.hostname.domain
2026-02-07 21:59:21 -05:00
Kelin
ce3fbdb244 Simplify Option 3: Remove local Traefik from additional servers
Major architectural simplification for headless additional servers:

Templates:
- Remove Traefik labels from dockge/docker-compose.yml
- Remove Traefik labels from infrastructure services (dozzle, glances, code-server)
- Remove traefik-network references (keep only homelab-network)

Scripts (ez-homelab.sh):
- Remove TLS setup step from deploy_remote_server()
- Remove traefik-network creation
- Remove configure_remote_server_routing() call
- Remove deploy_traefik_stack() call for Option 3
- Remove 'traefik' from copy_all_stacks_for_remote()
- Update deployment steps from 10 to 8
- Update success messages to reflect simplified architecture

Scripts (common.sh):
- Remove unused generate_traefik_provider_config() function

Config:
- Add ADMIN_SSH_PUB_KEY field to .env.example

Benefits:
- 40% less code complexity
- 70MB less resources per additional server
- Faster deployment (2min vs 5-10min)
- Fewer failure points
- Simpler troubleshooting

Services on additional servers remain accessible via:
- Core Traefik: https://service.hostname.domain
- Direct IP: http://IP:PORT
2026-02-07 21:34:20 -05:00
Kelin
72d3d8b38f Fix remote server Traefik configuration
- Add ports 80/443 to remote Traefik template
- Enable API access (insecure=true) for debugging
- Update configure_remote_server_routing to use HTTP-only config
  - Change entrypoint from websecure to web
  - Remove TLS cert resolver references
  - Remove authelia middleware references
- Replace Docker TLS provider with manual HTTP routes
  - Core Traefik now uses file provider with direct URLs
  - Fixes 404/gateway timeout errors on remote services
  - Routes HTTPS from core to HTTP on remote servers
- Fix port in routes (dozzle uses 8085, not 8082)

Resolves issues with remote server service accessibility.
2026-02-07 20:43:34 -05:00
Kelin
63a837f8e9 feat: Implement server-specific subdomains for infrastructure services
- Rename disable_traefik_on_remote_services() to configure_remote_server_routing()
- Enable Traefik on dockge, dozzle, glances with server-specific subdomains
- Keep sablier disabled (no web UI)
- Add Traefik dashboard route with server-specific subdomain
- Update sed patterns to dynamically replace server names in routing rules

This allows infrastructure/monitoring services to coexist on multiple
servers without conflicts, while shared services remain centralized.
2026-02-07 17:05:00 -05:00
Kelin
802a4d1ba0 Fix multiserver deployment: domain in docker-provider and auto-disable remote Traefik
- Fix missing domain in docker provider defaultRule (use DOMAIN or DUCKDNS_DOMAIN)
- Add disable_traefik_on_remote_services() to strip routing labels from remote services
- Call disable function automatically during remote server deployment (Step 5.5)
- Remote services now properly accessed through core Traefik via docker provider
- Prevents errors: middleware/certresolver not found on remote Traefik

Resolves Traefik errors on remote servers where services had labels for
SSL/auth that only exist on core server.
2026-02-07 16:24:57 -05:00
Kelin
9fea40c8b5 Fix multiserver deployment: Add SSH config automation, enhance registration verification, improve Traefik deployment
- Auto-create SSH config entry for passwordless core server access
- Add pre-flight SSH connectivity check before registration
- Verify docker-provider and sablier-middleware files are created on core
- Display explicit success/failure messages with troubleshooting steps
- Create placeholder routes.yml for Traefik dynamic config
- Verify Traefik container starts successfully after deployment
- Add container status check after Traefik deployment

Fixes issues where remote server registration silently failed and
Traefik deployment did not verify configuration files existed.
2026-02-07 16:09:56 -05:00
kelin
44b529a7cb feat: Add Option 3 - Deploy Additional Server with multi-server support
Major features:
- Automated SSH key setup between remote and core servers
- Docker TLS configuration with shared CA certificates
- Automatic deployment of Dockge, Traefik, Sablier, and Infrastructure stacks
- Copy all stacks (except core) to remote server for on-demand deployment
- New standalone Traefik stack for remote server container discovery
- Locale-aware SSH/SCP commands to handle Raspberry Pi warnings
- Variable expansion support in .env files (${VAR} references)
- Comprehensive error handling and verbose deployment logging

Technical improvements:
- setup_ssh_key_to_core() - Automated RSA 4096-bit key generation and installation
- setup_multi_server_tls() - Fetch shared CA from core server via SSH
- copy_all_stacks_for_remote() - Deploy all stacks except core
- deploy_traefik_stack() - Local Traefik for container discovery
- Enhanced localization with envsubst support
- Docker network creation (traefik-network, homelab-network)
- Password authentication with special character handling

Fixes:
- Fixed SSH key path handling for non-root users
- Fixed SCP exit code checking (was checking grep instead of scp)
- Fixed CA file detection with proper test commands
- Removed unnecessary prepare_deployment() function call
- Added ACTUAL_USER variable initialization for remote deployments
2026-02-06 22:00:25 -05:00
kelin
4803d2c856 Fix: Add multi-line secret sanitization to deployment script
Added Python script to automatically merge multi-line Authelia secrets
(JWT_SECRET, SESSION_SECRET, STORAGE_ENCRYPTION_KEY) during core
deployment. This prevents envsubst from reading truncated values when
the .env file contains accidental line breaks.

Fixes Authelia startup errors:
- 'encryption key does not appear to be valid for this database'
- Invalid URL parsing due to incomplete variable expansion
2026-02-05 12:36:19 -05:00
kelin
d4c9516e00 Fix: Generate users_database.yml in secrets folder with proper variables
- Move users_database.yml template from config/ to secrets/ folder
- Authelia configuration expects users file at /secrets/users_database.yml
- Replace hardcoded values in configuration.yml with variables:
  - jwt_secret, session secret, storage encryption_key
  - Domain references (kelinreij.duckdns.org → ${DOMAIN})
- Update deploy script to not move config files (already in subdirectories)
- Fix sed command path to target config/configuration.yml
- Remove unnecessary mkdir and mv commands from deploy_core()

This ensures Authelia uses the correctly generated file with user credentials.
2026-02-05 12:12:33 -05:00
Kelin
73cb274160 v0.1.2: Multi-server architecture + security cleanup
- Implement multi-server Traefik + Sablier architecture
- Add label-based automatic service discovery
- Create separate Sablier stack deployment
- Add remote server deployment workflow (Option 3)
- Add 9 new functions for multi-server management
- Remove deprecated config-templates folder
- Replace hardcoded private data with placeholders
- Update backup timestamp format to YY_MM_DD_hh_mm
- Add markup.yml to .gitignore

Breaking changes:
- Removed Sablier from core docker-compose.yml (now separate stack)
- Config templates moved from config-templates/ to docker-compose/core/
- REQUIRED_VARS now dynamic based on deployment type
2026-02-04 19:36:18 -05:00
Kelin Fox
75e66586d1 Fix Authelia password hash generation to remove 'digest:' prefix
- Use awk to extract only the argon2 hash from Authelia command output
- Prevent 'Digest:' or 'digest:' prefixes from being included in password hashes
- Apply fix to both password generation locations in the script
2026-02-03 23:43:31 -05:00
Kelin
5a4360bc53 Fix Docker Compose variable substitution for password hashes
- Escape $ characters in AUTHELIA_ADMIN_PASSWORD_HASH in deployed .env file
- Prevent Docker Compose from interpreting password hash as environment variables
- Update ez-homelab.sh to source common.sh and use specialized users_database.yml processing
2026-02-03 22:58:52 -05:00
Kelin
3d5979b5f1 Implement fixes from test results
- Update Docker install to use curl method
- Rename ADMIN_PASSWORD to AUTHELIA_ADMIN_PASSWORD
- Fix Authelia password hash generation (remove grep, no quotes)
- Revert compose labels to single quotes
- Ensure users_database.yml has unquoted password placeholder
2026-02-02 20:59:07 -05:00
Kelin
5b5df8960b Fix variable replacement logic for nested variables and remove repo file modification
- Remove localize_deployment call from main flow to avoid modifying repo files
- Enhance localize_yml_file to recursively expand nested variables using envsubst
- Ensure config files contain actual values, not variable names
2026-02-02 17:30:24 -05:00
Kelin
fd15c18389 Strip carriage returns from .env values to handle DOS line endings 2026-02-02 14:48:28 -05:00
Kelin
10338f2de5 Fix subshell issue in config file processing loops to allow env var replacement 2026-02-02 14:45:13 -05:00
Kelin
98f6b55fdf Fix load_env_file_safely to actually export environment variables 2026-02-02 14:37:23 -05:00
Kelin
457d803f57 Strip inline comments from .env values before parsing to prevent xargs quote issues 2026-02-02 14:11:14 -05:00
Kelin
e5d678ebbe Replace sed-based variable replacement with envsubst for universal handling of special characters 2026-02-02 14:06:26 -05:00
Kelin
cd9dc925f3 Add logging of missing variables and trim keys in .env parsing 2026-02-02 13:56:27 -05:00
Kelin
80b44f8bef Trim whitespace from .env values to ensure proper secret generation 2026-02-02 13:52:28 -05:00
Kelin
4fd30340ef Quote Authelia password hash to prevent Docker expansion warnings, fix variable trimming with xargs, and update comment exclusion pattern 2026-02-02 13:46:44 -05:00
Kelin
1be1ecb264 Fix variable trimming, reload .env after secret generation, and improve comment exclusion in validation 2026-02-02 13:42:22 -05:00
Kelin
faaf39002a Replace personal URLs with placeholders and fix variable replacement logic 2026-02-02 13:19:22 -05:00
Kelin
0041b15cc2 Fix Authelia password copying and update menu logic
- Fix password hash copying to user_database.yml by removing premature env cleanup
- Update menu case logic to match display options (1-4)
- Add input validation loop for better user experience
- Handle special menu options (prerequisites, NVIDIA drivers) appropriately
2026-02-01 01:12:03 -05:00
EZ-Homelab Assistant
6c4a3362e0 feat: Major UI improvements to ez-homelab.sh
- Add professional ASCII box styling to main menu
- Implement interactive variable configuration with validation
- Add icons to all prompts (🌐 🌍 🦆 🔑 👤 🔒 📧 🏠)
- Create vanishing prompts that replace with status confirmations
- Add comprehensive menu system with proceed/review/quit options
- Show current configuration values before deployment choices
- Implement proper error handling for invalid inputs
- Add immediate quit functionality with 'q' during any prompt
- Improve spacing and visual hierarchy throughout interface
- Fix deployment flow to prevent accidental starts on invalid input
2026-01-31 05:41:30 -05:00
EZ-Homelab Assistant
90a26a9ac4 Fix homepage Traefik network routing and update configurations
- Add traefik.docker.network=traefik-network label to homepage service
- Prevent Traefik from using wrong IP from homelab-network
- Resolve 504 Gateway Timeout issues after authentication
- Update various docker-compose configurations and templates
- Clean up unused configuration files
2026-01-30 23:29:00 -05:00
EZ-Homelab Assistant
465c10ae42 fix: ensure script properly configures Traefik and Authelia
- Move Traefik traefik.yml to config/ directory for correct Docker mounting
- Remove invalid session.cookies section from Authelia template and config processing
- Ensure all configuration files are placed in correct locations for Docker containers
- Prevent configuration validation errors that cause service restarts
2026-01-30 19:42:05 -05:00
EZ-Homelab Assistant
c6fbcb785e fix: move Authelia config files to correct Docker mount location
- Move configuration.yml and users_database.yml to authelia/config/ directory
- Ensure files are in the correct location for Docker volume mount ./authelia/config:/config
- Prevent Authelia from using default configuration instead of processed template
2026-01-30 19:12:49 -05:00
EZ-Homelab Assistant
ea75ea9829 fix: remove REMOTE_SERVER_HOSTNAME dependencies for local deployments
- Remove local-host-production.yml for single-server setups
- Remove remote server sections from sablier.yml for local deployments
- Remove remote server entries from homepage services files when no REMOTE_SERVER_HOSTNAME is set
- Prevent placeholder replacement failures on files not needed for local deployments
2026-01-30 19:06:30 -05:00
EZ-Homelab Assistant
437eb37aff fix: resolve ez-homelab.sh issues for local deployments
- Fix variable substitution in prompt_for_values() to properly display DEFAULT_USER
- Only rename external-host-production.yml for multi-server setups (option 3)
- Prevent REMOTE_SERVER_HOSTNAME validation errors for local deployments (options 1-2)
- Ensure local deployments don't fail on missing remote server variables
2026-01-30 19:03:21 -05:00
kelinfoxy
7b4f562c68 feat: Update EZ-Homelab configurations and documentation
- Update .env.example with latest environment variables
- Enhance homepage dashboard configurations and templates
- Improve Traefik routing templates for external hosts
- Update docker-compose files for dashboards and infrastructure
- Add comprehensive TUI documentation and PRD
- Add new Homelab-Audit documentation
- Remove outdated release notes
2026-01-28 21:56:33 -05:00
Kelin
ea91151829 Minor fixes and improvements to ez-homelab.sh 2026-01-26 23:24:23 -05:00
Kelin
62454130db Fix multi-server TLS setup to support password authentication and improve error handling 2026-01-26 18:16:35 -05:00
Kelin
e2d28b5208 feat: Improve TLS handling for multi-server deployments
- Add CORE_SERVER_IP variable for remote server configuration
- Implement setup_multi_server_tls() function for shared CA management
- Change TLS failure handling from exit-on-error to warning-based approach
- Add TLS_ISSUES_SUMMARY for end-of-deployment remediation guidance
- Update documentation for automated TLS setup process
- Add comprehensive AI assistant instructions for project management

This allows deployments to complete successfully even with TLS issues,
providing clear remediation steps instead of failing the entire setup.
2026-01-26 16:34:25 -05:00
Kelin
3076232e8f Fix permission errors in perform_deployment() function
- Use sudo for creating /opt directories during deployment
- Use sudo for copying files to /opt/dockge and /opt/stacks
- Ensure proper ownership of deployment directories
- Fix mkdir command for dashboards directory
2026-01-26 00:08:41 -05:00
Kelin
52e0697311 Fix share_certs_with_core() to handle missing shared CA gracefully
- Check if shared CA exists on core server before attempting to copy
- Generate local shared CA if core server doesn't have certificates
- Provide clear instructions for manual certificate synchronization
- Remove script failure when certificates can't be copied
- Allow infrastructure deployment to continue with local CA generation
2026-01-26 00:05:26 -05:00
Kelin
89ca29918b Implement shared CA certificate system for multi-server TLS security
- Generate shared CA during core deployment for consistent trust across servers
- Modify setup_docker_tls() to use shared CA instead of per-server CAs
- Update share_certs_with_core() to copy shared CA from core server
- Re-enable TLS verification (DOCKER_TLS_VERIFY=1) in Sablier
- Fix Sablier certificate mounting for proper TLS connection
- Add docker-tls/ to .gitignore to prevent certificate leaks
- Update documentation for shared CA approach
2026-01-25 23:08:01 -05:00
kelinfoxy
08b184aea7 Standardize Compose Files 2026-01-24 23:11:05 -05:00
kelinfoxy
a59862c988 Documentation updates 2026-01-24 21:40:51 -05:00
EZ-Homelab
9e5c09bd2a fix: Ensure ez-homelab.sh copies config directories for all stacks
- Add config directory copying to setup_stacks_for_dockge() function
- Add config directory copying to infrastructure deployment
- Fixes monitoring stack (prometheus/loki/promtail) config file issues
- Ensures all service configs are properly deployed

All stacks now have their configuration files copied during setup.
2026-01-24 21:16:30 -05:00