286 Commits

Author SHA1 Message Date
Kelin
9454b3862e Configure Traefik for wildcard SSL certificates
- Add DNS resolvers (1.1.1.1, 8.8.8.8) to traefik.yml for faster DNS challenge
- Create wildcard-cert.yml to request *.kelinreij.duckdns.org certificate
- Add comprehensive setup documentation for wildcard certificates
- Update to use production Let's Encrypt email

This configuration will automatically request a wildcard certificate
covering all subdomains once the Let's Encrypt rate limit resets
(Feb 13, 2026 at 21:33 UTC), eliminating the need for individual
per-service certificates.
2026-02-12 17:55:57 -05:00
kelin
82b34fd2b7 Add script improvements documentation and update Arcane compose
- Add Script improvements.md documenting refactoring changes
- Update Arcane docker-compose.yml configuration
v0.2.1
2026-02-11 21:10:38 -05:00
kelin
ae4c0ef20c Refactor: Major script reorganization and improvements
- Add 12 section headers for better code navigation
- Create 7 common helper functions (backup, directories, networks, etc.)
- Break down monolithic functions: setup_ssh_key_to_core (140→31 lines), main (213→70 lines)
- Consolidate redundant backup operations using common_backup()
- Consolidate directory/network creation with common helpers
- Extract menu handling and completion display into separate functions
- Improve visual consistency with ║-bordered output formatting
- Suppress verbose output from SSH and deployment operations
- Add deployment-ready feedback to install-prerequisites.sh

Total functions increased from ~50 to 58 for better modularity
Script now 2,824 lines with clearer structure and reduced redundancy
2026-02-11 20:41:18 -05:00
kelin
9ccfb36923 Add ez-assistant and kerberos service folders 2026-02-11 14:56:03 -05:00
e4e8ae1b87 Further improvements to deployment output and status checking 2026-02-11 00:16:13 -05:00
9c758c7135 Clean up deployment output formatting
- Hide individual Docker network creation messages (only show 'Creating Docker networks...')
- Remove excessive blank lines between deployment steps
- Maintain clean, concise output with 'Deploying X...' and 'Success' messages
2026-02-10 21:19:39 -05:00
f227772275 Show deployment messages in normal output, not verbose mode
- Changed deployment status messages from log_info to echo for always-visible output
- Updated network creation messages to use echo instead of log_success/log_info
- Clean deployment output now shows 'Deploying X Stack...' and 'Success' always
- Network creation shows 'Creating Docker networks...' and status messages always
2026-02-10 21:16:45 -05:00
299d008f82 Clean up Docker Compose deployment output
- Add --quiet flag to run_cmd for silent Docker deployments
- Suppress verbose Docker Compose output (progress bars, warnings)
- Show clean 'Deploying X Stack' and 'Success' messages instead
- Updated all deployment functions: core, infrastructure, dashboards, arcane, dockge, sablier, traefik
2026-02-10 21:11:19 -05:00
f9aa9d4f59 Update docker-compose files and ez-homelab script
- Modified arcane and core docker-compose.yml files
- Updated ez-homelab.sh script
2026-02-10 21:05:32 -05:00
9c41814089 Add /opt/arcane directory removal to reset script 2026-02-10 20:25:15 -05:00
5df66a3923 Refine .env.example files across all stacks
- Streamline variable organization and formatting
- Reduce redundant comments and improve clarity
- Maintain all required variables for each stack
2026-02-10 20:23:52 -05:00
4067ba9c38 Fix nested variable expansion in process_stack_env
- Add eval expansion of  references when populating stack .env files
- Resolves PROJECTS_DIR=${STACKS_DIR} to actual path value
- Ensures Docker Compose receives fully resolved paths, not variable references
2026-02-10 19:26:23 -05:00
f318170d02 Standardize .env.example files with complete variables and empty values
Changes:
- Reviewed all docker-compose files to identify required variables
- Updated all .env.example files to include ONLY variables used in their respective compose files
- Changed all placeholder values to empty strings for cleaner configuration
- Fixed variable name inconsistency: DOWNLOADS_DIR -> DOWNLOAD_DIR (to match main .env)
- Removed unnecessary variables that aren't used by the compose files

All .env.example files now:
- Contain only the variables actually used in the stack
- Have empty values instead of example placeholders
- Match the variable names used in docker-compose files

This ensures clean, minimal .env files for each stack without any extraneous variables or confusing placeholder values.
2026-02-10 19:14:53 -05:00
96c7ff39a4 Add missing variables to .env.example files
- Added STACKS_DIR to dashboards/.env.example (needed for homepage volume mount)
- Added HOMEPAGE_VAR_TITLE to dashboards/.env.example
- Added CODE_SERVER_PASSWORD and CODE_SERVER_SUDO_PASSWORD to infrastructure/.env.example

These variables are required by the docker-compose files but were missing from the .env.example files, causing Docker Compose to fail with 'variable is not set' errors.
2026-02-10 19:01:57 -05:00
4350e47696 Fix validation to not fail on intentionally unreplaced variables
Changes:
- Docker-compose files: Skip validation entirely since we intentionally leave environment/volume variables as ${VAR}
- Config files: Changed from error+exit to warning only
- Prevents false positives where variables like ${PUID}, ${PGID}, ${TZ} in environment sections were flagged as errors

Now all missing variables will warn but not cause script to exit.
2026-02-10 18:53:48 -05:00
90d81c63ca Fix Python script file path passing in localize_yml_file
The Python script wasn't receiving the file path correctly because heredoc syntax doesn't support command-line arguments the way it was written. Changed to use environment variable COMPOSE_FILE_PATH instead.

This fixes the SyntaxError that occurred during deployment where Python was trying to parse YAML as Python code.
2026-02-10 18:50:01 -05:00
013bdb969d Improve .env file handling and variable replacement in deployment
Major improvements to environment variable management:

1. Added .env.example files for ALL stacks
   - Each stack now has its own .env.example with only required variables
   - Variables include comments explaining their purpose
   - Examples: core, dockge, infrastructure, dashboards, arcane, media, media-management, transcoders, monitoring, sablier, utilities, productivity, wikis, vpn, homeassistant, alternatives

2. Created .env.global generation
   - Generates /opt/stacks/.env.global with all variables
   - Strips comments and blank lines for clean output
   - Available to all stacks for reference

3. Improved variable replacement strategy
   - Variable replacement now ONLY targets labels and x-dockge sections in docker-compose files
   - Configuration files (traefik, authelia) still get full variable replacement
   - Uses Python script for precise section detection
   - Preserves environment variables and volume mounts as-is

4. New deployment approach
   - Each stack copies .env.example to .env
   - Values populated from main ~/EZ-Homelab/.env file
   - No more manual sed commands to remove unused variables
   - Consistent approach across all deployment functions

5. Updated deployment functions
   - deploy_dockge, deploy_core, deploy_infrastructure, deploy_dashboards, deploy_arcane, deploy_sablier_stack
   - All now use process_stack_env() for clean .env handling
   - All use updated localize_yml_file() for targeted variable replacement

Benefits:
- Clean, minimal .env files for each stack
- No unnecessary variables or comments in deployed .env files
- Variables in compose files preserved for Docker Compose to handle
- Easier to understand what each stack needs
- Uniform deployment approach across all stacks
2026-02-10 18:43:56 -05:00
363530f395 Update docker-compose files to use configurable environment variables
- Replace hardcoded paths with variables: STACKS_DIR, PROJECTS_DIR, MEDIA_DIR, DOWNLOAD_DIR
- Update .env.example with new variable definitions
- Remove unused .template files
- Enable configurable directory paths for stacks, media, and downloads
2026-02-10 17:55:47 -05:00
5fcd10895a Fix AUTHELIA_ADMIN_PASSWORD placeholder handling and saving
- Add placeholder check for AUTHELIA_ADMIN_PASSWORD (like other secrets)
- Save AUTHELIA_ADMIN_PASSWORD to .env file after generation
- Now properly replaces 'generate-with-openssl-rand-hex-64' with DEFAULT_PASSWORD
- Ensures password is saved to both commented and uncommented versions in .env
2026-02-10 14:28:20 -05:00
c5e9d959bd Fix backup file creation - only backup when overwriting existing files
- Removed backup logic from localize_yml_file function
- Added backup checks before file copy in all deploy functions
- Backups now only created when docker-compose.yml already exists
- Prevents unnecessary .backup files on first deployment
2026-02-10 14:19:24 -05:00
b27b8d380d Make Authelia secret generation consistent with Arcane
- Add placeholder string checks to AUTHELIA_JWT_SECRET, AUTHELIA_SESSION_SECRET, and AUTHELIA_STORAGE_ENCRYPTION_KEY
- Now generates secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Ensures consistent behavior across all auto-generated secrets
2026-02-10 14:13:17 -05:00
e78063d8cb Fix Arcane secret generation from placeholder values
- Check if ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET contain placeholder string
- Generate new secrets even when placeholder 'generate-with-openssl-rand-hex-64' is loaded from .env
- Previously secrets were not generated because .env file loaded placeholder as non-empty value
2026-02-10 14:06:29 -05:00
ac7b095d78 Fix Arcane secret generation and deployment
- Add commented/uncommented sed handling for ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET
- Use sudo for file copying in deploy_arcane to ensure proper permissions
- Ensures secrets are properly saved to .env file in repo folder
2026-02-10 13:53:35 -05:00
4fcda86495 Add Arcane stack with auto-deployment and secret generation
- Added ARCANE_ENCRYPTION_KEY and ARCANE_JWT_SECRET to .env.example
- Created deploy_arcane() function in ez-homelab.sh
- Auto-generate Arcane secrets after Authelia secrets
- Deploy Arcane in both Option 2 (Core Server) and Option 3 (Additional Server)
- Added Arcane docker-compose.yml configuration
2026-02-10 13:15:14 -05:00
bd54a895ac Add Arcane Configuration Guide 2026-02-10 13:01:49 -05:00
Kelin
803165a1c8 Remove homelab-network from pihole service 2026-02-09 22:49:47 -05:00
Kelin
377ba6dc20 Add Arcane stack and clean up deprecated files
- Add new Arcane application stack
- Move aliases.sh to scripts/ directory
- Remove deprecated files (IMPLEMENTATION_COMPLETE.md, markup.yml, release notes)
- Remove standalone traefik docker-compose (now part of core stack)
- Update documentation (ai-vscode-setup.md, docker-guidelines.md, core README)
2026-02-09 22:22:40 -05:00
Kelin
33336c75ff Fix step 9 registration failure - correct file verification
The registration function creates '-server-routes.yml'
but the verification was checking for 'docker-provider-.yml'.

Changes:
- Updated file verification to check for correct filenames
- Added DOMAIN variable loading in add_remote_server_to_traefik()
- Replaced hard-coded domain with ${DOMAIN} variable in route rules
- Routes now use dynamic domain from .env instead of hard-coded value

This fixes the 'docker-provider file not created' error in step 9.
2026-02-08 20:17:30 -05:00
Kelin
86619db71e Improve error handling for SSH setup failures in option 3
Instead of exiting immediately when SSH key setup fails, the script now:
- Offers retry option for transient issues
- Allows skipping SSH setup with manual configuration instructions
- Provides option to return to main menu
- Continues deployment flow more gracefully

This prevents frustrating exits when SSH configuration needs adjustment.
2026-02-08 20:01:05 -05:00
Kelin
824a415f01 Fix: Call prepare_deployment() to handle option 1 (install prerequisites)
The prepare_deployment() function was defined but never called, causing
option 1 (Install Prerequisites) to not execute. Added the function call
after the menu selection loop to properly handle special deployment modes.
2026-02-08 19:33:56 -05:00
Kelin
30143d5f75 Update documentation files with recent tweaks 2026-02-08 18:42:16 -05:00
Kelin
0de82e55b6 Refactor proxying-external-hosts.md: Remove Method 2 (Docker labels) completely
Changes:
- Removed entire Method 2 section (Docker labels with dummy containers)
- Simplified to focus only on Method 1 (File Provider) as recommended approach
- Updated Quick Start section with cleaner step-by-step instructions
- Removed resource consumption warnings (no longer relevant)
- Updated AI Management section to remove Docker labels references
- Maintained all advanced configuration examples (WebSocket, HTTPS backend, IP whitelist)
- Kept all troubleshooting, security best practices, and complete example

The guide now focuses exclusively on the recommended YAML file approach,
which is simpler, more reliable, and doesn't consume unnecessary resources.
2026-02-08 15:59:44 -05:00
Kelin
7c8f128eaf Update architecture diagram to show public internet routing and HTTP vs HTTPS traffic
Changes:
- Added public internet layer showing HTTPS traffic flow
- Clarified that internal network uses HTTP (no SSL/TLS)
- Added traffic flow summary section
- Updated 'How It Works' section to explicitly mention HTTPS externally, HTTP internally
- Added 'Key Points' section explaining no double encryption

The diagram now clearly shows:
1. 🌐 Public Internet → HTTPS (ports 80/443)
2. Core Server (SSL termination)
3. Internal Network → HTTP (no encryption needed)
4. Additional Servers (direct port access)
2026-02-08 15:19:48 -05:00
Kelin
63b2ae8fe0 Update multi-server deployment docs for simplified architecture
Changes:
- Remove references to local Traefik on additional servers
- Update architecture diagram to show direct port exposure
- Clarify that additional servers are 'headless' (no local reverse proxy)
- Update traffic flow to show direct routing from core to services
- Update performance metrics (50MB vs 100MB, 2min vs 5-10min deployment)
- Rename 'Remote Server' to 'Additional Server' for consistency

The docs now accurately reflect the current simplified architecture where
additional servers only run Sablier and expose ports directly.
2026-02-08 15:13:44 -05:00
Kelin
9e71d202cb Move pihole from infrastructure to core stack
Changes:
- docker-compose/core/docker-compose.yml: Added pihole service with full Traefik configuration
- docker-compose/infrastructure/docker-compose.yml: Removed pihole service
- docker-compose/dockge/docker-compose.yml.template: Deleted (no longer needed)

Pihole is now part of core infrastructure alongside Traefik, Authelia, and DuckDNS.
This ensures DNS services are always available on the core server.
2026-02-08 15:11:52 -05:00
Kelin
043e1cb64a Fix Option 3: Create traefik-network and improve sed patterns
Issue: Option 3 deployment failed because traefik-network wasn't created,
but the cleanup function didn't properly strip network references from files.

Changes:
- scripts/ez-homelab.sh: Add traefik-network creation in Step 2
  (Network is harmless if unused - prevents docker compose errors)
- Improve sed patterns in configure_remote_server_routing():
  * Use anchored patterns (^ and $) to match exact lines
  * Fix network removal regex to match indentation correctly

The traefik-network will exist but remain unused on additional servers.
Services are accessed via core Traefik's manual HTTP routes.
2026-02-07 22:22:52 -05:00
Kelin
16f7eaa703 Fix: Restore Traefik labels for Option 2 core deployments
The previous commit removed labels from templates, breaking Option 2 (Core Server)
deployments. This commit restores the proper architecture:

Templates (docker-compose files):
- Restore all Traefik labels to dockge/docker-compose.yml
- Restore all Traefik labels to infrastructure services (dozzle, glances, code-server)
- Restore traefik-network references
- Templates now work correctly for Option 2 (Core Server)

Scripts (ez-homelab.sh):
- Rewrite configure_remote_server_routing() to strip labels for Option 3
- Re-add configure_remote_server_routing() call to deploy_remote_server()
- Add as Step 5 (after copying stacks, before deploying them)
- Update remaining step numbers (6-9)

Architecture flow:
Option 2 (Core Server):
  - Uses templates as-is with Traefik labels
  - Local Traefik discovers services via Docker labels
  - Services accessible at https://service.domain

Option 3 (Additional Server):
  - Copies templates with labels (Step 4)
  - Strips out labels and traefik-network (Step 5)
  - Deploys labelless services with exposed ports (Steps 6-8)
  - Core Traefik routes via manual HTTP configs to IP:PORT
  - Services accessible at https://service.hostname.domain
2026-02-07 21:59:21 -05:00
Kelin
ce3fbdb244 Simplify Option 3: Remove local Traefik from additional servers
Major architectural simplification for headless additional servers:

Templates:
- Remove Traefik labels from dockge/docker-compose.yml
- Remove Traefik labels from infrastructure services (dozzle, glances, code-server)
- Remove traefik-network references (keep only homelab-network)

Scripts (ez-homelab.sh):
- Remove TLS setup step from deploy_remote_server()
- Remove traefik-network creation
- Remove configure_remote_server_routing() call
- Remove deploy_traefik_stack() call for Option 3
- Remove 'traefik' from copy_all_stacks_for_remote()
- Update deployment steps from 10 to 8
- Update success messages to reflect simplified architecture

Scripts (common.sh):
- Remove unused generate_traefik_provider_config() function

Config:
- Add ADMIN_SSH_PUB_KEY field to .env.example

Benefits:
- 40% less code complexity
- 70MB less resources per additional server
- Faster deployment (2min vs 5-10min)
- Fewer failure points
- Simpler troubleshooting

Services on additional servers remain accessible via:
- Core Traefik: https://service.hostname.domain
- Direct IP: http://IP:PORT
2026-02-07 21:34:20 -05:00
Kelin
72d3d8b38f Fix remote server Traefik configuration
- Add ports 80/443 to remote Traefik template
- Enable API access (insecure=true) for debugging
- Update configure_remote_server_routing to use HTTP-only config
  - Change entrypoint from websecure to web
  - Remove TLS cert resolver references
  - Remove authelia middleware references
- Replace Docker TLS provider with manual HTTP routes
  - Core Traefik now uses file provider with direct URLs
  - Fixes 404/gateway timeout errors on remote services
  - Routes HTTPS from core to HTTP on remote servers
- Fix port in routes (dozzle uses 8085, not 8082)

Resolves issues with remote server service accessibility.
2026-02-07 20:43:34 -05:00
Kelin
63a837f8e9 feat: Implement server-specific subdomains for infrastructure services
- Rename disable_traefik_on_remote_services() to configure_remote_server_routing()
- Enable Traefik on dockge, dozzle, glances with server-specific subdomains
- Keep sablier disabled (no web UI)
- Add Traefik dashboard route with server-specific subdomain
- Update sed patterns to dynamically replace server names in routing rules

This allows infrastructure/monitoring services to coexist on multiple
servers without conflicts, while shared services remain centralized.
2026-02-07 17:05:00 -05:00
Kelin
802a4d1ba0 Fix multiserver deployment: domain in docker-provider and auto-disable remote Traefik
- Fix missing domain in docker provider defaultRule (use DOMAIN or DUCKDNS_DOMAIN)
- Add disable_traefik_on_remote_services() to strip routing labels from remote services
- Call disable function automatically during remote server deployment (Step 5.5)
- Remote services now properly accessed through core Traefik via docker provider
- Prevents errors: middleware/certresolver not found on remote Traefik

Resolves Traefik errors on remote servers where services had labels for
SSL/auth that only exist on core server.
2026-02-07 16:24:57 -05:00
Kelin
9fea40c8b5 Fix multiserver deployment: Add SSH config automation, enhance registration verification, improve Traefik deployment
- Auto-create SSH config entry for passwordless core server access
- Add pre-flight SSH connectivity check before registration
- Verify docker-provider and sablier-middleware files are created on core
- Display explicit success/failure messages with troubleshooting steps
- Create placeholder routes.yml for Traefik dynamic config
- Verify Traefik container starts successfully after deployment
- Add container status check after Traefik deployment

Fixes issues where remote server registration silently failed and
Traefik deployment did not verify configuration files existed.
2026-02-07 16:09:56 -05:00
kelin
44b529a7cb feat: Add Option 3 - Deploy Additional Server with multi-server support
Major features:
- Automated SSH key setup between remote and core servers
- Docker TLS configuration with shared CA certificates
- Automatic deployment of Dockge, Traefik, Sablier, and Infrastructure stacks
- Copy all stacks (except core) to remote server for on-demand deployment
- New standalone Traefik stack for remote server container discovery
- Locale-aware SSH/SCP commands to handle Raspberry Pi warnings
- Variable expansion support in .env files (${VAR} references)
- Comprehensive error handling and verbose deployment logging

Technical improvements:
- setup_ssh_key_to_core() - Automated RSA 4096-bit key generation and installation
- setup_multi_server_tls() - Fetch shared CA from core server via SSH
- copy_all_stacks_for_remote() - Deploy all stacks except core
- deploy_traefik_stack() - Local Traefik for container discovery
- Enhanced localization with envsubst support
- Docker network creation (traefik-network, homelab-network)
- Password authentication with special character handling

Fixes:
- Fixed SSH key path handling for non-root users
- Fixed SCP exit code checking (was checking grep instead of scp)
- Fixed CA file detection with proper test commands
- Removed unnecessary prepare_deployment() function call
- Added ACTUAL_USER variable initialization for remote deployments
v0.2.0
2026-02-06 22:00:25 -05:00
kelinfoxy
5b3c4a2c5b Chage homarr image & add SECRET_ENCRYPTION_KEY 2026-02-06 17:17:17 -05:00
kelinfoxy
5cbb106160 Add multi-server support and update docs
Introduce multi-server architecture documentation and reorganize README content. Top-level README now documents Core vs Remote server roles, links to local docs instead of wiki pages, and highlights Traefik/Sablier multi-server behavior. docker-compose/README.md was rewritten to be a template-style reference with single- and multi-server deployment guidance, Traefik label examples, and sablier usage; dockge README was moved into docker-compose/dockge/. docker-compose/core/README.md was updated to describe core responsibilities, shared CA artifacts, and startup order for multi-server deployments. Several obsolete/duplicated docs and action reports were removed and a new multi-server deployment doc was added to centralize on-demand/remote service guidance. Overall this cleans up legacy docs and documents the multi-server workflow and TLS/shared-CA requirements.
2026-02-05 22:30:52 -05:00
Kelin
84b2cabacc Merge pull request #2 from kelinfoxy/copilot/scan-and-replace-personal-data
Sanitize personal data from repository history
2026-02-05 14:09:23 -05:00
copilot-swe-agent[bot]
0052fa4ddc Complete personal data replacement in Traefik configs
- Replace remaining domain references in Traefik dynamic configs
- Replace kelinreij.duckdns.org with yourdomain.duckdns.org in:
  - external-host-homeassistant.yml
  - local-host-production.yml
  - sablier.yml
  - markup.yml service URLs

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
2026-02-05 18:41:42 +00:00
copilot-swe-agent[bot]
300d870a2b Replace personal data with variables and placeholders
- Replace hardcoded password in code-server config with ${CODE_SERVER_PASSWORD}
- Replace domain kelin-hass.duckdns.org with yourdomain.duckdns.org in docs
- Replace domain kelinreij.duckdns.org with yourdomain.duckdns.org in homepage config
- Replace personal emails with example addresses
- Replace DuckDNS token and credentials in markup.yml with placeholders
- Replace Let's Encrypt account numbers with placeholders

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
2026-02-05 18:40:44 +00:00
copilot-swe-agent[bot]
94b4906a6a Initial plan 2026-02-05 18:35:18 +00:00
kelin
10aee3b3b1 Fix: Correct Dozzle port and remove unnecessary template files
- Changed Dozzle service port from 8085 to 8080 (correct port)
- Removed authelia config template files (no longer needed)
- Removed db.sqlite3 runtime data file
2026-02-05 13:13:44 -05:00