Add multi-server support and update docs

Introduce multi-server architecture documentation and reorganize README content. Top-level README now documents Core vs Remote server roles, links to local docs instead of wiki pages, and highlights Traefik/Sablier multi-server behavior. docker-compose/README.md was rewritten to be a template-style reference with single- and multi-server deployment guidance, Traefik label examples, and sablier usage; dockge README was moved into docker-compose/dockge/. docker-compose/core/README.md was updated to describe core responsibilities, shared CA artifacts, and startup order for multi-server deployments. Several obsolete/duplicated docs and action reports were removed and a new multi-server deployment doc was added to centralize on-demand/remote service guidance. Overall this cleans up legacy docs and documents the multi-server workflow and TLS/shared-CA requirements.

README.md

@@ -31,6 +31,12 @@ cd EZ-Homelab
 ./scripts/ez-homelab.sh
 ```
 
+**Multi-Server Support:**
+- **Core Server**: Full deployment with ports 80/443 forwarded from router
+- **Remote Servers**: Infrastructure-only setup (option 3 in script)
+- Each server runs its own Traefik and Sablier for local container management
+- Core server Traefik routes to all servers via Docker TLS providers
+
 **What the script does:**
 - Installs Docker and required system packages
 - Guides you through configuration (domain, admin credentials, etc.)
@@ -44,41 +50,56 @@ cd EZ-Homelab
 
 ## 📚 Documentation
 
-For comprehensive documentation, see the [GitHub Wiki](https://github.com/kelinfoxy/EZ-Homelab/wiki):
+- **[Getting Started Guide](docs/getting-started.md)** - Step-by-step deployment and configuration
-
+- **[Automated Setup](docs/automated-setup.md)** - Guided installation with ez-homelab.sh script
-- **[Getting Started Guide](https://github.com/kelinfoxy/EZ-Homelab/wiki/Getting-Started-Guide)** - Step-by-step deployment and configuration
+- **[Manual Setup](docs/manual-setup.md)** - Step-by-step manual installation
-- **[Docker Guidelines](https://github.com/kelinfoxy/EZ-Homelab/wiki/Docker-Guidelines)** - Service management patterns and best practices
+- **[Docker Guidelines](docs/docker-guidelines.md)** - Service management patterns and best practices
-- **[Quick Reference](https://github.com/kelinfoxy/EZ-Homelab/wiki/Quick-Reference)** - Command cheat sheet and troubleshooting
+- **[Services Reference](docs/services-overview.md)** - All 50+ available services
-- **[Services Reference](https://github.com/kelinfoxy/EZ-Homelab/wiki/Services-Overview)** - All 70+ available services
+- **[Quick Reference](docs/quick-reference.md)** - Command cheat sheet and troubleshooting
-- **[Proxying External Hosts](https://github.com/kelinfoxy/EZ-Homelab/wiki/Proxying-External-Hosts)** - Connect non-Docker services (Raspberry Pi, NAS, etc.)
+- **[Proxying External Hosts](docs/proxying-external-hosts.md)** - Connect non-Docker services (Raspberry Pi, NAS, etc.)
+- **[Multi-Server Setup](docs/Ondemand-Remote-Services.md)** - Deploy services across multiple servers
 
 ## 🚀 Quick Navigation
 
-**New to EZ-Homelab?** → [Getting Started Guide](https://github.com/kelinfoxy/EZ-Homelab/wiki/Getting-Started-Guide)
+**New to EZ-Homelab?** → [Getting Started Guide](docs/getting-started.md)
 
-**Need Help Deploying?** → [Automated Setup](https://github.com/kelinfoxy/EZ-Homelab/wiki/Getting-Started-Guide#automated-setup)
+**Need Help Deploying?** → [Automated Setup](docs/automated-setup.md)
 
-**Want to Add Services?** → [Service Creation Guide](https://github.com/kelinfoxy/EZ-Homelab/wiki/Docker-Guidelines#service-creation-guidelines)
+**Want to Add Services?** → [Service Creation Guide](docs/docker-guidelines.md)
 
-**Having Issues?** → [Troubleshooting](https://github.com/kelinfoxy/EZ-Homelab/wiki/Quick-Reference#troubleshooting)
+**Having Issues?** → [Troubleshooting](docs/quick-reference.md)
 
-**Managing Services?** → [Dockge Dashboard](https://dockge.yourdomain.duckdns.org)
+**Multi-Server Setup?** → [Remote Services Guide](docs/Ondemand-Remote-Services.md)
+
+**Managing Services?** → Dockge Dashboard at `https://dockge.yourdomain.duckdns.org`
 
 ### Service Documentation
-Individual service documentation is available in the [GitHub Wiki](https://github.com/kelinfoxy/EZ-Homelab/wiki):
+Individual service documentation is available in [docs/service-docs/](docs/service-docs/):
-- [Authelia](https://github.com/kelinfoxy/EZ-Homelab/wiki/Authelia) - SSO authentication
+- [Authelia](docs/service-docs/authelia.md) - SSO authentication
-- [Traefik](https://github.com/kelinfoxy/EZ-Homelab/wiki/Traefik) - Reverse proxy and SSL
+- [Traefik](docs/service-docs/traefik.md) - Reverse proxy and SSL
-- [Dockge](https://github.com/kelinfoxy/EZ-Homelab/wiki/Dockge) - Stack management
+- [Sablier](docs/service-docs/sablier.md) - Lazy loading for on-demand containers
-- [Homepage](https://github.com/kelinfoxy/EZ-Homelab/wiki/Homepage) - Service dashboard
+- [DuckDNS](docs/service-docs/duckdns.md) - Dynamic DNS
-- And 50+ more services...
+- [Dockge](docs/service-docs/dockge.md) - Stack management
+- [Homepage](docs/service-docs/homepage.md) - Service dashboard
+- And 50+ more services in the docs/service-docs/ folder
 
 ## 🏗️ Architecture
 
-### Core Infrastructure
+### Core Infrastructure (Deploy on Main Server)
-- **Traefik** - Reverse proxy with automatic HTTPS termination
-- **Authelia** - Single sign-on (SSO) authentication
 - **DuckDNS** - Dynamic DNS with wildcard SSL certificates
-- **Sablier** - Lazy loading service for on-demand containers
+- **Traefik** - Reverse proxy with automatic HTTPS termination and multi-server routing
+- **Authelia** - Single sign-on (SSO) authentication
+
+### Per-Server Infrastructure (Deploy on Each Server)
+- **Traefik** - Local reverse proxy instance for container discovery
+- **Sablier** - Lazy loading service for on-demand local container startup
+
+### Multi-Server Architecture
+- **Core Server**: Only server with ports 80/443 forwarded from router
+- **Remote Servers**: Connect to core via Docker TLS (port 2376)
+- **Unified Access**: All services accessible through core server's domain
+- **Automatic Routing**: Core Traefik discovers services on all servers
+- **Lazy Loading**: Each server's Sablier manages local containers only
 
 ### VPN Services
 - **Gluetun** - VPN client for secure downloads
@@ -94,11 +115,13 @@ Individual service documentation is available in the [GitHub Wiki](https://githu
 
 ### Key Features
 - **File-based configuration** - AI-manageable YAML files
+- **Multi-server support** - Scale across multiple machines with unified access
 - **Automated SSL** - Wildcard certificates via Let's Encrypt
+- **Automatic routing** - Traefik discovers services across all servers
 - **VPN routing** - Secure download clients through Gluetun
 - **Resource limits** - Prevent resource exhaustion
 - **SSO protection** - Authelia integration with bypass options
-- **Lazy loading** - Sablier enables on-demand container startup
+- **Lazy loading** - Per-server Sablier enables on-demand container startup
 - **Automated backups** - Restic + Backrest for comprehensive data protection
 
 ## 🤖 AI Management
@@ -121,8 +144,8 @@ This homelab is designed to be managed by AI agents through VS Code with GitHub
 ## 🔧 Manual Setup
 
 If automated scripts fail, see:
-- **[Manual Setup Guide](https://github.com/kelinfoxy/EZ-Homelab/wiki/Manual-Setup)** - Step-by-step manual installation
+- **[Manual Setup Guide](docs/manual-setup.md)** - Step-by-step manual installation
-- **[Troubleshooting](https://github.com/kelinfoxy/EZ-Homelab/wiki/Troubleshooting)** - Common issues and solutions
+- **[Troubleshooting](docs/quick-reference.md)** - Common issues and solutions
 
 ## 🤝 Contributing
 

docker-compose/README.md

@@ -1,152 +1,108 @@
 # Docker Compose Stacks
 
-This directory contains Docker Compose files for managing your homelab services. Each stack is organized in its own folder for better organization and maintainability.
+This directory contains Docker Compose templates for managing your homelab services. Each stack is organized in its own folder for better organization and maintainability.
 
 ## Structure
 
 ```
 docker-compose/
-├── core/              # Core infrastructure (Traefik, Authelia, DuckDNS)
+├── core/              # Core infrastructure (MUST DEPLOY FIRST)
-├── infrastructure/    # Additional infrastructure (Pi-hole, Dockge, etc.)
+│   ├── docker-compose.yml
+│   ├── authelia/      # SSO configuration
+│   ├── duckdns/       # DNS configuration
+│   └── traefik/       # Reverse proxy configuration
+│       └── dynamic/   # External routing YAML files (multi-server)
+├── sablier/           # Lazy loading service (per-server)
+├── dockge/            # Docker management web UI
+├── infrastructure/    # Additional infrastructure (Pi-hole, etc.)
 ├── dashboards/        # Dashboard services (Homepage, Homarr)
-├── vpn/               # VPN services (Gluetun, qBittorrent)
 ├── media/             # Media services (Plex, Jellyfin, etc.)
+├── media-management/  # *arr services (Sonarr, Radarr, etc.)
 ├── monitoring/        # Observability stack (Prometheus, Grafana, etc.)
-├── alternatives/      # Alternative services (Authentik, etc.)
 ├── homeassistant/     # Home Assistant stack
-├── nextcloud/         # Nextcloud stack
+├── productivity/      # Productivity tools (Nextcloud, Gitea, etc.)
-├── productivity/      # Productivity tools
+├── utilities/         # Utility services (Duplicati, FreshRSS, etc.)
-├── utilities/         # Utility services
+├── wikis/             # Mediawiki, Dokuwiki, Bookstacks
-## ⚠️ Important: Core Services First
+└── vpn/               # VPN services (Gluetun, qBittorrent)
-
-**Before deploying any other stacks, ensure the `core/` services are running:**
-
-- **Traefik**: Reverse proxy and SSL termination
-- **Authelia**: Single sign-on authentication
-- **DuckDNS**: Dynamic DNS for domain resolution
-
-These services provide the foundation for all other services. Most stacks depend on Traefik for routing and Authelia for authentication.
-
-### Quick Start Core Services
-```bash
-cd core
-cp .env.template .env  # Edit with your values
-cp docker-compose.yml.template docker-compose.yml  # Or use the pre-configured version
-docker compose up -d
 ```
 
-### Starting Services
+## Multi-Server Architecture
 
-Start all services in a stack:
+EZ-Homelab supports two deployment models:
+
+### **Single Server:**
+- Core + all other stacks on one machine
+- Simplest setup for beginners
+
+### **Multi-Server:**
+- **Core Server**: DuckDNS, Traefik (multi-provider), Authelia
+- **Remote Servers**: Traefik (local-only), Sablier (local-only), application services
+- All services accessed through unified domain
+
+See [docs/Ondemand-Remote-Services.md](../docs/Ondemand-Remote-Services.md) for multi-server setup.
+
+## Deployment
+
+Use the unified setup script:
 ```bash
-cd docker-compose/core && docker compose up -d
+cd ~/EZ-Homelab
+./scripts/ez-homelab.sh
 ```
 
-Start a specific service:
+## Single Server Traefik service labels
-```bash
-cd docker-compose/vpn && docker compose up -d gluetun
-```
 
-### Stopping Services
-
-Stop all services in a stack:
-```bash
-cd docker-compose/core && docker compose down
-```
-
-Stop a specific service:
-```bash
-cd docker-compose/vpn && docker compose stop qbittorrent
-```
-
-### Viewing Status
-
-Check running services:
-```bash
-docker compose -f docker-compose/media.yml ps
-```
-
-View logs:
-```bash
-docker compose -f docker-compose/media.yml logs -f plex
-```
-
-### Updating Services
-
-Pull latest images:
-```bash
-docker compose -f docker-compose/media.yml pull
-```
-
-Update a specific service:
-```bash
-docker compose -f docker-compose/media.yml pull plex
-docker compose -f docker-compose/media.yml up -d plex
-```
-
-## Networks
-
-All services connect to a shared bridge network called `homelab-network`. Create it once:
-
-```bash
-docker network create homelab-network
-```
-
-Some services may use additional networks for security isolation:
-- `monitoring-network` - For monitoring stack
-- `database-network` - For database isolation
-- `media-network` - For media services
-
-Create them as needed:
-```bash
-docker network create monitoring-network
-docker network create database-network
-docker network create media-network
-```
-
-## Environment Variables
-
-Create a `.env` file in the root of your homelab directory with common variables:
-
-```bash
-# .env
-PUID=1000
-PGID=1000
-TZ=America/New_York
-USERDIR=/home/username/homelab
-DATADIR=/mnt/data
-
-```
-
-Never commit `.env` files to git! Use `.env.example` as a template instead.
-
-## Labels
-
-### To enable Authelia SSO
-```yaml
-
-
-```
-
-### Traefik routing labels
-
-If Traekif is on the same server add these labels.
-```yaml
-
-```
-
->If Traefik is on a seperate server, don't use traekfik labels in compose files, use an external host yaml file.
-
-
-### Sablier middleware labels
-
-Add these labels to enable ondemand functionality.
 ```yaml
+services:
+  myservice:
     labels:
-      - sablier.enable=true
+      # TRAEFIK CONFIGURATION
-      - sablier.group=<server>-<service name>
+      # ==========================================
-      - sablier.start-on-demand=true
+      # Service metadata
-      
+      - "com.centurylinklabs.watchtower.enable=true"
+      - "homelab.category=category-name"
+      - "homelab.description=Brief service description"
+      # Traefik labels
+      - "traefik.enable=true"
+      # Router configuration
+      - "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
+      - "traefik.http.routers.myservice.entrypoints=websecure"
+      - "traefik.http.routers.myservice.tls.certresolver=letsencrypt"
+      - "traefik.http.routers.myservice.middlewares=authelia@docker"  # SSO (remove to disable)
+      # Service configuration
+      - "traefik.http.services.myservice.loadbalancer.server.port=8080"
+      # Sablier configuration (lazy loading)
+      - "sablier.enable=true"
+      - "sablier.group=${SERVER_HOSTNAME}-myservice"
+      - "sablier.start-on-demand=true"
+```
+
+## Multi-Server Traefik
+
+### On Core Server
+
+
+## On Remote Server
+
+
+### Disabling SSO (Media Servers)
+
+Remove or comment the authelia middleware line:
+```yaml
+# SSO enabled (default):
+- "traefik.http.routers.myservice.middlewares=authelia@docker"
+
+# SSO disabled (for Plex, Jellyfin, etc.):
+# - "traefik.http.routers.myservice.middlewares=authelia@docker"
+```
+
+### Disabling Lazy Loading (Always-On Services)
+
+Remove Sablier labels and use `restart: unless-stopped`:
+```yaml
+services:
+  myservice:
+    restart: unless-stopped  # Always running
+    # No sablier labels
 ```
 
 ## Best Practices
@@ -171,6 +127,7 @@ services:
     restart: unless-stopped
     networks:
       - homelab-network
+      - traefik-network
     ports:
       - "host_port:container_port"
     volumes:
@@ -197,77 +154,3 @@ networks:
   homelab-network:
     external: true
 ```
-
-## Troubleshooting
-
-### Service won't start
-1. Check logs: `docker compose -f file.yml logs service-name`
-2. Validate config: `docker compose -f file.yml config`
-3. Check for port conflicts: `sudo netstat -tlnp | grep PORT`
-4. Verify volumes exist and have correct permissions
-
-### Permission errors
-1. Ensure PUID and PGID match your user: `id -u` and `id -g`
-2. Fix directory ownership: `sudo chown -R 1000:1000 ./config/service-name`
-
-### Network issues
-1. Verify network exists: `docker network ls`
-2. Check service is connected: `docker network inspect homelab-network`
-3. Test connectivity: `docker compose exec service1 ping service2`
-
-## Migration from Docker Run
-
-If you have services running via `docker run`, migrate them to compose:
-
-1. Get current configuration:
-   ```bash
-   docker inspect container-name > container-config.json
-   ```
-
-2. Convert to compose format (extract image, ports, volumes, environment)
-
-3. Test the compose configuration
-
-4. Stop old container:
-   ```bash
-   docker stop container-name
-   docker rm container-name
-   ```
-
-5. Start with compose:
-   ```bash
-   docker compose -f file.yml up -d
-   ```
-
-## Backup Strategy
-
-Regular backups are essential:
-
-```bash
-# Backup compose files (already in git)
-git add docker-compose/*.yml
-git commit -m "Update compose configurations"
-
-# Backup volumes
-docker run --rm \
-  -v volume-name:/data \
-  -v $(pwd)/backups:/backup \
-  busybox tar czf /backup/volume-name-$(date +%Y%m%d).tar.gz /data
-
-# Backup config directories
-tar czf backups/config-$(date +%Y%m%d).tar.gz config/
-```
-
-## Getting Help
-
-- Check the [Docker Guidelines](../docs/docker-guidelines.md) for detailed documentation
-- Review the [GitHub Copilot Instructions](../.github/copilot-instructions.md) for AI assistance
-- Consult service-specific documentation in `config/service-name/README.md`
-
-## Examples
-
-See the example compose files in this directory:
-- `infrastructure.yml` - Essential services like reverse proxy
-- `media.yml` - Media server stack
-- `monitoring.yml` - Observability and monitoring
-- `development.yml` - Development environments and tools

docker-compose/core/README.md

@@ -1,27 +1,65 @@
 # Core Infrastructure Services
 
-This directory contains the core infrastructure services that form the foundation of the homelab. These services should always be running and are critical for the operation of other services.
+This directory contains the core infrastructure services that form the foundation of the homelab. These services should be deployed **on the main server only** and are critical for the operation of all other services across all servers.
 
 ## Services
 
+### DuckDNS
+- **Purpose**: Dynamic DNS service for domain resolution and wildcard SSL certificates
+- **Subdomain**: Configurable via environment variables
+- **Token**: Configured in environment variables
+- **SSL Certificates**: Generates wildcard cert used by all services on all servers
+- **Deploy**: Core server only
+
 ### Traefik (v3)
-- **Purpose**: Reverse proxy and SSL termination
+- **Purpose**: Reverse proxy and SSL termination with multi-server routing
 - **Ports**: 80 (HTTP), 443 (HTTPS), 8080 (Dashboard)
 - **Configuration**: Located in `traefik/config/traefik.yml`
-- **SSL**: Let's Encrypt with DNS-01 challenge (configurable provider)
+- **Multi-Server**: Discovers services on all servers via Docker providers
+- **SSL**: Let's Encrypt with DNS-01 challenge (wildcard certificate)
 - **Dashboard**: Available at configured domain
+- **Deploy**: Core server (multi-provider), Remote servers (local-only)
+
+**Note**: Sablier has been moved to its own stack (`/opt/stacks/sablier/`) and should be deployed on each server individually. See [Sablier documentation](../../docs/service-docs/sablier.md) for details.
 
 ### Authelia (v4.37.5)
-- **Purpose**: Single sign-on authentication service
+- **Purpose**: Single sign-on authentication service for all services across all servers
 - **Port**: 9091 (internal)
 - **Access**: Configured authentication domain
 - **Configuration**: Located in `authelia/config/`
 - **Database**: SQLite database in `authelia/config/db.sqlite3`
+- **Deploy**: Core server only
 
 ### DuckDNS
-- **Purpose**: Dynamic DNS service for domain resolution
+- **Purpose**: Dynamic DNS service for domain resolution and wildcard SSL certificates
 - **Subdomain**: Configurable via environment variables
 - **Token**: Configured in environment variables
+- **SSL Certificates**: Generates wildcard cert used by all services on all servers
+- **Deploy**: Core server only
+
+## Multi-Server Architecture
+
+The core stack on the main server provides centralized services for the entire homelab:
+
+**Core Server Responsibilities:**
+- Receives all external traffic (ports 80/443 forwarded from router)
+- Runs DuckDNS for domain management and SSL certificates
+- Runs Authelia for centralized authentication
+- Runs multi-provider Traefik that discovers services on all servers
+- Generates shared CA for Docker TLS communication
+
+**Remote Server Setup:**
+- Remote servers run their own Traefik instance (local Docker provider only)
+- Remote servers run their own Sablier instance (local container management)
+- Remote servers expose Docker API on port 2376 with TLS
+- Core server Traefik connects to remote Docker APIs to discover services
+- No port forwarding needed on remote servers
+
+**Service Access:**
+- All services accessible via: `https://service.yourdomain.duckdns.org`
+- Core Traefik routes to appropriate server (local or remote)
+- Single wildcard SSL certificate used for all services
+- Authelia provides SSO for all protected services
 
 ## ⚠️ Version Pinning & Breaking Changes
 
@@ -64,15 +102,22 @@ core/
 │       ├── configuration.yml   # Authelia main config
 │       ├── users_database.yml  # User credentials
 │       └── db.sqlite3          # SQLite database
-└── traefik/
+├── duckdns/
-    ├── config/
+│   └── config/                 # DuckDNS configuration
-    │   └── traefik.yml         # Traefik static config
+├── traefik/
-    ├── dynamic/                # Dynamic configurations
+│   ├── config/
-    │   ├── routes.yml
+│   │   └── traefik.yml         # Traefik static config
-    │   ├── sablier.yml
+│   ├── dynamic/                # Dynamic configurations
-    │   └── external-host-*.yml
+│   │   ├── routes.yml
-    └── letsencrypt/
+│   │   ├── sablier.yml
-        └── acme.json           # SSL certificates
+│   │   └── external-host-*.yml # Remote server routing
+│   └── letsencrypt/
+│       └── acme.json           # SSL certificates
+└── shared-ca/                  # TLS certificates for multi-server
+    ├── ca.pem                  # Certificate Authority
+    ├── ca-key.pem              # CA private key
+    ├── cert.pem                # Client certificate
+    └── key.pem                 # Client key
 ```
 
 ### Environment Variables (.env)
@@ -100,10 +145,12 @@ PGID=1000
 4. Domain configured in DuckDNS
 
 ### Startup Order
-1. `duckdns` - For DNS updates
+1. `duckdns` - For DNS updates and SSL certificate generation
-2. `traefik` - Reverse proxy
+2. `traefik` - Reverse proxy (waits for SSL certificates)
 3. `authelia` - Authentication service
 
+**Note**: Sablier is now deployed separately in `/opt/stacks/sablier/` after core stack is running.
+
 ### Commands
 ```bash
 # Start all services

docs/Homelab-Audit.md

@@ -1,20 +0,0 @@
-# EZ-Homelab Configuration Audit
-
-## Purpose
-
-Validate the configuration for the entire homelab.  
-
-For each server:
-
-* Folder structure
-* File permissions
-* List all stacks in /opt
-* List all compose files with relevant .env file
-
-For standalone server:
-
-* All of the above plus
-*
-
-
-

docs/Ondemand-Remote-Services.md

@@ -1,458 +0,0 @@
-# On Demand Remote Services with Authelia, Sablier & Traefik
-
-## Overview
-
-This guide explains how to set up lazy-loading services on remote servers (like Raspberry Pi) that start automatically when accessed via Traefik. The core server runs Sablier, which connects to remote Docker daemons via TLS to manage container lifecycle.
-
-## Prerequisites
-
-- Core server with Traefik, Authelia, and Sablier deployed
-- Remote server with Docker installed
-- Shared TLS CA configured between core and remote servers
-
-## Automated Setup
-
-For new remote servers, use the automated script:
-
-1. On the remote server, run `ez-homelab.sh` and select option 3 (Infrastructure Only)
-2. When prompted, enter the core server IP for shared TLS CA
-3. The script will automatically:
-   - Copy shared CA from core server via SSH
-   - Configure Docker TLS with shared certificates
-   - Generate server certificates signed by shared CA
-   - Set up Docker daemon for TLS on port 2376
-
-**Important**: The script will fail if it cannot copy the shared CA from the core server. Ensure SSH access is configured between servers before running option 3.
-
-## Manual Setup (if automated fails)
-
-If the automated setup fails, manually configure TLS:
-
-### On Core Server:
-```bash
-# Generate server certificates for remote server
-cd /opt/stacks/core/shared-ca
-openssl genrsa -out server-key.pem 4096
-openssl req -subj "/CN=<REMOTE_IP>" -new -key server-key.pem -out server.csr
-echo "subjectAltName = DNS:<REMOTE_IP>,IP:<REMOTE_IP>,IP:127.0.0.1" > extfile.cnf
-openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf
-```
-
-### On Remote Server:
-```bash
-# Copy certificates
-scp user@core-server:/opt/stacks/core/shared-ca/ca.pem /opt/stacks/core/shared-ca/
-scp user@core-server:/opt/stacks/core/shared-ca/server-cert.pem /opt/stacks/core/shared-ca/
-scp user@core-server:/opt/stacks/core/shared-ca/server-key.pem /opt/stacks/core/shared-ca/
-
-# Update Docker daemon
-sudo tee /etc/docker/daemon.json > /dev/null <<EOF
-{
-  "tls": true,
-  "tlsverify": true,
-  "tlscacert": "/opt/stacks/core/shared-ca/ca.pem",
-  "tlscert": "/opt/stacks/core/shared-ca/server-cert.pem",
-  "tlskey": "/opt/stacks/core/shared-ca/server-key.pem"
-}
-EOF
-
-sudo systemctl restart docker
-```
-
-## 4 Step Process for Adding Services
-
-1. Add route & service in Traefik external hosts file
-2. Add middleware in Sablier config file (sablier.yml)
-3. Add labels to compose files on Remote Host
-4. Restart services
-
-## Required Information
-```bash
-<server> - the hostname of the remote server
-
-<service> - the application/container name
-
-<full domain> - the base url for your proxy host (my-subdomain.duckdns.org)
-
-<ip address> - the ip address of the remote server
-
-<port> - the external port exposed by the service
-
-<service display name> - how it will appear on the now loading page
-
-<group name> - use <service name> for a single service, or something descriptive for the group of services that will start together.
-  
-```  
-## Step 1: Add route & service in Traefik external hosts file
-
-### In /opt/stacks/core/traefik/dynamic/external-host-server_name.yml  
-
-```yaml
-http:
-    routers:
-        # Add a section under routers for each Route (Proxy Host)
-        <service>-<server>:
-            rule: "Host(`<service>.<full domain>`)"
-            entryPoints:
-                - websecure
-            service: <service>-<server>
-            tls:
-                certResolver: letsencrypt
-            middlewares:
-                - sablier-<server>-<service>@file
-                - authelia@docker   # comment this line to disable SSO login
-
-        # next route goes here
-
-# All Routes go above this line
-# Services section defines each service used above
-services:
-    <service>-<server>:
-    loadBalancer:
-        servers:
-        - url: "http://<ip address>:<port>"
-        passHostHeader: true
-    
-    # next service goes here
-```
-## Step 2: Add middlware to sablier config file
-
-### In /opt/stacks/core/traefik/dynamic/sablier.yml  
-
-```yaml
-http:
-    middlwares:
-        # Add a section under middlewares for each Route (Proxy Host)
-        sablier-<server>-<service>:
-            plugin:
-                sablier:
-                sablierUrl: http://sablier-service:10000
-                group: <server>-<group name>
-                sessionDuration: 2m     # Increase this for convience
-                ignoreUserAgent: curl   # Don't wake the service for a curl command
-                dynamic:
-                    displayName: <service display name>
-                    theme: ghost        # This can be changed
-                    show-details-by-default: true   # May want to disable for production
-
-        # Next middleware goes here
-```
-## Step 3: Add labels to compose files on Remote Host
-
-## On the Remote Server 
-
-### Apply lables to the services in the compose files
-
-```yaml
-    labels:
-      - sablier.enable=true
-      - sablier.group=<server>-<group name>
-      - sablier.start-on-demand=true
-
-```
-
->**Note**:  
-Traefik & Authelia labels are not used in the compose file for Remote Hosts
-
-## Step 4: Restart services
-
-### On host system
-
-```bash
-docker restart traefik
-docker restart sablier-service
-```
-
-### On the Remote Host
-
-```bash
-cd /opt/stacks/<service>
-docker compose down && docker compose up -d
-docker stop <service>
-```
-
-## Setup Complete
-
-Access your service by the proxy url.
-
----
-
-# Deployment Plan for Multi-Server Setup
-
-This section provides a complete deployment plan for scenarios where the core infrastructure (Traefik, Authelia, Sablier) runs on one server, and application services run on remote servers. This setup enables centralized control and routing while maintaining service isolation.
-
-## Architecture Overview
-
-- **Core Server**: Hosts Traefik (reverse proxy), Authelia (SSO), Sablier (lazy loading controller)
-- **Remote/Media Servers**: Host application containers controlled by Sablier
-- **Communication**: TLS-secured Docker API calls between servers
-
-## Prerequisites
-
-- Both servers must be on the same network and able to communicate
-- SSH access configured between servers (key-based or password authentication supported)
-- Domain configured with DuckDNS or similar
-- The EZ-Homelab script handles Docker TLS certificate generation on the core server and automatic copying to remote servers
-- Basic understanding of Docker concepts (optional - script guides you through setup)
-
-## Step 1: Configure Core Server
-
-### On Core Server Only
-
-1. **Install Docker** (if not already installed):
-   ```bash
-   curl -fsSL https://get.docker.com | sh
-   usermod -aG docker $USER
-   systemctl enable docker
-   systemctl start docker
-   # Log out and back in for group changes
-   ```
-
-2. **Configure Firewall** (optional, for security):
-   ```bash
-   sudo ufw allow 2376/tcp  # For Docker API access from remote servers
-   sudo ufw --force enable
-   ```
-
-The EZ-Homelab script will automatically generate TLS certificates and configure Docker daemon TLS when you deploy the core infrastructure.
-
-## Step 2: Configure Remote Servers
-
-### On Each Remote Server
-
-1. **Install Docker** (if not already installed):
-   ```bash
-   curl -fsSL https://get.docker.com | sh
-   usermod -aG docker $USER
-   systemctl enable docker
-   systemctl start docker
-   # Log out and back in for group changes
-   ```
-
-2. **Configure Firewall** (optional, for security):
-   ```bash
-   sudo ufw allow 2376/tcp  # For Docker API access from core server
-   sudo ufw --force enable
-   ```
-
-The EZ-Homelab script will automatically copy TLS certificates from the core server and configure Docker daemon TLS when you run the infrastructure-only deployment.
-
-## Certificate and Secret Sharing
-
-The EZ-Homelab script automatically handles certificate and secret sharing for infrastructure-only deployments:
-
-### Automatic Process (Recommended)
-
-1. **On Remote Server**: Run `./scripts/ez-homelab.sh` and select option 3
-2. **Script Actions**:
-   - Prompts for core server IP
-   - Tests SSH connectivity
-   - Copies shared CA and TLS certificates from core server
-   - Generates server-specific certificates signed by the shared CA
-   - Configures Docker daemon for TLS on port 2376
-
-### Manual Process (Fallback)
-
-If automatic sharing fails, manually share certificates:
-
-1. **On Core Server**:
-   ```bash
-   # Generate server certificates for remote server
-   cd /opt/stacks/core/shared-ca
-   openssl genrsa -out server-key.pem 4096
-   openssl req -subj "/CN=<REMOTE_IP>" -new -key server-key.pem -out server.csr
-   echo "subjectAltName = DNS:<REMOTE_IP>,IP:<REMOTE_IP>,IP:127.0.0.1" > extfile.cnf
-   openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf
-   ```
-
-2. **On Remote Server**:
-   ```bash
-   # Copy certificates
-   scp user@core-server:/opt/stacks/core/shared-ca/ca.pem /opt/stacks/core/shared-ca/
-   scp user@core-server:/opt/stacks/core/shared-ca/server-cert.pem /opt/stacks/core/shared-ca/
-   scp user@core-server:/opt/stacks/core/shared-ca/server-key.pem /opt/stacks/core/shared-ca/
-
-   # Update Docker daemon
-   sudo tee /etc/docker/daemon.json > /dev/null <<EOF
-   {
-     "tls": true,
-     "tlsverify": true,
-     "tlscacert": "/opt/stacks/core/shared-ca/ca.pem",
-     "tlscert": "/opt/stacks/core/shared-ca/server-cert.pem",
-     "tlskey": "/opt/stacks/core/shared-ca/server-key.pem"
-   }
-   EOF
-
-   sudo systemctl restart docker
-   ```
-
-## Step 3: Deploy Core Infrastructure
-
-### On Core Server
-
-1. **Run the EZ-Homelab script** with core deployment:
-   ```bash
-   cd ~/EZ-Homelab
-   ./scripts/ez-homelab.sh
-   # Select option 1 (Default Setup) or 2 (Core Only)
-   ```
-
-   The script will:
-   - Generate Authelia secrets automatically
-   - Generate shared TLS CA and certificates
-   - Configure Docker daemon TLS
-   - Deploy Traefik, Authelia, and Sablier
-   - Set up certificates for secure communication
-
-2. **Verify core deployment**:
-   ```bash
-   # Check services are running
-   docker ps --filter "label=com.docker.compose.project=core"
-   
-   # Test Authelia access
-   curl -k https://auth.<your-domain>
-   ```
-
-## Step 4: Deploy Remote Infrastructure
-
-### On Remote/Media Server
-
-1. **Run the EZ-Homelab script** with infrastructure-only deployment:
-   ```bash
-   cd ~/EZ-Homelab
-   ./scripts/ez-homelab.sh
-   # Select option 3 (Infrastructure Only)
-   ```
-
-   The script will automatically:
-   - Prompt for core server IP address
-   - Establish SSH connection to core server
-   - Copy shared CA and generate server-specific certificates
-   - Configure Docker daemon for TLS on port 2376
-   - Set up required networks and directories
-
-2. **Manual certificate setup** (if automatic fails):
-   If SSH connection fails, follow the manual process in the Certificate and Secret Sharing section above.
-
-## Step 5: Configure Sablier for Remote Control
-
-### On Core Server
-
-Sablier uses middleware configuration in Traefik's dynamic files to control remote Docker daemons. The middleware specifies the remote server connection details.
-
-1. **Create Sablier middleware configuration**:
-   `/opt/stacks/core/traefik/dynamic/sablier.yml`
-   ```yaml
-   http:
-     middlewares:
-       sablier-<remote_hostname>-<group>:
-         plugin:
-           sablier:
-             sablierUrl: http://sablier-service:10000
-             group: <remote_hostname>-<group>
-             sessionDuration: 2m
-             ignoreUserAgent: curl
-             dynamic:
-               displayName: "<Service Group Display Name>"
-               theme: ghost
-               show-details-by-default: true
-   ```
-
-2. **Restart Traefik** to load the middleware:
-   ```bash
-   docker restart traefik
-   ```
-
-## Step 6: Deploy Application Services
-
-### On Remote Server
-
-1. **Deploy application stacks** with Sablier labels:
-   ```yaml
-   # Example: /opt/stacks/media-management/docker-compose.yml
-   services:
-     sonarr:
-       labels:
-         - sablier.enable=true
-         - sablier.group=<REMOTE_HOSTNAME>-media
-         - sablier.start-on-demand=true
-   ```
-
-2. **Deploy and stop services** for lazy loading:
-   ```bash
-   cd /opt/stacks/media-management
-   docker compose up -d
-   docker compose stop
-   ```
-
-## Step 7: Configure Traefik Routing
-
-### On Core Server
-
-Since Traefik cannot auto-discover labels from remote Docker hosts, use the file provider method:
-
-1. **Create external host configuration**:
-   `/opt/stacks/core/traefik/dynamic/external-host-<remote_server>.yml`
-   ```yaml
-   http:
-     routers:
-       sonarr-remote:
-         rule: "Host(`sonarr.<DOMAIN>`)"
-         entrypoints:
-           - websecure
-         service: sonarr-remote
-         tls:
-           certResolver: letsencrypt
-         middlewares:
-           - sablier-<remote_hostname>-media@file
-           - authelia@docker
-
-     services:
-       sonarr-remote:
-         loadBalancer:
-           servers:
-             - url: "http://<REMOTE_IP>:8989"
-           passHostHeader: true
-   ```
-
-2. **Restart Traefik**:
-   ```bash
-   docker restart traefik
-   ```
-
-## Step 8: Verification and Testing
-
-1. **Check Sablier connection**:
-   ```bash
-   # On core server
-   docker logs sablier-service
-   # Should show groups from remote server
-   ```
-
-2. **Test lazy loading**:
-   - Access `https://sonarr.<DOMAIN>`
-   - Should show Sablier loading page
-   - Container should start on remote server
-
-3. **Verify Traefik routes**:
-   ```bash
-   curl -k https://localhost:8080/api/http/routers | jq
-   ```
-
-## Troubleshooting
-
-- **TLS Connection Issues**: Check certificate validity and paths on both core and remote servers
-- **Sablier Not Detecting Groups**: Verify middleware configuration in `/opt/stacks/core/traefik/dynamic/sablier.yml` and that remote services have correct Sablier labels
-- **Traefik Routing Problems**: Check external host YAML syntax and that Traefik has reloaded configuration
-- **Network Connectivity**: Ensure ports 2376 (Docker API), 80, 443 are open between servers
-- **Certificate Sharing Failed**: Verify SSH access between servers and that the shared CA exists on core server
-
-## Security Considerations
-
-- TLS certificates expire after 365 days - monitor and renew
-- Limit Docker API access to trusted networks
-- Use strong firewall rules
-- Regularly update all components
-
-This setup provides centralized management with distributed execution, optimal for resource management and security.
-

docs/Traefik Routing Quick Reference.md

@@ -1,111 +0,0 @@
-# Traefik Routing Quick Reference
-
-## Variables (used throughout):
-
-```yaml
-  ${infrastructure}
-  ${description}
-  ${watchtower_enable}
-  ${service}
-  ${sso_enable}
-  ${sablier_enable}
-  ${traefik_enable}
-  ${port}
-```
-
-## Compose file labels section
-
-### Service metadata
-```yaml
-- homelab.category=${infrastructure}
-- homelab.description=${description}
-- com.centurylinklabs.watchtower.enable=${watchtower_enable}
-```
-
-### Traefik labels 
-
->**Traefik labels** are used for services on the same machine  
-They are ignored when the service is on a different machine
-
-
-```yaml
-- "traefik.enable=${traefik_enable}"
-- "traefik.http.routers.${service}.rule=Host(`${service}.${DOMAIN}`)"
-- "traefik.http.routers.${service}.entrypoints=websecure"
-- "traefik.http.routers.${service}.tls.certresolver=letsencrypt"
-- "traefik.http.routers.${service}.middlewares=authelia@docker"
-- "traefik.http.services.${service}.loadbalancer.server.port=${port}"
-```
-
-### Sablier lazy loading
-
-```yaml
-- sablier.enable=${sablier_enable}
-- sablier.group=${SERVER_HOSTNAME}-${service}
-- sablier.start-on-demand=true
-```
-
-## External Host Yml Files
-
->**Recomended**: use 1 yml file per host  
-
-### external-host-production.yml
-```yaml
-http:
-  # Routes for External Host Services
-  routers:
-    # External Service Routing Template
-    ${service}-${SERVER_HOSTNAME}:
-      rule: "Host(`${service}.${DOMAIN}`)"
-      entryPoints:
-        - websecure
-      service: ${service}-${SERVER_HOSTNAME}
-      tls:
-        certResolver: letsencrypt
-      middlewares:
-        - sablier-${SERVER_HOSTNAME}-${service}@file
-        - authelia@docker
-
-  # Middleware Definitions
-  middlewares:
-
-
-  # Service Definitions
-  services:
-    ${service}-${SERVER_HOSTNAME}:
-      loadBalancer:
-        servers:
-          - url: "http://${SERVER_IP}:${port}"
-        passHostHeader: true
-
-```
-
-## sablier.yml
-
-```yaml
-# Session duration set to 5m for testing. Increase to 30m for production.
-http:
-  middlewares:
-    # Authelia SSO middleware
-    authelia:
-      forwardauth:
-        address: http://authelia:9091/api/verify?rd=https://auth.${DOMAIN}/
-        authResponseHeaders:
-          - X-Secret
-        trustForwardHeader: true
-
-    # Sablier enabled Service Template
-    sablier-${SERVER_HOSTNAME}-${service}:
-      plugin:
-        sablier:
-          sablierUrl: http://sablier-service:10000
-          group: ${SERVER_HOSTNAME}-${service}
-          sessionDuration: 5m
-          ignoreUserAgent: curl
-          dynamic:
-            displayName: ${service}
-            theme: ghost
-            show-details-by-default: true
-
-
-```

docs/action-reports/2026-01-12-ssl-wildcard-certificate-setup.md

@@ -1,480 +0,0 @@
-# Action Report: SSL Wildcard Certificate Setup
-
-**Date:** January 12, 2026  
-**Status:** ✅ Completed Successfully  
-**Impact:** All homelab services now have valid Let's Encrypt SSL certificates
-
----
-
-## Problem Statement
-
-Services were showing "not secure" warnings in browsers despite Traefik being configured for Let's Encrypt certificates. Multiple simultaneous certificate requests were failing due to DNS challenge conflicts.
-
-## Root Causes Identified
-
-### 1. **Multiple Simultaneous Certificate Requests**
-- **Issue:** Each service (dockge, dozzle, glances, pihole, authelia) had `traefik.http.routers.*.tls.certresolver=letsencrypt` labels
-- **Impact:** Traefik attempted to request individual certificates for each subdomain simultaneously
-- **Consequence:** DuckDNS DNS challenge can only handle ONE TXT record at `_acme-challenge.yourdomain.duckdns.org` at a time
-- **Result:** All certificate requests failed with "Incorrect TXT record" errors
-
-### 2. **DNS TXT Record Conflicts**
-- **Issue:** Multiple services tried to create different TXT records at the same DNS location
-- **Example:** 
-  - Service A creates: `_acme-challenge.yourdomain.duckdns.org` = "token1"
-  - Service B overwrites: `_acme-challenge.yourdomain.duckdns.org` = "token2"
-  - Let's Encrypt validates Service A but finds "token2" → validation fails
-- **DuckDNS Limitation:** Can only maintain ONE TXT record per domain
-
-### 3. **Authelia Configuration Error**
-- **Issue:** Environment variable `AUTHELIA_NOTIFIER_SMTP_PASSWORD` was set without corresponding SMTP configuration
-- **Impact:** Authelia crashed on startup with "please ensure only one of the 'smtp' or 'filesystem' notifier is configured"
-- **Consequence:** Services requiring Authelia authentication were inaccessible
-
-### 4. **Stale DNS Records**
-- **Issue:** Old TXT records from failed attempts persisted in DNS
-- **Impact:** New certificate attempts validated against old, incorrect TXT records
-
-## Solution Implemented
-
-### Phase 1: Identify Certificate Request Pattern
-
-**Actions:**
-1. Discovered Traefik logs at `/var/log/traefik/traefik.log` (not stdout)
-2. Analyzed logs showing multiple simultaneous DNS-01 challenges
-3. Confirmed DuckDNS TXT record conflicts
-
-**Command Used:**
-```bash
-docker exec traefik tail -f /var/log/traefik/traefik.log
-```
-
-### Phase 2: Configure Wildcard Certificate
-
-**Actions:**
-1. Removed `certresolver` labels from all services except Traefik
-2. Configured wildcard certificate on Traefik router only
-3. Added DNS propagation skip for faster validation
-
-**Changes Made:**
-
-**File:** `/home/kelin/AI-Homelab/docker-compose/core.yml`
-```yaml
-# Traefik - Only service with certresolver
-traefik:
-  labels:
-    - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
-    - "traefik.http.routers.traefik.tls.domains[0].main=${DOMAIN}"
-    - "traefik.http.routers.traefik.tls.domains[0].sans=*.${DOMAIN}"
-
-# Authelia - No certresolver, just tls=true
-authelia:
-  labels:
-    - "traefik.http.routers.authelia.tls=true"
-```
-
-**File:** `/home/kelin/AI-Homelab/docker-compose/infrastructure.yml`
-```yaml
-# All infrastructure services - No certresolver
-dockge:
-  labels:
-    - "traefik.http.routers.dockge.tls=true"
-
-dozzle:
-  labels:
-    - "traefik.http.routers.dozzle.tls=true"
-
-glances:
-  labels:
-    - "traefik.http.routers.glances.tls=true"
-
-pihole:
-  labels:
-    - "traefik.http.routers.pihole.tls=true"
-```
-
-**File:** `/opt/stacks/core/traefik/traefik.yml`
-```yaml
-certificatesResolvers:
-  letsencrypt:
-    acme:
-      email: your-email@example.com
-      storage: /acme.json
-      dnsChallenge:
-        provider: duckdns
-        disablePropagationCheck: true  # Added to skip DNS propagation wait
-        resolvers:
-          - "1.1.1.1:53"
-          - "8.8.8.8:53"
-```
-
-### Phase 3: Clear DNS and Reset Certificates
-
-**Actions:**
-1. Stopped all services to clear DNS TXT records
-2. Reset `acme.json` to force fresh certificate request
-3. Waited 60 seconds for DNS to fully clear
-4. Restarted services with wildcard-only configuration
-
-**Commands Executed:**
-```bash
-# Stop services
-cd /opt/stacks/core && docker compose down
-
-# Reset certificate storage
-rm /opt/stacks/core/traefik/acme.json
-touch /opt/stacks/core/traefik/acme.json
-chmod 600 /opt/stacks/core/traefik/acme.json
-chown kelin:kelin /opt/stacks/core/traefik/acme.json
-
-# Wait for DNS to clear
-sleep 60
-dig +short TXT _acme-challenge.yourdomain.duckdns.org  # Verified empty
-
-# Deploy updated configuration
-cp /home/kelin/AI-Homelab/docker-compose/core.yml /opt/stacks/core/docker-compose.yml
-cd /opt/stacks/core && docker compose up -d
-```
-
-### Phase 4: Fix Authelia Configuration
-
-**Issue Found:** Environment variable triggering SMTP configuration check
-
-**File:** `/opt/stacks/core/docker-compose.yml`
-
-**Removed:**
-```yaml
-environment:
-  - AUTHELIA_NOTIFIER_SMTP_PASSWORD=${SMTP_PASSWORD}  # ❌ Removed
-```
-
-**Command:**
-```bash
-cd /opt/stacks/core && docker compose up -d authelia
-```
-
-### Phase 5: Fix Infrastructure Services
-
-**Issue:** Missing `networks:` header in compose file
-
-**File:** `/opt/stacks/infrastructure/infrastructure.yml`
-
-**Fixed:**
-```yaml
-# Before (incorrect):
-  traefik-network:
-    external: true
-
-# After (correct):
-networks:
-  traefik-network:
-    external: true
-  homelab-network:
-    driver: bridge
-  dockerproxy-network:
-    driver: bridge
-```
-
-**Command:**
-```bash
-cd /opt/stacks/infrastructure && docker compose -f infrastructure.yml up -d
-```
-
-## Results
-
-### Certificate Obtained Successfully ✅
-
-**acme.json Contents:**
-```json
-{
-  "letsencrypt": {
-    "Account": {
-      "Email": "your-email@example.com",
-      "Registration": {
-        "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/XXXXXXXXXX"
-      }
-    },
-    "Certificates": [
-      {
-        "domain": {
-          "main": "dockge.yourdomain.duckdns.org"
-        }
-      },
-      {
-        "domain": {
-          "main": "yourdomain.duckdns.org",
-          "sans": ["*.yourdomain.duckdns.org"]
-        }
-      }
-    ]
-  }
-}
-```
-
-**Certificate Details:**
-- **Subject:** CN=yourdomain.duckdns.org
-- **Issuer:** C=US, O=Let's Encrypt, CN=R12
-- **Coverage:** Wildcard certificate covering all subdomains
-- **File Size:** 23KB (up from 0 bytes)
-
-### Services Status
-
-All services running with valid SSL certificates:
-
-| Service | Status | URL | Certificate |
-|---------|--------|-----|-------------|
-| Traefik | ✅ Up | https://traefik.yourdomain.duckdns.org | Valid |
-| Authelia | ✅ Up | https://auth.yourdomain.duckdns.org | Valid |
-| Dockge | ✅ Up | https://dockge.yourdomain.duckdns.org | Valid |
-| Dozzle | ✅ Up | https://dozzle.yourdomain.duckdns.org | Valid |
-| Glances | ✅ Up | https://glances.yourdomain.duckdns.org | Valid |
-| Pi-hole | ✅ Up | https://pihole.yourdomain.duckdns.org | Valid |
-
-## Best Practices & Prevention
-
-### 1. ✅ Use Wildcard Certificates with DuckDNS
-
-**Rule:** Only ONE service should request certificates with DuckDNS DNS challenge
-
-**Configuration:**
-```yaml
-# ✅ CORRECT: Only Traefik requests wildcard cert
-traefik:
-  labels:
-    - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
-    - "traefik.http.routers.traefik.tls.domains[0].main=${DOMAIN}"
-    - "traefik.http.routers.traefik.tls.domains[0].sans=*.${DOMAIN}"
-
-# ✅ CORRECT: Other services just enable TLS
-other-service:
-  labels:
-    - "traefik.http.routers.service.tls=true"  # Uses wildcard automatically
-
-# ❌ WRONG: Multiple services requesting certs
-other-service:
-  labels:
-    - "traefik.http.routers.service.tls.certresolver=letsencrypt"  # DON'T DO THIS
-```
-
-### 2. ✅ DuckDNS DNS Challenge Limitations
-
-**Understand the Constraint:**
-- DuckDNS can only maintain ONE TXT record at `_acme-challenge.yourdomain.duckdns.org`
-- Multiple simultaneous challenges WILL fail
-- Use wildcard certificate to avoid this limitation
-
-**Alternative Providers (if needed):**
-- Cloudflare: Supports multiple simultaneous DNS challenges
-- Route53: Supports multiple TXT records
-- Use HTTP challenge if DNS challenge isn't required
-
-### 3. ✅ Traefik Logging Configuration
-
-**Enable File Logging for Debugging:**
-
-**File:** `/opt/stacks/core/traefik/traefik.yml`
-```yaml
-log:
-  level: DEBUG  # Use DEBUG for troubleshooting, INFO for production
-  filePath: /var/log/traefik/traefik.log  # Easier to tail than docker logs
-
-# Mount in docker-compose.yml:
-volumes:
-  - /var/log/traefik:/var/log/traefik
-```
-
-**Useful Commands:**
-```bash
-# Monitor certificate acquisition
-docker exec traefik tail -f /var/log/traefik/traefik.log | grep -E "acme|certificate|DNS"
-
-# Check for errors
-docker exec traefik tail -100 /var/log/traefik/traefik.log | grep -E "error|Unable"
-
-# View specific domain
-docker exec traefik tail -200 /var/log/traefik/traefik.log | grep "yourdomain.duckdns.org"
-```
-
-### 4. ✅ Certificate Troubleshooting Workflow
-
-**When certificates aren't working:**
-
-```bash
-# 1. Check acme.json status
-cat /opt/stacks/core/traefik/acme.json | python3 -m json.tool | grep -A5 "Certificates"
-
-# 2. Check certificate count
-python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')"
-
-# 3. Test certificate being served
-echo | openssl s_client -connect auth.yourdomain.duckdns.org:443 -servername auth.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer
-
-# 4. Check DNS TXT records
-dig +short TXT _acme-challenge.yourdomain.duckdns.org
-
-# 5. Check Traefik logs
-docker exec traefik tail -50 /var/log/traefik/traefik.log
-```
-
-### 5. ✅ Environment Variable Hygiene
-
-**Principle:** Only set environment variables that are actually used
-
-**Example - Authelia:**
-```yaml
-# ✅ CORRECT: Only variables for configured features
-environment:
-  - AUTHELIA_JWT_SECRET=${AUTHELIA_JWT_SECRET}
-  - AUTHELIA_SESSION_SECRET=${AUTHELIA_SESSION_SECRET}
-  - AUTHELIA_STORAGE_ENCRYPTION_KEY=${AUTHELIA_STORAGE_ENCRYPTION_KEY}
-
-# ❌ WRONG: SMTP variable without SMTP configuration
-environment:
-  - AUTHELIA_NOTIFIER_SMTP_PASSWORD=${SMTP_PASSWORD}  # Causes crash if SMTP not in config.yml
-```
-
-### 6. ✅ Docker Compose File Validation
-
-**Before deploying:**
-```bash
-# Validate syntax
-docker compose -f /path/to/file.yml config
-
-# Check for common errors
-grep -n "^  [a-z]" file.yml  # Networks should have "networks:" header
-```
-
-### 7. ✅ Certificate Renewal Strategy
-
-**Automatic Renewal:**
-- Traefik automatically renews certificates 30 days before expiration
-- Wildcard certificate covers all subdomains (no individual renewals needed)
-- Monitor `acme.json` for certificate expiration dates
-
-**Backup acme.json:**
-```bash
-# Regular backup (e.g., daily cron)
-cp /opt/stacks/core/traefik/acme.json /opt/backups/acme.json.$(date +%Y%m%d)
-
-# Keep last 7 days
-find /opt/backups -name "acme.json.*" -mtime +7 -delete
-```
-
-## Key Learnings
-
-### Technical Insights
-
-1. **DuckDNS Limitation:** Single TXT record constraint requires wildcard certificate approach
-2. **DNS Propagation:** `disablePropagationCheck: true` speeds up validation but relies on fast DNS updates
-3. **Traefik Labels:** `tls=true` vs `tls.certresolver=letsencrypt` - use former for wildcard coverage
-4. **Environment Variables:** Can trigger configuration validation even without corresponding config file entries
-
-### Process Insights
-
-1. **Log Discovery:** Traefik logs to files by default, not always visible via `docker logs`
-2. **DNS Clearing:** Stopping services and waiting 60s ensures DNS records fully clear
-3. **Incremental Debugging:** Monitor logs during certificate acquisition to catch issues early
-4. **Configuration Synchronization:** Repository files must be copied to deployment locations
-
-## Documentation Updates
-
-### Files Modified
-
-**Repository:**
-- `/home/kelin/AI-Homelab/docker-compose/core.yml`
-- `/home/kelin/AI-Homelab/docker-compose/infrastructure.yml`
-
-**Deployed:**
-- `/opt/stacks/core/docker-compose.yml`
-- `/opt/stacks/core/traefik/traefik.yml`
-- `/opt/stacks/core/traefik/acme.json`
-- `/opt/stacks/infrastructure/infrastructure.yml`
-
-### Configuration Templates
-
-**Wildcard Certificate Template:**
-```yaml
-services:
-  traefik:
-    labels:
-      - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
-      - "traefik.http.routers.traefik.tls.domains[0].main=${DOMAIN}"
-      - "traefik.http.routers.traefik.tls.domains[0].sans=*.${DOMAIN}"
-  
-  any-other-service:
-    labels:
-      - "traefik.http.routers.service.tls=true"  # No certresolver!
-```
-
-## Future Recommendations
-
-### Short-term (Next Week)
-
-1. ✅ Monitor certificate auto-renewal (should happen automatically)
-2. ✅ Test browser access from different devices to verify SSL
-3. ✅ Update homelab documentation with wildcard certificate pattern
-4. ⚠️ Consider adding certificate monitoring alerts
-
-### Medium-term (Next Month)
-
-1. Set up automated `acme.json` backups
-2. Document certificate troubleshooting runbook
-3. Consider migrating to Cloudflare if more services are added
-4. Implement certificate expiration monitoring
-
-### Long-term (Next Quarter)
-
-1. Evaluate alternative DNS providers for better DNS challenge support
-2. Consider setting up staging Let's Encrypt for testing
-3. Implement centralized logging for all services
-4. Add Prometheus/Grafana monitoring for SSL certificate expiration
-
-## Quick Reference
-
-### Emergency Certificate Reset
-
-```bash
-# 1. Stop all services
-cd /opt/stacks/core && docker compose down
-cd /opt/stacks/infrastructure && docker compose -f infrastructure.yml down
-
-# 2. Reset acme.json
-rm /opt/stacks/core/traefik/acme.json
-touch /opt/stacks/core/traefik/acme.json
-chmod 600 /opt/stacks/core/traefik/acme.json
-
-# 3. Wait for DNS to clear
-sleep 60
-
-# 4. Restart
-cd /opt/stacks/core && docker compose up -d
-cd /opt/stacks/infrastructure && docker compose -f infrastructure.yml up -d
-
-# 5. Monitor
-docker exec traefik tail -f /var/log/traefik/traefik.log
-```
-
-### Verify Certificate Command
-
-```bash
-echo | openssl s_client -connect ${SUBDOMAIN}.yourdomain.duckdns.org:443 -servername ${SUBDOMAIN}.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer -dates
-```
-
-### Check All Service Certificates
-
-```bash
-for subdomain in auth traefik dockge dozzle glances pihole; do
-  echo "=== $subdomain.yourdomain.duckdns.org ==="
-  echo | openssl s_client -connect $subdomain.yourdomain.duckdns.org:443 -servername $subdomain.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer
-  echo
-done
-```
-
----
-
-## Summary
-
-Successfully implemented wildcard SSL certificate for all homelab services using Let's Encrypt DNS challenge via DuckDNS. Key success factor was recognizing DuckDNS's limitation of one TXT record at a time and configuring Traefik to request a single wildcard certificate instead of individual certificates per service. All services now accessible via HTTPS with valid certificates.
-
-**Status:** ✅ Production Ready  
-**Next Review:** 30 days before certificate expiration (March 13, 2026)

docs/automated-setup.md

@@ -6,9 +6,11 @@ For most users, the automated setup script handles everything from system prepar
 - **Fresh Debian/Ubuntu server** (or existing system)
 - **Root/sudo access**
 - **Internet connection**
-- **Ports 80 and 443 forwarded** from your router to your server (required for SSL certificates)
+- **Ports 80 and 443 forwarded** from your router to your **core server only** (required for SSL certificates)
 - **VS Code with GitHub Copilot** (for AI assistance)
 
+**Note**: For multi-server setups, only the core server needs ports forwarded. Remote servers connect via Docker TLS (port 2376).
+
 ## Simple Setup
 
 1. **Connect to your server** via SSH
@@ -17,11 +19,13 @@ For most users, the automated setup script handles everything from system prepar
 
 2. **Install git if needed**
    ```bash
-      sudo apt update && sudo apt upgrade -y && sudo apt install git
+   sudo apt update && sudo apt upgrade -y && sudo apt install git
+   ```
 3. **Clone the repository**:
    ```bash
-   git clone https://github.com/kelinfoxy/AI-Homelab.git
+   git clone https://github.com/kelinfoxy/EZ-Homelab.git
-   cd AI-Homelab
+   cd EZ-Homelab
+   ```
 4. **Configure environment**:
    ```bash
    cp .env.example .env
@@ -57,6 +61,78 @@ For most users, the automated setup script handles everything from system prepar
 **That's it!** Your homelab is ready.
 **Access Dockge at `https://dockge.yourdomain.duckdns.org`**
 
+## Multi-Server Setup
+
+To deploy services across multiple servers (e.g., Raspberry Pi, mini PCs):
+
+### Core Server Setup (First)
+1. Follow the main setup above (steps 1-5)
+2. This server gets ports 80/443 forwarded from your router
+3. This server generates the shared CA for Docker TLS communication
+
+### Remote Server Setup (After Core)
+1. **Clone repository on remote server**:
+   ```bash
+   git clone https://github.com/kelinfoxy/EZ-Homelab.git
+   cd EZ-Homelab
+   ```
+
+2. **Copy `.env` from core server**:
+   ```bash
+   # On core server
+   cd ~/EZ-Homelab
+   cat .env  # Copy the contents
+   
+   # On remote server
+   nano ~/EZ-Homelab/.env  # Paste and save
+   ```
+
+3. **Run setup with Infrastructure-Only option**:
+   ```bash
+   ./scripts/ez-homelab.sh
+   # Select option 3: "Deploy Infrastructure Only (Remote Server)"
+   ```
+
+4. **When prompted, provide core server IP** for CA import
+
+5. **Script automatically**:
+   - Copies shared CA from core server via SSH
+   - Configures Docker TLS with shared certificates
+   - Generates server certificates signed by shared CA
+   - Sets up Docker daemon for TLS on port 2376
+   - Deploys Traefik for local container discovery
+   - Deploys Sablier for local lazy loading
+
+### What Gets Deployed Where
+| Component | Core Server | Remote Servers |
+|-----------|-------------|----------------|
+| DuckDNS | ✅ Yes | ❌ No |
+| Authelia | ✅ Yes | ❌ No |
+| Traefik | ✅ Yes (multi-provider) | ✅ Yes (local only) |
+| Sablier | ✅ Yes (own stack) | ✅ Yes (own stack) |
+| Dockge | ✅ Yes | ✅ Yes |
+| Services | ✅ Any | ✅ Any |
+
+### Architecture Benefits
+- **Single Domain**: All services accessible via core server's domain
+- **No Port Forwarding**: Remote servers don't need router configuration
+- **Automatic Discovery**: Core Traefik finds services on all servers
+- **Local Control**: Each Sablier manages its own server's containers
+- **Secure Communication**: All inter-server traffic uses TLS encryption
+
+### Troubleshooting Multi-Server Setup
+
+If remote server setup fails:
+1. **Check SSH access** from remote to core server
+2. **Verify firewall** allows port 2376 on remote servers
+3. **Test TLS connection** from core:
+   ```bash
+   cd /opt/stacks/core/shared-ca
+   docker --tlsverify --tlscacert=ca.pem --tlscert=cert.pem \
+     --tlskey=key.pem --host=tcp://REMOTE_IP:2376 ps
+   ```
+4. **Check logs**: See setup script output for specific errors
+
 ## What the Unified Setup Script Does
 
 The `ez-homelab.sh` script is a comprehensive guided setup and deployment tool:
@@ -81,8 +157,10 @@ The `ez-homelab.sh` script is a comprehensive guided setup and deployment tool:
 **Infrastructure Setup & Deployment:**
 - ✅ Creates directory structure (`/opt/stacks/`)
 - ✅ Sets up Docker networks (homelab, traefik, dockerproxy, media)
-- ✅ Deploys selected service stacks
+- ✅ Deploys selected service stacks with individual deployment scripts
 - ✅ Obtains wildcard SSL certificate (*.yourdomain.duckdns.org)
+- ✅ Configures Traefik for multi-server support (if applicable)
+- ✅ Generates and distributes TLS certificates for Docker API (multi-server)
 - ✅ Detects NVIDIA GPU and offers driver installation
 - ✅ Opens Dockge when ready
 
@@ -94,7 +172,8 @@ The `ez-homelab.sh` script is a comprehensive guided setup and deployment tool:
 - Confirmation prompts for destructive actions
 
 ## Release-Specific Notes
-- **Version**: Based on v0.1.0—ensure you're using the latest scripts.
+- **Current Version**: Production-ready with comprehensive multi-server support
-- **Stacks**: Core, Infrastructure, and Dashboards deploy automatically. Others are inactive by default.
+- **Stacks**: Core, Infrastructure, Sablier, and Dashboards deploy automatically
-- **Dashboards**: Homepage is preconfigured at `homepage.yoursubdomain.duckdns.org`.
+- **Dashboards**: Homepage is preconfigured at `homepage.yourdomain.duckdns.org`
-- **Known Limitations**: Options 1 & 2 require additional testing; use Option 3 for production.
+- **Multi-Server**: Use option 3 for remote server infrastructure deployment
+- **Modular Deployment**: Individual scripts in `docker-compose/*/deploy-*.sh` called by ez-homelab.sh

docs/development-notes.txt

@@ -1,24 +0,0 @@
-Install Process Excellent
-Deploy Porcess Excellent
-Core, dashboards, infrastructure, & media deployed successfully!
-Install & Deply script status: Working on the test system, needs testing on an actual fresh install.
-
-Notes:
-
-Authelia timeouts way too low set to 24 hours
-
-media-extended stack was renamed to media-management, homepage refers to old stack name
-media-extended folder still exist, needs Removed
-
-media-management stack & utilities stack have many invalid image:tag errors, requires image:tag validation
-
-monitoring-stack, prometheus, loki & promtail have errors
-
-productivity stack came up without errors, but nextcloud gave untrusted domain error in browser, mealie works, but the rest of the stack gave error 404 in browser
-
-EVERY compose file should include a section for the dockge urls containing all the proxy urls for services in the stack
-
-homeassistant stack the zigbee2mqtt container gives an error cause it can't find the device, which doesn't exist on this system
-
-resource limiting on a per stack or per container basis need to be implimented
-

docs/docker-guidelines.md

@@ -4,6 +4,21 @@
 
 This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab using Dockge, Traefik, and Authelia. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
 
+## Multi-Server Architecture Support
+
+EZ-Homelab supports two deployment models:
+
+1. **Single Server**: All services on one server with Traefik and Sablier managing local containers
+2. **Multi-Server**: Core server handles external traffic; remote servers run their own Traefik/Sablier instances
+
+### Multi-Server Architecture Overview
+
+- **Core Server**: DuckDNS, Traefik (multi-provider), Authelia, ports 80/443 forwarded
+- **Remote Servers**: Traefik (local discovery only), Sablier (local containers only)
+- **Routing**: Core Traefik uses external YAML files to route traffic to remote servers
+- **Security**: Each server independently manages SSO and lazy loading for its services
+- **No Docker API**: Servers communicate via HTTP/HTTPS, not Docker API TLS
+
 ## Table of Contents
 
 1. [Philosophy](#philosophy)
@@ -100,6 +115,12 @@ AI will suggest `/mnt/` when data may exceed 50GB or grow continuously.
 
 ## Traefik and Authelia Integration
 
+### Routing Decision Tree
+
+**Is Traefik running on the SAME server as your service?**
+- **YES**: Use Docker labels in the service's compose file (see below)
+- **NO**: Comment out labels; add route to Traefik's external YAML file on the core server
+
 ### Every Local (on the same server) Service Needs Traefik Labels
 
 **Default Configuration**: All services should use authelia SSO, traefik routing, and sablier lazy loading by default.
@@ -564,9 +585,13 @@ networks:
     external: true
 ```
 
-If Traefik & Sablier are on a remote server:
+### Multi-Server Configuration
+
+If Traefik is on a DIFFERENT server (e.g., service on remote server, Traefik on core):
   * Comment out the traefik labels since they won't be used, don't delete them.
-  * Notify user to add the service and middleware to the traefic external host yml file, and the sablier.yml file.
+  * Keep Sablier labels (each server has its own Sablier for local containers)
+  * Add route to Traefik's external YAML file on the core server
+  * Authelia SSO is handled by core server (no need for Authelia on remote servers)
 
 **Example: Comment out Traefik labels in docker-compose.yml:**
 ```yaml

docs/getting-started.md

@@ -1,50 +1,53 @@
 # Getting Started Guide
 
-Welcome to your AI-powered homelab! This guide will walk you through setting up your production-ready infrastructure with Dockge, Traefik, Authelia, and [50+ services](services-overview.md).
+Welcome to your EZ-Homelab! This guide will walk you through setting up one or more homelab servers with Dockge, Traefik, Authelia, and [50+ services](services-overview.md).
 
-## Release v0.1.0 Highlights
+## How It All Works
-This is the first official release of EZ-Homelab, thoroughly tested on Debian 12. Key features include:
+
-- Automated SSL via DuckDNS and Let's Encrypt.
+Before diving in, See [How Your AI Homelab Works](how-it-works.md) for a comprehensive overview.
-- Authelia SSO enabled by default for security.
-- Sablier lazy loading to save resources.
-- 50+ preconfigured services across 10 stacks.
-- See [Release Notes](../release-notes-v0.1.0.md) for full details.
 
 ## Getting Started Checklist
 - [ ] Clone this repository to your home folder
-- [ ] Configure `.env` file with your domain and tokens ([see prerequisites](env-configuration.md))
+- [ ] (optional) Configure `.env` file with your configuration details
-- [ ] Forward ports 80 and 443 from your router to your server
+- [ ] Forward ports 80 and 443 from your router to your **core server only**
-- [ ] Run unified setup script (generates Authelia secrets and admin user, deploys services) ([ez-homelab.sh](../scripts/ez-homelab.sh))
+- [ ] Run  ([ez-homelab.sh](../scripts/ez-homelab.sh))
-- [ ] Access Dockge web UI (`https://dockge.${DOMAIN}`)
+- [ ] (Optional) Set up additional remote servers using option 3 in ez-homelab.sh
+- [ ] Access Dockge web UI (`https://dockge.servername.${DOMAIN}`)
 - [ ] Set up 2FA with Authelia ([Authelia setup guide](service-docs/authelia.md))
-- [ ] (optional) Deploy additional stacks as needed via Dockge ([services overview](services-overview.md))
+- [ ] Deploy additional stacks as needed via Dockge ([services overview](services-overview.md))
-- [ ] Configure and use VS Code with Github Copilot to manage the server ([AI management](.github/copilot-instructions.md) | [Example prompts](ai-management-prompts.md))
+- [ ] Configure VS Code with GitHub Copilot to manage services ([AI management](.github/copilot-instructions.md))
 
 ## Setup Options
 
 Choose the setup method that works best for you:
 
 ### 🚀 Automated Setup (Recommended)
-For most users, the automated scripts handle everything. See [Automated Setup Guide](automated-setup.md) for step-by-step instructions.
+For most users, the automated scripts handle everything.  
+See [Automated Setup Guide](automated-setup.md) for step-by-step instructions.
 
 ### 🔧 Manual Setup
-If you prefer manual control or the automated script fails, see the [Manual Setup Guide](manual-setup.md) for detailed instructions.
+If you prefer manual control or the automated script fails,  
+see the [Manual Setup Guide](manual-setup.md) for detailed instructions.
 
 ### 🤖 AI-Assisted Setup
-Learn how to use VS Code with GitHub Copilot for AI-powered homelab management. See [AI VS Code Setup](ai-vscode-setup.md).
+Learn how to use VS Code with GitHub Copilot for AI-powered homelab management.  
-
+See [AI VS Code Setup](ai-vscode-setup.md).
-## How It All Works
-
-Before diving in, understand how your homelab infrastructure works together. See [How Your AI Homelab Works](how-it-works.md) for a comprehensive overview.
 
 ## SSL Certificates
 
-Your homelab uses Let's Encrypt for automatic HTTPS certificates. See [SSL Certificates Guide](ssl-certificates.md) for details on certificate management and troubleshooting.
+Your homelab uses Let's Encrypt for automatic HTTPS certificates.  
+See [SSL Certificates Guide](ssl-certificates.md) for details on certificate management and troubleshooting.
 
 ## What Comes Next
 
-After setup, learn what to do with your running homelab. See [Post-Setup Guide](post-setup.md) for accessing services, customization, and maintenance.
+After setup, learn what to do with your running homelab. 
+See [Post-Setup Guide](post-setup.md) for accessing services, customization, and maintenance.
 
-## On-Demand Remote Services
+## Multi-Server Deployments
+>NOTE:  
+**Core Server** refers to the server that has ports 80 & 443 forwarded to it.  
+All other servers are refered to as **Remote Server(s)**
 
-For advanced users: Learn how to set up lazy-loading services on remote servers (like Raspberry Pi) that start automatically when accessed. See [On-Demand Remote Services](Ondemand-Remote-Services.md).
+Learn how to set up lazy-loading services on remote servers that start automatically when accessed. Each server runs its own Traefik and Sablier for local container management, while the core server handles all external routing.
+
+See [Multi-Server Deployment](multi-server-deployment.md) for detailed multi-server architecture and setup instructions.

docs/how-it-works.md

@@ -11,43 +11,67 @@ Your homelab is a **Docker-based infrastructure** that automatically:
 - Updates itself
 - Backs up your data
 - Monitors system health
+- **Scales across multiple servers**
 
 Everything runs in **containers** - like lightweight virtual machines - that are orchestrated by **Docker Compose** and managed through the **Dockge** web interface.
 
+## Single-Server vs Multi-Server
+
+EZ-Homelab supports two deployment architectures:
+
+### Single Server
+- All services run on one machine
+- Traefik and Sablier manage local services only
+- Simplest setup for homelab beginners
+- Ideal for: Single desktop/server, all services in one place
+
+### Multi-Server
+- **Core Server**: Handles external traffic (ports 80/443), runs DuckDNS/Traefik/Authelia
+- **Remote Servers**: Run their own Traefik/Sablier for local container management
+- Unified domain access: All services through `service.yourdomain.com`
+- Servers communicate via HTTP/HTTPS (no Docker API TLS needed)
+- Ideal for: Raspberry Pi clusters, NAS + desktop, distributed workloads
+
+**This guide covers both architectures**, with multi-server notes where relevant.
+
 ## Core Components
 
-### 🏠 **Homepage Dashboard** (`https://home.yourdomain.duckdns.org`)
+### 🏠 **Homepage Dashboard** (`https://homepage.yourdomain.duckdns.org`)
 Your central hub for accessing all services. Think of it as the "start menu" for your homelab.
 - **What it does**: Shows all your deployed services with quick links
 - **AI Integration**: The AI can automatically add new services and configure widgets
 - **Customization**: Add weather, system stats, and service-specific widgets
 - **Configuration**: [docker-compose/dashboards/](docker-compose/dashboards/) | [service-docs/homepage.md](service-docs/homepage.md)
 
-### 🐳 **Dockge** (`https://dockge.yourdomain.duckdns.org`)
+### 🐳 **Dockge** (`https://dockge.servername.yourdomain.duckdns.org`)
 Your primary management interface for deploying and managing services.
 - **What it does**: Web-based Docker Compose manager
 - **Stacks**: Groups services into logical units (media, monitoring, productivity)
 - **One-Click Deploy**: Upload compose files and deploy instantly
-- **Configuration**: [docker-compose/infrastructure/](docker-compose/infrastructure/) | [service-docs/dockge.md](service-docs/dockge.md)
+- **Multi-Server**: Deploy on core or remote servers from one interface
+- **Configuration**: [docker-compose/dockge/](docker-compose/dockge/) | [service-docs/dockge.md](service-docs/dockge.md)
 
 ### 🔐 **Authelia** (`https://auth.yourdomain.duckdns.org`)
 Your security gatekeeper that protects sensitive services.
 - **What it does**: Single sign-on (SSO) authentication
 - **Security**: Two-factor authentication, session management
 - **Smart Bypass**: Automatically bypasses auth for media apps (Plex, Jellyfin)
+- **Multi-Server**: Core server only; protects all services across all servers
 - **Configuration**: [docker-compose/core/](docker-compose/core/) | [service-docs/authelia.md](service-docs/authelia.md)
 
-### 🌐 **Traefik** (`https://traefik.yourdomain.duckdns.org`)
+### 🌐 **Traefik** (`https://traefik.servername.yourdomain.duckdns.org`)
 Your intelligent traffic director and SSL certificate manager.
 - **What it does**: Reverse proxy that routes web traffic to the right services
 - **SSL**: Automatically obtains and renews free HTTPS certificates
 - **Labels**: Services "advertise" themselves to Traefik via Docker labels
+- **Multi-Server**: Core uses multi-provider (labels + YAML files); remote servers use labels only
 - **Configuration**: [docker-compose/core/](docker-compose/core/) | [service-docs/traefik.md](service-docs/traefik.md)
 
 ### 🦆 **DuckDNS**
 Your dynamic DNS service that gives your homelab a consistent domain name.
 - **What it does**: Updates `yourdomain.duckdns.org` to point to your home IP
 - **Integration**: Works with Traefik to get wildcard SSL certificates
+- **Multi-Server**: Core server only (one domain for all servers)
 - **Configuration**: [docker-compose/core/](docker-compose/core/) | [service-docs/duckdns.md](service-docs/duckdns.md)
 
 ### 🛡️ **Gluetun (VPN)**
@@ -75,17 +99,26 @@ Your download traffic protector.
 
 ## Network Architecture
 
-### Internal Networks
+### Internal Networks (Per Server)
 - **`traefik-network`**: All web-facing services connect here
 - **`homelab-network`**: Internal service communication
 - **`media-network`**: Media services (Plex, Jellyfin, etc.)
 - **VPN Networks**: Download services route through Gluetun
 
 ### External Access
+
+**Single Server:**
 - **Port 80/443**: Only Traefik exposes these to the internet
 - **Domain**: `*.yourdomain.duckdns.org` points to your home
 - **SSL**: Wildcard certificate covers all subdomains automatically
 
+**Multi-Server:**
+- **Core Server Ports**: Only core forwards 80/443 to internet
+- **Remote Servers**: No port forwarding needed; accessed through core
+- **Traffic Flow**: Internet → Core Traefik → Remote Traefik → Service
+- **SSL**: Core handles all SSL termination
+- **Unified Domain**: `service.yourdomain.com` works for all servers
+
 ## Storage Strategy
 
 ### Configuration Files
@@ -123,6 +156,44 @@ Some services start **on-demand** to save resources:
 - **How it works**: Service starts when you first access it
 - **Benefits**: Saves RAM and CPU when services aren't in use
 - **Configuration**: AI manages the lazy loading rules
+- **Multi-Server**: Each server runs its own Sablier for LOCAL container management
+
+## Multi-Server Architecture (Optional)
+
+If you deploy across multiple servers:
+
+### Core Server Responsibilities
+- **External Traffic**: Only server with ports 80/443 forwarded
+- **SSL Termination**: Handles all HTTPS certificates
+- **SSO Gateway**: Authelia protects all services across servers
+- **DNS Updates**: DuckDNS points domain to this server
+- **Route Configuration**: Traefik YAML files route to remote servers
+
+### Remote Server Responsibilities
+- **Local Services**: Runs media, downloads, or specialized workloads
+- **Local Management**: Own Traefik and Sablier instances
+- **Internal Access**: Services listen on Docker networks only
+- **Lazy Loading**: Sablier starts/stops containers locally
+
+### Traffic Flow Example
+```
+User requests: https://sonarr.yourdomain.com
+    ↓
+1. Internet → Core Server (ports 80/443)
+2. Core Traefik → Authelia (check login)
+3. Core Traefik → Remote Server (http://192.168.1.100:8989)
+4. Remote Traefik → Sablier (is service running?)
+5. Remote Sablier → Starts Container (if needed)
+6. Service Response → Back through chain to user
+```
+
+### Setup Process
+1. **Core**: Deploy with `ez-homelab.sh` option 1 or 2
+2. **Remote**: Deploy with `ez-homelab.sh` option 3 per server
+3. **Configure**: Add external YAML files on core for remote services
+4. **Access**: All services available through core domain
+
+See [Ondemand-Remote-Services.md](Ondemand-Remote-Services.md) for detailed setup.
 
 ## Monitoring & Maintenance
 
@@ -154,53 +225,19 @@ Some services start **on-demand** to save resources:
 ## Scaling & Customization
 
 ### Adding Services
+
+**Single Server:**
 - **Pre-built**: [50+ services](services-overview.md) ready to deploy
 - **Custom**: AI can create configurations for any Docker service
-- **External**: Proxy services on other devices (Raspberry Pi, NAS)
+- **External Devices**: Proxy services on Raspberry Pi, NAS via Traefik YAML files
+
+**Multi-Server:**
+- **Remote Services**: Deploy on any server with Traefik labels
+- **Core Routing**: Add external YAML file on core to route traffic
+- **Unified Access**: All services appear under same domain
+- **See**: [Ondemand-Remote-Services.md](Ondemand-Remote-Services.md) for patterns
 
 ### Resource Management
 - **Limits**: CPU, memory, and I/O limits prevent resource exhaustion
 - **Reservations**: Guaranteed minimum resources
 - **GPU Support**: Automatic NVIDIA GPU detection and configuration
-
-## Troubleshooting Philosophy
-
-- **Logs First**: Every service provides detailed logs. The AI can help analyze them.
-- **Isolation Testing**: Deploy services one at a time to identify conflicts.
-- **Configuration Validation**: AI validates Docker Compose syntax before deployment.
-- **Rollback Ready**: Previous configurations are preserved for quick recovery.
-
-## Getting Help
-
-### Documentation Links
-- **[Automated Setup](automated-setup.md)**: Step-by-step deployment
-- **[SSL Certificates](ssl-certificates.md)**: HTTPS configuration details
-- **[Post-Setup](post-setup.md)**: What to do after deployment
-- **[AI VS Code Setup](ai-vscode-setup.md)**: Configure AI assistance
-- **[AI Management Prompts](ai-management-prompts.md)**: Example commands for AI assistant
-- **[Services Overview](../docs/services-overview.md)**: All available services
-- **[Docker Guidelines](../docs/docker-guidelines.md)**: Technical details
-
-### AI Assistance
-- **In VS Code**: Use GitHub Copilot Chat for instant help
-- **Examples**:
-  - "Add a new service to my homelab"
-  - "Fix SSL certificate issues"
-  - "Configure backup for my data"
-  - "Set up monitoring dashboard"
-
-### Community Resources
-- **GitHub Issues**: Report bugs or request features
-- **Discussions**: Ask questions and share configurations
-- **Wiki**: Community-contributed guides and tutorials
-
-## Architecture Summary
-
-Your homelab follows these principles:
-- **Infrastructure as Code**: Everything defined in files
-- **GitOps**: Version control for all configurations
-- **Security First**: SSO protection by default
-- **AI-Assisted**: Intelligent management and troubleshooting
-- **Production Ready**: Monitoring, backups, and high availability
-
-The result is a powerful, secure, and easy-to-manage homelab that grows with your needs!

docs/implementation-plan-corrections.md

@@ -1,202 +0,0 @@
-# Implementation Plan Corrections
-
-## User Feedback Summary
-
-### 1. Remove `prompt_for_server_role()` Function
-**Issue**: Unnecessary function - should integrate with existing validation  
-**Solution**: Use existing menu structure (Option 2: Deploy Core, Option 3: Deploy Additional Server) and integrate role detection into existing `validate_and_prompt_variables()` function via dynamic REQUIRED_VARS
-
-### 2. Remove `validate_env_file()` Function  
-**Issue**: Duplicates existing REQUIRED_VARS mechanism  
-**Solution**: Enhance existing system:
-- Add `set_required_vars_for_deployment(type)` function to dynamically set REQUIRED_VARS
-- Reuse existing `validate_and_prompt_variables()` (line 562) - already validates and prompts
-- No new validation function needed
-
-### 3. config-templates Folder is Deprecated
-**Issue**: Plan references `config-templates/` which should be deleted  
-**Solution**: All working configs are in `docker-compose/` folder
-- Update references: `docker-compose/core/traefik/traefik.yml` (not config-templates)
-- Delete config-templates folder if still exists
-
-### 4. .env Editing Should Be Optional
-**Issue**: Plan suggests users manually edit .env in step 4.2  
-**Solution**: 
-- Script ALWAYS prompts for ALL required variables for the deployment option
-- When user selects Option 3 (Deploy Additional Server):
-  - Call `set_required_vars_for_deployment("remote")`
-  - Sets REQUIRED_VARS to include: REMOTE_SERVER_IP, REMOTE_SERVER_HOSTNAME, REMOTE_SERVER_USER
-  - Call `validate_and_prompt_variables()` - prompts for all
-  - Save complete .env via `save_env_file()`
-
-### 5. Deploy Scripts Should Auto-Backup
-**Issue**: Migration path has manual backup steps  
-**Solution**:
-- Deploy scripts MUST backup automatically before changes
-- **Critical**: Verify backups are from `/opt/stacks/*/` (deployed location), NOT `~/EZ-Homelab/docker-compose/*/` (repo source)
-- Expected backup pattern: `/opt/stacks/core/traefik.backup.TIMESTAMP/`
-  where TIMESTAMP is like YY_MM_DD_hh_mm 
-- Review all deploy functions for correct backup logic
-
-### 6. Traefik Dynamic Folder Files Need Replacement
-**Issue**: Existing `external-host-*.yml` files are for old method  
-**Solution**:
-- During core deployment, replace entire `traefik/dynamic/` folder contents
-- New files:
-  - `sablier.yml` (updated middleware format)
-  - Auto-generated provider-specific configs
-- Deploy script should:
-  1. Backup `/opt/stacks/core/traefik/dynamic/` to timestamped folder
-  2. Copy new configs from `docker-compose/core/traefik/dynamic/`
-  3. Process variables via `localize_config_file()`
-
----
-
-## Corrected Function List
-
-### Functions to ADD (New):
-
-**In common.sh:**
-- `detect_server_role()` - Check .env to determine core vs remote
-- `generate_traefik_provider_config()` - Generate YAML for remote provider
-- `generate_sablier_middleware_config()` - Generate YAML for remote Sablier
-- `add_remote_server_to_traefik()` - Register remote server with core
-
-**In ez-homelab.sh:**
-- `check_docker_installed()` - Silent Docker check with timeout (Pi-safe)
-- `set_required_vars_for_deployment(type)` - Set REQUIRED_VARS dynamically
-- `deploy_remote_server()` - Deploy Docker TLS + Sablier on remote servers
-
-### Functions to MODIFY (Existing):
-
-**In ez-homelab.sh:**
-- `REQUIRED_VARS` (line 398) - Make dynamic via `set_required_vars_for_deployment()`
-- `main()` - Add Docker pre-check, call `set_required_vars_for_deployment()` before validation
-- `deploy_core()` - Auto-deploy Sablier stack after core stack
-- `validate_and_prompt_variables()` (line 562) - NO CHANGES (already does what we need)
-
-### Functions NOT NEEDED:
-- ~~`prompt_for_server_role()`~~ - Use existing menu structure
-- ~~`validate_env_file()`~~ - Use existing REQUIRED_VARS mechanism
-
----
-
-## Corrected Workflow
-
-### Option 2: Deploy Core (Existing Option)
-```bash
-main() {
-    case $DEPLOY_CHOICE in
-        2)
-            # Deploy Core
-            set_required_vars_for_deployment "core"  # NEW
-            validate_and_prompt_variables            # EXISTING - reuse
-            save_env_file                            # EXISTING
-            DEPLOY_CORE=true
-            # In deploy_core(): auto-deploy Sablier after core
-            ;;
-```
-
-### Option 3: Deploy Additional Server (New Option)
-```bash
-        3)
-            # Deploy Additional Server
-            set_required_vars_for_deployment "remote"  # NEW - sets REQUIRED_VARS
-            validate_and_prompt_variables              # EXISTING - prompts for all
-            save_env_file                              # EXISTING
-            deploy_remote_server                       # NEW function
-            ;;
-```
-
-### REQUIRED_VARS Dynamic Setting
-```bash
-set_required_vars_for_deployment() {
-    local deployment_type="${1:-core}"
-    
-    if [ "$deployment_type" == "core" ]; then
-        REQUIRED_VARS=("SERVER_IP" "SERVER_HOSTNAME" "DUCKDNS_SUBDOMAINS" "DUCKDNS_TOKEN" "DOMAIN" "DEFAULT_USER" "DEFAULT_PASSWORD" "DEFAULT_EMAIL")
-    elif [ "$deployment_type" == "remote" ]; then
-        REQUIRED_VARS=("SERVER_IP" "SERVER_HOSTNAME" "REMOTE_SERVER_IP" "REMOTE_SERVER_HOSTNAME" "REMOTE_SERVER_USER" "DEFAULT_USER" "DEFAULT_EMAIL")
-    fi
-}
-```
-
----
-
-## Corrected File Structure
-
-### Repo Source Files (docker-compose/):
-```
-docker-compose/
-├── core/
-│   ├── docker-compose.yml          # Remove Sablier section
-│   ├── traefik/
-│   │   ├── traefik.yml             # Static config with provider template
-│   │   └── dynamic/
-│   │       ├── sablier.yml         # Updated middleware configs
-│   │       └── (other dynamic configs)
-│   └── authelia/
-├── sablier/                        # NEW STACK
-│   ├── docker-compose.yml          # Container name: sablier
-│   └── README.md                   # Stack documentation
-└── (other stacks)/
-```
-
-### Deployed Files (/opt/stacks/):
-```
-/opt/stacks/
-├── core/                           # Core stack only (no Sablier)
-│   ├── docker-compose.yml
-│   ├── traefik/
-│   │   ├── config/traefik.yml
-│   │   └── dynamic/
-│   │       ├── sablier.yml
-│   │       └── (auto-generated provider configs)
-│   └── shared-ca/                  # Shared CA certificates
-├── sablier/                        # Sablier stack (all servers)
-│   ├── docker-compose.yml
-│   └── .env                        # Environment variables (copied from repo)
-└── (other stacks)/
-```
-
----
-
-## Critical Backup Check
-
-**Problem**: Deploy scripts may backup from repo instead of deployed location
-
-**Incorrect (if found)**:
-```bash
-# DON'T do this - backs up repo source, not deployed config
-cp -r ~/EZ-Homelab/docker-compose/core/traefik ~/backups/
-```
-
-**Correct**:
-```bash
-# DO this - backs up deployed configuration
-# Format: traefik.backup.YY_MM_DD_hh_mm
-cp -r /opt/stacks/core/traefik /opt/stacks/core/traefik.backup.$(date +%y_%m_%d_%H_%M)
-```
-
-**Action**: Review `deploy_core()` and all deploy functions for correct backup paths
-
----
-
-## Implementation Priority
-
-1. ✅ **Delete** `config-templates/` folder
-2. ✅ **Verify** deploy scripts backup from `/opt/stacks/` not repo
-3. ✅ **Add** `set_required_vars_for_deployment()` function
-4. ✅ **Add** `check_docker_installed()` function
-5. ✅ **Modify** `main()` to use dynamic REQUIRED_VARS
-6. ✅ **Create** `docker-compose/sablier/` stack
-7. ✅ **Remove** Sablier from `docker-compose/core/docker-compose.yml`
-8. ✅ **Add** common.sh functions for multi-server support
-9. ✅ **Add** `deploy_remote_server()` function
-10. ✅ **Update** `docker-compose/core/traefik/dynamic/` files
-
----
-
-*Corrections Version: 1.0*  
-*Date: February 4, 2026*  
-*Based on: User feedback on implementation plan v2.0*

docs/manual-setup.md

@@ -17,26 +17,8 @@ sudo apt update && sudo apt upgrade -y
 ## Step 2: Install Docker
 
 ```bash
-# Install dependencies
+curl -fsSL https://get.docker.com | sudo sh
-sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
+sudo usermod -aG docker your-username
-
-# Add Docker's official GPG key
-curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
-
-# Set up repository
-echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-
-# Install Docker Engine
-sudo apt update
-sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
-
-# Or use convenience script
-curl -fsSL https://get.docker.com -o get-docker.sh
-sudo sh get-docker.sh
-
-# Add user to docker group
-sudo usermod -aG docker $USER
-sudo usermod -aG sudo $USER
 
 # Log out and back in, or run: newgrp docker
 ```
@@ -45,32 +27,50 @@ sudo usermod -aG sudo $USER
 
 ```bash
 cd ~
-git clone https://github.com/kelinfoxy/AI-Homelab.git
+git clone https://github.com/kelinfoxy/EZ-Homelab.git
-cd AI-Homelab
+cd EZ-Homelab
 ```
 
-## Step 4: Configure Environment
+## Step 4: Generate Secrets
+```yaml
+# Generate 3 Secrets for Authelia
+
+# Generate hash for your password
+docker run --rm authelia/authelia:4.37 authelia crypto hash generate argon2 --password 'YourSecurePassword'
+
+```
+
+## Step 5: Configure Environment
 
 ```bash
 cp .env.example .env
 nano .env  # Edit all required variables
 ```
 
-**Required variables:**
+**Minimum Required variables:**
-- `DOMAIN` - Your DuckDNS domain
+* PUID
-- `DUCKDNS_TOKEN` - Your DuckDNS token
+* PGID
-- `ACME_EMAIL` - Your email for Let's Encrypt
+* TZ
-- `AUTHELIA_JWT_SECRET` - Generate with: `openssl rand -hex 64`
+* SERVER_IP                       
-- `AUTHELIA_SESSION_SECRET` - Generate with: `openssl rand -hex 64`
+* SERVER_HOSTNAME                         
-- `AUTHELIA_STORAGE_ENCRYPTION_KEY` - Generate with: `openssl rand -hex 64`
+* DUCKDNS_SUBDOMAINS
-- `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN
+* DUCKDNS_TOKEN
+* DOMAIN
+* DEFAULT_USER                             
+* DEFAULT_PASSWORD
+* DEFAULT_EMAIL  
+**If using VPN**:
+* SURFSHARK_USERNAM
+* SURFSHARK_PASSWORD
+* VPN_SERVER_COUNTRIES
 
-## Step 5: Create Infrastructure
+## Step 6: Create Infrastructure
 
 ```bash
 # Create directories
-sudo mkdir -p /opt/stacks /mnt/{media,database,downloads,backups}
+sudo mkdir -p /opt/stacks /opt/dockge 
-sudo chown -R $USER:$USER /opt/stacks /mnt
+sudo chown -R $USER:$USER /opt/stacks
+sudo chown -R $USER:$USER /opt/dockge
 
 # Create networks
 docker network create traefik-network
@@ -79,200 +79,17 @@ docker network create dockerproxy-network
 docker network create media-network
 ```
 
-## Step 6: Generate Authelia Password Hash
+## Step 7: Copy folders
-
+```yaml
-```bash
+sudo cp ~/EZ-Homelab/docker-compose /opt/stacks
-# Generate password hash (takes 30-60 seconds)
+# Move the dockge stack outside the stacks folder so you can't accidently stop dockge from dockge's webui
-docker run --rm authelia/authelia:4.37 authelia crypto hash generate argon2 --password 'YourSecurePassword'
+sudo mv /opt/stacks/dockge /opt/dockge
-
-# Copy the hash starting with $argon2id$...
 ```
 
-## Step 7: Configure Authelia
+## Step 8: Configure Stacks
 
-```bash
+Some services require manualy editing the configuration files.  
-# Copy Authelia config templates
+Each stack has a script `deploy-stackname.sh` which will replace variables in the config files and the traefik labels in the compose file.
-mkdir -p /opt/stacks/core/authelia
-cp config-templates/authelia/* /opt/stacks/core/authelia/
 
-# Edit users_database.yml
+See the `Readme.md` file in each stack folder for manual instructions for each stack.
-nano /opt/stacks/core/authelia/users_database.yml
 
-# Replace password hash and email in the users section:
-users:
-  admin:
-    displayname: "Admin User"
-    password: "$argon2id$v=19$m=65536,t=3,p=4$..."  # Your hash here
-    email: your.email@example.com
-    groups:
-      - admins
-      - users
-```
-
-## Step 8: Deploy Core Services
-
-```bash
-# Deploy core infrastructure
-sudo mkdir -p /opt/stacks/core
-cp docker-compose/core/docker-compose.yml /opt/stacks/core/docker-compose.yml
-cp -r config-templates/traefik /opt/stacks/core/
-cp .env /opt/stacks/core/
-
-# Update Traefik email
-sed -i "s/admin@example.com/$ACME_EMAIL/" /opt/stacks/core/traefik/traefik.yml
-
-cd /opt/stacks/core
-docker compose up -d
-```
-
-## Step 9: Deploy Infrastructure Stack
-
-```bash
-sudo mkdir -p /opt/stacks/infrastructure
-cp docker-compose/infrastructure/docker-compose.yml /opt/stacks/infrastructure/docker-compose.yml
-cp .env /opt/stacks/infrastructure/
-cd /opt/stacks/infrastructure
-docker compose up -d
-```
-
-## Step 10: Deploy Dashboards
-
-```bash
-sudo mkdir -p /opt/stacks/dashboards
-cp docker-compose/dashboards/docker-compose.yml /opt/stacks/dashboards/docker-compose.yml
-cp -r config-templates/homepage /opt/stacks/dashboards/
-cp .env /opt/stacks/dashboards/
-
-# Replace Homepage domain variables
-find /opt/stacks/dashboards/homepage -type f \( -name "*.yaml" -o -name "*.yml" \) -exec sed -i "s/{{HOMEPAGE_VAR_DOMAIN}}/$DOMAIN/g" {} \;
-
-cd /opt/stacks/dashboards
-docker compose up -d
-```
-
-## Step 10.5: Multi-Server TLS Setup (Optional)
-
-If you plan to deploy services on remote servers (like Raspberry Pi) that will be managed by Sablier for lazy loading, set up shared TLS certificates.
-
-### On Core Server (where Traefik/Authelia run):
-
-```bash
-# Create shared CA directory
-sudo mkdir -p /opt/stacks/core/shared-ca
-sudo chown $USER:$USER /opt/stacks/core/shared-ca
-
-# Generate shared CA certificate
-cd /opt/stacks/core/shared-ca
-openssl genrsa -out ca-key.pem 4096
-openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=US/ST=State/L=City/O=Homelab/CN=Homelab-CA"
-
-# Set proper permissions
-chmod 600 ca-key.pem
-chmod 644 ca.pem
-```
-
-### On Remote Servers:
-
-```bash
-# Create TLS directory
-sudo mkdir -p /opt/stacks/core/shared-ca
-sudo chown $USER:$USER /opt/stacks/core/shared-ca
-
-# Copy shared CA from core server (replace CORE_IP with your core server IP)
-scp user@CORE_IP:/opt/stacks/core/shared-ca/ca.pem /opt/stacks/core/shared-ca/
-scp user@CORE_IP:/opt/stacks/core/shared-ca/ca-key.pem /opt/stacks/core/shared-ca/
-
-# Generate client certificate for Docker client connections
-openssl genrsa -out client-key.pem 4096
-openssl req -subj "/CN=client" -new -key client-key.pem -out client.csr
-openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem
-
-# Configure Docker TLS
-sudo tee /etc/docker/daemon.json > /dev/null <<EOF
-{
-  "tls": true,
-  "tlsverify": true,
-  "tlscacert": "/opt/stacks/core/shared-ca/ca.pem",
-  "tlscert": "/opt/stacks/core/shared-ca/server-cert.pem",
-  "tlskey": "/opt/stacks/core/shared-ca/server-key.pem"
-}
-EOF
-
-# Update Docker service to listen on TLS port
-sudo sed -i 's|-H fd://|-H fd:// -H tcp://0.0.0.0:2376|' /lib/systemd/system/docker.service
-sudo systemctl daemon-reload
-sudo systemctl restart docker
-
-# Test TLS connection from core server
-# On core server, run: docker --tlsverify --tlscacert /opt/stacks/core/shared-ca/ca.pem --tlscert /opt/stacks/core/shared-ca/client-cert.pem --tlskey /opt/stacks/core/shared-ca/client-key.pem -H tcp://REMOTE_IP:2376 ps
-```
-
-## Step 11: Verify Deployment
-
-```bash
-# Check running containers
-docker ps
-
-# Check logs if any service fails
-docker logs container-name
-
-# Access services
-echo "Dockge: https://dockge.$DOMAIN"
-echo "Authelia: https://auth.$DOMAIN"
-echo "Traefik: https://traefik.$DOMAIN"
-```
-
-## Troubleshooting
-
-**Permission issues:**
-```bash
-# Ensure proper ownership
-sudo chown -R $USER:$USER /opt/stacks
-
-# Check group membership
-groups $USER
-```
-
-**Container won't start:**
-```bash
-# Check logs
-docker logs container-name
-
-# Check compose file syntax
-docker compose config
-```
-
-**Network conflicts:**
-```bash
-# List existing networks
-docker network ls
-
-# Remove and recreate if needed
-docker network rm network-name
-docker network create network-name
-```
-
-## When to Use Manual Setup
-
-- Automated script fails on your system
-- You want full control over each step
-- You're using a non-Debian/Ubuntu distribution
-- You need custom configurations
-- You're troubleshooting deployment issues
-
-## Post-Setup Notes
-- **Lazy Loading**: Services use Sablier for resource efficiency. Expect short delays on first access—refresh if you see timeouts.
-- **Security**: SSO is enabled by default; bypass only for media apps like Plex/Jellyfin.
-- **Testing**: This release has been thoroughly tested, but manual setup may need tweaks—report issues on GitHub.
-
-## Switching to Automated
-
-If manual setup works, you can switch to the unified automated script for future updates:
-
-```bash
-# Just run the unified setup script
-cd ~/EZ-Homelab
-./scripts/ez-homelab.sh
-```
-
-The unified script is idempotent - it won't break existing configurations.

docs/multi-server-deployment.md

@@ -0,0 +1,98 @@
+# Multi-Server Deployment with On-Demand Services
+
+## Overview
+
+This guide explains the **current multi-server architecture** where:
+- **Core Server**: Handles external traffic (ports 80/443); runs DuckDNS, Traefik (multi-provider), Authelia
+- **Remote Servers**: Run their own Traefik (local-only) and Sablier (local containers)
+- **No Docker API**: Servers communicate via HTTP/HTTPS; no TLS Docker API connections needed
+- **Independent Management**: Each server manages its own containers with lazy loading
+
+> **Note**: This document describes the current simplified architecture. For the legacy centralized Sablier approach, see the git history.
+
+## Architecture Diagram
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                         CORE SERVER                              │
+│  ┌────────────┐  ┌──────────┐  ┌──────────┐  ┌──────────────┐ │
+│  │  DuckDNS   │  │ Traefik  │  │ Authelia │  │ Core Services │ │
+│  │ (SSL DNS)  │  │ (multi-  │  │  (SSO)   │  │   (local)     │ │
+│  │            │  │ provider)│  │          │  │               │ │
+│  └────────────┘  └────┬─────┘  └──────────┘  └──────────────┘ │
+│                       │                                          │
+│          ┌──────────────┼──────────────┐                        │
+│          │  Routes:     │              │                        │
+│          │  • Local     │  (labels)    │                        │
+│          │  • Remote    │  (YAML files)│                        │
+└──────────┼──────────────┼──────────────┼────────────────────────┘
+           │              │              │
+    Ports  │  HTTP/HTTPS  │              │
+    80/443 │              │              │
+           ▼              ▼              ▼
+    ┌─────────────────────────────────────────┐
+    │         REMOTE SERVER (e.g., Pi)        │
+    │  ┌──────────┐  ┌──────────┐  ┌───────┐ │
+    │  │ Traefik  │  │ Sablier  │  │ Media │ │
+    │  │ (local)  │  │ (local)  │  │ Apps  │ │
+    │  └──────────┘  └──────────┘  └───────┘ │
+    └─────────────────────────────────────────┘
+```
+
+# Deployment Process
+
+## Step 1: Deploy Core Server
+
+On the core server, run `ez-homelab.sh`  
+* Use Option 1 to Install Prerequesites
+* Then Option 2 to Deploy Core Server
+
+This deploys:  DuckDNS, Traefik(core), Authelia, Dashboards & Infrastructure
+
+From Dockge you can start/stop any of the stacks or containers.
+
+**Port Forwarding**:
+- Forward ports 80 & 443 from your router
+- Only this server requires port forwarding
+
+## Step 2: Deploy Remote Server
+
+On the remote server, run `ez-homelab.sh`  
+* Use Option 1 to Install Prerequesites
+* Then Option 3 to Deploy Remote Server
+
+This deploys:  Traefik(local), Dashboards & Infrastructure
+
+From Dockge you can start/stop any of the stacks or containers.
+
+## How It Works
+
+### Traffic Flow
+
+1. **User accesses** `https://sonarr.yourdomain.duckdns.org`
+2. **Core Traefik** receives request:
+   - Checks Authelia for authentication (SSO)
+   - Routes to remote server: `http://192.168.1.100:8989`
+3. **Remote Traefik** receives forwarded request:
+   - Discovers service is stopped (lazy loaded)
+   - Forwards to local Sablier
+4. **Remote Sablier**:
+   - Starts Sonarr container
+   - Shows loading page
+   - Redirects to service when ready
+5. **Service responds** through the chain back to user
+
+### Key Benefits
+
+- **Independent Management**: Each server controls its own containers
+- **Centralized Access**: All services accessed through one domain
+- **Unified SSO**: Authelia on core server protects all services
+- **Local Lazy Loading**: Sablier manages containers on the same server
+
+## Performance Considerations
+
+- **Latency**: One additional hop (core → remote) adds minimal latency
+- **Resource Usage**: Each server runs lightweight Traefik + Sablier (~100MB combined)
+- **Scalability**: Can add unlimited remote servers without complexity
+- **Network**: Internal 1Gbps+ recommended between servers
+

docs/multi-server-implementation-plan.md

@@ -1,5 +1,28 @@
 # Multi-Server Traefik and Sablier Implementation Plan
 
+> **⚠️ IMPLEMENTATION STATUS: COMPLETED (with modified approach)**
+> 
+> **Date**: December 2024  
+> **Status**: The multi-server architecture has been successfully implemented, but with a **simplified approach** that differs from this original plan.
+>
+> **What Changed:**
+> - ❌ **Traefik Multi-Docker Provider**: NOT implemented (complexity, Docker API security concerns)
+> - ✅ **Per-Server Traefik**: Each server runs its own Traefik for LOCAL discovery only
+> - ✅ **Per-Server Sablier**: Each server runs its own Sablier for LOCAL lazy loading
+> - ✅ **Core Routing**: Core Traefik uses file provider (YAML) to route to remote services
+> - ✅ **No Docker API TLS**: Servers communicate via HTTP/HTTPS, not Docker API
+>
+> **Current Documentation:**
+> - **Architecture**: See [Ondemand-Remote-Services.md](Ondemand-Remote-Services.md) for the implemented approach
+> - **Service Creation**: See [docker-guidelines.md](docker-guidelines.md) for multi-server patterns
+> - **How It Works**: See [how-it-works.md](how-it-works.md) for architecture overview
+>
+> **This Document**: Preserved for historical reference and future consideration. The original plan's centralized Traefik approach may be revisited in the future if label-based remote discovery becomes a priority.
+
+---
+
+# Original Implementation Plan (Historical Reference)
+
 ## Executive Summary
 
 This document outlines the implementation plan for enabling label-based automatic routing and lazy loading across multiple servers in the EZ-Homelab infrastructure. The goals are to:

docs/post-setup.md

@@ -4,6 +4,8 @@ Congratulations! Your AI-powered homelab is now running. Here's what to do next.
 
 ## Access Your Services
 
+### Single Server Deployment
+
 - **Homepage**: `https://homepage.yourdomain.duckdns.org`
   - Great place to start exploring your services
   - After configuring your services, come back and add widgets with API keys (optional)
@@ -22,12 +24,35 @@ Congratulations! Your AI-powered homelab is now running. Here's what to do next.
 
 - **VS Code**: `https://code.yourdomain.duckdns.org`
    - Install GitHub Copilot Chat extension
-   - Open the AI-Homelab repository
+   - Open the EZ-Homelab repository
    - Use AI assistance for:
       - Adding new services
       - Configuring Traefik routing
       - Managing Docker stacks
 
+### Multi-Server Deployment
+
+If you deployed across multiple servers:
+
+**Core Server Services** (accessed from anywhere):
+- All services above (Homepage, Dockge, Authelia, Traefik, code-server)
+- Access these through your domain: `service.yourdomain.duckdns.org`
+
+**Remote Server Services** (accessed through core):
+- Remote services automatically routed through core Traefik
+- Example: `https://sonarr.yourdomain.duckdns.org` → Core → Remote Server
+- SSO protection applied by core Authelia
+
+**Direct Local Access** (on remote server network):
+- Services also accessible via local IP: `http://192.168.1.100:8989`
+- Useful for troubleshooting or local management
+- No SSO protection on local access
+
+**Management Tips:**
+- Use core Dockge to manage all servers (if configured)
+- Each server has its own local Traefik/Sablier for container management
+- All services appear under unified domain through core routing
+
 ## Monitoring Services
 
 - Use Dockge to easily view live container logs

docs/script-audit-report.md

@@ -1,240 +0,0 @@
-# EZ-Homelab Script Audit Report
-## Generated: 30 January 2026
-
-### Executive Summary
-The `ez-homelab.sh` script is a comprehensive Bash-based deployment tool for the EZ-Homelab project. It handles system setup, Docker configuration, and multi-stage service deployment. The script supports three main deployment modes with varying complexity. While functional for infrastructure-only deployments (Option 3), the core-only deployment (Option 2) has critical issues with Authelia secret generation and configuration that prevent successful deployment.
-
-### Script Architecture
-
-#### Global Variables & Constants
-- **Color codes**: RED, GREEN, YELLOW, BLUE, NC for console output formatting
-- **Logging functions**: `log_info()`, `log_success()`, `log_warning()`, `log_error()`
-- **Deployment flags**: DEPLOY_CORE, DEPLOY_INFRASTRUCTURE, DEPLOY_DASHBOARDS, SETUP_STACKS
-- **Configuration variables**: DOMAIN, SERVER_IP, ADMIN_USER, etc.
-- **Path variables**: SCRIPT_DIR, REPO_DIR, ACTUAL_USER
-
-#### Core Functions
-
-##### 1. `replace_env_placeholders()`
-**Purpose**: Replaces `${VAR}` placeholders in files with actual environment variable values
-**Process**:
-- Takes file path as argument
-- Uses `grep` to find all `${VAR}` patterns
-- Checks if each variable exists in environment
-- Uses `sed` for replacement: `s|\${VAR}|${!VAR}|g`
-- Accumulates missing variables in `MISSING_VARS_SUMMARY`
-**Issues**: 
-- Only reports missing variables at end, doesn't fail deployment
-- No validation of replacement success
-
-##### 2. `generate_shared_ca()`
-**Purpose**: Creates shared Certificate Authority for multi-server TLS
-**Process**:
-- Creates `/opt/stacks/core/shared-ca/` directory
-- Generates 4096-bit RSA CA key and certificate (365 days validity)
-- Sets ownership to `$ACTUAL_USER:$ACTUAL_USER`
-**Output**: ca.pem, ca-key.pem files
-
-##### 3. `setup_multi_server_tls()`
-**Purpose**: Configures TLS for remote Docker access using shared CA
-**Process**:
-- Prompts for core server IP if not set
-- Tests SSH connectivity (key auth first, then password)
-- Fetches CA certificates from core server via SCP
-- Calls `setup_docker_tls()` if successful
-**Issues**:
-- Complex SSH authentication logic
-- No fallback if CA fetch fails
-- TLS_ISSUES_SUMMARY populated but not always accurate
-
-##### 4. `load_env_file()`
-**Purpose**: Loads existing configuration from `.env` file
-**Process**:
-- Checks for `$REPO_DIR/.env` existence
-- Sources the file if found
-- Displays current configuration values
-- Returns 0 if file exists, 1 if not
-**Issues**: No validation of loaded values
-
-##### 5. `save_env_file()`
-**Purpose**: Persists configuration to `.env` file
-**Process**:
-- Creates `.env` from `.env.example` if needed
-- Updates values using `sed` replacements
-- For core deployment: generates Authelia secrets and password hash
-**Critical Issue**: Authelia secret generation is here, but may not be called in all deployment paths
-
-##### 6. `prompt_for_values()`
-**Purpose**: Interactive configuration collection
-**Process**:
-- Shows current/default values
-- Allows user to accept defaults or enter custom values
-- Handles sensitive inputs (passwords) with `-s` flag
-- Sets ADMIN_* variables for core deployment
-**Issues**: Complex logic with many conditional branches
-
-##### 7. `system_setup()`
-**Purpose**: Performs initial system configuration (requires root)
-**Process**:
-1. System package updates
-2. Installs prerequisites (curl, wget, git, etc.)
-3. Installs/configures Docker and Docker Compose
-4. Generates shared CA
-5. Configures Docker TLS
-6. Sets up UFW firewall
-7. Configures automatic updates
-8. Creates Docker networks
-9. Sets directory ownership
-**Issues**: 
-- Requires logout/login for Docker group changes
-- No rollback on failure
-
-##### 8. `deploy_dockge()`
-**Purpose**: Deploys Dockge stack management interface
-**Process**:
-- Copies compose file and .env to `/opt/dockge/`
-- Replaces placeholders
-- Runs `docker compose up -d`
-**Output**: Dockge service running
-
-##### 9. `deploy_core()`
-**Purpose**: Deploys core infrastructure stack
-**Process**:
-1. Copies compose file and .env to `/opt/stacks/core/`
-2. Copies Traefik and Authelia config templates
-3. Replaces placeholders in all config files
-4. Generates shared CA
-5. Replaces Authelia-specific secrets and user data
-6. Runs `docker compose up -d`
-**Critical Issues**:
-- Assumes Authelia secrets exist in environment
-- No validation that secrets were generated
-- Complex placeholder replacement logic
-
-##### 10. `deploy_infrastructure()` / `deploy_dashboards()`
-**Purpose**: Deploy additional service stacks
-**Process**: Similar to deploy_core but simpler
-- Copy files, replace placeholders, deploy
-**Issues**: Conditional Authelia middleware removal when core not deployed
-
-##### 11. `setup_docker_tls()`
-**Purpose**: Configures Docker daemon for TLS
-**Process**:
-1. Creates TLS directory
-2. Uses shared CA or generates local CA
-3. Generates server and client certificates
-4. Updates Docker daemon.json
-5. Modifies systemd service for TCP 2376
-6. Restarts Docker service
-
-##### 12. `setup_stacks_for_dockge()`
-**Purpose**: Prepares all service stacks for Dockge management
-**Process**:
-- Iterates through predefined stack list
-- Copies compose files and configs
-- Replaces placeholders
-- Prepares but doesn't deploy stacks
-
-### Deployment Flow Analysis
-
-#### Option 1: Default Setup
-**Flags**: DEPLOY_CORE=true, DEPLOY_INFRASTRUCTURE=true, DEPLOY_DASHBOARDS=true, SETUP_STACKS=true
-**Flow**:
-1. System setup (if needed)
-2. Prompt for values
-3. Save env file (generates Authelia secrets)
-4. Deploy Dockge
-5. Deploy core (uses generated secrets)
-6. Deploy infrastructure
-7. Deploy dashboards
-8. Setup stacks for Dockge
-
-#### Option 2: Core Only
-**Flags**: DEPLOY_CORE=true, DEPLOY_INFRASTRUCTURE=false, DEPLOY_DASHBOARDS=true, SETUP_STACKS=true
-**Flow**:
-1. System setup (if needed)
-2. Prompt for values
-3. Save env file (generates Authelia secrets)
-4. Deploy Dockge
-5. Deploy core (uses generated secrets)
-6. Deploy dashboards
-7. Setup stacks for Dockge
-
-#### Option 3: Infrastructure Only
-**Flags**: DEPLOY_CORE=false, DEPLOY_INFRASTRUCTURE=true, DEPLOY_DASHBOARDS=false, SETUP_STACKS=true
-**Flow**:
-1. System setup (if needed)
-2. Prompt for values
-3. Save env file (no Authelia secrets generated)
-4. Setup multi-server TLS
-5. Deploy Dockge
-6. Deploy infrastructure
-7. Setup stacks for Dockge
-
-### Critical Issues Identified
-
-#### 1. Authelia Secret Generation Timing (Option 2)
-**Problem**: In Option 2, `save_env_file()` is called and should generate Authelia secrets, but the deployment may fail if secrets aren't properly set.
-**Root Cause**: The `save_env_file()` function generates secrets only when `DEPLOY_CORE=true`, but the generation logic may not execute or persist correctly.
-**Impact**: Authelia container fails to start due to missing JWT_SECRET, SESSION_SECRET, or STORAGE_ENCRYPTION_KEY
-
-#### 2. Environment Variable Persistence
-**Problem**: After `save_env_file()`, the script sources the .env file, but there may be a timing issue where variables aren't available for `deploy_core()`.
-**Evidence**: The script does `source "$REPO_DIR/.env"` in `perform_deployment()`, but if secrets weren't saved properly, they'll be empty.
-
-#### 3. Placeholder Replacement Order
-**Problem**: `replace_env_placeholders()` is called during deployment, but if environment variables are missing, replacements fail silently.
-**Impact**: Configuration files contain literal `${VAR}` strings instead of actual values.
-
-#### 4. Authelia Password Hash Generation
-**Problem**: Password hash generation happens in `save_env_file()`, but requires Docker to be running and Authelia image to be available.
-**Issues**: 
-- May fail if Docker isn't ready
-- Uses complex docker run command that could timeout
-- No fallback if hash generation fails
-
-#### 5. Multi-Server TLS Complexity
-**Problem**: `setup_multi_server_tls()` has complex SSH logic that can fail in multiple ways.
-**Issues**:
-- SSH key vs password detection unreliable
-- No retry logic for connection failures
-- Error reporting doesn't clearly indicate resolution steps
-
-#### 6. Directory Creation Race Conditions
-**Problem**: Script creates directories with sudo, then tries to write files as regular user.
-**Potential Issue**: Permission conflicts if ownership isn't set correctly.
-
-### Recommendations
-
-#### Immediate Fixes for Option 2
-1. **Add Secret Validation**: After `save_env_file()`, validate that all required Authelia secrets exist before proceeding with deployment.
-
-2. **Improve Error Handling**: Make `replace_env_placeholders()` fail deployment if critical variables are missing.
-
-3. **Add Authelia Health Check**: After core deployment, verify Authelia container is running and healthy.
-
-#### Structural Improvements
-1. **Separate Secret Generation**: Move Authelia secret generation to a dedicated function called before deployment.
-
-2. **Add Pre-deployment Validation**: Create a validation function that checks all required environment variables and Docker state before starting deployment.
-
-3. **Simplify TLS Setup**: Reduce complexity in multi-server TLS setup with better error handling and user guidance.
-
-4. **Add Rollback Capability**: Implement cleanup functions for failed deployments.
-
-#### Code Quality
-1. **Reduce Function Complexity**: Break down large functions like `deploy_core()` into smaller, testable units.
-
-2. **Add Logging**: Increase verbosity for debugging deployment issues.
-
-3. **Configuration Management**: Consider using a configuration file format (YAML/JSON) instead of .env for complex setups.
-
-### Testing Recommendations
-1. **Unit Test Functions**: Test individual functions like `replace_env_placeholders()` and `generate_shared_ca()` in isolation.
-
-2. **Integration Testing**: Test each deployment option in a clean environment.
-
-3. **Error Scenario Testing**: Test failure modes (missing Docker, network issues, invalid credentials).
-
-### Conclusion
-The `ez-homelab.sh` script is a solid foundation for automated homelab deployment, but Option 2 (Core Only) has critical issues with Authelia secret management that prevent reliable deployment. The script needs focused improvements in error handling, validation, and secret generation to achieve the reliability required for critical infrastructure deployment.

docs/service-docs/authelia.md

@@ -17,10 +17,12 @@
 
 **Category:** Core Infrastructure  
 **Docker Image:** [authelia/authelia](https://hub.docker.com/r/authelia/authelia)  
-**Default Stack:** `core.yml`  
+**Default Stack:** `core` (deployed on core server only)  
 **Web UI:** `https://auth.${DOMAIN}`  
 **Authentication:** Self-authenticating (login portal)
 
+**Multi-Server Note:** Authelia runs only on the core server and provides centralized SSO authentication for all services across all servers. Remote server services use the `authelia@docker` middleware to authenticate against the core server's Authelia instance.
+
 ## What is Authelia?
 
 Authelia is an open-source authentication and authorization server providing single sign-on (SSO) and two-factor authentication (2FA) for your applications via a web portal. It acts as a gatekeeper between Traefik and your services.

docs/service-docs/duckdns.md

@@ -15,10 +15,12 @@
 
 **Category:** Core Infrastructure  
 **Docker Image:** [linuxserver/duckdns](https://hub.docker.com/r/linuxserver/duckdns)  
-**Default Stack:** `core.yml`  
+**Default Stack:** `core` (deployed on core server only)  
 **Web UI:** No web interface (runs silently)  
 **Authentication:** Not applicable
 
+**Multi-Server Note:** DuckDNS runs only on the core server where Traefik generates the wildcard SSL certificate. This single certificate is used for all services across all servers in your homelab.
+
 ## What is DuckDNS?
 
 DuckDNS is a free dynamic DNS (DDNS) service that provides you with a memorable subdomain under `duckdns.org` and keeps it updated with your current IP address. It's perfect for homelabs where your ISP provides a dynamic IP address that changes periodically.

docs/service-docs/sablier.md

@@ -15,17 +15,25 @@
 
 ## Overview
 
-**Category:** Core Infrastructure
+**Category:** Resource Management
-**Docker Image:** [sablierapp/sablier](https://hub.docker.com/r/sablierapp/sablier)
+**Docker Image:** [acouvreur/sablier](https://hub.docker.com/r/acouvreur/sablier)
-**Default Stack:** `core.yml`
+**Default Stack:** `sablier` (separate stack, deployed on each server)
-**Web UI:** No web UI (API only)
+**Web UI:** `https://sablier.${DOMAIN}` (on core server)
-**Authentication:** None required
+**Authentication:** Protected by Authelia (SSO)
 **Purpose:** On-demand container startup and resource management
 
+**Multi-Server Note:** Each server runs its own Sablier instance that only manages local containers. This eliminates the need for remote Docker API connections and creates a more resilient architecture.
+
 ## What is Sablier?
 
 Sablier is a lightweight service that enables lazy loading for Docker containers. It automatically starts containers when they're accessed through Traefik and stops them after a period of inactivity, helping to conserve system resources and reduce power consumption.
 
+**In EZ-Homelab's multi-server architecture**, each server runs its own Sablier instance managing only local containers, providing:
+- **Decentralized Control**: No single point of failure
+- **Simplified Networking**: No remote Docker API connections needed
+- **Better Security**: Each Sablier only accesses local Docker socket
+- **Independent Operation**: Remote servers function even if core server is down
+
 ### Key Features
 - **On-Demand Startup:** Containers start automatically when accessed
 - **Automatic Shutdown:** Containers stop after configurable inactivity periods
@@ -63,24 +71,43 @@ When a request comes in for a service with Sablier enabled:
 
 ## Configuration in AI-Homelab
 
-Sablier is deployed as part of the core infrastructure stack and requires no additional configuration for basic operation. It automatically discovers services with the appropriate labels.
+Sablier is deployed as a **separate stack** (`/opt/stacks/sablier/`) on each server and requires no additional configuration for basic operation. It automatically discovers services with the appropriate labels.
+
+### Deployment Location
+
+**Core Server:** `/opt/stacks/sablier/docker-compose.yml`  
+**Remote Servers:** `/opt/stacks/sablier/docker-compose.yml`  
+
+Each Sablier instance:
+- Runs independently on its server
+- Connects only to local Docker socket (`/var/run/docker.sock`)
+- Manages containers on the same server only
+- Has its own web dashboard (if enabled)
 
 ### Service Integration
 
 Add these labels to any service that should use lazy loading:
 
+**Local Service (same server as Sablier):**
+
 ```yaml
 services:
   myservice:
     # ... other configuration ...
     labels:
       - "sablier.enable=true"
-      - "sablier.group=core-myservice"  # Optional: group related services
+      - "sablier.group=${SERVER_HOSTNAME}-myservice"  # Include server hostname
+      - "sablier.start-on-demand=true"
       - "traefik.enable=true"
       - "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
       # ... other Traefik labels ...
 ```
 
+**Note on Group Naming:** Always prefix group names with `${SERVER_HOSTNAME}` to avoid conflicts between servers:
+- Core server: `core-myservice`
+- Remote Pi: `pi-myservice`
+- Remote NAS: `nas-myservice`
+
 ### Advanced Configuration
 
 For services requiring custom timeouts or group management:

docs/service-docs/traefik.md

@@ -16,10 +16,12 @@
 
 **Category:** Core Infrastructure  
 **Docker Image:** [traefik](https://hub.docker.com/_/traefik)  
-**Default Stack:** `core.yml`  
+**Default Stack:** `core` (multi-provider on core server), `infrastructure` or separate (local-only on remote servers)  
 **Web UI:** `https://traefik.${DOMAIN}`  
 **Authentication:** Protected by Authelia (SSO)
 
+**Multi-Server Note:** The core server runs a multi-provider Traefik instance that discovers services across all servers. Remote servers run their own local Traefik instances for container discovery only.
+
 ## What is Traefik?
 
 Traefik is a modern HTTP reverse proxy and load balancer designed for microservices and containerized applications. It automatically discovers services and configures routing, making it ideal for Docker environments.
@@ -45,6 +47,7 @@ Traefik is a modern HTTP reverse proxy and load balancer designed for microservi
 
 ## How It Works
 
+**Single Server Architecture:**
 ```
 Internet → Your Domain → Router (Port 80/443) → Traefik
                                                    ├→ Plex (plex.domain.com)
@@ -53,6 +56,18 @@ Internet → Your Domain → Router (Port 80/443) → Traefik
                                                    └→ [Other Services]
 ```
 
+**Multi-Server Architecture:**
+```
+Internet → Router (Port 80/443) → Core Server Traefik (Multi-Provider)
+                                           ├→ Local Services (plex.domain.com)
+                                           ├→ Remote Server 1 via Docker TLS :2376
+                                           │   ├→ Remote Traefik (discovery only)
+                                           │   └→ Services (pi-service.domain.com)
+                                           └→ Remote Server 2 via Docker TLS :2376
+                                               ├→ Remote Traefik (discovery only)
+                                               └→ Services (nas-service.domain.com)
+```
+
 ### Request Flow
 
 1. **User visits** `https://plex.yourdomain.duckdns.org`
@@ -83,6 +98,20 @@ Traefik automatically:
 
 ## Configuration in AI-Homelab
 
+### Multi-Server Deployment
+
+**Core Server (Primary Traefik):**
+- Multi-provider configuration discovers services on all servers
+- Receives all external traffic (ports 80/443 forwarded)
+- Manages SSL certificates for all services
+- Routes traffic to local and remote services
+
+**Remote Servers (Secondary Traefik):**
+- Local Docker provider only (discovers containers on same server)
+- No external ports exposed
+- Services use labels for local discovery
+- Core Traefik connects via Docker TLS (port 2376)
+
 ### Directory Structure
 
 ```
@@ -96,6 +125,8 @@ Traefik automatically:
 
 ### Static Configuration (`traefik.yml`)
 
+**Core Server Configuration (Multi-Provider):**
+
 ```yaml
 api:
   dashboard: true  # Enable web dashboard
@@ -119,20 +150,46 @@ certificatesResolvers:
     acme:
       email: your-email@example.com
       storage: /acme.json
-      # For testing environments: Use Let's Encrypt staging to avoid rate limits
-      # caServer: https://acme-staging-v02.api.letsencrypt.org/directory
       dnsChallenge:
         provider: duckdns
-        # Note: Explicit resolvers can cause DNS propagation check failures
+
-        # Remove resolvers to use system's DNS for better DuckDNS TXT record resolution
+providers:
+  docker:
+    # Local Docker provider
+    endpoint: "unix:///var/run/docker.sock"
+    exposedByDefault: false
+    network: traefik-network
+  
+  # Optional: Additional Docker providers for remote servers
+  # Uncomment and configure for multi-server setup
+  # docker:
+  #   endpoint: "tcp://remote-server-ip:2376"
+  #   tls:
+  #     ca: /path/to/ca.pem
+  #     cert: /path/to/cert.pem
+  #     key: /path/to/key.pem
+  #   exposedByDefault: false
+  
+  file:
+    directory: /dynamic
+    watch: true
+```
+
+**Remote Server Configuration (Local Only):**
+
+```yaml
+api:
+  dashboard: false  # Disable dashboard on remote servers
+
+entryPoints:
+  websecure:
+    address: ":443"  # Internal only, not exposed
 
 providers:
   docker:
     endpoint: "unix:///var/run/docker.sock"
     exposedByDefault: false
-  file:
+    network: traefik-network
-    directory: /dynamic
-    watch: true
 ```
 
 ### Environment Variables
@@ -144,6 +201,8 @@ ACME_EMAIL=your-email@example.com
 
 ### Service Labels Example
 
+**For Local Services (same server as Traefik):**
+
 ```yaml
 services:
   myservice:
@@ -159,6 +218,42 @@ services:
       - traefik-network
 ```
 
+**For Remote Services (different server):**
+
+Don't use Traefik labels. Instead, create an external host file on the core server:
+
+```yaml
+# /opt/stacks/core/traefik/dynamic/external-host-remoteserver.yml
+http:
+  routers:
+    myservice-remote:
+      rule: "Host(`myservice.${DOMAIN}`)"
+      entryPoints:
+        - websecure
+      service: myservice-remote
+      tls:
+        certResolver: letsencrypt
+      middlewares:
+        - authelia@docker
+  
+  services:
+    myservice-remote:
+      loadBalancer:
+        servers:
+          - url: "http://remote-server-ip:8080"
+        passHostHeader: true
+```
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
+      - "traefik.http.routers.myservice.entrypoints=websecure"
+      - "traefik.http.routers.myservice.tls=true"  # Uses wildcard cert automatically
+      - "traefik.http.routers.myservice.middlewares=authelia@docker"
+      - "traefik.http.services.myservice.loadbalancer.server.port=8080"
+    networks:
+      - traefik-network
+```
+
 ## Official Resources
 
 - **Website:** https://traefik.io

docs/services-overview-clean.md

@@ -1,235 +0,0 @@
-# Services Overview
-
-This document provides a comprehensive overview of all 50+ pre-configured services available in the AI-Homelab repository.
-
-## Services Overview
-
-| Stacks (12) | Services (70 + 6db) | SSO | Storage | Access URLs |
-|-------|----------|-----|---------|-------------|
-| **📦 core.yaml (4)** | **Deploy First** | | | |
-| ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI |
-| ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} |
-| ├─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
-| └─ Sablier | Lazy loading service | - | /opt/stacks/core/sablier | No UI |
-| **🔒 vpn.yaml (2)** | **VPN Services** | | | |
-| ├─ Gluetun | VPN (Surfshark) | - | /opt/stacks/vpn/gluetun | No UI |
-| └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} |
-| **🔧 infrastructure.yaml** (6)** | | | | |
-| ├─ Pi-hole | DNS + Ad blocking | ✓ | /opt/stacks/infrastructure | pihole.${DOMAIN} |
-| ├─ Watchtower | Auto container updates | - | /opt/stacks/infrastructure | No UI |
-| ├─ Dozzle | Docker log viewer | ✓ | /opt/stacks/infrastructure | dozzle.${DOMAIN} |
-| ├─ Glances | System monitoring | ✓ | /opt/stacks/infrastructure | glances.${DOMAIN} |
-| ├─ Code Server | VS Code in browser | ✓ | /opt/stacks/infrastructure | code.${DOMAIN} |
-| └─ Docker Proxy | Secure socket access | - | /opt/stacks/infrastructure | No UI |
-| **📊 dashboards.yaml** (2) | | | | |
-| ├─ Homepage | App dashboard (AI cfg) | ✓ | /opt/stacks/dashboards | home.${DOMAIN} |
-| └─ Homarr | Modern dashboard | ✓ | /opt/stacks/dashboards | homarr.${DOMAIN} |
-| **🎬 media.yaml** (2) | | | | |
-| ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} |
-| └─ Calibre-Web | Ebook reader | ✓ | /opt/stacks/media, /mnt/media | calibre.${DOMAIN} |
-| **📺 media-management.yaml** (9) | | | | |
-| ├─ Sonarr | TV automation | ✓ | /opt/stacks/media-management, /mnt/media | sonarr.${DOMAIN} |
-| ├─ Radarr | Movie automation | ✓ | /opt/stacks/media-management, /mnt/media | radarr.${DOMAIN} |
-| ├─ Prowlarr | Indexer manager | ✓ | /opt/stacks/media-management | prowlarr.${DOMAIN} |
-| ├─ Readarr | Ebooks/Audiobooks | ✓ | /opt/stacks/media-management, /mnt/media | readarr.${DOMAIN} |
-| ├─ Lidarr | Music manager | ✓ | /opt/stacks/media-management, /mnt/media | lidarr.${DOMAIN} |
-| ├─ Lazy Librarian | Book automation | ✓ | /opt/stacks/media-management, /mnt/media | lazylibrarian.${DOMAIN} |
-| ├─ Mylar3 | Comic manager | ✓ | /opt/stacks/media-management, /mnt/media | mylar.${DOMAIN} |
-| ├─ Jellyseerr | Media requests | ✓ | /opt/stacks/media-management | jellyseerr.${DOMAIN} |
-| └─ FlareSolverr | Cloudflare bypass | - | /opt/stacks/media-management | No UI |
-| **🔄 transcoders.yaml** (3) | | | | |
-| ├─ Tdarr Server | Transcoding server | ✓ | /opt/stacks/transcoders, /mnt/transcode | tdarr.${DOMAIN} |
-| ├─ Tdarr Node | Transcoding worker | - | /mnt/transcode-cache | No UI |
-| └─ Unmanic | Library optimizer | ✓ | /opt/stacks/transcoders, /mnt/transcode | unmanic.${DOMAIN} |
-| **📖 wikis.yaml** (4) | | | | |
-| ├─ DokuWiki | File-based wiki | ✓ | /opt/stacks/wikis | dokuwiki.${DOMAIN} |
-| ├─ BookStack | Documentation | ✓ | /opt/stacks/wikis | docs.${DOMAIN} |
-| │  └─ bookstack-db | MariaDB | - | /opt/stacks/wikis | No UI |
-| └─ MediaWiki | Wiki platform | ✓ | /opt/stacks/wikis | mediawiki.${DOMAIN} |
-| **🏠 homeassistant.yaml** (6) | | | | |
-| ├─ Home Assistant | HA platform | ✗ | /opt/stacks/homeassistant | ha.${DOMAIN} |
-| ├─ ESPHome | ESP firmware mgr | ✓ | /opt/stacks/homeassistant | esphome.${DOMAIN} |
-| ├─ TasmoAdmin | Tasmota device mgr | ✓ | /opt/stacks/homeassistant | tasmoadmin.${DOMAIN} |
-| ├─ Node-RED | Automation flows | ✓ | /opt/stacks/homeassistant | nodered.${DOMAIN} |
-| ├─ Mosquitto | MQTT broker | - | /opt/stacks/homeassistant | Ports 1883, 9001 |
-| └─ Zigbee2MQTT | Zigbee bridge | ✓ | /opt/stacks/homeassistant | zigbee2mqtt.${DOMAIN} |
-| **💼 productivity.yaml** (8 + 6 DBs) | | | | |
-| ├─ Nextcloud | File sync platform | ✓ | /opt/stacks/productivity, /mnt/nextcloud | nextcloud.${DOMAIN} |
-| │  └─ nextcloud-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| ├─ Mealie | Recipe manager | ✗ | /opt/stacks/productivity | mealie.${DOMAIN} |
-| ├─ WordPress | Blog platform | ✗ | /opt/stacks/productivity | blog.${DOMAIN} |
-| │  └─ wordpress-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| ├─ Gitea | Git service | ✓ | /opt/stacks/productivity, /mnt/git | git.${DOMAIN} |
-| │  └─ gitea-db | PostgreSQL | - | /opt/stacks/productivity | No UI |
-| └─ Jupyter Lab | Notebooks | ✓ | /opt/stacks/productivity | jupyter.${DOMAIN} |
-| **🛠️ utilities.yaml** (5) | | | | |
-| ├─ Vaultwarden | Password manager | ✗ | /opt/stacks/utilities | bitwarden.${DOMAIN} |
-| ├─ Backrest | Backup (restic) | ✓ | /opt/stacks/utilities, /mnt/backups | backrest.${DOMAIN} |
-| ├─ Duplicati | Encrypted backups | ✓ | /opt/stacks/utilities, /mnt/backups | duplicati.${DOMAIN} |
-| ├─ Form.io | Form builder | ✓ | /opt/stacks/utilities | forms.${DOMAIN} |
-| │  └─ formio-mongo | MongoDB | - | /opt/stacks/utilities | No UI |
-| └─ Authelia-Redis | Session storage | - | /opt/stacks/utilities | No UI |
-| **📈 monitoring.yaml** (8) | | | | |
-| ├─ Prometheus | Metrics collection | ✓ | /opt/stacks/monitoring | prometheus.${DOMAIN} |
-| ├─ Grafana | Visualization | ✓ | /opt/stacks/monitoring | grafana.${DOMAIN} |
-| ├─ Loki | Log aggregation | - | /opt/stacks/monitoring | Via Grafana |
-| ├─ Promtail | Log shipper | - | /opt/stacks/monitoring | No UI |
-| ├─ Node Exporter | Host metrics | - | /opt/stacks/monitoring | No UI |
-| ├─ cAdvisor | Container metrics | - | /opt/stacks/monitoring | Internal :8080 |
-| └─ Uptime Kuma | Uptime monitoring | ✓ | /opt/stacks/monitoring | status.${DOMAIN} |
-| **🔧 alternatives.yaml** (6) | | | | |
-| ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/alternatives | dockge.${DOMAIN} |
-| ├─ Portainer | Container management | ✓ | /opt/stacks/alternatives | portainer.${DOMAIN} |
-| ├─ Authentik Server | SSO with web UI | ✓ | /opt/stacks/alternatives | authentik.${DOMAIN} |
-| │  ├─ authentik-worker | Background tasks | - | /opt/stacks/alternatives | No UI |
-| │  ├─ authentik-db | PostgreSQL | - | /opt/stacks/alternatives | No UI |
-| │  └─ authentik-redis | Cache/messaging | - | /opt/stacks/alternatives | No UI |
-| └─ Plex | Media server | ✗ | /mnt/media, /mnt/transcode | plex.${DOMAIN} |
-| **🏠 homeassistant.yaml** (7) | | | | |
-| ├─ Home Assistant | HA platform | ✗ | /opt/stacks/homeassistant | ha.${DOMAIN} |
-| ├─ ESPHome | ESP firmware mgr | ✓ | /opt/stacks/homeassistant | esphome.${DOMAIN} |
-| ├─ TasmoAdmin | Tasmota device mgr | ✓ | /opt/stacks/homeassistant | tasmoadmin.${DOMAIN} |
-| ├─ Node-RED | Automation flows | ✓ | /opt/stacks/homeassistant | nodered.${DOMAIN} |
-| ├─ Mosquitto | MQTT broker | - | /opt/stacks/homeassistant | Ports 1883, 9001 |
-| ├─ Zigbee2MQTT | Zigbee bridge | ✓ | /opt/stacks/homeassistant | zigbee2mqtt.${DOMAIN} |
-| └─ MotionEye | Video surveillance | ✓ | /opt/stacks/homeassistant, /mnt/surveillance | motioneye.${DOMAIN} |
-| **💼 productivity.yaml** (8 + 6 DBs) | | | | |
-| ├─ Nextcloud | File sync platform | ✓ | /opt/stacks/productivity, /mnt/nextcloud | nextcloud.${DOMAIN} |
-| │  └─ nextcloud-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| ├─ Mealie | Recipe manager | ✗ | /opt/stacks/productivity | mealie.${DOMAIN} |
-| ├─ WordPress | Blog platform | ✗ | /opt/stacks/productivity | blog.${DOMAIN} |
-| │  └─ wordpress-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| ├─ Gitea | Git service | ✓ | /opt/stacks/productivity, /mnt/git | git.${DOMAIN} |
-| │  └─ gitea-db | PostgreSQL | - | /opt/stacks/productivity | No UI |
-| ├─ DokuWiki | File-based wiki | ✓ | /opt/stacks/productivity | wiki.${DOMAIN} |
-| ├─ BookStack | Documentation | ✓ | /opt/stacks/productivity | docs.${DOMAIN} |
-| │  └─ bookstack-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| ├─ MediaWiki | Wiki platform | ✓ | /opt/stacks/productivity | mediawiki.${DOMAIN} |
-| │  └─ mediawiki-db | MariaDB | - | /opt/stacks/productivity | No UI |
-| └─ Form.io | Form builder | ✓ | /opt/stacks/productivity | forms.${DOMAIN} |
-|    └─ formio-mongo | MongoDB | - | /opt/stacks/productivity | No UI |
-| **🛠️ utilities.yaml** (7) | | | | |
-| ├─ Vaultwarden | Password manager | ✗ | /opt/stacks/utilities | bitwarden.${DOMAIN} |
-| ├─ Backrest | Backup (restic) | ✓ | /opt/stacks/utilities, /mnt/backups | backrest.${DOMAIN} |
-| ├─ Duplicati | Encrypted backups | ✓ | /opt/stacks/utilities, /mnt/backups | duplicati.${DOMAIN} |
-| ├─ Code Server | VS Code in browser | ✓ | /opt/stacks/utilities | code.${DOMAIN} |
-| ├─ Form.io | Form platform | ✓ | /opt/stacks/utilities | forms.${DOMAIN} |
-| │  └─ formio-mongo | MongoDB | - | /opt/stacks/utilities | No UI |
-| └─ Authelia-Redis | Session storage | - | /opt/stacks/utilities | No UI |
-| **📈 monitoring.yaml** (8) | | | | |
-| ├─ Prometheus | Metrics collection | ✓ | /opt/stacks/monitoring | prometheus.${DOMAIN} |
-| ├─ Grafana | Visualization | ✓ | /opt/stacks/monitoring | grafana.${DOMAIN} |
-| ├─ Loki | Log aggregation | - | /opt/stacks/monitoring | Via Grafana |
-| ├─ Promtail | Log shipper | - | /opt/stacks/monitoring | No UI |
-| ├─ Node Exporter | Host metrics | - | /opt/stacks/monitoring | No UI |
-| ├─ cAdvisor | Container metrics | - | /opt/stacks/monitoring | Internal :8080 |
-| └─ Uptime Kuma | Uptime monitoring | ✓ | /opt/stacks/monitoring | status.${DOMAIN} |
-| **👨‍💻 development.yaml** (6) | | | | |
-| ├─ GitLab CE | Git + CI/CD | ✓ | /opt/stacks/development, /mnt/git | gitlab.${DOMAIN} |
-| ├─ PostgreSQL | SQL database | - | /opt/stacks/development | Port 5432 |
-| ├─ Redis | In-memory store | - | /opt/stacks/development | Port 6379 |
-| ├─ pgAdmin | PostgreSQL UI | ✓ | /opt/stacks/development | pgadmin.${DOMAIN} |
-| ├─ Jupyter Lab | Notebooks | ✓ | /opt/stacks/development | jupyter.${DOMAIN} |
-| └─ Code Server | VS Code | ✓ | /opt/stacks/development | code.${DOMAIN} |
-
-**Legend:** ✓ = Protected by SSO | ✗ = Bypasses SSO | - = No web UI
-
-## Quick Deployment Order
-
-1. **Create Networks** (one-time setup)
-   ```bash
-   docker network create traefik-network
-   docker network create homelab-network
-   docker network create dockerproxy-network
-   ```
-
-2. **Deploy Core Stack** (required first)
-   ```bash
-   cd /opt/stacks/core/
-   docker compose up -d
-   ```
-
-3. **Deploy Infrastructure**
-   ```bash
-   cd /opt/stacks/infrastructure/
-   docker compose up -d
-   ```
-
-4. **Deploy Dashboards**
-   ```bash
-   cd /opt/stacks/dashboards/
-   docker compose up -d
-   ```
-
-5. **Deploy Additional Stacks** (as needed)
-   - Media: `/opt/stacks/media/`
-   - Media Management: `/opt/stacks/media-management/`
-   - Transcoders: `/opt/stacks/transcoders/`
-   - Wikis: `/opt/stacks/wikis/`
-   - Home Automation: `/opt/stacks/homeassistant/`
-   - Productivity: `/opt/stacks/productivity/`
-   - Utilities: `/opt/stacks/utilities/`
-   - Monitoring: `/opt/stacks/monitoring/`
-   - Alternatives: `/opt/stacks/alternatives/`
-
-## Toggling SSO (Authelia) On/Off
-
-You can easily enable or disable SSO protection for any service by modifying its Traefik labels in the docker-compose.yml file.
-
-### To Enable SSO on a Service
-
-Add the Authelia middleware to the service's Traefik labels:
-
-```yaml
-labels:
-  - "traefik.enable=true"
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Add this line
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-### To Disable SSO on a Service
-
-Comment out (don't remove) the middleware line:
-
-```yaml
-labels:
-  - "traefik.enable=true"
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  # - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Commented out (not removed)
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-After making changes, redeploy the service:
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose up -d
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml up -d
-```
-
-**Stopping a Service:**
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose down
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml down
-```
-
-**Use Cases for Development/Production:**
-- **Security First**: All services start with SSO enabled by default for maximum security
-- **Development**: Keep SSO enabled to protect services during testing
-- **Production**: Disable SSO only for services needing direct app/API access (Plex, Jellyfin)
-- **Gradual Exposure**: Comment out SSO only when ready to expose a service
-- **Quick Toggle**: AI assistant can modify these labels automatically when you ask
-

docs/services-overview.md

@@ -6,22 +6,18 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 
 | Stacks (12) | Services (50 + 6db) | SSO | Storage | Access URLs |
 |-------|----------|-----|---------|-------------|
-| **🔀 alternatives.yaml** (6 + 3 DBs) | | | | |
+| **📦 core.yaml (3)** | **Deploy First** | | | |
-| ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/alternatives | dockge.${DOMAIN} |
-| ├─ Portainer | Container management | ✓ | /opt/stacks/alternatives | portainer.${DOMAIN} |
-| ├─ Authentik Server | SSO with web UI | ✓ | /opt/stacks/alternatives | authentik.${DOMAIN} |
-| │  ├─ authentik-worker | Background tasks | - | /opt/stacks/alternatives | No UI |
-| │  ├─ authentik-db | PostgreSQL | - | /opt/stacks/alternatives | No UI |
-| │  └─ authentik-redis | Cache/messaging | - | /opt/stacks/alternatives | No UI |
-| └─ Plex | Media server | ✗ | /mnt/media, /mnt/transcode | plex.${DOMAIN} |
-| **📦 core.yaml (4)** | **Deploy First** | | | |
 | ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI |
 | ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} |
-| ├─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
+| └─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
-| └─ Sablier | Lazy loading service | - | /opt/stacks/core/sablier | No UI |
+|  
+| **🔄 sablier.yaml (1)** | **Deploy on Each Server** | | | |
+| └─ Sablier | Lazy loading service | - | /opt/stacks/sablier | sablier.${DOMAIN} |
+|  
 | **📊 dashboards.yaml** (2) | | | | |
 | ├─ Homepage | App dashboard (AI cfg) | ✓ | /opt/stacks/dashboards | home.${DOMAIN} |
 | └─ Homarr | Modern dashboard | ✓ | /opt/stacks/dashboards | homarr.${DOMAIN} |
+|  
 | **🏠 homeassistant.yaml** (7) | | | | |
 | ├─ Home Assistant | HA platform | ✗ | /opt/stacks/homeassistant | ha.${DOMAIN} |
 | ├─ ESPHome | ESP firmware mgr | ✓ | /opt/stacks/homeassistant | esphome.${DOMAIN} |
@@ -30,6 +26,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Mosquitto | MQTT broker | - | /opt/stacks/homeassistant | Ports 1883, 9001 |
 | ├─ Zigbee2MQTT | Zigbee bridge | ✓ | /opt/stacks/homeassistant | zigbee2mqtt.${DOMAIN} |
 | └─ MotionEye | Video surveillance | ✓ | /opt/stacks/homeassistant, /mnt/surveillance | motioneye.${DOMAIN} |
+|  
 | **🔧 infrastructure.yaml** (6)** | | | | |
 | ├─ Pi-hole | DNS + Ad blocking | ✓ | /opt/stacks/infrastructure | pihole.${DOMAIN} |
 | ├─ Watchtower | Auto container updates | - | /opt/stacks/infrastructure | No UI |
@@ -37,6 +34,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Glances | System monitoring | ✓ | /opt/stacks/infrastructure | glances.${DOMAIN} |
 | ├─ Code Server | VS Code in browser | ✓ | /opt/stacks/infrastructure | code.${DOMAIN} |
 | └─ Docker Proxy | Secure socket access | - | /opt/stacks/infrastructure | No UI |
+|  
 | **📺 media-management.yaml** (9) | | | | |
 | ├─ Sonarr | TV automation | ✓ | /opt/stacks/media-management, /mnt/media | sonarr.${DOMAIN} |
 | ├─ Radarr | Movie automation | ✓ | /opt/stacks/media-management, /mnt/media | radarr.${DOMAIN} |
@@ -47,9 +45,11 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Mylar3 | Comic manager | ✓ | /opt/stacks/media-management, /mnt/media | mylar.${DOMAIN} |
 | ├─ Jellyseerr | Media requests | ✓ | /opt/stacks/media-management | jellyseerr.${DOMAIN} |
 | └─ FlareSolverr | Cloudflare bypass | - | /opt/stacks/media-management | No UI |
+|  
 | **🎬 media.yaml** (2) | | | | |
 | ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} |
 | └─ Calibre-Web | Ebook reader | ✓ | /opt/stacks/media, /mnt/media | calibre.${DOMAIN} |
+|  
 | **📈 monitoring.yaml** (8) | | | | |
 | ├─ Prometheus | Metrics collection | ✓ | /opt/stacks/monitoring | prometheus.${DOMAIN} |
 | ├─ Grafana | Visualization | ✓ | /opt/stacks/monitoring | grafana.${DOMAIN} |
@@ -58,6 +58,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Node Exporter | Host metrics | - | /opt/stacks/monitoring | No UI |
 | ├─ cAdvisor | Container metrics | - | /opt/stacks/monitoring | Internal :8080 |
 | └─ Uptime Kuma | Uptime monitoring | ✓ | /opt/stacks/monitoring | status.${DOMAIN} |
+|  
 | **💼 productivity.yaml** (5 + 4 DBs) | | | | |
 | ├─ Nextcloud | File sync platform | ✓ | /opt/stacks/productivity, /mnt/nextcloud | nextcloud.${DOMAIN} |
 | │  └─ nextcloud-db | MariaDB | - | /opt/stacks/productivity | No UI |
@@ -67,10 +68,12 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Gitea | Git service | ✓ | /opt/stacks/productivity, /mnt/git | git.${DOMAIN} |
 | │  └─ gitea-db | PostgreSQL | - | /opt/stacks/productivity | No UI |
 | └─ Jupyter Lab | Notebooks | ✓ | /opt/stacks/productivity | jupyter.${DOMAIN} |
+|  
 | **🔄 transcoders.yaml** (3) | | | | |
 | ├─ Tdarr Server | Transcoding server | ✓ | /opt/stacks/transcoders, /mnt/transcode | tdarr.${DOMAIN} |
 | ├─ Tdarr Node | Transcoding worker | - | /mnt/transcode-cache | No UI |
 | └─ Unmanic | Library optimizer | ✓ | /opt/stacks/transcoders, /mnt/transcode | unmanic.${DOMAIN} |
+|  
 | **🛠️ utilities.yaml** (7) | | | | |
 | ├─ Vaultwarden | Password manager | ✗ | /opt/stacks/utilities | bitwarden.${DOMAIN} |
 | ├─ Backrest | Backup (restic) | ✓ | /opt/stacks/utilities, /mnt/backups | backrest.${DOMAIN} |
@@ -79,151 +82,55 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Form.io | Form platform | ✓ | /opt/stacks/utilities | forms.${DOMAIN} |
 | │  └─ formio-mongo | MongoDB | - | /opt/stacks/utilities | No UI |
 | └─ Authelia-Redis | Session storage | - | /opt/stacks/utilities | No UI |
+|  
 | **🔒 vpn.yaml (2)** | **VPN Services** | | | |
 | ├─ Gluetun | VPN (Surfshark) | - | /opt/stacks/vpn/gluetun | No UI |
 | └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} |
+|  
 | **📖 wikis.yaml** (4) | | | | |
 | ├─ DokuWiki | File-based wiki | ✓ | /opt/stacks/wikis | dokuwiki.${DOMAIN} |
 | ├─ BookStack | Documentation | ✓ | /opt/stacks/wikis | docs.${DOMAIN} |
 | │  └─ bookstack-db | MariaDB | - | /opt/stacks/wikis | No UI |
 | └─ MediaWiki | Wiki platform | ✓ | /opt/stacks/wikis | mediawiki.${DOMAIN} |
+|  
+| **🔀 alternatives.yaml** (6 + 3 DBs) | | | | |
+| ├─ Portainer | Container management | ✓ | /opt/stacks/alternatives | portainer.${DOMAIN} |
+| ├─ Authentik Server | SSO with web UI | ✓ | /opt/stacks/alternatives | authentik.${DOMAIN} |
+| │  ├─ authentik-worker | Background tasks | - | /opt/stacks/alternatives | No UI |
+| │  ├─ authentik-db | PostgreSQL | - | /opt/stacks/alternatives | No UI |
+| │  └─ authentik-redis | Cache/messaging | - | /opt/stacks/alternatives | No UI |
+| └─ Plex | Media server | ✗ | /mnt/media, /mnt/transcode | plex.${DOMAIN} |
 
 **Legend:** ✓ = Protected by SSO | ✗ = Bypasses SSO | - = No web UI
 
-## Quick Deployment Order
+## Service Configuration
 
-1. **Create Networks** (one-time setup)
+Some services require no initial configuration (ie. creates it on first run if needed)
-   ```bash
-   docker network create traefik-network
-   docker network create homelab-network
-   docker network create dockerproxy-network
-   ```
 
-2. **Deploy Core Stack** (required first)
+### Other services have config files/folders in the stack folder for easy deployment.  
-   ```bash
+* These typicaly use values from variables in .env however **can not access the variables.**  
-   cd /opt/stacks/core/
+* `ez-homelab.sh` handles variable replacement on deployment.    
-   docker compose up -d
+* Each stack folder contains a `deploy-stacksname.sh` that will do the same for that stack.
-   ```
 
-3. **Deploy Infrastructure**
-   ```bash
-   cd /opt/stacks/infrastructure/
-   docker compose up -d
-   ```
 
-4. **Deploy Dashboards**
+## Multi-Server Deployment Notes
-   ```bash
-   cd /opt/stacks/dashboards/
-   docker compose up -d
-   ```
 
-5. **Deploy Additional Stacks** (as needed)
+### Core Server `There can be only one`
-   - Alternatives: `/opt/stacks/alternatives/`
-   - Core: `/opt/stacks/core/` (deploy first)
-   - Dashboards: `/opt/stacks/dashboards/`
-   - Home Automation: `/opt/stacks/homeassistant/`
-   - Infrastructure: `/opt/stacks/infrastructure/`
-   - Media: `/opt/stacks/media/`
-   - Media Management: `/opt/stacks/media-management/`
-   - Monitoring: `/opt/stacks/monitoring/`
-   - Productivity: `/opt/stacks/productivity/`
-   - Transcoders: `/opt/stacks/transcoders/`
-   - Utilities: `/opt/stacks/utilities/`
-   - VPN: `/opt/stacks/vpn/`
-   - Wikis: `/opt/stacks/wikis/`
 
-## Toggling SSO (Authelia) On/Off
+`Forward ports 80 & 443 from your router`
 
-You can easily enable or disable SSO protection for any service by modifying its Traefik labels in the docker-compose.yml file.
+- **DuckDNS**: Dynamic DNS and SSL certificate management
+- **Authelia**: Centralized SSO authentication for all servers
+- **Traefik** (core): Multi-provider configuration to route to all servers
 
-### To Enable SSO on a Service
+### Remote Server Services (Deploy on Each Server)
+- **Traefik** (local): Discovers containers through labels
+- **Sablier**: Manages lazy loading for local containers only
+- **Dockge**: Stack management interface
 
-Add the Authelia middleware to the service's Traefik labels:
+### Architecture Overview
-
+- **TLS Communication**: Remote servers connect via Docker TLS (port 2376)
-```yaml
+- **Unified Access**: All services accessible through core server domain
-labels:
+- **Service Routing**: Having Traefik on every server enables label based service discovery
-  - "traefik.enable=true"
+- **Lazy Loading**: Having Sablier on every server enables label based lazy loading configuration 
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Add this line
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-### To Disable SSO on a Service
-
-Comment out (don't remove) the middleware line:
-
-```yaml
-labels:
-  - "traefik.enable=true"
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  # - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Commented out (not removed)
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-After making changes, redeploy the service:
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose up -d
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml up -d
-```
-
-**Stopping a Service:**
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose down
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml down
-```
-
-**Use Cases for Development/Production:**
-- **Security First**: All services start with SSO enabled by default for maximum security
-- **Development**: Keep SSO enabled to protect services during testing
-- **Production**: Disable SSO only for services needing direct app/API access (Plex, Jellyfin)
-- **Gradual Exposure**: Comment out SSO only when ready to expose a service
-- **Quick Toggle**: AI assistant can modify these labels automatically when you ask
-
-## Storage Recommendations
-
-| Data Type | Recommended Location | Reason |
-|-----------|---------------------|--------|
-| Configuration files | `/opt/stacks/stack-name/` | Easy access, version control |
-| Small databases (< 10GB) | `/opt/stacks/stack-name/db/` | Manageable on system drive |
-| Media files (movies, TV, music) | `/mnt/media/` | Large, continuous growth |
-| Downloads | `/mnt/downloads/` | Temporary, high throughput |
-| Backups | `/mnt/backups/` | Large, separate from system |
-| Surveillance footage | `/mnt/surveillance/` | Continuous recording |
-| Large databases (> 10GB) | `/mnt/databases/` | Growth over time |
-| Transcoding cache | `/mnt/transcode-cache/` | High I/O, large temporary files |
-| Git repositories | `/mnt/git/` | Can grow large |
-| Nextcloud data | `/mnt/nextcloud/` | User files, photos |
-
-## Configuration Templates
-
-All configuration templates are available in `config-templates/`:
-- `traefik/` - Static and dynamic Traefik configuration
-- `authelia/` - Complete Authelia setup with user database
-- `homepage/` - Dashboard services, widgets, and Docker integration
-- `prometheus/` - Metrics scrape configurations
-- `loki/` - Log aggregation settings
-- `promtail/` - Log shipping configuration
-- `redis/` - Redis server configuration
-
-## Additional Resources
-
-- **Getting Started**: See [docs/getting-started.md](getting-started.md) for detailed deployment
-- **Docker Guidelines**: See [docs/docker-guidelines.md](docker-guidelines.md) for management patterns
-- **Quick Reference**: See [docs/quick-reference.md](quick-reference.md) for common commands
-- **Proxying External Hosts**: See [docs/proxying-external-hosts.md](proxying-external-hosts.md) for Raspberry Pi, NAS, etc.
-- **AI Assistant**: Use GitHub Copilot in VS Code with `.github/copilot-instructions.md` for intelligent homelab management
 

docs/todo.md

@@ -1,56 +0,0 @@
-# EZ-Homelab Script Fixes & Improvements
-
-## Critical Fixes (Implement First)
-- [x] **Secret Validation**: Add validation after `save_env_file()` to ensure Authelia secrets exist before deployment
-- [x] **Better Placeholder Error Handling**: Make `replace_env_placeholders()` fail deployment if critical variables are missing
-- [x] **Debug Logging**: Add toggleable comprehensive logging to file for troubleshooting
-- [x] **Simplified Placeholder Logic**: Streamline the replacement process in `deploy_core()`
-- [x] **Standardized .env Placeholders**: Update .env.example and .env with consistent placeholder format
-- [x] **File Permission Issues**: Fix ownership problems when copying files as root then accessing as user
-- [x] **REMOTE_SERVER_HOSTNAME Error**: Remove multi-server config files from core deployments to prevent critical errors
-- [x] **Docker Compose Variable Expansion**: Remove AUTHELIA_ADMIN_PASSWORD from core stack .env to prevent argon2id hash expansion warnings
-
-## High Priority Issues
-- [ ] **Authelia Password Hash Generation Reliability**
-  - Issue: Docker-based password hash generation can fail if Docker isn't ready or Authelia image pull fails
-  - Impact: Deployment fails with cryptic errors
-  - Fix: Add retry logic and fallback to local hash generation
-
-- [x] **Environment Variable Persistence Issues**
-  - Issue: Timing issues with when .env is sourced vs when variables are needed
-  - Impact: Variables not available when functions expect them
-  - Fix: Implemented safe .env loading that doesn't expand special characters + filtered .env files per stack
-
-## Medium Priority Issues
-- [ ] **Multi-Server TLS Setup Complexity**
-  - Issue: Complex SSH authentication logic with multiple failure points
-  - Impact: TLS setup often fails, preventing remote Docker access
-  - Fix: Simplify to use SSH config files and better error messages
-
-- [ ] **Directory Permission Race Conditions**
-  - Issue: Script creates directories with sudo then writes as regular user
-  - Impact: Permission conflicts during file operations
-  - Fix: Consistent ownership handling throughout
-
-- [ ] **Missing Pre-deployment Validation**
-  - Issue: No comprehensive checks before starting deployment
-  - Impact: Failures occur mid-deployment after time investment
-  - Fix: Add validation phase checking Docker, networks, environment
-
-## Low Priority Issues
-- [ ] **Function Complexity**
-  - Issue: Large functions like `deploy_core()` and `prompt_for_values()` are hard to test/debug
-  - Impact: Bugs are harder to isolate and fix
-  - Fix: Break down into smaller, focused functions
-
-- [ ] **No Rollback Capability**
-  - Issue: Failed deployments leave partial state
-  - Impact: Manual cleanup required, risk of inconsistent state
-  - Fix: Add cleanup functions for failed deployments
-
-## Implementation Notes
-- Start with Critical Fixes to make Option 2 deployment reliable
-- Test each fix individually before moving to next
-- Use debug logging to validate fixes work correctly
-- Update documentation after each major change
-- Consider backward compatibility with existing deployments

get-docker.sh

@@ -1,764 +0,0 @@
-#!/bin/sh
-set -e
-# Docker Engine for Linux installation script.
-#
-# This script is intended as a convenient way to configure docker's package
-# repositories and to install Docker Engine, This script is not recommended
-# for production environments. Before running this script, make yourself familiar
-# with potential risks and limitations, and refer to the installation manual
-# at https://docs.docker.com/engine/install/ for alternative installation methods.
-#
-# The script:
-#
-# - Requires `root` or `sudo` privileges to run.
-# - Attempts to detect your Linux distribution and version and configure your
-#   package management system for you.
-# - Doesn't allow you to customize most installation parameters.
-# - Installs dependencies and recommendations without asking for confirmation.
-# - Installs the latest stable release (by default) of Docker CLI, Docker Engine,
-#   Docker Buildx, Docker Compose, containerd, and runc. When using this script
-#   to provision a machine, this may result in unexpected major version upgrades
-#   of these packages. Always test upgrades in a test environment before
-#   deploying to your production systems.
-# - Isn't designed to upgrade an existing Docker installation. When using the
-#   script to update an existing installation, dependencies may not be updated
-#   to the expected version, resulting in outdated versions.
-#
-# Source code is available at https://github.com/docker/docker-install/
-#
-# Usage
-# ==============================================================================
-#
-# To install the latest stable versions of Docker CLI, Docker Engine, and their
-# dependencies:
-#
-# 1. download the script
-#
-#   $ curl -fsSL https://get.docker.com -o install-docker.sh
-#
-# 2. verify the script's content
-#
-#   $ cat install-docker.sh
-#
-# 3. run the script with --dry-run to verify the steps it executes
-#
-#   $ sh install-docker.sh --dry-run
-#
-# 4. run the script either as root, or using sudo to perform the installation.
-#
-#   $ sudo sh install-docker.sh
-#
-# Command-line options
-# ==============================================================================
-#
-# --version <VERSION>
-# Use the --version option to install a specific version, for example:
-#
-#   $ sudo sh install-docker.sh --version 23.0
-#
-# --channel <stable|test>
-#
-# Use the --channel option to install from an alternative installation channel.
-# The following example installs the latest versions from the "test" channel,
-# which includes pre-releases (alpha, beta, rc):
-#
-#   $ sudo sh install-docker.sh --channel test
-#
-# Alternatively, use the script at https://test.docker.com, which uses the test
-# channel as default.
-#
-# --mirror <Aliyun|AzureChinaCloud>
-#
-# Use the --mirror option to install from a mirror supported by this script.
-# Available mirrors are "Aliyun" (https://mirrors.aliyun.com/docker-ce), and
-# "AzureChinaCloud" (https://mirror.azure.cn/docker-ce), for example:
-#
-#   $ sudo sh install-docker.sh --mirror AzureChinaCloud
-#
-# --setup-repo
-#
-# Use the --setup-repo option to configure Docker's package repositories without
-# installing Docker packages. This is useful when you want to add the repository
-# but install packages separately:
-#
-#   $ sudo sh install-docker.sh --setup-repo
-#
-# Automatic Service Start
-#
-# By default, this script automatically starts the Docker daemon and enables the docker
-# service after installation if systemd is used as init.
-#
-# If you prefer to start the service manually, use the --no-autostart option:
-#
-#   $ sudo sh install-docker.sh --no-autostart
-#
-# Note: Starting the service requires appropriate privileges to manage system services.
-#
-# ==============================================================================
-
-
-# Git commit from https://github.com/docker/docker-install when
-# the script was uploaded (Should only be modified by upload job):
-SCRIPT_COMMIT_SHA="f381ee68b32e515bb4dc034b339266aff1fbc460"
-
-# strip "v" prefix if present
-VERSION="${VERSION#v}"
-
-# The channel to install from:
-#   * stable
-#   * test
-DEFAULT_CHANNEL_VALUE="stable"
-if [ -z "$CHANNEL" ]; then
-	CHANNEL=$DEFAULT_CHANNEL_VALUE
-fi
-
-DEFAULT_DOWNLOAD_URL="https://download.docker.com"
-if [ -z "$DOWNLOAD_URL" ]; then
-	DOWNLOAD_URL=$DEFAULT_DOWNLOAD_URL
-fi
-
-DEFAULT_REPO_FILE="docker-ce.repo"
-if [ -z "$REPO_FILE" ]; then
-	REPO_FILE="$DEFAULT_REPO_FILE"
-	# Automatically default to a staging repo fora
-	# a staging download url (download-stage.docker.com)
-	case "$DOWNLOAD_URL" in
-		*-stage*) REPO_FILE="docker-ce-staging.repo";;
-	esac
-fi
-
-mirror=''
-DRY_RUN=${DRY_RUN:-}
-REPO_ONLY=${REPO_ONLY:-0}
-NO_AUTOSTART=${NO_AUTOSTART:-0}
-while [ $# -gt 0 ]; do
-	case "$1" in
-		--channel)
-			CHANNEL="$2"
-			shift
-			;;
-		--dry-run)
-			DRY_RUN=1
-			;;
-		--mirror)
-			mirror="$2"
-			shift
-			;;
-		--version)
-			VERSION="${2#v}"
-			shift
-			;;
-		--setup-repo)
-			REPO_ONLY=1
-			shift
-			;;
-		--no-autostart)
-			NO_AUTOSTART=1
-			;;
-		--*)
-			echo "Illegal option $1"
-			;;
-	esac
-	shift $(( $# > 0 ? 1 : 0 ))
-done
-
-case "$mirror" in
-	Aliyun)
-		DOWNLOAD_URL="https://mirrors.aliyun.com/docker-ce"
-		;;
-	AzureChinaCloud)
-		DOWNLOAD_URL="https://mirror.azure.cn/docker-ce"
-		;;
-	"")
-		;;
-	*)
-		>&2 echo "unknown mirror '$mirror': use either 'Aliyun', or 'AzureChinaCloud'."
-		exit 1
-		;;
-esac
-
-case "$CHANNEL" in
-	stable|test)
-		;;
-	*)
-		>&2 echo "unknown CHANNEL '$CHANNEL': use either stable or test."
-		exit 1
-		;;
-esac
-
-command_exists() {
-	command -v "$@" > /dev/null 2>&1
-}
-
-# version_gte checks if the version specified in $VERSION is at least the given
-# SemVer (Maj.Minor[.Patch]), or CalVer (YY.MM) version.It returns 0 (success)
-# if $VERSION is either unset (=latest) or newer or equal than the specified
-# version, or returns 1 (fail) otherwise.
-#
-# examples:
-#
-# VERSION=23.0
-# version_gte 23.0  // 0 (success)
-# version_gte 20.10 // 0 (success)
-# version_gte 19.03 // 0 (success)
-# version_gte 26.1  // 1 (fail)
-version_gte() {
-	if [ -z "$VERSION" ]; then
-			return 0
-	fi
-	version_compare "$VERSION" "$1"
-}
-
-# version_compare compares two version strings (either SemVer (Major.Minor.Path),
-# or CalVer (YY.MM) version strings. It returns 0 (success) if version A is newer
-# or equal than version B, or 1 (fail) otherwise. Patch releases and pre-release
-# (-alpha/-beta) are not taken into account
-#
-# examples:
-#
-# version_compare 23.0.0 20.10 // 0 (success)
-# version_compare 23.0 20.10   // 0 (success)
-# version_compare 20.10 19.03  // 0 (success)
-# version_compare 20.10 20.10  // 0 (success)
-# version_compare 19.03 20.10  // 1 (fail)
-version_compare() (
-	set +x
-
-	yy_a="$(echo "$1" | cut -d'.' -f1)"
-	yy_b="$(echo "$2" | cut -d'.' -f1)"
-	if [ "$yy_a" -lt "$yy_b" ]; then
-		return 1
-	fi
-	if [ "$yy_a" -gt "$yy_b" ]; then
-		return 0
-	fi
-	mm_a="$(echo "$1" | cut -d'.' -f2)"
-	mm_b="$(echo "$2" | cut -d'.' -f2)"
-
-	# trim leading zeros to accommodate CalVer
-	mm_a="${mm_a#0}"
-	mm_b="${mm_b#0}"
-
-	if [ "${mm_a:-0}" -lt "${mm_b:-0}" ]; then
-		return 1
-	fi
-
-	return 0
-)
-
-is_dry_run() {
-	if [ -z "$DRY_RUN" ]; then
-		return 1
-	else
-		return 0
-	fi
-}
-
-is_wsl() {
-	case "$(uname -r)" in
-	*microsoft* ) true ;; # WSL 2
-	*Microsoft* ) true ;; # WSL 1
-	* ) false;;
-	esac
-}
-
-is_darwin() {
-	case "$(uname -s)" in
-	*darwin* ) true ;;
-	*Darwin* ) true ;;
-	* ) false;;
-	esac
-}
-
-deprecation_notice() {
-	distro=$1
-	distro_version=$2
-	echo
-	printf "\033[91;1mDEPRECATION WARNING\033[0m\n"
-	printf "    This Linux distribution (\033[1m%s %s\033[0m) reached end-of-life and is no longer supported by this script.\n" "$distro" "$distro_version"
-	echo   "    No updates or security fixes will be released for this distribution, and users are recommended"
-	echo   "    to upgrade to a currently maintained version of $distro."
-	echo
-	printf   "Press \033[1mCtrl+C\033[0m now to abort this script, or wait for the installation to continue."
-	echo
-	sleep 10
-}
-
-get_distribution() {
-	lsb_dist=""
-	# Every system that we officially support has /etc/os-release
-	if [ -r /etc/os-release ]; then
-		lsb_dist="$(. /etc/os-release && echo "$ID")"
-	fi
-	# Returning an empty string here should be alright since the
-	# case statements don't act unless you provide an actual value
-	echo "$lsb_dist"
-}
-
-start_docker_daemon() {
-	# Use systemctl if available (for systemd-based systems)
-	if command_exists systemctl; then
-		is_dry_run || >&2 echo "Using systemd to manage Docker service"
-		if (
-			is_dry_run || set -x
-			$sh_c systemctl enable --now docker.service 2>/dev/null
-		); then
-			is_dry_run || echo "INFO: Docker daemon enabled and started" >&2
-		else
-			is_dry_run || echo "WARNING: unable to enable the docker service" >&2
-		fi
-	else
-		# No service management available (container environment)
-		if ! is_dry_run; then
-			>&2 echo "Note: Running in a container environment without service management"
-			>&2 echo "Docker daemon cannot be started automatically in this environment"
-			>&2 echo "The Docker packages have been installed successfully"
-		fi
-	fi
-	>&2 echo
-}
-
-echo_docker_as_nonroot() {
-	if is_dry_run; then
-		return
-	fi
-	if command_exists docker && [ -e /var/run/docker.sock ]; then
-		(
-			set -x
-			$sh_c 'docker version'
-		) || true
-	fi
-
-	# intentionally mixed spaces and tabs here -- tabs are stripped by "<<-EOF", spaces are kept in the output
-	echo
-	echo "================================================================================"
-	echo
-	if version_gte "20.10"; then
-		echo "To run Docker as a non-privileged user, consider setting up the"
-		echo "Docker daemon in rootless mode for your user:"
-		echo
-		echo "    dockerd-rootless-setuptool.sh install"
-		echo
-		echo "Visit https://docs.docker.com/go/rootless/ to learn about rootless mode."
-		echo
-	fi
-	echo
-	echo "To run the Docker daemon as a fully privileged service, but granting non-root"
-	echo "users access, refer to https://docs.docker.com/go/daemon-access/"
-	echo
-	echo "WARNING: Access to the remote API on a privileged Docker daemon is equivalent"
-	echo "         to root access on the host. Refer to the 'Docker daemon attack surface'"
-	echo "         documentation for details: https://docs.docker.com/go/attack-surface/"
-	echo
-	echo "================================================================================"
-	echo
-}
-
-# Check if this is a forked Linux distro
-check_forked() {
-
-	# Check for lsb_release command existence, it usually exists in forked distros
-	if command_exists lsb_release; then
-		# Check if the `-u` option is supported
-		set +e
-		lsb_release -a -u > /dev/null 2>&1
-		lsb_release_exit_code=$?
-		set -e
-
-		# Check if the command has exited successfully, it means we're in a forked distro
-		if [ "$lsb_release_exit_code" = "0" ]; then
-			# Print info about current distro
-			cat <<-EOF
-			You're using '$lsb_dist' version '$dist_version'.
-			EOF
-
-			# Get the upstream release info
-			lsb_dist=$(lsb_release -a -u 2>&1 | tr '[:upper:]' '[:lower:]' | grep -E 'id' | cut -d ':' -f 2 | tr -d '[:space:]')
-			dist_version=$(lsb_release -a -u 2>&1 | tr '[:upper:]' '[:lower:]' | grep -E 'codename' | cut -d ':' -f 2 | tr -d '[:space:]')
-
-			# Print info about upstream distro
-			cat <<-EOF
-			Upstream release is '$lsb_dist' version '$dist_version'.
-			EOF
-		else
-			if [ -r /etc/debian_version ] && [ "$lsb_dist" != "ubuntu" ] && [ "$lsb_dist" != "raspbian" ]; then
-				if [ "$lsb_dist" = "osmc" ]; then
-					# OSMC runs Raspbian
-					lsb_dist=raspbian
-				else
-					# We're Debian and don't even know it!
-					lsb_dist=debian
-				fi
-				dist_version="$(sed 's/\/.*//' /etc/debian_version | sed 's/\..*//')"
-				case "$dist_version" in
-					13|14|forky)
-						dist_version="trixie"
-					;;
-					12)
-						dist_version="bookworm"
-					;;
-					11)
-						dist_version="bullseye"
-					;;
-					10)
-						dist_version="buster"
-					;;
-					9)
-						dist_version="stretch"
-					;;
-					8)
-						dist_version="jessie"
-					;;
-				esac
-			fi
-		fi
-	fi
-}
-
-do_install() {
-	echo "# Executing docker install script, commit: $SCRIPT_COMMIT_SHA"
-
-	if command_exists docker; then
-		cat >&2 <<-'EOF'
-			Warning: the "docker" command appears to already exist on this system.
-
-			If you already have Docker installed, this script can cause trouble, which is
-			why we're displaying this warning and provide the opportunity to cancel the
-			installation.
-
-			If you installed the current Docker package using this script and are using it
-			again to update Docker, you can ignore this message, but be aware that the
-			script resets any custom changes in the deb and rpm repo configuration
-			files to match the parameters passed to the script.
-
-			You may press Ctrl+C now to abort this script.
-		EOF
-		( set -x; sleep 20 )
-	fi
-
-	user="$(id -un 2>/dev/null || true)"
-
-	sh_c='sh -c'
-	if [ "$user" != 'root' ]; then
-		if command_exists sudo; then
-			sh_c='sudo -E sh -c'
-		elif command_exists su; then
-			sh_c='su -c'
-		else
-			cat >&2 <<-'EOF'
-			Error: this installer needs the ability to run commands as root.
-			We are unable to find either "sudo" or "su" available to make this happen.
-			EOF
-			exit 1
-		fi
-	fi
-
-	if is_dry_run; then
-		sh_c="echo"
-	fi
-
-	# perform some very rudimentary platform detection
-	lsb_dist=$( get_distribution )
-	lsb_dist="$(echo "$lsb_dist" | tr '[:upper:]' '[:lower:]')"
-
-	if is_wsl; then
-		echo
-		echo "WSL DETECTED: We recommend using Docker Desktop for Windows."
-		echo "Please get Docker Desktop from https://www.docker.com/products/docker-desktop/"
-		echo
-		cat >&2 <<-'EOF'
-
-			You may press Ctrl+C now to abort this script.
-		EOF
-		( set -x; sleep 20 )
-	fi
-
-	case "$lsb_dist" in
-
-		ubuntu)
-			if command_exists lsb_release; then
-				dist_version="$(lsb_release --codename | cut -f2)"
-			fi
-			if [ -z "$dist_version" ] && [ -r /etc/lsb-release ]; then
-				dist_version="$(. /etc/lsb-release && echo "$DISTRIB_CODENAME")"
-			fi
-		;;
-
-		debian|raspbian)
-			dist_version="$(sed 's/\/.*//' /etc/debian_version | sed 's/\..*//')"
-			case "$dist_version" in
-				13)
-					dist_version="trixie"
-				;;
-				12)
-					dist_version="bookworm"
-				;;
-				11)
-					dist_version="bullseye"
-				;;
-				10)
-					dist_version="buster"
-				;;
-				9)
-					dist_version="stretch"
-				;;
-				8)
-					dist_version="jessie"
-				;;
-			esac
-		;;
-
-		centos|rhel)
-			if [ -z "$dist_version" ] && [ -r /etc/os-release ]; then
-				dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
-			fi
-		;;
-
-		*)
-			if command_exists lsb_release; then
-				dist_version="$(lsb_release --release | cut -f2)"
-			fi
-			if [ -z "$dist_version" ] && [ -r /etc/os-release ]; then
-				dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
-			fi
-		;;
-
-	esac
-
-	# Check if this is a forked Linux distro
-	check_forked
-
-	# Print deprecation warnings for distro versions that recently reached EOL,
-	# but may still be commonly used (especially LTS versions).
-	case "$lsb_dist.$dist_version" in
-		centos.8|centos.7|rhel.7)
-			deprecation_notice "$lsb_dist" "$dist_version"
-			;;
-		debian.buster|debian.stretch|debian.jessie)
-			deprecation_notice "$lsb_dist" "$dist_version"
-			;;
-		raspbian.buster|raspbian.stretch|raspbian.jessie)
-			deprecation_notice "$lsb_dist" "$dist_version"
-			;;
-		ubuntu.focal|ubuntu.bionic|ubuntu.xenial|ubuntu.trusty)
-			deprecation_notice "$lsb_dist" "$dist_version"
-			;;
-		ubuntu.oracular|ubuntu.mantic|ubuntu.lunar|ubuntu.kinetic|ubuntu.impish|ubuntu.hirsute|ubuntu.groovy|ubuntu.eoan|ubuntu.disco|ubuntu.cosmic)
-			deprecation_notice "$lsb_dist" "$dist_version"
-			;;
-		fedora.*)
-			if [ "$dist_version" -lt 41 ]; then
-				deprecation_notice "$lsb_dist" "$dist_version"
-			fi
-			;;
-	esac
-
-	# Run setup for each distro accordingly
-	case "$lsb_dist" in
-		ubuntu|debian|raspbian)
-			pre_reqs="ca-certificates curl"
-			apt_repo="deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] $DOWNLOAD_URL/linux/$lsb_dist $dist_version $CHANNEL"
-			(
-				if ! is_dry_run; then
-					set -x
-				fi
-				$sh_c 'apt-get -qq update >/dev/null'
-				$sh_c "DEBIAN_FRONTEND=noninteractive apt-get -y -qq install $pre_reqs >/dev/null"
-				$sh_c 'install -m 0755 -d /etc/apt/keyrings'
-				$sh_c "curl -fsSL \"$DOWNLOAD_URL/linux/$lsb_dist/gpg\" -o /etc/apt/keyrings/docker.asc"
-				$sh_c "chmod a+r /etc/apt/keyrings/docker.asc"
-				$sh_c "echo \"$apt_repo\" > /etc/apt/sources.list.d/docker.list"
-				$sh_c 'apt-get -qq update >/dev/null'
-			)
-
-			if [ "$REPO_ONLY" = "1" ]; then
-				exit 0
-			fi
-
-			pkg_version=""
-			if [ -n "$VERSION" ]; then
-				if is_dry_run; then
-					echo "# WARNING: VERSION pinning is not supported in DRY_RUN"
-				else
-					# Will work for incomplete versions IE (17.12), but may not actually grab the "latest" if in the test channel
-					pkg_pattern="$(echo "$VERSION" | sed 's/-ce-/~ce~.*/g' | sed 's/-/.*/g')"
-					search_command="apt-cache madison docker-ce | grep '$pkg_pattern' | head -1 | awk '{\$1=\$1};1' | cut -d' ' -f 3"
-					pkg_version="$($sh_c "$search_command")"
-					echo "INFO: Searching repository for VERSION '$VERSION'"
-					echo "INFO: $search_command"
-					if [ -z "$pkg_version" ]; then
-						echo
-						echo "ERROR: '$VERSION' not found amongst apt-cache madison results"
-						echo
-						exit 1
-					fi
-					if version_gte "18.09"; then
-							search_command="apt-cache madison docker-ce-cli | grep '$pkg_pattern' | head -1 | awk '{\$1=\$1};1' | cut -d' ' -f 3"
-							echo "INFO: $search_command"
-							cli_pkg_version="=$($sh_c "$search_command")"
-					fi
-					pkg_version="=$pkg_version"
-				fi
-			fi
-			(
-				pkgs="docker-ce${pkg_version%=}"
-				if version_gte "18.09"; then
-						# older versions didn't ship the cli and containerd as separate packages
-						pkgs="$pkgs docker-ce-cli${cli_pkg_version%=} containerd.io"
-				fi
-				if version_gte "20.10"; then
-						pkgs="$pkgs docker-compose-plugin docker-ce-rootless-extras$pkg_version"
-				fi
-				if version_gte "23.0"; then
-						pkgs="$pkgs docker-buildx-plugin"
-				fi
-				if version_gte "28.2"; then
-						pkgs="$pkgs docker-model-plugin"
-				fi
-				if ! is_dry_run; then
-					set -x
-				fi
-				$sh_c "DEBIAN_FRONTEND=noninteractive apt-get -y -qq install $pkgs >/dev/null"
-			)
-			if [ "$NO_AUTOSTART" != "1" ]; then
-				start_docker_daemon
-			fi
-			echo_docker_as_nonroot
-			exit 0
-			;;
-		centos|fedora|rhel)
-			if [ "$(uname -m)" = "s390x" ]; then
-				echo "Effective v27.5, please consult RHEL distro statement for s390x support."
-				exit 1
-			fi
-			repo_file_url="$DOWNLOAD_URL/linux/$lsb_dist/$REPO_FILE"
-			(
-				if ! is_dry_run; then
-					set -x
-				fi
-				if command_exists dnf5; then
-					$sh_c "dnf -y -q --setopt=install_weak_deps=False install dnf-plugins-core"
-					$sh_c "dnf5 config-manager addrepo --overwrite --save-filename=docker-ce.repo --from-repofile='$repo_file_url'"
-
-					if [ "$CHANNEL" != "stable" ]; then
-						$sh_c "dnf5 config-manager setopt \"docker-ce-*.enabled=0\""
-						$sh_c "dnf5 config-manager setopt \"docker-ce-$CHANNEL.enabled=1\""
-					fi
-					$sh_c "dnf makecache"
-				elif command_exists dnf; then
-					$sh_c "dnf -y -q --setopt=install_weak_deps=False install dnf-plugins-core"
-					$sh_c "rm -f /etc/yum.repos.d/docker-ce.repo  /etc/yum.repos.d/docker-ce-staging.repo"
-					$sh_c "dnf config-manager --add-repo $repo_file_url"
-
-					if [ "$CHANNEL" != "stable" ]; then
-						$sh_c "dnf config-manager --set-disabled \"docker-ce-*\""
-						$sh_c "dnf config-manager --set-enabled \"docker-ce-$CHANNEL\""
-					fi
-					$sh_c "dnf makecache"
-				else
-					$sh_c "yum -y -q install yum-utils"
-					$sh_c "rm -f /etc/yum.repos.d/docker-ce.repo  /etc/yum.repos.d/docker-ce-staging.repo"
-					$sh_c "yum-config-manager --add-repo $repo_file_url"
-
-					if [ "$CHANNEL" != "stable" ]; then
-						$sh_c "yum-config-manager --disable \"docker-ce-*\""
-						$sh_c "yum-config-manager --enable \"docker-ce-$CHANNEL\""
-					fi
-					$sh_c "yum makecache"
-				fi
-			)
-
-			if [ "$REPO_ONLY" = "1" ]; then
-				exit 0
-			fi
-
-			pkg_version=""
-			if command_exists dnf; then
-				pkg_manager="dnf"
-				pkg_manager_flags="-y -q --best"
-			else
-				pkg_manager="yum"
-				pkg_manager_flags="-y -q"
-			fi
-			if [ -n "$VERSION" ]; then
-				if is_dry_run; then
-					echo "# WARNING: VERSION pinning is not supported in DRY_RUN"
-				else
-					if [ "$lsb_dist" = "fedora" ]; then
-						pkg_suffix="fc$dist_version"
-					else
-						pkg_suffix="el"
-					fi
-					pkg_pattern="$(echo "$VERSION" | sed 's/-ce-/\\\\.ce.*/g' | sed 's/-/.*/g').*$pkg_suffix"
-					search_command="$pkg_manager list --showduplicates docker-ce | grep '$pkg_pattern' | tail -1 | awk '{print \$2}'"
-					pkg_version="$($sh_c "$search_command")"
-					echo "INFO: Searching repository for VERSION '$VERSION'"
-					echo "INFO: $search_command"
-					if [ -z "$pkg_version" ]; then
-						echo
-						echo "ERROR: '$VERSION' not found amongst $pkg_manager list results"
-						echo
-						exit 1
-					fi
-					if version_gte "18.09"; then
-						# older versions don't support a cli package
-						search_command="$pkg_manager list --showduplicates docker-ce-cli | grep '$pkg_pattern' | tail -1 | awk '{print \$2}'"
-						cli_pkg_version="$($sh_c "$search_command" | cut -d':' -f 2)"
-					fi
-					# Cut out the epoch and prefix with a '-'
-					pkg_version="-$(echo "$pkg_version" | cut -d':' -f 2)"
-				fi
-			fi
-			(
-				pkgs="docker-ce$pkg_version"
-				if version_gte "18.09"; then
-					# older versions didn't ship the cli and containerd as separate packages
-					if [ -n "$cli_pkg_version" ]; then
-						pkgs="$pkgs docker-ce-cli-$cli_pkg_version containerd.io"
-					else
-						pkgs="$pkgs docker-ce-cli containerd.io"
-					fi
-				fi
-				if version_gte "20.10"; then
-					pkgs="$pkgs docker-compose-plugin docker-ce-rootless-extras$pkg_version"
-				fi
-				if version_gte "23.0"; then
-						pkgs="$pkgs docker-buildx-plugin docker-model-plugin"
-				fi
-				if ! is_dry_run; then
-					set -x
-				fi
-				$sh_c "$pkg_manager $pkg_manager_flags install $pkgs"
-			)
-			if [ "$NO_AUTOSTART" != "1" ]; then
-				start_docker_daemon
-			fi
-			echo_docker_as_nonroot
-			exit 0
-			;;
-		sles)
-			echo "Effective v27.5, please consult SLES distro statement for s390x support."
-			exit 1
-			;;
-		*)
-			if [ -z "$lsb_dist" ]; then
-				if is_darwin; then
-					echo
-					echo "ERROR: Unsupported operating system 'macOS'"
-					echo "Please get Docker Desktop from https://www.docker.com/products/docker-desktop"
-					echo
-					exit 1
-				fi
-			fi
-			echo
-			echo "ERROR: Unsupported distribution '$lsb_dist'"
-			echo
-			exit 1
-			;;
-	esac
-	exit 1
-}
-
-# wrapped up in a function so that we have some protection against only getting
-# half the file during "curl | sh"
-do_install