Add multi-server support and update docs

Introduce multi-server architecture documentation and reorganize README content. Top-level README now documents Core vs Remote server roles, links to local docs instead of wiki pages, and highlights Traefik/Sablier multi-server behavior. docker-compose/README.md was rewritten to be a template-style reference with single- and multi-server deployment guidance, Traefik label examples, and sablier usage; dockge README was moved into docker-compose/dockge/. docker-compose/core/README.md was updated to describe core responsibilities, shared CA artifacts, and startup order for multi-server deployments. Several obsolete/duplicated docs and action reports were removed and a new multi-server deployment doc was added to centralize on-demand/remote service guidance. Overall this cleans up legacy docs and documents the multi-server workflow and TLS/shared-CA requirements.
2026-02-05 22:30:52 -05:00
parent 84b2cabacc
commit 5cbb106160
27 changed files with 803 additions and 3300 deletions
--- a/docs/services-overview.md
+++ b/docs/services-overview.md
@@ -6,22 +6,18 @@ This document provides a comprehensive overview of all 50+ pre-configured servic

 | Stacks (12) | Services (50 + 6db) | SSO | Storage | Access URLs |
 |-------|----------|-----|---------|-------------|
-| **🔀 alternatives.yaml** (6 + 3 DBs) | | | | |
-| ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/alternatives | dockge.${DOMAIN} |
-| ├─ Portainer | Container management | ✓ | /opt/stacks/alternatives | portainer.${DOMAIN} |
-| ├─ Authentik Server | SSO with web UI | ✓ | /opt/stacks/alternatives | authentik.${DOMAIN} |
-| │  ├─ authentik-worker | Background tasks | - | /opt/stacks/alternatives | No UI |
-| │  ├─ authentik-db | PostgreSQL | - | /opt/stacks/alternatives | No UI |
-| │  └─ authentik-redis | Cache/messaging | - | /opt/stacks/alternatives | No UI |
-| └─ Plex | Media server | ✗ | /mnt/media, /mnt/transcode | plex.${DOMAIN} |
-| **📦 core.yaml (4)** | **Deploy First** | | | |
+| **📦 core.yaml (3)** | **Deploy First** | | | |
 | ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI |
 | ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} |
-| ├─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
-| └─ Sablier | Lazy loading service | - | /opt/stacks/core/sablier | No UI |
+| └─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
+| &nbsp;
+| **🔄 sablier.yaml (1)** | **Deploy on Each Server** | | | |
+| └─ Sablier | Lazy loading service | - | /opt/stacks/sablier | sablier.${DOMAIN} |
+| &nbsp;
 | **📊 dashboards.yaml** (2) | | | | |
 | ├─ Homepage | App dashboard (AI cfg) | ✓ | /opt/stacks/dashboards | home.${DOMAIN} |
 | └─ Homarr | Modern dashboard | ✓ | /opt/stacks/dashboards | homarr.${DOMAIN} |
+| &nbsp;
 | **🏠 homeassistant.yaml** (7) | | | | |
 | ├─ Home Assistant | HA platform | ✗ | /opt/stacks/homeassistant | ha.${DOMAIN} |
 | ├─ ESPHome | ESP firmware mgr | ✓ | /opt/stacks/homeassistant | esphome.${DOMAIN} |
@@ -30,6 +26,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Mosquitto | MQTT broker | - | /opt/stacks/homeassistant | Ports 1883, 9001 |
 | ├─ Zigbee2MQTT | Zigbee bridge | ✓ | /opt/stacks/homeassistant | zigbee2mqtt.${DOMAIN} |
 | └─ MotionEye | Video surveillance | ✓ | /opt/stacks/homeassistant, /mnt/surveillance | motioneye.${DOMAIN} |
+| &nbsp;
 | **🔧 infrastructure.yaml** (6)** | | | | |
 | ├─ Pi-hole | DNS + Ad blocking | ✓ | /opt/stacks/infrastructure | pihole.${DOMAIN} |
 | ├─ Watchtower | Auto container updates | - | /opt/stacks/infrastructure | No UI |
@@ -37,6 +34,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Glances | System monitoring | ✓ | /opt/stacks/infrastructure | glances.${DOMAIN} |
 | ├─ Code Server | VS Code in browser | ✓ | /opt/stacks/infrastructure | code.${DOMAIN} |
 | └─ Docker Proxy | Secure socket access | - | /opt/stacks/infrastructure | No UI |
+| &nbsp;
 | **📺 media-management.yaml** (9) | | | | |
 | ├─ Sonarr | TV automation | ✓ | /opt/stacks/media-management, /mnt/media | sonarr.${DOMAIN} |
 | ├─ Radarr | Movie automation | ✓ | /opt/stacks/media-management, /mnt/media | radarr.${DOMAIN} |
@@ -47,9 +45,11 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Mylar3 | Comic manager | ✓ | /opt/stacks/media-management, /mnt/media | mylar.${DOMAIN} |
 | ├─ Jellyseerr | Media requests | ✓ | /opt/stacks/media-management | jellyseerr.${DOMAIN} |
 | └─ FlareSolverr | Cloudflare bypass | - | /opt/stacks/media-management | No UI |
+| &nbsp;
 | **🎬 media.yaml** (2) | | | | |
 | ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} |
 | └─ Calibre-Web | Ebook reader | ✓ | /opt/stacks/media, /mnt/media | calibre.${DOMAIN} |
+| &nbsp;
 | **📈 monitoring.yaml** (8) | | | | |
 | ├─ Prometheus | Metrics collection | ✓ | /opt/stacks/monitoring | prometheus.${DOMAIN} |
 | ├─ Grafana | Visualization | ✓ | /opt/stacks/monitoring | grafana.${DOMAIN} |
@@ -58,6 +58,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Node Exporter | Host metrics | - | /opt/stacks/monitoring | No UI |
 | ├─ cAdvisor | Container metrics | - | /opt/stacks/monitoring | Internal :8080 |
 | └─ Uptime Kuma | Uptime monitoring | ✓ | /opt/stacks/monitoring | status.${DOMAIN} |
+| &nbsp;
 | **💼 productivity.yaml** (5 + 4 DBs) | | | | |
 | ├─ Nextcloud | File sync platform | ✓ | /opt/stacks/productivity, /mnt/nextcloud | nextcloud.${DOMAIN} |
 | │  └─ nextcloud-db | MariaDB | - | /opt/stacks/productivity | No UI |
@@ -67,10 +68,12 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Gitea | Git service | ✓ | /opt/stacks/productivity, /mnt/git | git.${DOMAIN} |
 | │  └─ gitea-db | PostgreSQL | - | /opt/stacks/productivity | No UI |
 | └─ Jupyter Lab | Notebooks | ✓ | /opt/stacks/productivity | jupyter.${DOMAIN} |
+| &nbsp;
 | **🔄 transcoders.yaml** (3) | | | | |
 | ├─ Tdarr Server | Transcoding server | ✓ | /opt/stacks/transcoders, /mnt/transcode | tdarr.${DOMAIN} |
 | ├─ Tdarr Node | Transcoding worker | - | /mnt/transcode-cache | No UI |
 | └─ Unmanic | Library optimizer | ✓ | /opt/stacks/transcoders, /mnt/transcode | unmanic.${DOMAIN} |
+| &nbsp;
 | **🛠️ utilities.yaml** (7) | | | | |
 | ├─ Vaultwarden | Password manager | ✗ | /opt/stacks/utilities | bitwarden.${DOMAIN} |
 | ├─ Backrest | Backup (restic) | ✓ | /opt/stacks/utilities, /mnt/backups | backrest.${DOMAIN} |
@@ -79,151 +82,55 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
 | ├─ Form.io | Form platform | ✓ | /opt/stacks/utilities | forms.${DOMAIN} |
 | │  └─ formio-mongo | MongoDB | - | /opt/stacks/utilities | No UI |
 | └─ Authelia-Redis | Session storage | - | /opt/stacks/utilities | No UI |
+| &nbsp;
 | **🔒 vpn.yaml (2)** | **VPN Services** | | | |
 | ├─ Gluetun | VPN (Surfshark) | - | /opt/stacks/vpn/gluetun | No UI |
 | └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} |
+| &nbsp;
 | **📖 wikis.yaml** (4) | | | | |
 | ├─ DokuWiki | File-based wiki | ✓ | /opt/stacks/wikis | dokuwiki.${DOMAIN} |
 | ├─ BookStack | Documentation | ✓ | /opt/stacks/wikis | docs.${DOMAIN} |
 | │  └─ bookstack-db | MariaDB | - | /opt/stacks/wikis | No UI |
 | └─ MediaWiki | Wiki platform | ✓ | /opt/stacks/wikis | mediawiki.${DOMAIN} |
+| &nbsp;
+| **🔀 alternatives.yaml** (6 + 3 DBs) | | | | |
+| ├─ Portainer | Container management | ✓ | /opt/stacks/alternatives | portainer.${DOMAIN} |
+| ├─ Authentik Server | SSO with web UI | ✓ | /opt/stacks/alternatives | authentik.${DOMAIN} |
+| │  ├─ authentik-worker | Background tasks | - | /opt/stacks/alternatives | No UI |
+| │  ├─ authentik-db | PostgreSQL | - | /opt/stacks/alternatives | No UI |
+| │  └─ authentik-redis | Cache/messaging | - | /opt/stacks/alternatives | No UI |
+| └─ Plex | Media server | ✗ | /mnt/media, /mnt/transcode | plex.${DOMAIN} |

 **Legend:** ✓ = Protected by SSO | ✗ = Bypasses SSO | - = No web UI

-## Quick Deployment Order
+## Service Configuration

-1. **Create Networks** (one-time setup)
-   ```bash
-   docker network create traefik-network
-   docker network create homelab-network
-   docker network create dockerproxy-network
-   ```
+Some services require no initial configuration (ie. creates it on first run if needed)

-2. **Deploy Core Stack** (required first)
-   ```bash
-   cd /opt/stacks/core/
-   docker compose up -d
-   ```
+### Other services have config files/folders in the stack folder for easy deployment.  
+* These typicaly use values from variables in .env however **can not access the variables.**  
+* `ez-homelab.sh` handles variable replacement on deployment.    
+* Each stack folder contains a `deploy-stacksname.sh` that will do the same for that stack.

-3. **Deploy Infrastructure**
-   ```bash
-   cd /opt/stacks/infrastructure/
-   docker compose up -d
-   ```

-4. **Deploy Dashboards**
-   ```bash
-   cd /opt/stacks/dashboards/
-   docker compose up -d
-   ```
+## Multi-Server Deployment Notes

-5. **Deploy Additional Stacks** (as needed)
-   - Alternatives: `/opt/stacks/alternatives/`
-   - Core: `/opt/stacks/core/` (deploy first)
-   - Dashboards: `/opt/stacks/dashboards/`
-   - Home Automation: `/opt/stacks/homeassistant/`
-   - Infrastructure: `/opt/stacks/infrastructure/`
-   - Media: `/opt/stacks/media/`
-   - Media Management: `/opt/stacks/media-management/`
-   - Monitoring: `/opt/stacks/monitoring/`
-   - Productivity: `/opt/stacks/productivity/`
-   - Transcoders: `/opt/stacks/transcoders/`
-   - Utilities: `/opt/stacks/utilities/`
-   - VPN: `/opt/stacks/vpn/`
-   - Wikis: `/opt/stacks/wikis/`
+### Core Server `There can be only one`

-## Toggling SSO (Authelia) On/Off
+`Forward ports 80 & 443 from your router`

-You can easily enable or disable SSO protection for any service by modifying its Traefik labels in the docker-compose.yml file.
+- **DuckDNS**: Dynamic DNS and SSL certificate management
+- **Authelia**: Centralized SSO authentication for all servers
+- **Traefik** (core): Multi-provider configuration to route to all servers

-### To Enable SSO on a Service
+### Remote Server Services (Deploy on Each Server)
+- **Traefik** (local): Discovers containers through labels
+- **Sablier**: Manages lazy loading for local containers only
+- **Dockge**: Stack management interface

-Add the Authelia middleware to the service's Traefik labels:
-
-```yaml
-labels:
-  - "traefik.enable=true"
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Add this line
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-### To Disable SSO on a Service
-
-Comment out (don't remove) the middleware line:
-
-```yaml
-labels:
-  - "traefik.enable=true"
-  - "traefik.http.routers.servicename.rule=Host(`servicename.${DOMAIN}`)"
-  - "traefik.http.routers.servicename.entrypoints=websecure"
-  - "traefik.http.routers.servicename.tls.certresolver=letsencrypt"
-  # - "traefik.http.routers.servicename.middlewares=authelia@docker"  # ← Commented out (not removed)
-  - "traefik.http.services.servicename.loadbalancer.server.port=8080"
-```
-
-After making changes, redeploy the service:
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose up -d
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml up -d
-```
-
-**Stopping a Service:**
-
-```bash
-# From inside the stack directory
-cd /opt/stacks/stack-name/
-docker compose down
-
-# Or from anywhere, using the full path
-docker compose -f /opt/stacks/stack-name/docker-compose.yml down
-```
-
-**Use Cases for Development/Production:**
- **Security First**: All services start with SSO enabled by default for maximum security
- **Development**: Keep SSO enabled to protect services during testing
- **Production**: Disable SSO only for services needing direct app/API access (Plex, Jellyfin)
- **Gradual Exposure**: Comment out SSO only when ready to expose a service
- **Quick Toggle**: AI assistant can modify these labels automatically when you ask
-
-## Storage Recommendations
-
-| Data Type | Recommended Location | Reason |
-|-----------|---------------------|--------|
-| Configuration files | `/opt/stacks/stack-name/` | Easy access, version control |
-| Small databases (< 10GB) | `/opt/stacks/stack-name/db/` | Manageable on system drive |
-| Media files (movies, TV, music) | `/mnt/media/` | Large, continuous growth |
-| Downloads | `/mnt/downloads/` | Temporary, high throughput |
-| Backups | `/mnt/backups/` | Large, separate from system |
-| Surveillance footage | `/mnt/surveillance/` | Continuous recording |
-| Large databases (> 10GB) | `/mnt/databases/` | Growth over time |
-| Transcoding cache | `/mnt/transcode-cache/` | High I/O, large temporary files |
-| Git repositories | `/mnt/git/` | Can grow large |
-| Nextcloud data | `/mnt/nextcloud/` | User files, photos |
-
-## Configuration Templates
-
-All configuration templates are available in `config-templates/`:
- `traefik/` - Static and dynamic Traefik configuration
- `authelia/` - Complete Authelia setup with user database
- `homepage/` - Dashboard services, widgets, and Docker integration
- `prometheus/` - Metrics scrape configurations
- `loki/` - Log aggregation settings
- `promtail/` - Log shipping configuration
- `redis/` - Redis server configuration
-
-## Additional Resources
-
- **Getting Started**: See [docs/getting-started.md](getting-started.md) for detailed deployment
- **Docker Guidelines**: See [docs/docker-guidelines.md](docker-guidelines.md) for management patterns
- **Quick Reference**: See [docs/quick-reference.md](quick-reference.md) for common commands
- **Proxying External Hosts**: See [docs/proxying-external-hosts.md](proxying-external-hosts.md) for Raspberry Pi, NAS, etc.
- **AI Assistant**: Use GitHub Copilot in VS Code with `.github/copilot-instructions.md` for intelligent homelab management
+### Architecture Overview
+- **TLS Communication**: Remote servers connect via Docker TLS (port 2376)
+- **Unified Access**: All services accessible through core server domain
+- **Service Routing**: Having Traefik on every server enables label based service discovery
+- **Lazy Loading**: Having Sablier on every server enables label based lazy loading configuration