kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add multi-server support and update docs

Browse Source 
Introduce multi-server architecture documentation and reorganize README content. Top-level README now documents Core vs Remote server roles, links to local docs instead of wiki pages, and highlights Traefik/Sablier multi-server behavior. docker-compose/README.md was rewritten to be a template-style reference with single- and multi-server deployment guidance, Traefik label examples, and sablier usage; dockge README was moved into docker-compose/dockge/. docker-compose/core/README.md was updated to describe core responsibilities, shared CA artifacts, and startup order for multi-server deployments. Several obsolete/duplicated docs and action reports were removed and a new multi-server deployment doc was added to centralize on-demand/remote service guidance. Overall this cleans up legacy docs and documents the multi-server workflow and TLS/shared-CA requirements.
This commit is contained in:
kelinfoxy
2026-02-05 22:30:52 -05:00
parent 84b2cabacc
commit 5cbb106160
27 changed files with 803 additions and 3300 deletions
Download Patch File Download Diff File Expand all files Collapse all files

289
docker-compose/README.md
View File

@@ -1,152 +1,108 @@
# Docker Compose Stacks
This directory contains Docker Compose files for managing your homelab services. Each stack is organized in its own folder for better organization and maintainability.
This directory contains Docker Compose templates for managing your homelab services. Each stack is organized in its own folder for better organization and maintainability.
## Structure
```
docker-compose/
├── core/ # Core infrastructure (Traefik, Authelia, DuckDNS)
├── infrastructure/ # Additional infrastructure (Pi-hole, Dockge, etc.)
├── core/ # Core infrastructure (MUST DEPLOY FIRST)
│ ├── docker-compose.yml
│ ├── authelia/ # SSO configuration
│ ├── duckdns/ # DNS configuration
│ └── traefik/ # Reverse proxy configuration
│ └── dynamic/ # External routing YAML files (multi-server)
├── sablier/ # Lazy loading service (per-server)
├── dockge/ # Docker management web UI
├── infrastructure/ # Additional infrastructure (Pi-hole, etc.)
├── dashboards/ # Dashboard services (Homepage, Homarr)
├── vpn/ # VPN services (Gluetun, qBittorrent)
├── media/ # Media services (Plex, Jellyfin, etc.)
├── media-management/ # *arr services (Sonarr, Radarr, etc.)
├── monitoring/ # Observability stack (Prometheus, Grafana, etc.)
├── alternatives/ # Alternative services (Authentik, etc.)
├── homeassistant/ # Home Assistant stack
├── nextcloud/ # Nextcloud stack
├── productivity/ # Productivity tools
├── utilities/ # Utility services
## ⚠️ Important: Core Services First
**Before deploying any other stacks, ensure the `core/` services are running:**
- **Traefik**: Reverse proxy and SSL termination
- **Authelia**: Single sign-on authentication
- **DuckDNS**: Dynamic DNS for domain resolution
These services provide the foundation for all other services. Most stacks depend on Traefik for routing and Authelia for authentication.
### Quick Start Core Services
```bash
cd core
cp .env.template .env # Edit with your values
cp docker-compose.yml.template docker-compose.yml # Or use the pre-configured version
docker compose up -d
├── productivity/ # Productivity tools (Nextcloud, Gitea, etc.)
├── utilities/ # Utility services (Duplicati, FreshRSS, etc.)
├── wikis/ # Mediawiki, Dokuwiki, Bookstacks
└── vpn/ # VPN services (Gluetun, qBittorrent)
```
### Starting Services
## Multi-Server Architecture
Start all services in a stack:
EZ-Homelab supports two deployment models:
### **Single Server:**
- Core + all other stacks on one machine
- Simplest setup for beginners
### **Multi-Server:**
- **Core Server**: DuckDNS, Traefik (multi-provider), Authelia
- **Remote Servers**: Traefik (local-only), Sablier (local-only), application services
- All services accessed through unified domain
See [docs/Ondemand-Remote-Services.md](../docs/Ondemand-Remote-Services.md) for multi-server setup.
## Deployment
Use the unified setup script:
```bash
cd docker-compose/core && docker compose up -d
cd ~/EZ-Homelab
./scripts/ez-homelab.sh
```
Start a specific service:
```bash
cd docker-compose/vpn && docker compose up -d gluetun
```
## Single Server Traefik service labels
### Stopping Services
Stop all services in a stack:
```bash
cd docker-compose/core && docker compose down
```
Stop a specific service:
```bash
cd docker-compose/vpn && docker compose stop qbittorrent
```
### Viewing Status
Check running services:
```bash
docker compose -f docker-compose/media.yml ps
```
View logs:
```bash
docker compose -f docker-compose/media.yml logs -f plex
```
### Updating Services
Pull latest images:
```bash
docker compose -f docker-compose/media.yml pull
```
Update a specific service:
```bash
docker compose -f docker-compose/media.yml pull plex
docker compose -f docker-compose/media.yml up -d plex
```
## Networks
All services connect to a shared bridge network called `homelab-network`. Create it once:
```bash
docker network create homelab-network
```
Some services may use additional networks for security isolation:
- `monitoring-network` - For monitoring stack
- `database-network` - For database isolation
- `media-network` - For media services
Create them as needed:
```bash
docker network create monitoring-network
docker network create database-network
docker network create media-network
```
## Environment Variables
Create a `.env` file in the root of your homelab directory with common variables:
```bash
# .env
PUID=1000
PGID=1000
TZ=America/New_York
USERDIR=/home/username/homelab
DATADIR=/mnt/data
```
Never commit `.env` files to git! Use `.env.example` as a template instead.
## Labels
### To enable Authelia SSO
```yaml
```
### Traefik routing labels
If Traekif is on the same server add these labels.
```yaml
```
>If Traefik is on a seperate server, don't use traekfik labels in compose files, use an external host yaml file.
### Sablier middleware labels
Add these labels to enable ondemand functionality.
```yaml
services:
myservice:
labels:
- sablier.enable=true
- sablier.group=<server>-<service name>
- sablier.start-on-demand=true
# TRAEFIK CONFIGURATION
# ==========================================
# Service metadata
- "com.centurylinklabs.watchtower.enable=true"
- "homelab.category=category-name"
- "homelab.description=Brief service description"
# Traefik labels
- "traefik.enable=true"
# Router configuration
- "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
- "traefik.http.routers.myservice.entrypoints=websecure"
- "traefik.http.routers.myservice.tls.certresolver=letsencrypt"
- "traefik.http.routers.myservice.middlewares=authelia@docker" # SSO (remove to disable)
# Service configuration
- "traefik.http.services.myservice.loadbalancer.server.port=8080"
# Sablier configuration (lazy loading)
- "sablier.enable=true"
- "sablier.group=${SERVER_HOSTNAME}-myservice"
- "sablier.start-on-demand=true"
```
## Multi-Server Traefik
### On Core Server
## On Remote Server
### Disabling SSO (Media Servers)
Remove or comment the authelia middleware line:
```yaml
# SSO enabled (default):
- "traefik.http.routers.myservice.middlewares=authelia@docker"
# SSO disabled (for Plex, Jellyfin, etc.):
# - "traefik.http.routers.myservice.middlewares=authelia@docker"
```
### Disabling Lazy Loading (Always-On Services)
Remove Sablier labels and use `restart: unless-stopped`:
```yaml
services:
myservice:
restart: unless-stopped # Always running
# No sablier labels
```
## Best Practices
@@ -171,6 +127,7 @@ services:
restart: unless-stopped
networks:
- homelab-network
- traefik-network
ports:
- "host_port:container_port"
volumes:
@@ -197,77 +154,3 @@ networks:
homelab-network:
external: true
```
## Troubleshooting
### Service won't start
1. Check logs: `docker compose -f file.yml logs service-name`
2. Validate config: `docker compose -f file.yml config`
3. Check for port conflicts: `sudo netstat -tlnp | grep PORT`
4. Verify volumes exist and have correct permissions
### Permission errors
1. Ensure PUID and PGID match your user: `id -u` and `id -g`
2. Fix directory ownership: `sudo chown -R 1000:1000 ./config/service-name`
### Network issues
1. Verify network exists: `docker network ls`
2. Check service is connected: `docker network inspect homelab-network`
3. Test connectivity: `docker compose exec service1 ping service2`
## Migration from Docker Run
If you have services running via `docker run`, migrate them to compose:
1. Get current configuration:
```bash
docker inspect container-name > container-config.json
```
2. Convert to compose format (extract image, ports, volumes, environment)
3. Test the compose configuration
4. Stop old container:
```bash
docker stop container-name
docker rm container-name
```
5. Start with compose:
```bash
docker compose -f file.yml up -d
```
## Backup Strategy
Regular backups are essential:
```bash
# Backup compose files (already in git)
git add docker-compose/*.yml
git commit -m "Update compose configurations"
# Backup volumes
docker run --rm \
-v volume-name:/data \
-v $(pwd)/backups:/backup \
busybox tar czf /backup/volume-name-$(date +%Y%m%d).tar.gz /data
# Backup config directories
tar czf backups/config-$(date +%Y%m%d).tar.gz config/
```
## Getting Help
- Check the [Docker Guidelines](../docs/docker-guidelines.md) for detailed documentation
- Review the [GitHub Copilot Instructions](../.github/copilot-instructions.md) for AI assistance
- Consult service-specific documentation in `config/service-name/README.md`
## Examples
See the example compose files in this directory:
- `infrastructure.yml` - Essential services like reverse proxy
- `media.yml` - Media server stack
- `monitoring.yml` - Observability and monitoring
- `development.yml` - Development environments and tools

79
docker-compose/core/README.md
View File

@@ -1,27 +1,65 @@
# Core Infrastructure Services
This directory contains the core infrastructure services that form the foundation of the homelab. These services should always be running and are critical for the operation of other services.
This directory contains the core infrastructure services that form the foundation of the homelab. These services should be deployed **on the main server only** and are critical for the operation of all other services across all servers.
## Services
### DuckDNS
- **Purpose**: Dynamic DNS service for domain resolution and wildcard SSL certificates
- **Subdomain**: Configurable via environment variables
- **Token**: Configured in environment variables
- **SSL Certificates**: Generates wildcard cert used by all services on all servers
- **Deploy**: Core server only
### Traefik (v3)
- **Purpose**: Reverse proxy and SSL termination
- **Purpose**: Reverse proxy and SSL termination with multi-server routing
- **Ports**: 80 (HTTP), 443 (HTTPS), 8080 (Dashboard)
- **Configuration**: Located in `traefik/config/traefik.yml`
- **SSL**: Let's Encrypt with DNS-01 challenge (configurable provider)
- **Multi-Server**: Discovers services on all servers via Docker providers
- **SSL**: Let's Encrypt with DNS-01 challenge (wildcard certificate)
- **Dashboard**: Available at configured domain
- **Deploy**: Core server (multi-provider), Remote servers (local-only)
**Note**: Sablier has been moved to its own stack (`/opt/stacks/sablier/`) and should be deployed on each server individually. See [Sablier documentation](../../docs/service-docs/sablier.md) for details.
### Authelia (v4.37.5)
- **Purpose**: Single sign-on authentication service
- **Purpose**: Single sign-on authentication service for all services across all servers
- **Port**: 9091 (internal)
- **Access**: Configured authentication domain
- **Configuration**: Located in `authelia/config/`
- **Database**: SQLite database in `authelia/config/db.sqlite3`
- **Deploy**: Core server only
### DuckDNS
- **Purpose**: Dynamic DNS service for domain resolution
- **Purpose**: Dynamic DNS service for domain resolution and wildcard SSL certificates
- **Subdomain**: Configurable via environment variables
- **Token**: Configured in environment variables
- **SSL Certificates**: Generates wildcard cert used by all services on all servers
- **Deploy**: Core server only
## Multi-Server Architecture
The core stack on the main server provides centralized services for the entire homelab:
**Core Server Responsibilities:**
- Receives all external traffic (ports 80/443 forwarded from router)
- Runs DuckDNS for domain management and SSL certificates
- Runs Authelia for centralized authentication
- Runs multi-provider Traefik that discovers services on all servers
- Generates shared CA for Docker TLS communication
**Remote Server Setup:**
- Remote servers run their own Traefik instance (local Docker provider only)
- Remote servers run their own Sablier instance (local container management)
- Remote servers expose Docker API on port 2376 with TLS
- Core server Traefik connects to remote Docker APIs to discover services
- No port forwarding needed on remote servers
**Service Access:**
- All services accessible via: `https://service.yourdomain.duckdns.org`
- Core Traefik routes to appropriate server (local or remote)
- Single wildcard SSL certificate used for all services
- Authelia provides SSO for all protected services
## ⚠️ Version Pinning & Breaking Changes
@@ -64,15 +102,22 @@ core/
│ ├── configuration.yml # Authelia main config
│ ├── users_database.yml # User credentials
│ └── db.sqlite3 # SQLite database
└── traefik/
├── config/
│ └── traefik.yml # Traefik static config
├── dynamic/ # Dynamic configurations
│ ├── routes.yml
│ ├── sablier.yml
│ └── external-host-*.yml
└── letsencrypt/
└── acme.json # SSL certificates
├── duckdns/
│ └── config/ # DuckDNS configuration
├── traefik/
│ ├── config/
│ │ └── traefik.yml # Traefik static config
│ ├── dynamic/ # Dynamic configurations
│ │ ├── routes.yml
│ │ ├── sablier.yml
│ │ └── external-host-*.yml # Remote server routing
│ └── letsencrypt/
│ └── acme.json # SSL certificates
└── shared-ca/ # TLS certificates for multi-server
├── ca.pem # Certificate Authority
├── ca-key.pem # CA private key
├── cert.pem # Client certificate
└── key.pem # Client key
```
### Environment Variables (.env)
@@ -100,10 +145,12 @@ PGID=1000
4. Domain configured in DuckDNS
### Startup Order
1. `duckdns` - For DNS updates
2. `traefik` - Reverse proxy
1. `duckdns` - For DNS updates and SSL certificate generation
2. `traefik` - Reverse proxy (waits for SSL certificates)
3. `authelia` - Authentication service
**Note**: Sablier is now deployed separately in `/opt/stacks/sablier/` after core stack is running.
### Commands
```bash
# Start all services

0
docker-compose/README-dockge.md → docker-compose/dockge/README.md
View File