kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Arcane stack and clean up deprecated files

Browse Source 
- Add new Arcane application stack
- Move aliases.sh to scripts/ directory
- Remove deprecated files (IMPLEMENTATION_COMPLETE.md, markup.yml, release notes)
- Remove standalone traefik docker-compose (now part of core stack)
- Update documentation (ai-vscode-setup.md, docker-guidelines.md, core README)
This commit is contained in:
Kelin
2026-02-09 22:22:40 -05:00
parent 33336c75ff
commit 377ba6dc20
11 changed files with 136 additions and 764 deletions
Download Patch File Download Diff File Expand all files Collapse all files

285
IMPLEMENTATION_COMPLETE.md
View File

@@ -1,285 +0,0 @@
# Multi-Server Implementation - COMPLETED
**Implementation Date:** February 4, 2026
**Status:** ✅ COMPLETE - All changes implemented and validated
---
## Implementation Summary
Successfully implemented multi-server Traefik + Sablier architecture for EZ-Homelab. The system now supports:
1. **Label-based automatic service discovery** - No manual YAML editing required
2. **Multi-server Docker provider** - Traefik discovers containers on remote servers via TLS
3. **Per-server Sablier instances** - Each server controls local lazy loading independently
4. **Unified domain management** - All services under one DuckDNS wildcard domain
5. **Secure Docker TLS** - Shared CA certificates for multi-server communication
---
## Changes Implemented
### 1. File Structure Changes
#### Deleted:
- ✅ `config-templates/` folder (deprecated)
#### Created:
- ✅ `docker-compose/sablier/` - New standalone Sablier stack
- `docker-compose.yml` - Sablier container with local Docker socket
- `README.md` - Complete documentation
#### Modified:
- ✅ `docker-compose/core/docker-compose.yml` - Removed embedded Sablier service
- ✅ `scripts/common.sh` - Added 4 new multi-server functions
- ✅ `scripts/ez-homelab.sh` - Added 5 new functions + updated workflow
- ✅ `.env.example` - Already contained REMOTE_SERVER_* variables
---
### 2. New Functions Added
#### common.sh (4 functions)
```bash
detect_server_role() # Detects if server is core or remote
generate_traefik_provider_config() # Creates Docker provider config for remote server
generate_sablier_middleware_config() # Creates Sablier middleware for remote server
add_remote_server_to_traefik() # Registers remote server with core Traefik
```
#### ez-homelab.sh (5 functions)
```bash
check_docker_installed() # Pre-flight check for Docker
set_required_vars_for_deployment() # Dynamic REQUIRED_VARS based on deployment type
deploy_remote_server() # Complete remote server deployment workflow
register_remote_server_with_core() # SSH to core server for registration
deploy_sablier_stack() # Deploy Sablier stack (used by both core and remote)
```
---
### 3. Workflow Changes
#### main() Function Updates:
- ✅ Added Docker pre-check before Options 2 and 3
- ✅ Calls `set_required_vars_for_deployment()` dynamically
- ✅ Option 2: Sets `REQUIRED_VARS` for core deployment
- ✅ Option 3: Sets `REQUIRED_VARS` for remote deployment, calls `deploy_remote_server()`
#### deploy_core() Function Updates:
- ✅ Automatically deploys Sablier stack after core deployment
- ✅ Updated config paths from `config-templates/*` to `docker-compose/core/*`
- ✅ Fixed backup timestamp format: `YY_MM_DD_hh_mm`
#### Backup Logic Verification:
- ✅ Backups correctly create from `/opt/stacks/core/` (deployed location, not repo)
- ✅ Format: `traefik.backup.26_02_04_14_30/`
---
## Architecture Overview
### Core Server (Option 2)
```
Core Server
├── Traefik (discovers all servers)
│ ├── Local Docker provider (this server)
│ ├── Remote Docker provider (auto-registered)
│ └── Dynamic configs in /opt/stacks/core/traefik/dynamic/
├── Authelia (SSO for all servers)
├── DuckDNS (wildcard domain)
└── Sablier (manages local lazy loading)
```
### Remote Server (Option 3)
```
Remote Server
├── Docker API (TLS port 2376)
│ └── Shares CA with core server
├── Sablier (manages local lazy loading)
└── Services with Traefik labels
└── Auto-discovered by core Traefik
```
### Service Discovery Flow
```
1. Remote server deployed → Docker TLS configured → Sablier deployed
2. Remote server registers with core → Creates Traefik provider config
3. Traefik polls remote Docker API → Discovers labeled containers
4. User accesses https://service.domain.duckdns.org
5. Core Traefik routes to remote service
6. SSL certificate issued by core Traefik
```
---
## Required Variables by Deployment Type
### Core Deployment (Option 2):
```bash
SERVER_IP
SERVER_HOSTNAME
DUCKDNS_SUBDOMAINS
DUCKDNS_TOKEN
DOMAIN
DEFAULT_USER
DEFAULT_PASSWORD
DEFAULT_EMAIL
```
### Remote Deployment (Option 3):
```bash
SERVER_IP # This remote server
SERVER_HOSTNAME # This remote server
DUCKDNS_DOMAIN # Shared domain
DEFAULT_USER # Local user
REMOTE_SERVER_IP # Core server IP
REMOTE_SERVER_HOSTNAME # Core server hostname
REMOTE_SERVER_USER # Core server SSH user
```
---
## Testing Checklist
### Pre-Implementation Tests:
- ✅ Bash syntax validation (`bash -n scripts/*.sh`)
- ✅ Docker Compose syntax validation
- ✅ No errors in VS Code
### Post-Implementation Tests Required:
- ⏳ Deploy core server (Option 2)
- ⏳ Verify Sablier stack auto-deployed
- ⏳ Verify shared CA generated
- ⏳ Deploy remote server (Option 3)
- ⏳ Verify Docker TLS configured
- ⏳ Verify registration with core
- ⏳ Deploy test service on remote with labels
- ⏳ Verify Traefik discovers service
- ⏳ Verify SSL certificate issued
- ⏳ Verify lazy loading works
---
## Key Implementation Details
### 1. Sablier Container Name
- Changed from `sablier-service` to `sablier` (consistent naming)
- Only connects to local Docker socket (no remote DOCKER_HOST)
- Each server runs independent Sablier instance
### 2. REQUIRED_VARS Mechanism
- Reused existing `validate_and_prompt_variables()` function
- Made REQUIRED_VARS dynamic via `set_required_vars_for_deployment()`
- No duplicate validation functions created
### 3. Docker Pre-Check
- Added `check_docker_installed()` before deployment options
- Prevents confusing errors during deployment
- Guides users to Option 1 if Docker missing
### 4. Traefik Provider Configuration
- Auto-generated in `/opt/stacks/core/traefik/dynamic/`
- Format: `docker-provider-{hostname}.yml`
- Traefik auto-reloads within 2 seconds
### 5. Remote Server Registration
- Uses SSH to run functions on core server
- Sources common.sh on core to access functions
- Creates provider and Sablier middleware configs
- Restarts Traefik to apply changes
---
## Files Modified Summary
| File | Lines Changed | Status |
|------|---------------|--------|
| `scripts/common.sh` | +130 | ✅ Complete |
| `scripts/ez-homelab.sh` | +200 | ✅ Complete |
| `docker-compose/core/docker-compose.yml` | -38 | ✅ Complete |
| `docker-compose/sablier/docker-compose.yml` | +19 | ✅ Created |
| `docker-compose/sablier/README.md` | +77 | ✅ Created |
| `config-templates/` | Entire folder | ✅ Deleted |
**Total Lines of Code:** ~430 lines added/modified
---
## Documentation Updates Needed
The following documentation should be updated:
- [ ] README.md - Add multi-server architecture section
- [ ] Quick reference guide - Update deployment options
- [ ] Troubleshooting guide - Add multi-server scenarios
---
## Next Steps
1. **Test on Raspberry Pi 4** - Verify resource constraints handled properly
2. **Create example service** - Document label structure for remote services
3. **Update RoadMap.md** - Mark investigation items as complete
4. **Performance testing** - Verify timeout handling on Pi 4
---
## Notes for Future Maintenance
### Adding New Remote Server:
1. Run Option 3 on new server
2. Script automatically registers with core
3. Deploy services with proper labels
### Removing Remote Server:
1. Delete provider config: `/opt/stacks/core/traefik/dynamic/docker-provider-{hostname}.yml`
2. Delete Sablier config: `/opt/stacks/core/traefik/dynamic/sablier-middleware-{hostname}.yml`
3. Traefik auto-reloads
### Debugging:
- Check Traefik logs: `docker logs traefik`
- Check dynamic configs: `/opt/stacks/core/traefik/dynamic/`
- Verify Docker TLS: `docker -H tcp://remote-ip:2376 --tlsverify ps`
- Check Sablier logs: `docker logs sablier`
---
## Implementation Validation
### Syntax Checks:
```bash
✅ bash -n scripts/ez-homelab.sh
✅ bash -n scripts/common.sh
✅ docker compose -f docker-compose/core/docker-compose.yml config -q
✅ docker compose -f docker-compose/sablier/docker-compose.yml config -q
```
### Code Quality:
- ✅ No VS Code errors/warnings
- ✅ Follows existing code patterns
- ✅ Reuses existing functions appropriately
- ✅ Proper error handling
- ✅ Debug logging included
- ✅ User-friendly messages
---
## Success Criteria - ALL MET ✅
- [x] Sablier in separate stack (not embedded in core)
- [x] Container named "sablier" (not "sablier-service")
- [x] No prompt_for_server_role() function (unnecessary)
- [x] Reused existing validate_and_prompt_variables()
- [x] Dynamic REQUIRED_VARS based on deployment type
- [x] Compose changes in repo files (not script overrides)
- [x] Backup from /opt/stacks/ (not repo)
- [x] Timestamp format: YY_MM_DD_hh_mm
- [x] Docker pre-check before deployment
- [x] Config-templates folder deleted
- [x] All functions properly documented
---
**Implementation Complete!** 🎉
Ready for deployment testing on target hardware (Raspberry Pi 4 4GB).

43
docker-compose/arcane/docker-compose.yml Normal file
View File

@@ -0,0 +1,43 @@
services:
arcane:
image: ghcr.io/getarcaneapp/arcane:latest
container_name: arcane
ports:
- '3552:3552'
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- arcane-data:/app/data
- /opt/stacks:/opt/stacks
environment:
- APP_URL=http://192.168.4.12:3552
- PROJECTS_DIRECTORY=/opt/stacks
- PUID=1000
- PGID=1000
- ENCRYPTION_KEY=5Db7OZ8TtiRNnXi09oh4WN27igS8YohGo45bnBycN7U=
- JWT_SECRET=xwe+pZrRvv18OprdQS+8O5R3fhNekn4pOmikSTJzQIg=
restart: unless-stopped
networks:
- traefik-network
# arcane-agent:
# image: ghcr.io/getarcaneapp/arcane-headless:latest
# container_name: arcane-agent
# restart: unless-stopped
# environment:
# - AGENT_MODE=true
# - AGENT_TOKEN=
# - MANAGER_API_URL=http://192.168.4.4:3552
# ports:
# - "3553:3553"
# volumes:
# - /var/run/docker.sock:/var/run/docker.sock
# - arcane-data:/app/data
# networks:
# - traefik-network
volumes:
arcane-data:
networks:
traefik-network:
external: true

144
docker-compose/core/README.md
View File

@@ -1,6 +1,6 @@
# Core Infrastructure Services # Core Infrastructure Services
This directory contains the core infrastructure services that form the foundation of the homelab. These services should be deployed **on the main server only** and are critical for the operation of all other services across all servers. This directory contains the core infrastructure services that form the foundation of the homelab. These services should be deployed **on the core server only** and are critical for the operation of all other services across all servers.
## Services ## Services
@@ -18,9 +18,7 @@ This directory contains the core infrastructure services that form the foundatio
- **Multi-Server**: Discovers services on all servers via Docker providers - **Multi-Server**: Discovers services on all servers via Docker providers
- **SSL**: Let's Encrypt with DNS-01 challenge (wildcard certificate) - **SSL**: Let's Encrypt with DNS-01 challenge (wildcard certificate)
- **Dashboard**: Available at configured domain - **Dashboard**: Available at configured domain
- **Deploy**: Core server (multi-provider), Remote servers (local-only) - **Deploy**: Core server only
**Note**: Sablier has been moved to its own stack (`/opt/stacks/sablier/`) and should be deployed on each server individually. See [Sablier documentation](../../docs/service-docs/sablier.md) for details.
### Authelia (v4.37.5) ### Authelia (v4.37.5)
- **Purpose**: Single sign-on authentication service for all services across all servers - **Purpose**: Single sign-on authentication service for all services across all servers
@@ -30,30 +28,24 @@ This directory contains the core infrastructure services that form the foundatio
- **Database**: SQLite database in `authelia/config/db.sqlite3` - **Database**: SQLite database in `authelia/config/db.sqlite3`
- **Deploy**: Core server only - **Deploy**: Core server only
### DuckDNS
- **Purpose**: Dynamic DNS service for domain resolution and wildcard SSL certificates
- **Subdomain**: Configurable via environment variables
- **Token**: Configured in environment variables
- **SSL Certificates**: Generates wildcard cert used by all services on all servers
- **Deploy**: Core server only
## Multi-Server Architecture ## Multi-Server Architecture
The core stack on the main server provides centralized services for the entire homelab: The core stack on the main server provides centralized services for the entire homelab:
**Core Server Responsibilities:** **Core Server:**
- Receives all external traffic (ports 80/443 forwarded from router) - Receives all external traffic (ports 80/443 forwarded from router)
- Runs DuckDNS for domain management and SSL certificates - Runs DuckDNS for domain management and SSL certificates
- Runs Authelia for centralized authentication - Runs Authelia for centralized authentication
- Runs multi-provider Traefik that discovers services on all servers - Runs Sablier for lazyloading local containers
- Generates shared CA for Docker TLS communication - Traefik lables route servies on the core server
- Traekif external-host-servername.yml defines routes for Remote Servers
**Remote Server Setup:** **Remote Server:**
- Remote servers run their own Traefik instance (local Docker provider only) - Each container exposes ports
- Remote servers run their own Sablier instance (local container management) - No port forwarding from router needed
- Remote servers expose Docker API on port 2376 with TLS - No Traefik lables
- Core server Traefik connects to remote Docker APIs to discover services - Traefik configured by an external-host yaml file on Core Server
- No port forwarding needed on remote servers - Runs Sablier for lazyloading local containers
**Service Access:** **Service Access:**
- All services accessible via: `https://service.yourdomain.duckdns.org` - All services accessible via: `https://service.yourdomain.duckdns.org`
@@ -98,10 +90,11 @@ core/
├── docker-compose.yml # Main service definitions ├── docker-compose.yml # Main service definitions
├── .env # Environment variables ├── .env # Environment variables
├── authelia/ ├── authelia/
│ └── config/ │ ├── config/
│ ├── configuration.yml # Authelia main config │ | ├── configuration.yml # Authelia main config
│ ├── users_database.yml # User credentials │ | └── notification.txt
│ └── db.sqlite3 # SQLite database | └── secrets/
| └── users_database.yml # User credentials
├── duckdns/ ├── duckdns/
│ └── config/ # DuckDNS configuration │ └── config/ # DuckDNS configuration
├── traefik/ ├── traefik/
@@ -113,106 +106,5 @@ core/
│ │ └── external-host-*.yml # Remote server routing │ │ └── external-host-*.yml # Remote server routing
│ └── letsencrypt/ │ └── letsencrypt/
│ └── acme.json # SSL certificates │ └── acme.json # SSL certificates
└── shared-ca/ # TLS certificates for multi-server
├── ca.pem # Certificate Authority
├── ca-key.pem # CA private key
├── cert.pem # Client certificate
└── key.pem # Client key
``` ```
### Environment Variables (.env)
```bash
# Required for proper operation
DUCKDNS_TOKEN=your_duckdns_token_here
DUCKDNS_SUBDOMAINS=your_subdomain
DOMAIN=yourdomain.duckdns.org
TZ=America/New_York
PUID=1000
PGID=1000
```
### Network Requirements
- Docker network: `traefik-network`
- External ports: 80, 443 must be accessible
- DNS resolution: Domain must point to server IP
## Deployment
### Prerequisites
1. Docker and Docker Compose installed
2. Ports 80/443 forwarded to server
3. DuckDNS account with valid token
4. Domain configured in DuckDNS
### Startup Order
1. `duckdns` - For DNS updates and SSL certificate generation
2. `traefik` - Reverse proxy (waits for SSL certificates)
3. `authelia` - Authentication service
**Note**: Sablier is now deployed separately in `/opt/stacks/sablier/` after core stack is running.
### Commands
```bash
# Start all services
docker-compose up -d
# Check status
docker-compose ps
# View logs
docker-compose logs -f [service-name]
# Restart specific service
docker-compose restart [service-name]
```
## Troubleshooting
### Common Issues
1. **Connection Refused**: Check if Traefik config file is in correct location (`traefik/config/traefik.yml`)
2. **SSL Certificate Issues**: Verify DuckDNS token and domain configuration
3. **Authelia Login Issues**: Check database file exists and configuration is valid
4. **Service Not Starting**: Check Docker logs for error messages
### Backup Strategy
- Configuration files are backed up automatically (see backup directories)
- Database should be backed up regularly
- SSL certificates are stored in `letsencrypt/acme.json`
- Use `backup.sh` script for automated backups
## Security Notes
- Authelia provides authentication for protected services
- All external traffic goes through Traefik with SSL termination
- Internal services communicate via Docker networks
- Dashboard access is protected by Authelia middleware
## Maintenance
- Monitor SSL certificate expiration (Let's Encrypt auto-renews)
- Keep Authelia version pinned until tested upgrades are available
- Regularly backup configuration and database files
- Check logs for security issues or errors
- Run `./backup.sh` regularly to backup critical files
## Customization
### Domain Configuration
Update the following files with your domain:
- `docker-compose.yml`: Traefik labels and Authelia configuration
- `authelia/config/configuration.yml`: Domain settings
- `.env`: Domain environment variables
### SSL Certificate Provider
Modify `traefik/config/traefik.yml` to use different DNS providers:
```yaml
certificatesResolvers:
letsencrypt:
acme:
dnsChallenge:
provider: cloudflare # or other supported provider
```
### Adding New Services
1. Add service definition to `docker-compose.yml`
2. Configure Traefik labels for routing
3. Add middleware for authentication if needed
4. Update network configuration

42
docker-compose/traefik/docker-compose.yml
View File

@@ -1,42 +0,0 @@
# Traefik Service for Remote Servers
# This standalone Traefik instance runs on remote servers to discover local containers
# and communicate with the core Traefik on the core server via Docker TLS
services:
traefik:
# Local Traefik instance for container discovery on this remote server
image: traefik:v3
container_name: traefik
restart: unless-stopped
command:
- '--api.dashboard=true'
- '--api.insecure=true' # Allow API access on port 8080
- '--providers.docker=true'
- '--providers.docker.exposedbydefault=false'
- '--providers.docker.network=traefik-network'
- '--providers.file.directory=/dynamic'
- '--providers.file.watch=true'
- '--log.level=INFO'
- '--accesslog=true'
- '--entrypoints.web.address=:80'
- '--entrypoints.websecure.address=:443'
environment:
- TZ=America/New_York
ports:
- '80:80' # HTTP entrypoint
- '443:443' # HTTPS entrypoint
- '8080:8080' # Dashboard (optional, for debugging)
volumes:
- ./config:/config
- ./dynamic:/dynamic
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- traefik-network
labels:
- 'homelab.category=infrastructure'
- 'homelab.description=Local reverse proxy for container discovery'
- 'traefik.enable=false' # This Traefik doesn't route itself
networks:
traefik-network:
external: true

8
docs/ai-vscode-setup.md
View File

@@ -31,7 +31,7 @@ This guide shows you how to use VS Code with GitHub Copilot on your local PC to
With VS Code connected to your server, you can now use GitHub Copilot to guide you through the entire setup process: With VS Code connected to your server, you can now use GitHub Copilot to guide you through the entire setup process:
### Initial Server Setup ### Initial Server Setup
- **Clone repository**: Ask Copilot "Help me clone the AI-Homelab repository" - **Clone repository**: Ask Copilot "Help me clone the EZ-Homelab repository"
- **Configure environment**: "Guide me through setting up the .env file" - **Configure environment**: "Guide me through setting up the .env file"
- **Run setup scripts**: "Walk me through running the ez-homelab.sh script" - **Run setup scripts**: "Walk me through running the ez-homelab.sh script"
- **Deploy services**: "Help me run the deployment script" - **Deploy services**: "Help me run the deployment script"
@@ -43,14 +43,14 @@ The AI will help you:
- Set up user accounts and permissions - Set up user accounts and permissions
- Troubleshoot any issues that arise - Troubleshoot any issues that arise
## Step 4: Open the AI-Homelab Repository ## Step 4: Open the EZ-Homelab Repository
1. Once connected to your server, open the terminal in VS Code (Ctrl+`) 1. Once connected to your server, open the terminal in VS Code (Ctrl+`)
2. Navigate to your repository: 2. Navigate to your repository:
```bash ```bash
cd ~/AI-Homelab cd ~/EZ-Homelab
``` ```
3. Open the folder in VS Code: `File > Open Folder` and select `/home/your-user/AI-Homelab` 3. Open the folder in VS Code: `File > Open Folder` and select `/home/your-user/EZ-Homelab`
## Step 5: Enable GitHub Copilot ## Step 5: Enable GitHub Copilot

60
docs/core-stack-readme.md
View File

@@ -1,60 +0,0 @@
# Core Stack
## The core stack contains the critical infrastructure for a homelab.
>There are always other options.
I chose these for easy of use and file based configuration (so AI can do it all)
>For a multi server homelab, only install core services on one server.
* DuckDNS with LetsEncrypt- Free subdomains and wildcard SSL Certificates
* Authelia - Single Sign On (SSO) Authentication with optional 2 Factor Authentication
* Traefik - Proxy Host Routing to access your services through your duckdns url
* Sablier - Both a container and a Traefik plugin, enables ondemand services (saving resources and power)
## DuckDNS with LetsEncrypt
Get your free subdomain at http://www.duckdns.org
>Copy your DuckDNS Token, you'll need to add that to the .env file
DuckDNS service will keep your subdomain pointing to your IP address, even if it changes.
LetsEncrypt service will generate 2 certificates, one for your duckdns subdomain, and one wildcard for the same domain.
The wildcard certificate is used by all the services. No need for an individual certificate per service.
Certificates will auto renew. This is a set it and forget service, no webui, no changes needed when adding a new service.
>It can take 2-5 minutes for the certificates to be generated and applied. Until then it will use a self signed certificate. You just need to accept the browser warnings. Once it's applied the browser warnings will go away.
## Authelia
Provides Single Sign On (SSO) Authentication for your services.
>Some services you may not want behind SSO, like Jellyfin/Plex if you watch your media through an app on a smart TV, firestick or phone.
* Optional 2 Factor Authentication
* Easy to enable/disable - Comment out 1 line in the compose file then bring the stack down and back up to disable (not restart)
* Configured on a per service basis
* A public service (like wordpress) can bypass authelia login and let the service handle it's own authentication
* Admin services (like dockge/portainer) can use 2 Factor Authorization for an extra layer of security
## Traefik
Provides proxy routing for your services so you can access them at a url like wordpress.my-subdoain.duckdns.org
For services on a Remote Host, create a file in the traefik/dynamic folder like my-server-external-host.yml
Create 1 file per remote host.
## Sablier
Provides ondemand container management (start/pause/stop)
>Important: The stack must be up and the service stopped for Sablier to work.
Tip: If you have a lot of services, use a script to get the services to that state.
If a service is down and anything requests the url for that service, Sablier will start the service and redirect after a short delay (however long it takes the service to come up), usually less than 20 seconds.
Once the set inactivity period has elapsed, Sablier will pause or stop the container according to the settings.
This saves resources and electricity. Allowing you have more services installed, configured, and ready to use even if the server is incapable of running all the services simultaniously. Great for single board computers, mini PCs, low resource systems, and your power bill.

178
docs/docker-guidelines.md
View File

@@ -67,32 +67,43 @@ All stacks live in `/opt/stacks/stack-name/`:
``` ```
/opt/stacks/ /opt/stacks/
├── traefik/ ├── traefik/
│ ├── docker-compose.yml │ ├── docker-compose.yml
│ ├── traefik.yml # Static config │ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes │ ├── dynamic/ # Dynamic routes
│ │ ├── routes.yml │ │ ├── routes.yml
│ │ └── external.yml # External host proxying │ │ └── external.yml # External host proxying
│ ├── acme.json # SSL certificates (chmod 600) │ ├── acme.json # SSL certificates (chmod 600)
│ └── .env │ └── .env
├── authelia/ ├── authelia/
│ ├── docker-compose.yml │ ├── config/
│ ├── configuration.yml # Authelia settings │ | ├── configuration.yml # Authelia main config
│ ├── users_database.yml # User accounts │ | └── notification.txt
│ └── .env | └── secrets/
├── media/ | └── users_database.yml # User credentials
│ ├── docker-compose.yml ├── media/
│ └── .env │ ├── docker-compose.yml
└── ... │ └── .env
└── ...
``` ```
### Why Dockge? ### Why Dockge?
- **Visual Management**: Web UI at `https://dockge.${DOMAIN}` - **Visual Management**: Simple Web UI at `https://dockge.${DOMAIN}`
- **Direct File Editing**: Edit compose files in-place - **Beginner Friendly**: No advancted options, simple control & edit functionality
- **Stack Organization**: Each service stack is independent - **Stack Organization**: Keep dependant services together, or just 1 service per stack
- **AI Compatible**: Files can be managed by AI - **AI Compatible**: Files can be managed by AI
- **Git Integration**: Easy to version control
### **x-dockge Section:**
At the bottom of the compose file, add a top-level `x-dockge` section for service discovery in Dockge:
```yaml
x-dockge:
urls:
- https://myservice.${DOMAIN}
- http://${SERVER_IP}:8080 # Direct local access
```
### Storage Strategy ### Storage Strategy
@@ -113,15 +124,8 @@ volumes:
AI will suggest `/mnt/` when data may exceed 50GB or grow continuously. AI will suggest `/mnt/` when data may exceed 50GB or grow continuously.
## Traefik and Authelia Integration
### Routing Decision Tree ## Traefik Labels for services on Core Server only
**Is Traefik running on the SAME server as your service?**
- **YES**: Use Docker labels in the service's compose file (see below)
- **NO**: Comment out labels; add route to Traefik's external YAML file on the core server
### Every Local (on the same server) Service Needs Traefik Labels
**Default Configuration**: All services should use authelia SSO, traefik routing, and sablier lazy loading by default. **Default Configuration**: All services should use authelia SSO, traefik routing, and sablier lazy loading by default.
@@ -179,42 +183,8 @@ services:
- `sablier.group=${SERVER_HOSTNAME}-myservice` - Groups containers for coordinated startup - `sablier.group=${SERVER_HOSTNAME}-myservice` - Groups containers for coordinated startup
- `sablier.start-on-demand=true` - Starts containers only when accessed - `sablier.start-on-demand=true` - Starts containers only when accessed
**x-dockge Section:** ## **Services on all Remote Servers**
At the bottom of the compose file, add a top-level `x-dockge` section for service discovery in Dockge: - Create external-host-servername.yml on the Core Server in Traefik's dynamic folder
```yaml
x-dockge:
urls:
- https://myservice.${DOMAIN}
- http://localhost:8080 # Direct local access
```
### If Traefik is on a Remote Server, configure routes & services on the Remote Server
When Traefik runs on a separate server from your application services, you cannot use Docker labels for configuration. Instead, create YAML files in the Traefik server's `dynamic/` directory to define routes and services.
#### When to Use Remote Traefik Configuration
Use this approach when:
- Traefik runs on a dedicated reverse proxy server
- Application services run on separate application servers
- You want centralized routing configuration
- Docker labels cannot be used (different servers)
#### File Organization
Create one YAML file per application server in `/opt/stacks/traefik/dynamic/`:
```
/opt/stacks/traefik/dynamic/
├── server1.example.com.yml # Services on server1
├── server2.example.com.yml # Services on server2
├── shared-services.yml # Common services
└── sablier.yml # Sablier middlewares
```
#### YAML File Structure
Each server-specific YAML file should contain: Each server-specific YAML file should contain:
```yaml ```yaml
@@ -259,6 +229,17 @@ http:
passhostheader: true passhostheader: true
``` ```
#### File Organization
Create one YAML file per application server in `/opt/stacks/traefik/dynamic/`:
```
/opt/stacks/traefik/dynamic/
├── external-host-server1.yml # Services on server1
├── external-host-server2.yml # Services on server2
└── sablier.yml # Sablier middlewares
```
#### Complete Example for a Media Server #### Complete Example for a Media Server
```yaml ```yaml
@@ -361,7 +342,7 @@ When moving from Docker labels to YAML configuration:
This approach provides centralized, version-controllable routing configuration while maintaining the same security and performance benefits as Docker label-based configuration. This approach provides centralized, version-controllable routing configuration while maintaining the same security and performance benefits as Docker label-based configuration.
### When to Use Authelia SSO ## When to Use Authelia SSO
**Protect with Authelia** (Default for all services): **Protect with Authelia** (Default for all services):
- Admin interfaces (Sonarr, Radarr, Prowlarr, etc.) - Admin interfaces (Sonarr, Radarr, Prowlarr, etc.)
@@ -388,7 +369,7 @@ access_control:
policy: bypass policy: bypass
``` ```
### Routing Through VPN (Gluetun) ## Routing Through VPN (Gluetun)
For services that need VPN (downloads): For services that need VPN (downloads):
@@ -421,27 +402,6 @@ Use Docker Compose for:
- Services with complex configurations - Services with complex configurations
- Anything you want to keep long-term - Anything you want to keep long-term
**Example:**
```yaml
# docker-compose/plex.yml
services:
plex:
image: plexinc/pms-docker:1.40.0.7998-f68041501
container_name: plex
restart: unless-stopped
networks:
- media-network
ports:
- "32400:32400"
volumes:
- ./config/plex:/config
- /media:/media:ro
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
```
### Docker Run: For Temporary Operations Only ### Docker Run: For Temporary Operations Only
Use `docker run` for: Use `docker run` for:
@@ -585,30 +545,34 @@ networks:
external: true external: true
``` ```
**Required: Add to Sablier YAML file (e.g., `/opt/stacks/traefik/dynamic/sablier.yml`):**
```yaml
http:
middlewares:
sablier-server1-servicename:
plugin:
sablier:
sablierUrl: http://sablier-service:10000
group: server1-servicename
sessionDuration: 5m
ignoreUserAgent: curl
dynamic:
displayName: Service Name
theme: ghost
show-details-by-default: true
```
### Multi-Server Configuration ### Multi-Server Configuration
If Traefik is on a DIFFERENT server (e.g., service on remote server, Traefik on core): On Remote Servers:
* Comment out the traefik labels since they won't be used, don't delete them. * Remove traefik labels
* Keep Sablier labels (each server has its own Sablier for local containers) * Keep Sablier labels (each server has its own Sablier for local containers)
* Add route to Traefik's external YAML file on the core server * Add route to external-host yml file in Traefik's dynamic folder on the core server
* Authelia SSO is handled by core server (no need for Authelia on remote servers) * Authelia SSO is handled by the external-host yml file
**Example: Comment out Traefik labels in docker-compose.yml:** **Example: Remove Traefik labels in docker-compose.yml:**
```yaml ```yaml
labels: labels:
# TRAEFIK CONFIGURATION
# ==========================================
# Service metadata
- "com.centurylinklabs.watchtower.enable=true"
- "homelab.category=category-name"
- "homelab.description=Brief service description"
# Traefik labels - COMMENTED OUT for remote server
# - "traefik.enable=true"
# - "traefik.http.routers.service-name.rule=Host(`service-name.${DOMAIN}`)"
# - "traefik.http.routers.service-name.entrypoints=websecure"
# - "traefik.http.routers.service-name.tls.certresolver=letsencrypt"
# - "traefik.http.routers.service-name.middlewares=authelia@docker"
# - "traefik.http.services.service-name.loadbalancer.server.port=8080"
# Sablier configuration # Sablier configuration
- "sablier.enable=true" - "sablier.enable=true"
- "sablier.group=${SERVER_HOSTNAME}-service-name" - "sablier.group=${SERVER_HOSTNAME}-service-name"
@@ -638,7 +602,7 @@ http:
passhostheader: true passhostheader: true
``` ```
**Required: Add to Sablier YAML file (e.g., `/opt/stacks/traefik/dynamic/sablier.yml`):** **Required: Add to Sablier YAML file (e.g., `/opt/stacks/sablier/config/sablier.yml`):**
```yaml ```yaml
sablier-server1-servicename: sablier-server1-servicename:
plugin: plugin:

60
markup.yml
View File

@@ -1,60 +0,0 @@
echo "╔═════════════════════════════════════════════════════════════╗"
echo "║ EZ-HOMELAB SETUP & DEPLOYMENT ║"
echo "║ ║"
echo "║ 1) Install Docker ║"
echo "║ 2) Deploy Core Server ║"
echo "║ 3) Deploy Additional Server ║"
echo "║ 4) Install NVIDIA Drivers ║"
echo "║ ║"
echo "║ q) Quit ║"
echo "╚═════════════════════════════════════════════════════════════╝"
echo "╔═════════════════════════════════════════════════════════════╗
echo "║ ✅ SERVER_IP: 192.168.4.4 ║
echo "║ ✅ SERVER_HOSTNAME: jasper ║
echo "║ ✅ DUCKDNS_SUBDOMAINS: yourdomain ║
echo "║ ✅ DUCKDNS_TOKEN: your-duckdns-token ║
echo "║ ✅ DOMAIN: yourdomain.duckdns.org ║
echo "║ ✅ DEFAULT_USER: admin ║
echo "║ ✅ DEFAULT_PASSWORD: changeme ║
echo "║ ✅ DEFAULT_EMAIL: admin@example.com ║
echo "╚═════════════════════════════════════════════════════════════╝
echo "╔═════════════════════════════════════════════════════════════╗
echo "║ Use These Values ? (Y/n) ║
echo "╚═════════════════════════════════════════════════════════════╝
echo "╔═════════════════════════════════════════════════════════════╗
echo "║ Deployment Complete! ║
echo "║ SSL Certificates may take a few minutes to be issued. ║
echo "║ ║
echo "║ https://dockge.yourdomain.duckdns.org ║
echo "║ http://192.168.4.4:5001 ║
echo "║ ║
echo "║ https://homepage.yourdomain.duckdns.org ║
echo "║ http://192.168.4.4:3003 ║
echo "║ ║
echo "║ https://authelia.yourdomain.duckdns.org ║
echo "║ http://192.168.4.4:9091 ║
echo "║ ║
echo "║ https://traefik.yourdomain.duckdns.org ║
echo "║ http://192.168.4.4:8080 ║
echo "║ ║
echo "╚═════════════════════════════════════════════════════════════╝
echo "╔═════════════════════════════════════════════════════════════╗
echo "║ ⚠️ WARNING ⚠️ ║
echo "║ The following variables were not defined ║
echo "║ If something isn't working as expected check these first ║
echo "║ ║
echo "║ WATCHTOWER_NOTIFICATION_URL WATCHTOWER_NOTIFICATION_URL ║
echo "║ AUTHENTIK_DB_NAME AUTHENTIK_DB_PASSWORD ║
echo "║ ║
echo "╚═════════════════════════════════════════════════════════════╝

38
release-notes-v0.1.0.md
View File

@@ -1,38 +0,0 @@
# EZ-Homelab Release Notes - v0.1.0
## Overview
EZ-Homelab v0.1.0 is the first official release of this Docker homelab infrastructure. Tested on Debian 12, it deploys 50+ services with automated SSL, SSO authentication, and resource-efficient lazy loading. This release focuses on ease of setup, security, and scalability for self-hosted environments.
## What's New
- 🚀 **Sablier Lazy Loading**: Automatically starts services on-demand to save resources and reduce power costs. Enabled by default on most services; dependent services (e.g., *arr apps) load as groups.
- 🔒 **Security Enhancements**: Authelia SSO enabled by default (with optional 2FA); TLS certificates for Docker proxy; secure routing via Traefik.
- 🌐 **DNS & Proxy**: DuckDNS integration with Let's Encrypt wildcard SSL; Traefik routing for local services (via labels) and remote servers (via external host files). Subdomains like `service.yoursubdomain.duckdns.org` for web UIs, with multi-server support (e.g., `dockge.serverhostname.yoursubdomain.duckdns.org`).
- 📊 **Dashboards**: Preconfigured Homepage at `homepage.yoursubdomain.duckdns.org` for easy service access. Lazy loading requires stacks to be up.
- 🛠️ **Setup Improvements**: Unified `ez-homelab.sh` script with refined options; detailed UX for fresh OS installs.
## Services Included
Preconfigured with Traefik and Sablier (most require initial web UI setup):
- Core, Infrastructure, Dashboards, Media, Media Management, Productivity, Transcoders, Utilities, VPN, and Wikis stacks.
- **Notes**: Monitoring stack not yet configured for Traefik/Sablier. Alternatives stack is untested.
## Installation & Setup
- **Automated (Recommended)**: Run `./ez-homelab.sh` (Option 3 confirmed working on fresh Debian 12 with existing core server; Options 1 & 2 need additional testing).
- **Manual**: Follow [Manual Setup Guide](docs/manual-setup.md)—may require refinement.
- **Fresh OS Steps** (e.g., Debian):
1. As root: `apt update && apt upgrade -y && apt install git sudo -y && usermod -aG sudo yourusername`.
2. Exit and log in as user: `cd ~ && git clone https://github.com/kelinfoxy/EZ-Homelab.git`.
3. Install Docker: `sudo ./scripts/ez-homelab.sh`.
4. Exit/login and run: `./scripts/ez-homelab.sh` (without sudo).
- **Post-Setup**: Script provides Dockge link. Core, Infrastructure, and Dashboards stacks run automatically; others are inactive.
## Known Issues & Limitations
- **ez-homelab.sh**: Options 1 & 2 require additional testing.
- **Sablier**: May cause short delays, timeouts, or Bad Gateway errors on first access—refresh the page once the container is healthy.
- **Manual Install**: Instructions may need refinement.
- **GitHub Wiki**: Mostly accurate but needs updates.
## Upgrading from Previous Versions
No previous versions exist—this is the initial release.
## Thanks & Feedback
Thanks to the community for early feedback! Report issues or contribute via GitHub. See [Getting Started](docs/getting-started.md) for more details.

42
release-notes-v0.1.1.md
View File

@@ -1,42 +0,0 @@
# EZ-Homelab Release Notes - v0.1.1
## Overview
EZ-Homelab v0.1.1 includes significant improvements to configuration templates, documentation, and infrastructure setup. This maintenance release focuses on enhanced user experience, better organization, and comprehensive service configurations.
## What's New
- 📚 **Enhanced Documentation**: Added comprehensive TUI deployment script documentation and product requirements
- 🏠 **Homepage Improvements**: Complete dashboard configuration templates with custom CSS, widgets, and service integration
- 🔀 **Traefik Enhancements**: Updated dynamic routing configurations for better external host proxying and local service management
- 🐳 **Docker Compose Updates**: Improved infrastructure and dashboard stack configurations
- 📋 **Environment Templates**: Updated .env.example with latest variables and configurations
- 🔧 **Script Refinements**: Enhanced ez-homelab.sh with better error handling and configuration management
## Configuration Improvements
- **Homepage Dashboard**: Complete service catalog with bookmarks, widgets, and custom styling
- **Traefik Routing**: Enhanced external host proxying with improved middleware configurations
- **Service Templates**: Updated docker-compose files for better resource management and networking
- **Documentation**: Added Homelab-Audit documentation for system monitoring and maintenance
## Technical Updates
- Improved Traefik dynamic configuration templates
- Enhanced Sablier lazy loading middleware setup
- Updated environment variable handling
- Better error handling in deployment scripts
## Installation & Setup
No changes to installation process. Follow the same steps as v0.1.0:
- Run `./ez-homelab.sh` for automated setup
- Access services through Dockge at `dockge.yoursubdomain.duckdns.org`
## Upgrading from v0.1.0
- Pull latest changes: `git pull origin main`
- Update configurations: Copy new templates from `config-templates/`
- Restart services if needed: Use Dockge UI or docker-compose commands
## Known Issues
- Same as v0.1.0, with improved error handling for configuration issues
- Sablier lazy loading may cause initial access delays (refresh page after container starts)
## Thanks & Feedback
Continued improvements based on community feedback. Report issues or contribute via GitHub.</content>
<parameter name="filePath">c:\Users\kelin\Documents\Apps\GitHub\EZ-Homelab\release-notes-v0.1.1.md

0
aliases.sh → scripts/aliases.sh
View File