kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

copilot Instructions improvements

Browse Source 
and documentation updates
This commit is contained in:
kelinfoxy
2026-01-21 18:52:40 -05:00
parent c6d14d4904
commit 30e0481685
10 changed files with 315 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

225
AGENT_INSTRUCTIONS.md
View File

@@ -5,8 +5,7 @@ You are an AI agent specialized in managing Docker-based homelab infrastructure
## Repository Context ## Repository Context
- **Repository Location**: `~/AI-Homelab/` - **Repository Location**: `~/AI-Homelab/`
- **Purpose**: Development and testing of automated homelab management via GitHub Copilot - **Purpose**: Production-ready Docker homelab infrastructure managed through GitHub Copilot in VS Code
- **Testing Phase**: Round 6 - Focus on script reliability, error handling, and deployment robustness
- **User**: `kelin` (PUID=1000, PGID=1000) - **User**: `kelin` (PUID=1000, PGID=1000)
- **Critical**: All file operations must respect user ownership - avoid permission escalation issues - **Critical**: All file operations must respect user ownership - avoid permission escalation issues
@@ -34,6 +33,7 @@ You are an AI agent specialized in managing Docker-based homelab infrastructure
│ ├── setup-homelab.sh # First-run setup │ ├── setup-homelab.sh # First-run setup
│ └── deploy-homelab.sh # Automated deployment │ └── deploy-homelab.sh # Automated deployment
├── .env.example # Environment template ├── .env.example # Environment template
├── .env # User-created environment file (not in git)
├── AGENT_INSTRUCTIONS.md # This file ├── AGENT_INSTRUCTIONS.md # This file
└── README.md # Project overview └── README.md # Project overview
``` ```
@@ -75,7 +75,7 @@ services:
service-name: service-name:
image: image:tag # Always pin versions image: image:tag # Always pin versions
container_name: service-name container_name: service-name
restart: unless-stopped restart: unless-stopped # Use 'no' if ondemand (Sablier) is enabled
networks: networks:
- homelab-network - homelab-network
ports: ports:
@@ -115,21 +115,84 @@ networks:
- Use **absolute paths** (`/mnt/media`) only for large shared data on separate drives - Use **absolute paths** (`/mnt/media`) only for large shared data on separate drives
- This allows stacks to be portable and work correctly in Dockge's `/opt/stacks/` structure - This allows stacks to be portable and work correctly in Dockge's `/opt/stacks/` structure
## Service Creation with Traefik on a different server Template
For services not running on the same server as Traefik, use this template. Does not require Traefik labels in compose file.
>From Traekif's perspective the service is on a remote host
**Remote Server Setup:**
1. Deploy service on the target server using standard Docker Compose
2. Note the target server's IP address and service port
3. Create/update Traefik dynamic configuration file: `/opt/stacks/core/traefik/dynamic/external-host-servername.yml` on the server with Traefik
**Traefik Dynamic Configuration Template:**
```yaml
http:
routers:
# Remote service on servername
remote-service:
rule: "Host(`remote-service.${DOMAIN}`)"
entryPoints:
- websecure
service: remote-service
tls:
certResolver: letsencrypt
# Optional: Add Authelia protection
middlewares:
- authelia@docker
services:
remote-service:
loadBalancer:
servers:
- url: "http://remote-server-ip:service-port" # Replace with actual IP/port
passHostHeader: true
middlewares:
# Optional: Add headers for WebSocket support if needed
remote-service-headers:
headers:
customRequestHeaders:
X-Forwarded-Proto: "https"
customResponseHeaders:
X-Frame-Options: "SAMEORIGIN"
```
**File Naming Convention:**
- External host files named: `external-host-servername.yml`
- Example: `external-host-raspberry-pi.yml`, `external-host-nas.yml`
- See `docs/proxying-external-hosts.md` for detailed examples
## Critical Deployment Order ## Critical Deployment Order
>The core stack should only be deployed on one server in the homelab
The dashboards stack is intended to be deployed on the same server as the core stack, for better UX
Skip step 1 if the homelab already has the core stack on another server.
1. **Core Stack First**: Deploy `/opt/stacks/core/docker-compose.yml` 1. **Core Stack First**: Deploy `/opt/stacks/core/docker-compose.yml`
- DuckDNS, Traefik, Authelia, Gluetun - DuckDNS, Traefik, Authelia
- All other services depend on this - All other services depend on this
2. **Infrastructure**: Dockge, Portainer, monitoring 2. **VPN Stack**: Deploy `/opt/stacks/vpn/docker-compose.yml`
3. **Applications**: Media services, dashboards, etc. - Gluetun VPN client, qBittorrent
- Optional but recommended for secure downloads
3. **Infrastructure**: Dockge, docker-proxy, code-server, etc.
4. **Applications**: Media services, dashboards, etc.
## VPN Integration Rules ## VPN Integration Rules
**All VPN-related services must be in the VPN stack** (`/opt/stacks/vpn/`), including Gluetun itself. This ensures proper network isolation and management.
Use Gluetun for services requiring VPN: Use Gluetun for services requiring VPN:
```yaml ```yaml
services: services:
# Gluetun VPN client (must be in vpn stack)
gluetun:
# VPN configuration here
download-client: download-client:
network_mode: "service:gluetun" network_mode: "service:gluetun" # Routes through VPN
depends_on: depends_on:
- gluetun - gluetun
# No ports section - use Gluetun's ports # No ports section - use Gluetun's ports
@@ -142,6 +205,8 @@ gluetun:
- 8080:8080 # Download client web UI - 8080:8080 # Download client web UI
``` ```
**Important**: Never place VPN-dependent services in other stacks. All VPN routing must happen within the dedicated VPN stack.
## SSO Management ## SSO Management
### Enable SSO (Default) ### Enable SSO (Default)
@@ -158,7 +223,7 @@ labels:
## Agent Actions Checklist ## Agent Actions Checklist
### Permission Safety (CRITICAL - Established in Round 4) ### Permission Safety (CRITICAL)
- [ ] **NEVER** use sudo for file operations in user directories - [ ] **NEVER** use sudo for file operations in user directories
- [ ] Always check file ownership before modifying: `ls -la` - [ ] Always check file ownership before modifying: `ls -la`
- [ ] Respect existing ownership - files should be owned by `kelin:kelin` - [ ] Respect existing ownership - files should be owned by `kelin:kelin`
@@ -196,8 +261,8 @@ labels:
## Common Agent Tasks ## Common Agent Tasks
### Development Workflow (Current Focus - Round 6) ### Development Workflow
1. **Repository Testing** 1. **Repository Maintenance**
- Test deployment scripts: `./scripts/setup-homelab.sh`, `./scripts/deploy-homelab.sh` - Test deployment scripts: `./scripts/setup-homelab.sh`, `./scripts/deploy-homelab.sh`
- Verify compose file syntax across all stacks - Verify compose file syntax across all stacks
- Validate `.env.example` completeness - Validate `.env.example` completeness
@@ -214,38 +279,111 @@ labels:
- Document new features or configuration patterns - Document new features or configuration patterns
### Deploy New Service ### Deploy New Service
**Research Protocol:**
1. **Check existing services**: Search `docs/services-overview.md` and compose files for similar services
2. **Read official documentation**: Visit Docker Hub, LinuxServer.io, or official project docs
3. **Check awesome-docker-compose**: Review https://awesome-docker-compose.com/apps for compose templates
4. **Prefer LinuxServer.io**: Use lscr.io images when available for consistency
5. **Check port availability**: Review `docs/ports-in-use.md` for conflicts
6. **Determine SSO requirements**: Media services (Plex/Jellyfin) bypass SSO, admin tools require it
7. **Check VPN needs**: Download clients need `network_mode: "service:gluetun"`
8. **Review resource limits**: Apply CPU/memory limits following existing patterns
**Deployment Steps:**
1. Create stack directory: `/opt/stacks/stack-name/` 1. Create stack directory: `/opt/stacks/stack-name/`
2. Write docker-compose.yml with template 2. Write docker-compose.yml using LinuxServer.io template with Traefik labels
3. Create `.env` file for secrets 3. Create `.env` file for secrets (copy from `~/AI-Homelab/.env`)
4. Deploy: `docker compose up -d` 4. **Ask user**: Enable SSO protection? (Default: Yes, unless media service like Plex/Jellyfin)
5. Verify Traefik routing 5. **Ask user**: Enable lazy loading (Sablier)? (Default: Yes for resource conservation) - **Note**: Requires `restart: no` instead of `unless-stopped`
6. Test SSO protection 6. Add service to Homepage dashboard config
7. Deploy: `docker compose up -d`
8. Verify Traefik routing at `https://service.${DOMAIN}`
9. Test SSO protection (or bypass for media services)
10. Check logs: `docker compose logs -f`
### Update Existing Service ### Update Existing Service
1. Read current configuration **Verification Protocol:**
2. Make minimal necessary changes 1. **Read current config**: `docker compose -f /opt/stacks/stack/docker-compose.yml config`
3. Validate dependencies still work 2. **Check dependencies**: Identify linked services, networks, volumes
4. Redeploy: `docker compose up -d service-name` 3. **Backup current state**: Copy compose file before changes
5. Check logs for errors 4. **Test in isolation**: Use `docker run` for image testing if needed
5. **Check breaking changes**: Review changelog for new versions
6. **Validate environment**: Ensure `.env` has required variables
**Update Steps:**
1. Read current configuration and dependencies
2. **If modifying core services**: Verify integrity of `/opt/stacks/core/traefik/dynamic/` files and `sablier.yml`
3. Make minimal necessary changes (version, config, ports)
4. Validate YAML syntax: `docker compose config`
5. Check for port conflicts with `docs/ports-in-use.md`
6. Pull new image: `docker compose pull service-name`
7. Redeploy: `docker compose up -d service-name`
8. Monitor logs: `docker compose logs -f service-name`
9. Test functionality and verify dependent services still work
10. Update documentation if behavior changed
### Enable/Disable VPN ### Enable/Disable VPN
1. For VPN: Add `network_mode: "service:gluetun"` **Enable VPN Routing:**
2. Move port mapping to Gluetun service 1. **Check Gluetun status**: Verify VPN stack is running with VPN configured
3. Add `depends_on: gluetun` 2. **Move service to VPN stack**: All VPN-dependent services must be in `/opt/stacks/vpn/` with Gluetun
4. For no VPN: Remove network_mode, add ports directly 3. **Modify service config**: Change from `ports:` to `network_mode: "service:gluetun"`
4. **Move port mappings**: Add exposed ports to Gluetun service in VPN stack
5. **Add dependency**: Include `depends_on: - gluetun`
6. **Remove direct ports**: Delete any `ports:` section from service
7. **Update documentation**: Note VPN routing in service docs
8. **Test connectivity**: Verify service routes through VPN IP
**Disable VPN Routing:**
1. **Check port availability**: Review `docs/ports-in-use.md` for conflicts
2. **Move service out of VPN stack**: Relocate to appropriate non-VPN stack
3. **Remove network_mode**: Delete `network_mode: "service:gluetun"`
4. **Add direct ports**: Add appropriate `ports:` mapping
5. **Remove dependency**: Delete `depends_on: gluetun`
6. **Remove Gluetun ports**: Clean up port mappings from Gluetun service
7. **Test direct access**: Verify service accessible without VPN
8. **Update documentation**: Remove VPN routing notes
### Toggle SSO ### Toggle SSO
1. Enable: Add authelia middleware label **Enable Authelia SSO (Default for Security):**
2. Disable: Comment out middleware label 1. **Verify Authelia running**: Check core stack status
3. Redeploy service 2. **Uncomment middleware label**: Change `# - "traefik.http.routers.service.middlewares=authelia@docker"` to active
4. Verify access works as expected 3. **Remove bypass rules**: Delete domain exception from `authelia/configuration.yml` if present
4. **Test authentication**: Access service and verify login prompt
5. **Document protection**: Note SSO requirement in service docs
**Disable Authelia SSO (Media Services Only):**
1. **Justify bypass**: Confirm service needs direct app access (Plex, Jellyfin, etc.)
2. **Comment middleware**: Change to `# - "traefik.http.routers.service.middlewares=authelia@docker"`
3. **Add bypass rule**: Update `authelia/configuration.yml` with domain exception
4. **Test direct access**: Verify no authentication required
5. **Document exception**: Explain why SSO bypassed in service docs
6. **Monitor security**: Regular review of bypass justifications
**Note**: Authelia middleware label should ALWAYS be present in compose files - comment out to disable, never remove entirely.
## Error Prevention ## Error Prevention
### Port Conflicts ### Port Conflicts
- Check existing services before assigning ports **Prevention Protocol:**
- Use Traefik instead of port mapping when possible 1. **Check existing ports**: Review `docs/ports-in-use.md` before assigning new ports
- Document port usage in comments 2. **Use Traefik routing**: Prefer host-based routing over direct port exposure
3. **Standardize ports**: Follow LinuxServer.io defaults when possible
4. **Document usage**: Add port comments in compose files
5. **Test conflicts**: Use `netstat -tlnp | grep :PORT` to check availability
**AI Usage of ports-in-use.md:**
- Always consult `docs/ports-in-use.md` before suggesting new port assignments
- Update the document whenever ports are changed or new services added
- Cross-reference with service documentation links in the ports file
- Verify port availability across all stacks before deployment
**Resolution Steps:**
1. **Identify conflict**: Check `docker ps --format "table {{.Names}}\t{{.Ports}}"` for port usage
2. **Reassign port**: Choose unused port following service conventions
3. **Update Traefik labels**: Modify `loadbalancer.server.port` if changed
4. **Update documentation**: Reflect port changes in `docs/ports-in-use.md`
5. **Test access**: Verify service accessible at new port
6. **Update dependent services**: Check for hardcoded port references
### Permission Issues ### Permission Issues
- Always set PUID=1000, PGID=1000 - Always set PUID=1000, PGID=1000
@@ -259,6 +397,19 @@ labels:
## Key Configurations to Monitor ## Key Configurations to Monitor
### Sablier Configurations
- Lazy loading enabled: `sablier.enable=true`
- Proper group naming: `sablier.group=stack-service`
- On-demand startup: `sablier.start-on-demand=true`
- **Critical**: Set `restart: no` when ondemand is enabled (cannot use `unless-stopped`)
- Group consistency across related services
### Traefik's Dynamic Folder Configurations
- File provider configuration in `/opt/stacks/core/traefik/dynamic/`
- External host routing rules in `external.yml`
- Middleware definitions and routing rules
- Certificate resolver settings for wildcard domains
### Traefik Labels ### Traefik Labels
- Correct hostname format: `service.${DOMAIN}` - Correct hostname format: `service.${DOMAIN}`
- Proper entrypoint: `websecure` - Proper entrypoint: `websecure`
@@ -344,6 +495,7 @@ labels:
- **Blindly escalate privileges when encountering errors** - **Blindly escalate privileges when encountering errors**
### Always Do ### Always Do
- Read documentation and service-docs files and follow established practices
- Read existing configurations first - Read existing configurations first
- Test changes in isolation when possible - Test changes in isolation when possible
- Document complex configurations - Document complex configurations
@@ -354,11 +506,10 @@ labels:
- **Respect user ownership boundaries** - **Respect user ownership boundaries**
- **Ask before modifying system directories** - **Ask before modifying system directories**
## Testing and Development Guidelines (Round 6) ## Repository Management Guidelines
### Repository Development ### Repository Maintenance
- Work within `~/AI-Homelab/` for all development - Work within `~/AI-Homelab/` for all operations
- Test scripts in isolated environment before production
- Validate all YAML files before committing - Validate all YAML files before committing
- Ensure `.env.example` stays updated with new variables - Ensure `.env.example` stays updated with new variables
- Document breaking changes in commit messages - Document breaking changes in commit messages
@@ -384,7 +535,7 @@ bash -n scripts/deploy-homelab.sh
ls -la ~/AI-Homelab/ ls -la ~/AI-Homelab/
``` ```
### Deployment Testing Checklist ### Deployment Checklist
- [ ] Fresh system: Test `setup-homelab.sh` - [ ] Fresh system: Test `setup-homelab.sh`
- [ ] Core stack: Deploy and verify DuckDNS, Traefik, Authelia, Gluetun - [ ] Core stack: Deploy and verify DuckDNS, Traefik, Authelia, Gluetun
- [ ] Infrastructure: Deploy Dockge and verify web UI access - [ ] Infrastructure: Deploy Dockge and verify web UI access
@@ -394,7 +545,7 @@ ls -la ~/AI-Homelab/
- [ ] VPN: Test Gluetun routing - [ ] VPN: Test Gluetun routing
- [ ] Documentation: Validate all steps in docs/ - [ ] Documentation: Validate all steps in docs/
### Round 6 Success Criteria ### Production Success Criteria
- [ ] No permission-related crashes - [ ] No permission-related crashes
- [ ] All deployment scripts work on fresh Debian install - [ ] All deployment scripts work on fresh Debian install
- [ ] Documentation matches actual implementation - [ ] Documentation matches actual implementation

8
README.md
View File

@@ -72,11 +72,15 @@ Individual service documentation is available in [`docs/service-docs/`](docs/ser
- **Traefik** - Reverse proxy with automatic HTTPS termination - **Traefik** - Reverse proxy with automatic HTTPS termination
- **Authelia** - Single sign-on (SSO) authentication - **Authelia** - Single sign-on (SSO) authentication
- **DuckDNS** - Dynamic DNS with wildcard SSL certificates - **DuckDNS** - Dynamic DNS with wildcard SSL certificates
- **Gluetun** - VPN client for secure downloads
- **Sablier** - Lazy loading service for on-demand containers - **Sablier** - Lazy loading service for on-demand containers
### VPN Services
- **Gluetun** - VPN client for secure downloads
- **qBittorrent** - Torrent client routed through VPN
### Service Categories ### Service Categories
- **Media** - Plex, Jellyfin, Sonarr, Radarr, qBittorrent - **Media** - Plex, Jellyfin, Sonarr, Radarr
- **VPN** - qBittorrent (VPN-routed downloads)
- **Productivity** - Nextcloud, Gitea, BookStack, OnlyOffice - **Productivity** - Nextcloud, Gitea, BookStack, OnlyOffice
- **Monitoring** - Grafana, Prometheus, Uptime Kuma - **Monitoring** - Grafana, Prometheus, Uptime Kuma
- **Home Automation** - Home Assistant, Node-RED, Zigbee2MQTT - **Home Automation** - Home Assistant, Node-RED, Zigbee2MQTT

70
docker-compose/downloaders.yml
View File

@@ -1,70 +0,0 @@
# Downloaders Stack
# VPN-routed download clients
# Place in /opt/stacks/downloaders/docker-compose.yml
# Service Access URLs:
# - qBittorrent: https://qbit.${DOMAIN}
services:
# Gluetun - VPN client (Surfshark)
# Routes download clients through VPN for security
gluetun:
image: qmcgaw/gluetun:latest
container_name: gluetun
restart: unless-stopped
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun:/dev/net/tun
networks:
- homelab-network
- traefik-network
ports:
- "8888:8888/tcp" # HTTP proxy
- "8388:8388/tcp" # Shadowsocks
- "8388:8388/udp" # Shadowsocks
- "8081:8080" # qBittorrent web UI
- "6881:6881" # qBittorrent
- "6881:6881/udp" # qBittorrent
volumes:
- ./gluetun:/gluetun
environment:
- VPN_SERVICE_PROVIDER=surfshark
- VPN_TYPE=openvpn
- OPENVPN_USER=${SURFSHARK_USERNAME}
- OPENVPN_PASSWORD=${SURFSHARK_PASSWORD}
- SERVER_COUNTRIES=${VPN_SERVER_COUNTRIES:-Netherlands}
- TZ=${TZ}
labels:
- "homelab.category=downloaders"
- "homelab.description=VPN client for secure downloads"
- "traefik.enable=true"
- "traefik.http.routers.qbittorrent.rule=Host(`qbit.${DOMAIN}`)"
- "traefik.http.routers.qbittorrent.entrypoints=websecure"
- "traefik.http.routers.qbittorrent.tls=true"
- "traefik.http.routers.qbittorrent.middlewares=authelia@docker"
- "traefik.http.services.qbittorrent.loadbalancer.server.port=8080"
# qBittorrent - Torrent client
# Routes through Gluetun VPN
qbittorrent:
image: lscr.io/linuxserver/qbittorrent:latest
container_name: qbittorrent
restart: unless-stopped
network_mode: "service:gluetun" # Routes through VPN in same compose file
volumes:
- ./qbittorrent/config:/config
- /mnt/downloads:/downloads
environment:
- PUID=${PUID:-1000}
- PGID=${PGID:-1000}
- TZ=${TZ}
- WEBUI_PORT=8080
depends_on:
- gluetun
networks:
homelab-network:
external: true
traefik-network:
external: true

8
docker-compose/downloaders/docker-compose.yml → docker-compose/vpn/docker-compose.yml
View File

@@ -1,6 +1,6 @@
# Downloaders Stack # VPN Stack
# VPN-routed download clients # VPN client and VPN-routed download clients
# Place in /opt/stacks/downloaders/docker-compose.yml # Place in /opt/stacks/vpn/docker-compose.yml
# Service Access URLs: # Service Access URLs:
# - qBittorrent: https://qbit.${DOMAIN} # - qBittorrent: https://qbit.${DOMAIN}
@@ -76,4 +76,4 @@ networks:
homelab-network: homelab-network:
external: true external: true
traefik-network: traefik-network:
external: true external: true

98
docs/ports-in-use.md Normal file
View File

@@ -0,0 +1,98 @@
# Ports in Use
This document tracks all ports used by services in the AI-Homelab. Update this document whenever services are added or ports are changed.
## Core Stack ([core.yml](../docker-compose/core.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [Traefik](../service-docs/traefik.md) | 80 | TCP | HTTP (redirects to HTTPS) | 80 |
| [Traefik](../service-docs/traefik.md) | 443 | TCP | HTTPS | 443 |
| [Traefik](../service-docs/traefik.md) | 8080 | TCP | Dashboard (protected) | 8080 |
## Infrastructure Stack ([infrastructure.yml](../docker-compose/infrastructure.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [Dockge](../service-docs/dockge.md) | 5001 | TCP | Web UI | 5001 |
| [Pi-hole](../service-docs/pihole.md) | 53 | TCP/UDP | DNS | 53 |
| [Docker Proxy](../service-docs/docker-proxy.md) | 127.0.0.1:2375 | TCP | Docker API proxy | 2375 |
## Development Stack ([development.yml](../docker-compose/development.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [PostgreSQL](../service-docs/postgresql.md) | 5432 | TCP | Database | 5432 |
| [Redis](../service-docs/redis.md) | 6379 | TCP | Cache/Database | 6379 |
## Home Assistant Stack ([homeassistant.yml](../docker-compose/homeassistant.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [MotionEye](../service-docs/motioneye.md) | 8765 | TCP | Web UI | 8765 |
| [Mosquitto](../service-docs/mosquitto.md) | 1883 | TCP | MQTT | 1883 |
| [Mosquitto](../service-docs/mosquitto.md) | 9001 | TCP | MQTT Websockets | 9001 |
## Monitoring Stack ([monitoring.yml](../docker-compose/monitoring.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [Prometheus](../service-docs/prometheus.md) | 9090 | TCP | Web UI/Metrics | 9090 |
## VPN Stack ([vpn.yml](../docker-compose/vpn.yml))
| Service | Port | Protocol | Purpose | Internal Port |
|---------|------|----------|---------|---------------|
| [Gluetun](../service-docs/gluetun.md) | 8888 | TCP | HTTP proxy | 8888 |
| [Gluetun](../service-docs/gluetun.md) | 8388 | TCP/UDP | Shadowsocks | 8388 |
| [Gluetun](../service-docs/gluetun.md) | 8081 | TCP | qBittorrent Web UI | 8080 |
| [Gluetun](../service-docs/gluetun.md) | 6881 | TCP/UDP | qBittorrent | 6881 |
## Port Range Reference
| Range | Usage |
|-------|-------|
| 1-1023 | System ports (well-known) |
| 1024-49151 | Registered ports |
| 49152-65535 | Dynamic/private ports |
## Common Port Conflicts
- **Port 80/443**: Used by Traefik for HTTP/HTTPS
- **Port 53**: Used by Pi-hole for DNS
- **Port 2375**: Used by Docker Proxy (localhost only)
- **Port 5001**: Used by Dockge
- **Port 5432**: Used by PostgreSQL
- **Port 6379**: Used by Redis
- **Port 8080**: Used by Traefik dashboard
- **Port 9090**: Used by Prometheus
## Adding New Services
When adding new services:
1. Check this document for available ports
2. Choose ports that don't conflict with existing services
3. Update this document with new port mappings
4. Consider using Traefik labels instead of direct port exposure for web services
## Port Planning Guidelines
- **Web services**: Use Traefik labels (no direct ports needed)
- **Databases**: Use internal networking only (no external ports)
- **VPN services**: Route through Gluetun for security
- **Development tools**: Consider localhost-only binding (127.0.0.1:port)
- **Monitoring**: Use high-numbered ports (9000+ range)
## Updating This Document
This document should be updated whenever:
- New services are added to any stack
- Existing services change their port mappings
- Services are removed from stacks
- Network configurations change
Run this command to find all port mappings in compose files:
```bash
grep -r "ports:" docker-compose/ | grep -v "^#" | sort
```

43
docs/proxying-external-hosts.md
View File

@@ -14,7 +14,7 @@ Traefik can proxy services that aren't running in Docker, such as:
### Step 1: Create External Service Configuration ### Step 1: Create External Service Configuration
Create a file in `/opt/stacks/traefik/dynamic/external-hosts.yml`: Create a file in `/opt/stacks/traefik/dynamic/` with the format 'external-host-servername.yml'
```yaml ```yaml
http: http:
@@ -71,7 +71,11 @@ Visit `https://ha.yourdomain.duckdns.org` - Traefik will:
## Method 2: Using Docker Labels (Dummy Container) ## Method 2: Using Docker Labels (Dummy Container)
If you prefer managing routes via Docker labels (so the AI can modify them), create a dummy container: If you prefer managing routes via Docker labels, create a dummy container:
>This can be resource intensive with serveral services running.
Not recomended due to unnecessary resource/power consumption.
Don't try it on a Raspberry Pi
### Create a Label Container ### Create a Label Container
@@ -108,13 +112,6 @@ cd /opt/stacks/external-proxies
docker compose up -d docker compose up -d
``` ```
## Method 3: Hybrid Approach (File + Docker Discovery)
Combine both methods for maximum flexibility:
- Use file provider for static external hosts
- Use Docker labels for frequently changing services
- AI can manage both!
## Common External Services to Proxy ## Common External Services to Proxy
### Home Assistant (Raspberry Pi) ### Home Assistant (Raspberry Pi)
@@ -134,34 +131,6 @@ router-admin:
- authelia@docker # Add SSO protection - authelia@docker # Add SSO protection
``` ```
### Proxmox Server
```yaml
proxmox:
rule: "Host(`proxmox.yourdomain.duckdns.org`)"
service: https://192.168.1.100:8006
middlewares:
- authelia@docker
# Note: Use https:// if backend uses HTTPS
```
### TrueNAS/FreeNAS
```yaml
truenas:
rule: "Host(`nas.yourdomain.duckdns.org`)"
service: http://192.168.1.200:80
middlewares:
- authelia@docker
```
### Security Camera NVR
```yaml
nvr:
rule: "Host(`cameras.yourdomain.duckdns.org`)"
service: http://192.168.1.10:80
middlewares:
- authelia@docker # Definitely protect cameras!
```
## Advanced Configuration ## Advanced Configuration
### WebSocket Support ### WebSocket Support

2
docs/resource-limits-template.md
View File

@@ -139,7 +139,7 @@ calibre_web: # Ebook manager
template: web_service template: web_service
notes: "Light web app with database" notes: "Light web app with database"
# Downloaders Stack # VPN Stack
qbittorrent: # Torrent client qbittorrent: # Torrent client
template: downloader_service template: downloader_service
notes: "Network I/O heavy, moderate CPU for hashing" notes: "Network I/O heavy, moderate CPU for hashing"

11
docs/services-overview.md
View File

@@ -6,11 +6,13 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
| Stacks (10) | Services (70 + 6db) | SSO | Storage | Access URLs | | Stacks (10) | Services (70 + 6db) | SSO | Storage | Access URLs |
|-------|----------|-----|---------|-------------| |-------|----------|-----|---------|-------------|
| **📦 core.yaml (4)** | **Deploy First** | | | | | **📦 core.yaml (3)** | **Deploy First** | | | |
| ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI | | ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI |
| ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} | | ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} |
| ├─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} | | └─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} |
| └─ Gluetun | VPN (Surfshark) | - | /opt/stacks/core/gluetun | No UI | | **🔒 vpn.yaml (2)** | **VPN Services** | | | |
| ├─ Gluetun | VPN (Surfshark) | - | /opt/stacks/vpn/gluetun | No UI |
| └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} |
| **🔧 infrastructure.yaml** (12) | | | | | | **🔧 infrastructure.yaml** (12) | | | | |
| ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/infrastructure | dockge.${DOMAIN} | | ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/infrastructure | dockge.${DOMAIN} |
| ├─ Portainer | Container management | ✓ | /opt/stacks/infrastructure | portainer.${DOMAIN} | | ├─ Portainer | Container management | ✓ | /opt/stacks/infrastructure | portainer.${DOMAIN} |
@@ -31,8 +33,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic
| ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} | | ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} |
| ├─ Sonarr | TV automation | ✓ | /opt/stacks/media, /mnt/media | sonarr.${DOMAIN} | | ├─ Sonarr | TV automation | ✓ | /opt/stacks/media, /mnt/media | sonarr.${DOMAIN} |
| ├─ Radarr | Movie automation | ✓ | /opt/stacks/media, /mnt/media | radarr.${DOMAIN} | | ├─ Radarr | Movie automation | ✓ | /opt/stacks/media, /mnt/media | radarr.${DOMAIN} |
| ├─ Prowlarr | Indexer manager | ✓ | /opt/stacks/media | prowlarr.${DOMAIN} | | └─ Prowlarr | Indexer manager | ✓ | /opt/stacks/media | prowlarr.${DOMAIN} |
| └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} |
| **📚 media-extended.yaml** (10) | | | | | | **📚 media-extended.yaml** (10) | | | | |
| ├─ Readarr | Ebooks/Audiobooks | ✓ | /opt/stacks/media-ext, /mnt/media | readarr.${DOMAIN} | | ├─ Readarr | Ebooks/Audiobooks | ✓ | /opt/stacks/media-ext, /mnt/media | readarr.${DOMAIN} |
| ├─ Lidarr | Music manager | ✓ | /opt/stacks/media-ext, /mnt/media | lidarr.${DOMAIN} | | ├─ Lidarr | Music manager | ✓ | /opt/stacks/media-ext, /mnt/media | lidarr.${DOMAIN} |

2
scripts/README.md
View File

@@ -89,7 +89,7 @@ Automated deployment script that deploys the core infrastructure and Dockge. Run
1. **Validate Prerequisites** - Checks for Docker, .env file, and proper configuration 1. **Validate Prerequisites** - Checks for Docker, .env file, and proper configuration
2. **Create Directories** - Sets up `/opt/stacks/core` and `/opt/stacks/infrastructure` 2. **Create Directories** - Sets up `/opt/stacks/core` and `/opt/stacks/infrastructure`
3. **Create Docker Networks** - Ensures homelab-network, traefik-network, and media-network exist 3. **Create Docker Networks** - Ensures homelab-network, traefik-network, and media-network exist
4. **Deploy Core Stack** - Deploys DuckDNS, Traefik, Authelia, and Gluetun 4. **Deploy Core Stack** - Deploys DuckDNS, Traefik, and Authelia
5. **Deploy Infrastructure Stack** - Deploys Dockge, Portainer, Pi-hole, and monitoring tools 5. **Deploy Infrastructure Stack** - Deploys Dockge, Portainer, Pi-hole, and monitoring tools
6. **Wait for Dockge** - Waits for Dockge web UI to become accessible 6. **Wait for Dockge** - Waits for Dockge web UI to become accessible
7. **Open Browser** - Automatically opens Dockge in your default browser 7. **Open Browser** - Automatically opens Dockge in your default browser

15
tasks.txt
View File

@@ -1,14 +1,9 @@
Tasks Tasks
.env.example Completed:
✅ create new variables - Created VPN stack and moved Gluetun & qBittorrent from downloaders stack
DAFAULT_USER: admin - Removed downloaders stack entirely (directory and .yml file)
DEFAULT_PASSWORD: changeme - Updated all documentation and AGENT_INSTRUCTIONS.md for VPN stack structure
DEFAULT_EMAIL: admin@example.com - Updated resource limits template to reflect VPN stack
✅ modify existing variables to use these 3 new variables like
NEXTCLOUD_ADMIN_USER=${DEFAULT_USER}
except for QBITTORRENT_USER=admin should stay as is
the goal is ease of setup for the user
✅ Then update env-configuration.md