From 30e0481685bbc07d3eac0d67bf019e193a4bf4db Mon Sep 17 00:00:00 2001 From: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com> Date: Wed, 21 Jan 2026 18:52:40 -0500 Subject: [PATCH] copilot Instructions improvements and documentation updates --- AGENT_INSTRUCTIONS.md | 225 +++++++++++++++--- README.md | 8 +- docker-compose/downloaders.yml | 70 ------ .../{downloaders => vpn}/docker-compose.yml | 8 +- docs/ports-in-use.md | 98 ++++++++ docs/proxying-external-hosts.md | 43 +--- docs/resource-limits-template.md | 2 +- docs/services-overview.md | 11 +- scripts/README.md | 2 +- tasks.txt | 15 +- 10 files changed, 315 insertions(+), 167 deletions(-) delete mode 100644 docker-compose/downloaders.yml rename docker-compose/{downloaders => vpn}/docker-compose.yml (94%) create mode 100644 docs/ports-in-use.md diff --git a/AGENT_INSTRUCTIONS.md b/AGENT_INSTRUCTIONS.md index d5555d1..422b778 100644 --- a/AGENT_INSTRUCTIONS.md +++ b/AGENT_INSTRUCTIONS.md @@ -5,8 +5,7 @@ You are an AI agent specialized in managing Docker-based homelab infrastructure ## Repository Context - **Repository Location**: `~/AI-Homelab/` -- **Purpose**: Development and testing of automated homelab management via GitHub Copilot -- **Testing Phase**: Round 6 - Focus on script reliability, error handling, and deployment robustness +- **Purpose**: Production-ready Docker homelab infrastructure managed through GitHub Copilot in VS Code - **User**: `kelin` (PUID=1000, PGID=1000) - **Critical**: All file operations must respect user ownership - avoid permission escalation issues @@ -34,6 +33,7 @@ You are an AI agent specialized in managing Docker-based homelab infrastructure │ ├── setup-homelab.sh # First-run setup │ └── deploy-homelab.sh # Automated deployment ├── .env.example # Environment template +├── .env # User-created environment file (not in git) ├── AGENT_INSTRUCTIONS.md # This file └── README.md # Project overview ``` @@ -75,7 +75,7 @@ services: service-name: image: image:tag # Always pin versions container_name: service-name - restart: unless-stopped + restart: unless-stopped # Use 'no' if ondemand (Sablier) is enabled networks: - homelab-network ports: @@ -115,21 +115,84 @@ networks: - Use **absolute paths** (`/mnt/media`) only for large shared data on separate drives - This allows stacks to be portable and work correctly in Dockge's `/opt/stacks/` structure +## Service Creation with Traefik on a different server Template + +For services not running on the same server as Traefik, use this template. Does not require Traefik labels in compose file. + +>From Traekif's perspective the service is on a remote host + +**Remote Server Setup:** +1. Deploy service on the target server using standard Docker Compose +2. Note the target server's IP address and service port +3. Create/update Traefik dynamic configuration file: `/opt/stacks/core/traefik/dynamic/external-host-servername.yml` on the server with Traefik + +**Traefik Dynamic Configuration Template:** +```yaml +http: + routers: + # Remote service on servername + remote-service: + rule: "Host(`remote-service.${DOMAIN}`)" + entryPoints: + - websecure + service: remote-service + tls: + certResolver: letsencrypt + # Optional: Add Authelia protection + middlewares: + - authelia@docker + + services: + remote-service: + loadBalancer: + servers: + - url: "http://remote-server-ip:service-port" # Replace with actual IP/port + passHostHeader: true + + middlewares: + # Optional: Add headers for WebSocket support if needed + remote-service-headers: + headers: + customRequestHeaders: + X-Forwarded-Proto: "https" + customResponseHeaders: + X-Frame-Options: "SAMEORIGIN" +``` + +**File Naming Convention:** +- External host files named: `external-host-servername.yml` +- Example: `external-host-raspberry-pi.yml`, `external-host-nas.yml` +- See `docs/proxying-external-hosts.md` for detailed examples + ## Critical Deployment Order +>The core stack should only be deployed on one server in the homelab +The dashboards stack is intended to be deployed on the same server as the core stack, for better UX + +Skip step 1 if the homelab already has the core stack on another server. + 1. **Core Stack First**: Deploy `/opt/stacks/core/docker-compose.yml` - - DuckDNS, Traefik, Authelia, Gluetun + - DuckDNS, Traefik, Authelia - All other services depend on this -2. **Infrastructure**: Dockge, Portainer, monitoring -3. **Applications**: Media services, dashboards, etc. +2. **VPN Stack**: Deploy `/opt/stacks/vpn/docker-compose.yml` + - Gluetun VPN client, qBittorrent + - Optional but recommended for secure downloads +3. **Infrastructure**: Dockge, docker-proxy, code-server, etc. +4. **Applications**: Media services, dashboards, etc. ## VPN Integration Rules +**All VPN-related services must be in the VPN stack** (`/opt/stacks/vpn/`), including Gluetun itself. This ensures proper network isolation and management. + Use Gluetun for services requiring VPN: ```yaml services: + # Gluetun VPN client (must be in vpn stack) + gluetun: + # VPN configuration here + download-client: - network_mode: "service:gluetun" + network_mode: "service:gluetun" # Routes through VPN depends_on: - gluetun # No ports section - use Gluetun's ports @@ -142,6 +205,8 @@ gluetun: - 8080:8080 # Download client web UI ``` +**Important**: Never place VPN-dependent services in other stacks. All VPN routing must happen within the dedicated VPN stack. + ## SSO Management ### Enable SSO (Default) @@ -158,7 +223,7 @@ labels: ## Agent Actions Checklist -### Permission Safety (CRITICAL - Established in Round 4) +### Permission Safety (CRITICAL) - [ ] **NEVER** use sudo for file operations in user directories - [ ] Always check file ownership before modifying: `ls -la` - [ ] Respect existing ownership - files should be owned by `kelin:kelin` @@ -196,8 +261,8 @@ labels: ## Common Agent Tasks -### Development Workflow (Current Focus - Round 6) -1. **Repository Testing** +### Development Workflow +1. **Repository Maintenance** - Test deployment scripts: `./scripts/setup-homelab.sh`, `./scripts/deploy-homelab.sh` - Verify compose file syntax across all stacks - Validate `.env.example` completeness @@ -214,38 +279,111 @@ labels: - Document new features or configuration patterns ### Deploy New Service +**Research Protocol:** +1. **Check existing services**: Search `docs/services-overview.md` and compose files for similar services +2. **Read official documentation**: Visit Docker Hub, LinuxServer.io, or official project docs +3. **Check awesome-docker-compose**: Review https://awesome-docker-compose.com/apps for compose templates +4. **Prefer LinuxServer.io**: Use lscr.io images when available for consistency +5. **Check port availability**: Review `docs/ports-in-use.md` for conflicts +6. **Determine SSO requirements**: Media services (Plex/Jellyfin) bypass SSO, admin tools require it +7. **Check VPN needs**: Download clients need `network_mode: "service:gluetun"` +8. **Review resource limits**: Apply CPU/memory limits following existing patterns + +**Deployment Steps:** 1. Create stack directory: `/opt/stacks/stack-name/` -2. Write docker-compose.yml with template -3. Create `.env` file for secrets -4. Deploy: `docker compose up -d` -5. Verify Traefik routing -6. Test SSO protection +2. Write docker-compose.yml using LinuxServer.io template with Traefik labels +3. Create `.env` file for secrets (copy from `~/AI-Homelab/.env`) +4. **Ask user**: Enable SSO protection? (Default: Yes, unless media service like Plex/Jellyfin) +5. **Ask user**: Enable lazy loading (Sablier)? (Default: Yes for resource conservation) - **Note**: Requires `restart: no` instead of `unless-stopped` +6. Add service to Homepage dashboard config +7. Deploy: `docker compose up -d` +8. Verify Traefik routing at `https://service.${DOMAIN}` +9. Test SSO protection (or bypass for media services) +10. Check logs: `docker compose logs -f` ### Update Existing Service -1. Read current configuration -2. Make minimal necessary changes -3. Validate dependencies still work -4. Redeploy: `docker compose up -d service-name` -5. Check logs for errors +**Verification Protocol:** +1. **Read current config**: `docker compose -f /opt/stacks/stack/docker-compose.yml config` +2. **Check dependencies**: Identify linked services, networks, volumes +3. **Backup current state**: Copy compose file before changes +4. **Test in isolation**: Use `docker run` for image testing if needed +5. **Check breaking changes**: Review changelog for new versions +6. **Validate environment**: Ensure `.env` has required variables + +**Update Steps:** +1. Read current configuration and dependencies +2. **If modifying core services**: Verify integrity of `/opt/stacks/core/traefik/dynamic/` files and `sablier.yml` +3. Make minimal necessary changes (version, config, ports) +4. Validate YAML syntax: `docker compose config` +5. Check for port conflicts with `docs/ports-in-use.md` +6. Pull new image: `docker compose pull service-name` +7. Redeploy: `docker compose up -d service-name` +8. Monitor logs: `docker compose logs -f service-name` +9. Test functionality and verify dependent services still work +10. Update documentation if behavior changed ### Enable/Disable VPN -1. For VPN: Add `network_mode: "service:gluetun"` -2. Move port mapping to Gluetun service -3. Add `depends_on: gluetun` -4. For no VPN: Remove network_mode, add ports directly +**Enable VPN Routing:** +1. **Check Gluetun status**: Verify VPN stack is running with VPN configured +2. **Move service to VPN stack**: All VPN-dependent services must be in `/opt/stacks/vpn/` with Gluetun +3. **Modify service config**: Change from `ports:` to `network_mode: "service:gluetun"` +4. **Move port mappings**: Add exposed ports to Gluetun service in VPN stack +5. **Add dependency**: Include `depends_on: - gluetun` +6. **Remove direct ports**: Delete any `ports:` section from service +7. **Update documentation**: Note VPN routing in service docs +8. **Test connectivity**: Verify service routes through VPN IP + +**Disable VPN Routing:** +1. **Check port availability**: Review `docs/ports-in-use.md` for conflicts +2. **Move service out of VPN stack**: Relocate to appropriate non-VPN stack +3. **Remove network_mode**: Delete `network_mode: "service:gluetun"` +4. **Add direct ports**: Add appropriate `ports:` mapping +5. **Remove dependency**: Delete `depends_on: gluetun` +6. **Remove Gluetun ports**: Clean up port mappings from Gluetun service +7. **Test direct access**: Verify service accessible without VPN +8. **Update documentation**: Remove VPN routing notes ### Toggle SSO -1. Enable: Add authelia middleware label -2. Disable: Comment out middleware label -3. Redeploy service -4. Verify access works as expected +**Enable Authelia SSO (Default for Security):** +1. **Verify Authelia running**: Check core stack status +2. **Uncomment middleware label**: Change `# - "traefik.http.routers.service.middlewares=authelia@docker"` to active +3. **Remove bypass rules**: Delete domain exception from `authelia/configuration.yml` if present +4. **Test authentication**: Access service and verify login prompt +5. **Document protection**: Note SSO requirement in service docs + +**Disable Authelia SSO (Media Services Only):** +1. **Justify bypass**: Confirm service needs direct app access (Plex, Jellyfin, etc.) +2. **Comment middleware**: Change to `# - "traefik.http.routers.service.middlewares=authelia@docker"` +3. **Add bypass rule**: Update `authelia/configuration.yml` with domain exception +4. **Test direct access**: Verify no authentication required +5. **Document exception**: Explain why SSO bypassed in service docs +6. **Monitor security**: Regular review of bypass justifications + +**Note**: Authelia middleware label should ALWAYS be present in compose files - comment out to disable, never remove entirely. ## Error Prevention ### Port Conflicts -- Check existing services before assigning ports -- Use Traefik instead of port mapping when possible -- Document port usage in comments +**Prevention Protocol:** +1. **Check existing ports**: Review `docs/ports-in-use.md` before assigning new ports +2. **Use Traefik routing**: Prefer host-based routing over direct port exposure +3. **Standardize ports**: Follow LinuxServer.io defaults when possible +4. **Document usage**: Add port comments in compose files +5. **Test conflicts**: Use `netstat -tlnp | grep :PORT` to check availability + +**AI Usage of ports-in-use.md:** +- Always consult `docs/ports-in-use.md` before suggesting new port assignments +- Update the document whenever ports are changed or new services added +- Cross-reference with service documentation links in the ports file +- Verify port availability across all stacks before deployment + +**Resolution Steps:** +1. **Identify conflict**: Check `docker ps --format "table {{.Names}}\t{{.Ports}}"` for port usage +2. **Reassign port**: Choose unused port following service conventions +3. **Update Traefik labels**: Modify `loadbalancer.server.port` if changed +4. **Update documentation**: Reflect port changes in `docs/ports-in-use.md` +5. **Test access**: Verify service accessible at new port +6. **Update dependent services**: Check for hardcoded port references ### Permission Issues - Always set PUID=1000, PGID=1000 @@ -259,6 +397,19 @@ labels: ## Key Configurations to Monitor +### Sablier Configurations +- Lazy loading enabled: `sablier.enable=true` +- Proper group naming: `sablier.group=stack-service` +- On-demand startup: `sablier.start-on-demand=true` +- **Critical**: Set `restart: no` when ondemand is enabled (cannot use `unless-stopped`) +- Group consistency across related services + +### Traefik's Dynamic Folder Configurations +- File provider configuration in `/opt/stacks/core/traefik/dynamic/` +- External host routing rules in `external.yml` +- Middleware definitions and routing rules +- Certificate resolver settings for wildcard domains + ### Traefik Labels - Correct hostname format: `service.${DOMAIN}` - Proper entrypoint: `websecure` @@ -344,6 +495,7 @@ labels: - **Blindly escalate privileges when encountering errors** ### Always Do +- Read documentation and service-docs files and follow established practices - Read existing configurations first - Test changes in isolation when possible - Document complex configurations @@ -354,11 +506,10 @@ labels: - **Respect user ownership boundaries** - **Ask before modifying system directories** -## Testing and Development Guidelines (Round 6) +## Repository Management Guidelines -### Repository Development -- Work within `~/AI-Homelab/` for all development -- Test scripts in isolated environment before production +### Repository Maintenance +- Work within `~/AI-Homelab/` for all operations - Validate all YAML files before committing - Ensure `.env.example` stays updated with new variables - Document breaking changes in commit messages @@ -384,7 +535,7 @@ bash -n scripts/deploy-homelab.sh ls -la ~/AI-Homelab/ ``` -### Deployment Testing Checklist +### Deployment Checklist - [ ] Fresh system: Test `setup-homelab.sh` - [ ] Core stack: Deploy and verify DuckDNS, Traefik, Authelia, Gluetun - [ ] Infrastructure: Deploy Dockge and verify web UI access @@ -394,7 +545,7 @@ ls -la ~/AI-Homelab/ - [ ] VPN: Test Gluetun routing - [ ] Documentation: Validate all steps in docs/ -### Round 6 Success Criteria +### Production Success Criteria - [ ] No permission-related crashes - [ ] All deployment scripts work on fresh Debian install - [ ] Documentation matches actual implementation diff --git a/README.md b/README.md index 6607116..6b560e6 100644 --- a/README.md +++ b/README.md @@ -72,11 +72,15 @@ Individual service documentation is available in [`docs/service-docs/`](docs/ser - **Traefik** - Reverse proxy with automatic HTTPS termination - **Authelia** - Single sign-on (SSO) authentication - **DuckDNS** - Dynamic DNS with wildcard SSL certificates -- **Gluetun** - VPN client for secure downloads - **Sablier** - Lazy loading service for on-demand containers +### VPN Services +- **Gluetun** - VPN client for secure downloads +- **qBittorrent** - Torrent client routed through VPN + ### Service Categories -- **Media** - Plex, Jellyfin, Sonarr, Radarr, qBittorrent +- **Media** - Plex, Jellyfin, Sonarr, Radarr +- **VPN** - qBittorrent (VPN-routed downloads) - **Productivity** - Nextcloud, Gitea, BookStack, OnlyOffice - **Monitoring** - Grafana, Prometheus, Uptime Kuma - **Home Automation** - Home Assistant, Node-RED, Zigbee2MQTT diff --git a/docker-compose/downloaders.yml b/docker-compose/downloaders.yml deleted file mode 100644 index a872bbd..0000000 --- a/docker-compose/downloaders.yml +++ /dev/null @@ -1,70 +0,0 @@ -# Downloaders Stack -# VPN-routed download clients -# Place in /opt/stacks/downloaders/docker-compose.yml - -# Service Access URLs: -# - qBittorrent: https://qbit.${DOMAIN} - -services: - # Gluetun - VPN client (Surfshark) - # Routes download clients through VPN for security - gluetun: - image: qmcgaw/gluetun:latest - container_name: gluetun - restart: unless-stopped - cap_add: - - NET_ADMIN - devices: - - /dev/net/tun:/dev/net/tun - networks: - - homelab-network - - traefik-network - ports: - - "8888:8888/tcp" # HTTP proxy - - "8388:8388/tcp" # Shadowsocks - - "8388:8388/udp" # Shadowsocks - - "8081:8080" # qBittorrent web UI - - "6881:6881" # qBittorrent - - "6881:6881/udp" # qBittorrent - volumes: - - ./gluetun:/gluetun - environment: - - VPN_SERVICE_PROVIDER=surfshark - - VPN_TYPE=openvpn - - OPENVPN_USER=${SURFSHARK_USERNAME} - - OPENVPN_PASSWORD=${SURFSHARK_PASSWORD} - - SERVER_COUNTRIES=${VPN_SERVER_COUNTRIES:-Netherlands} - - TZ=${TZ} - labels: - - "homelab.category=downloaders" - - "homelab.description=VPN client for secure downloads" - - "traefik.enable=true" - - "traefik.http.routers.qbittorrent.rule=Host(`qbit.${DOMAIN}`)" - - "traefik.http.routers.qbittorrent.entrypoints=websecure" - - "traefik.http.routers.qbittorrent.tls=true" - - "traefik.http.routers.qbittorrent.middlewares=authelia@docker" - - "traefik.http.services.qbittorrent.loadbalancer.server.port=8080" - - # qBittorrent - Torrent client - # Routes through Gluetun VPN - qbittorrent: - image: lscr.io/linuxserver/qbittorrent:latest - container_name: qbittorrent - restart: unless-stopped - network_mode: "service:gluetun" # Routes through VPN in same compose file - volumes: - - ./qbittorrent/config:/config - - /mnt/downloads:/downloads - environment: - - PUID=${PUID:-1000} - - PGID=${PGID:-1000} - - TZ=${TZ} - - WEBUI_PORT=8080 - depends_on: - - gluetun - -networks: - homelab-network: - external: true - traefik-network: - external: true diff --git a/docker-compose/downloaders/docker-compose.yml b/docker-compose/vpn/docker-compose.yml similarity index 94% rename from docker-compose/downloaders/docker-compose.yml rename to docker-compose/vpn/docker-compose.yml index e83b598..f8f3e5f 100644 --- a/docker-compose/downloaders/docker-compose.yml +++ b/docker-compose/vpn/docker-compose.yml @@ -1,6 +1,6 @@ -# Downloaders Stack -# VPN-routed download clients -# Place in /opt/stacks/downloaders/docker-compose.yml +# VPN Stack +# VPN client and VPN-routed download clients +# Place in /opt/stacks/vpn/docker-compose.yml # Service Access URLs: # - qBittorrent: https://qbit.${DOMAIN} @@ -76,4 +76,4 @@ networks: homelab-network: external: true traefik-network: - external: true + external: true \ No newline at end of file diff --git a/docs/ports-in-use.md b/docs/ports-in-use.md new file mode 100644 index 0000000..36f920e --- /dev/null +++ b/docs/ports-in-use.md @@ -0,0 +1,98 @@ +# Ports in Use + +This document tracks all ports used by services in the AI-Homelab. Update this document whenever services are added or ports are changed. + +## Core Stack ([core.yml](../docker-compose/core.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [Traefik](../service-docs/traefik.md) | 80 | TCP | HTTP (redirects to HTTPS) | 80 | +| [Traefik](../service-docs/traefik.md) | 443 | TCP | HTTPS | 443 | +| [Traefik](../service-docs/traefik.md) | 8080 | TCP | Dashboard (protected) | 8080 | + +## Infrastructure Stack ([infrastructure.yml](../docker-compose/infrastructure.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [Dockge](../service-docs/dockge.md) | 5001 | TCP | Web UI | 5001 | +| [Pi-hole](../service-docs/pihole.md) | 53 | TCP/UDP | DNS | 53 | +| [Docker Proxy](../service-docs/docker-proxy.md) | 127.0.0.1:2375 | TCP | Docker API proxy | 2375 | + +## Development Stack ([development.yml](../docker-compose/development.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [PostgreSQL](../service-docs/postgresql.md) | 5432 | TCP | Database | 5432 | +| [Redis](../service-docs/redis.md) | 6379 | TCP | Cache/Database | 6379 | + +## Home Assistant Stack ([homeassistant.yml](../docker-compose/homeassistant.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [MotionEye](../service-docs/motioneye.md) | 8765 | TCP | Web UI | 8765 | +| [Mosquitto](../service-docs/mosquitto.md) | 1883 | TCP | MQTT | 1883 | +| [Mosquitto](../service-docs/mosquitto.md) | 9001 | TCP | MQTT Websockets | 9001 | + +## Monitoring Stack ([monitoring.yml](../docker-compose/monitoring.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [Prometheus](../service-docs/prometheus.md) | 9090 | TCP | Web UI/Metrics | 9090 | + +## VPN Stack ([vpn.yml](../docker-compose/vpn.yml)) + +| Service | Port | Protocol | Purpose | Internal Port | +|---------|------|----------|---------|---------------| +| [Gluetun](../service-docs/gluetun.md) | 8888 | TCP | HTTP proxy | 8888 | +| [Gluetun](../service-docs/gluetun.md) | 8388 | TCP/UDP | Shadowsocks | 8388 | +| [Gluetun](../service-docs/gluetun.md) | 8081 | TCP | qBittorrent Web UI | 8080 | +| [Gluetun](../service-docs/gluetun.md) | 6881 | TCP/UDP | qBittorrent | 6881 | + +## Port Range Reference + +| Range | Usage | +|-------|-------| +| 1-1023 | System ports (well-known) | +| 1024-49151 | Registered ports | +| 49152-65535 | Dynamic/private ports | + +## Common Port Conflicts + +- **Port 80/443**: Used by Traefik for HTTP/HTTPS +- **Port 53**: Used by Pi-hole for DNS +- **Port 2375**: Used by Docker Proxy (localhost only) +- **Port 5001**: Used by Dockge +- **Port 5432**: Used by PostgreSQL +- **Port 6379**: Used by Redis +- **Port 8080**: Used by Traefik dashboard +- **Port 9090**: Used by Prometheus + +## Adding New Services + +When adding new services: + +1. Check this document for available ports +2. Choose ports that don't conflict with existing services +3. Update this document with new port mappings +4. Consider using Traefik labels instead of direct port exposure for web services + +## Port Planning Guidelines + +- **Web services**: Use Traefik labels (no direct ports needed) +- **Databases**: Use internal networking only (no external ports) +- **VPN services**: Route through Gluetun for security +- **Development tools**: Consider localhost-only binding (127.0.0.1:port) +- **Monitoring**: Use high-numbered ports (9000+ range) + +## Updating This Document + +This document should be updated whenever: +- New services are added to any stack +- Existing services change their port mappings +- Services are removed from stacks +- Network configurations change + +Run this command to find all port mappings in compose files: +```bash +grep -r "ports:" docker-compose/ | grep -v "^#" | sort +``` \ No newline at end of file diff --git a/docs/proxying-external-hosts.md b/docs/proxying-external-hosts.md index 5e5d3d0..6bea30e 100644 --- a/docs/proxying-external-hosts.md +++ b/docs/proxying-external-hosts.md @@ -14,7 +14,7 @@ Traefik can proxy services that aren't running in Docker, such as: ### Step 1: Create External Service Configuration -Create a file in `/opt/stacks/traefik/dynamic/external-hosts.yml`: +Create a file in `/opt/stacks/traefik/dynamic/` with the format 'external-host-servername.yml' ```yaml http: @@ -71,7 +71,11 @@ Visit `https://ha.yourdomain.duckdns.org` - Traefik will: ## Method 2: Using Docker Labels (Dummy Container) -If you prefer managing routes via Docker labels (so the AI can modify them), create a dummy container: +If you prefer managing routes via Docker labels, create a dummy container: + +>This can be resource intensive with serveral services running. +Not recomended due to unnecessary resource/power consumption. +Don't try it on a Raspberry Pi ### Create a Label Container @@ -108,13 +112,6 @@ cd /opt/stacks/external-proxies docker compose up -d ``` -## Method 3: Hybrid Approach (File + Docker Discovery) - -Combine both methods for maximum flexibility: -- Use file provider for static external hosts -- Use Docker labels for frequently changing services -- AI can manage both! - ## Common External Services to Proxy ### Home Assistant (Raspberry Pi) @@ -134,34 +131,6 @@ router-admin: - authelia@docker # Add SSO protection ``` -### Proxmox Server -```yaml -proxmox: - rule: "Host(`proxmox.yourdomain.duckdns.org`)" - service: https://192.168.1.100:8006 - middlewares: - - authelia@docker - # Note: Use https:// if backend uses HTTPS -``` - -### TrueNAS/FreeNAS -```yaml -truenas: - rule: "Host(`nas.yourdomain.duckdns.org`)" - service: http://192.168.1.200:80 - middlewares: - - authelia@docker -``` - -### Security Camera NVR -```yaml -nvr: - rule: "Host(`cameras.yourdomain.duckdns.org`)" - service: http://192.168.1.10:80 - middlewares: - - authelia@docker # Definitely protect cameras! -``` - ## Advanced Configuration ### WebSocket Support diff --git a/docs/resource-limits-template.md b/docs/resource-limits-template.md index d190f5e..8281216 100644 --- a/docs/resource-limits-template.md +++ b/docs/resource-limits-template.md @@ -139,7 +139,7 @@ calibre_web: # Ebook manager template: web_service notes: "Light web app with database" -# Downloaders Stack +# VPN Stack qbittorrent: # Torrent client template: downloader_service notes: "Network I/O heavy, moderate CPU for hashing" diff --git a/docs/services-overview.md b/docs/services-overview.md index 62d13ed..f923f27 100644 --- a/docs/services-overview.md +++ b/docs/services-overview.md @@ -6,11 +6,13 @@ This document provides a comprehensive overview of all 50+ pre-configured servic | Stacks (10) | Services (70 + 6db) | SSO | Storage | Access URLs | |-------|----------|-----|---------|-------------| -| **📦 core.yaml (4)** | **Deploy First** | | | | +| **📦 core.yaml (3)** | **Deploy First** | | | | | ├─ DuckDNS | Dynamic DNS updater | - | /opt/stacks/core/duckdns | No UI | | ├─ Traefik | Reverse proxy + SSL | ✓ | /opt/stacks/core/traefik | traefik.${DOMAIN} | -| ├─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} | -| └─ Gluetun | VPN (Surfshark) | - | /opt/stacks/core/gluetun | No UI | +| └─ Authelia | SSO authentication | - | /opt/stacks/core/authelia | auth.${DOMAIN} | +| **🔒 vpn.yaml (2)** | **VPN Services** | | | | +| ├─ Gluetun | VPN (Surfshark) | - | /opt/stacks/vpn/gluetun | No UI | +| └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} | | **🔧 infrastructure.yaml** (12) | | | | | | ├─ Dockge | Stack manager (PRIMARY) | ✓ | /opt/stacks/infrastructure | dockge.${DOMAIN} | | ├─ Portainer | Container management | ✓ | /opt/stacks/infrastructure | portainer.${DOMAIN} | @@ -31,8 +33,7 @@ This document provides a comprehensive overview of all 50+ pre-configured servic | ├─ Jellyfin | Media server (OSS) | ✗ | /mnt/media, /mnt/transcode | jellyfin.${DOMAIN} | | ├─ Sonarr | TV automation | ✓ | /opt/stacks/media, /mnt/media | sonarr.${DOMAIN} | | ├─ Radarr | Movie automation | ✓ | /opt/stacks/media, /mnt/media | radarr.${DOMAIN} | -| ├─ Prowlarr | Indexer manager | ✓ | /opt/stacks/media | prowlarr.${DOMAIN} | -| └─ qBittorrent | Torrent (via VPN) | ✓ | /mnt/downloads | qbit.${DOMAIN} | +| └─ Prowlarr | Indexer manager | ✓ | /opt/stacks/media | prowlarr.${DOMAIN} | | **📚 media-extended.yaml** (10) | | | | | | ├─ Readarr | Ebooks/Audiobooks | ✓ | /opt/stacks/media-ext, /mnt/media | readarr.${DOMAIN} | | ├─ Lidarr | Music manager | ✓ | /opt/stacks/media-ext, /mnt/media | lidarr.${DOMAIN} | diff --git a/scripts/README.md b/scripts/README.md index 81aa372..c5b3b30 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -89,7 +89,7 @@ Automated deployment script that deploys the core infrastructure and Dockge. Run 1. **Validate Prerequisites** - Checks for Docker, .env file, and proper configuration 2. **Create Directories** - Sets up `/opt/stacks/core` and `/opt/stacks/infrastructure` 3. **Create Docker Networks** - Ensures homelab-network, traefik-network, and media-network exist -4. **Deploy Core Stack** - Deploys DuckDNS, Traefik, Authelia, and Gluetun +4. **Deploy Core Stack** - Deploys DuckDNS, Traefik, and Authelia 5. **Deploy Infrastructure Stack** - Deploys Dockge, Portainer, Pi-hole, and monitoring tools 6. **Wait for Dockge** - Waits for Dockge web UI to become accessible 7. **Open Browser** - Automatically opens Dockge in your default browser diff --git a/tasks.txt b/tasks.txt index d6805f0..c1e1c56 100644 --- a/tasks.txt +++ b/tasks.txt @@ -1,14 +1,9 @@ Tasks -.env.example - ✅ create new variables - DAFAULT_USER: admin - DEFAULT_PASSWORD: changeme - DEFAULT_EMAIL: admin@example.com +Completed: +- Created VPN stack and moved Gluetun & qBittorrent from downloaders stack +- Removed downloaders stack entirely (directory and .yml file) +- Updated all documentation and AGENT_INSTRUCTIONS.md for VPN stack structure +- Updated resource limits template to reflect VPN stack - ✅ modify existing variables to use these 3 new variables like - NEXTCLOUD_ADMIN_USER=${DEFAULT_USER} - except for QBITTORRENT_USER=admin should stay as is - the goal is ease of setup for the user -✅ Then update env-configuration.md \ No newline at end of file