diff --git a/.env.example b/.env.example index b3134a6..0f9f452 100644 --- a/.env.example +++ b/.env.example @@ -37,12 +37,17 @@ ADMIN_EMAIL=your-email@example.com # Used for admin user account # ==================================== # AUTHELIA SSO CONFIGURATION # ==================================== -# These secrets are AUTO-GENERATED by setup-homelab.sh -# DO NOT manually set these - the setup script will create them! +# Generate these secrets with: openssl rand -hex 64 +# The deploy script will use these to configure Authelia -AUTHELIA_JWT_SECRET=will-be-auto-generated-by-setup-script -AUTHELIA_SESSION_SECRET=will-be-auto-generated-by-setup-script -AUTHELIA_STORAGE_ENCRYPTION_KEY=will-be-auto-generated-by-setup-script +AUTHELIA_JWT_SECRET=generate-with-openssl-rand-hex-64 +AUTHELIA_SESSION_SECRET=generate-with-openssl-rand-hex-64 +AUTHELIA_STORAGE_ENCRYPTION_KEY=generate-with-openssl-rand-hex-64 + +# Authelia Admin Password (OPTIONAL) +# If not provided, deploy script will generate a random password +# and save it to /opt/stacks/core/authelia/ADMIN_PASSWORD.txt +# AUTHELIA_ADMIN_PASSWORD=your-secure-password-here # SMTP for Authelia Notifications (OPTIONAL) # If not configured, notifications are saved to file instead diff --git a/AGENT_INSTRUCTIONS.md b/AGENT_INSTRUCTIONS.md index 76af1e3..7e746ed 100644 --- a/AGENT_INSTRUCTIONS.md +++ b/AGENT_INSTRUCTIONS.md @@ -3,6 +3,41 @@ ## Primary Directive You are an AI agent specialized in managing Docker-based homelab infrastructure using Dockge. Always prioritize security, consistency, and stability across the entire server stack. +## Repository Context +- **Repository Location**: `/home/kelin/AI-Homelab/` +- **Purpose**: Development and testing of automated homelab management via GitHub Copilot +- **Testing Phase**: Round 4 - Focus on stability, permission handling, and production readiness +- **User**: `kelin` (PUID=1000, PGID=1000) +- **Critical**: All file operations must respect user ownership - avoid permission escalation issues + +## Repository Structure +``` +~/AI-Homelab/ +├── .github/ +│ └── copilot-instructions.md # GitHub Copilot guidelines +├── docker-compose/ # Compose file templates +│ ├── core/ # Core infrastructure (deploy first) +│ ├── infrastructure/ # Management tools +│ ├── dashboards/ # Dashboard services +│ ├── media/ # Media server stack +│ ├── monitoring/ # Monitoring stack +│ ├── productivity/ # Productivity tools +│ └── *.yml # Individual service stacks +├── config-templates/ # Service configuration templates +├── docs/ # Comprehensive documentation +│ ├── getting-started.md +│ ├── services-reference.md +│ ├── docker-guidelines.md +│ ├── proxying-external-hosts.md +│ └── troubleshooting/ +├── scripts/ # Automation scripts +│ ├── setup-homelab.sh # First-run setup +│ └── deploy-homelab.sh # Automated deployment +├── .env.example # Environment template +├── AGENT_INSTRUCTIONS.md # This file +└── README.md # Project overview +``` + ## Core Operating Principles ### 1. Docker Compose First @@ -28,7 +63,7 @@ You are an AI agent specialized in managing Docker-based homelab infrastructure ``` ### 4. Storage Strategy -- **Config files**: `/opt/stacks/stack-name/config/` +- **Config files**: Use relative paths `./service/config:/config` in compose files - **Large data**: Separate drives (`/mnt/media`, `/mnt/downloads`) - **Small data**: Docker named volumes - **Secrets**: `.env` files (never commit) @@ -46,7 +81,7 @@ services: ports: - "host:container" # Only if not using Traefik volumes: - - /opt/stacks/stack-name/config:/config + - ./service-name/config:/config # Relative to stack directory - service-data:/data # Large data on separate drives: # - /mnt/media:/media @@ -75,6 +110,11 @@ networks: external: true ``` +**Important Volume Path Convention:** +- Use **relative paths** (`.//config`) for service configs within the stack directory +- Use **absolute paths** (`/mnt/media`) only for large shared data on separate drives +- This allows stacks to be portable and work correctly in Dockge's `/opt/stacks/` structure + ## Critical Deployment Order 1. **Core Stack First**: Deploy `/opt/stacks/core/docker-compose.yml` @@ -118,6 +158,13 @@ labels: ## Agent Actions Checklist +### Permission Safety (CRITICAL for Round 4) +- [ ] **NEVER** use sudo for file operations in user directories +- [ ] Always check file ownership before modifying: `ls -la` +- [ ] Respect existing ownership - files should be owned by `kelin:kelin` +- [ ] If permission denied, diagnose first - don't escalate privileges blindly +- [ ] Docker operations may need sudo, but file edits in `/home/kelin/` should not + ### Before Any Change - [ ] Read existing compose files for context - [ ] Check port availability @@ -142,12 +189,30 @@ labels: ### File Management - [ ] Store configs in `/opt/stacks/stack-name/` +- [ ] Use relative paths for configs: `./service/config` - [ ] Use `/mnt/` for large data (>50GB) - [ ] Create `.env.example` templates - [ ] Document non-obvious configurations ## Common Agent Tasks +### Development Workflow (Round 4 Focus) +1. **Repository Testing** + - Test deployment scripts: `./scripts/setup-homelab.sh`, `./scripts/deploy-homelab.sh` + - Verify compose file syntax across all stacks + - Validate `.env.example` completeness + - Check documentation accuracy + +2. **Configuration Updates** + - Modify compose files in `docker-compose/` directory + - Update config templates in `config-templates/` + - Ensure changes maintain backward compatibility + +3. **Documentation Maintenance** + - Keep `docs/` synchronized with compose changes + - Update service lists when adding new services + - Document new features or configuration patterns + ### Deploy New Service 1. Create stack directory: `/opt/stacks/stack-name/` 2. Write docker-compose.yml with template @@ -213,6 +278,29 @@ labels: ## Emergency Procedures +### Permission-Related Crashes (Recent Issue) +1. **Diagnose**: Check recent file operations + - Review which files were modified + - Check ownership: `ls -la /path/to/files` + - Identify what triggered permission errors + +2. **Fix Ownership Issues** + ```bash + # For /opt/ directory (if modified during testing) + sudo chown -R kelin:kelin /opt/stacks + + # For repository files + chown -R kelin:kelin ~/AI-Homelab # No sudo needed in home dir + + # For Docker-managed directories, leave as root + # (e.g., /opt/stacks/*/data/ created by containers) + ``` + +3. **Prevent Future Issues** + - Edit files in `~/AI-Homelab/` without sudo + - Only use sudo for Docker commands + - Don't change ownership of Docker-created volumes + ### Service Won't Start 1. Check logs: `docker compose logs service-name` 2. Verify YAML syntax @@ -251,6 +339,9 @@ labels: - Use `:latest` tags in production - Bypass security without explicit request - Modify core stack without understanding dependencies +- **Use sudo for operations in `/home/kelin/` directory** +- **Change file ownership without explicit permission** +- **Blindly escalate privileges when encountering errors** ### Always Do - Read existing configurations first @@ -259,6 +350,59 @@ labels: - Follow established naming patterns - Prioritize security over convenience - Maintain consistency across the stack +- **Check file permissions before operations** +- **Respect user ownership boundaries** +- **Ask before modifying system directories** + +## Testing and Development Guidelines (Round 4) + +### Repository Development +- Work within `~/AI-Homelab/` for all development +- Test scripts in isolated environment before production +- Validate all YAML files before committing +- Ensure `.env.example` stays updated with new variables +- Document breaking changes in commit messages + +### Permission Best Practices +- Repository files: Owned by `kelin:kelin` +- Docker socket: Requires docker group membership +- `/opt/stacks/`: Owned by user, some subdirs by containers +- Never use sudo for editing files in home directory + +### Pre-deployment Validation +```bash +# Validate compose files +docker compose -f docker-compose/core.yml config + +# Check environment variables +grep -v '^#' .env | grep -v '^$' + +# Test script syntax +bash -n scripts/deploy-homelab.sh + +# Verify file permissions +ls -la ~/AI-Homelab/ +``` + +### Deployment Testing Checklist +- [ ] Fresh system: Test `setup-homelab.sh` +- [ ] Core stack: Deploy and verify DuckDNS, Traefik, Authelia, Gluetun +- [ ] Infrastructure: Deploy Dockge and verify web UI access +- [ ] Additional stacks: Test individual stack deployment +- [ ] SSO: Verify authentication works +- [ ] SSL: Check certificate generation +- [ ] VPN: Test Gluetun routing +- [ ] Documentation: Validate all steps in docs/ + +### Round 4 Success Criteria +- [ ] No permission-related crashes +- [ ] All deployment scripts work on fresh Debian install +- [ ] Documentation matches actual implementation +- [ ] All 60+ services deploy successfully +- [ ] Traefik routes all services correctly +- [ ] Authelia protects appropriate services +- [ ] Gluetun routes download clients through VPN +- [ ] No sudo required for repository file editing ## Communication Guidelines diff --git a/README.md b/README.md index 95ac7aa..d6503e2 100644 --- a/README.md +++ b/README.md @@ -75,29 +75,51 @@ The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia f cp .env.example .env nano .env # Edit with your domain, API keys, and passwords ``` - > Alternativly you can ssh in from VS Code using the Remote-ssh plugin and edit in a nice editor + > Alternatively you can ssh in from VS Code using the Remote-SSH plugin and edit in a nice editor + + **Required variables:** + - `DOMAIN` - Your DuckDNS domain (e.g., yourdomain.duckdns.org) + - `DUCKDNS_TOKEN` - Your DuckDNS token + - `ACME_EMAIL` - Your email for Let's Encrypt certificates + - `AUTHELIA_JWT_SECRET` - Generate with: `openssl rand -hex 64` + - `AUTHELIA_SESSION_SECRET` - Generate with: `openssl rand -hex 64` + - `AUTHELIA_STORAGE_ENCRYPTION_KEY` - Generate with: `openssl rand -hex 64` + - `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN + + > See [Getting Started](docs/getting-started.md) for detailed instructions - Required variables: DOMAIN, DUCKDNS_TOKEN, TZ, Authelia user credentials, API keys for services you plan to use. - > See [Getting Started](docs/getting-started.md) for more details 4. **Run deployment script:** - This automated script will create required directories, verify Docker networks exist, deploy core stack (DuckDNS, Traefik, Authelia, Gluetun), deploy the infrastructure stack and open Dockge in your browser when ready. + This automated script will: + - Create Docker networks + - Configure Traefik with your email + - Generate Authelia admin password (saved to `/opt/stacks/core/authelia/ADMIN_PASSWORD.txt`) + - Deploy core stack (DuckDNS, Traefik, Authelia, Gluetun) + - Deploy infrastructure stack (Dockge, Pi-hole, monitoring tools) + - Open Dockge in your browser ```bash ./scripts/deploy-homelab.sh ``` + **Login credentials:** Username: `admin` | Password: Check `/opt/stacks/core/authelia/ADMIN_PASSWORD.txt` 5. **Deploy additional stacks through Dockge:** - Log in to Dockge with your Authelia credentials and deploy additional stacks: dashboards.yml, media.yml, media-extended.yml, homeassistant.yml, productivity.yml, monitoring.yml, utilities.yml. + Log in to Dockge at `https://dockge.yourdomain.duckdns.org` and deploy additional stacks from the repository's `docker-compose/` directory: + - `dashboards.yml` - Homepage, Homarr + - `media.yml` - Plex, Jellyfin, Sonarr, Radarr, etc. + - `media-extended.yml` - Readarr, Lidarr, etc. + - `homeassistant.yml` - Home Assistant and accessories + - `productivity.yml` - Nextcloud, Gitea, wikis + - `monitoring.yml` - Grafana, Prometheus, etc. + - `utilities.yml` - Backups, code editors, etc. -6. **Configure VS Code to control the server via Github Copilot** +6. **Configure VS Code to control the server via GitHub Copilot** - Log into VS Code, install and configure the Github Copilot extension with your api key. - Use the Copilot chat window to manage your homelab + Install and configure the GitHub Copilot extension in VS Code, then use the Copilot chat window to manage your homelab. - > Tip: If you have a paid account use the free models to perform simple tasks like starting/stopping a service, and premium models to do more advanced tasks. + > Tip: Use free models for simple tasks like starting/stopping services, and premium models for complex configurations. # # diff --git a/config-templates/traefik/traefik.yml b/config-templates/traefik/traefik.yml index 60529fb..3f6eba0 100644 --- a/config-templates/traefik/traefik.yml +++ b/config-templates/traefik/traefik.yml @@ -27,12 +27,14 @@ entryPoints: certificatesResolvers: letsencrypt: acme: - email: ${ACME_EMAIL} + email: ACME_EMAIL_PLACEHOLDER # Will be replaced by deploy script storage: /acme.json - # Use HTTP challenge (port 80 must be accessible) + # HTTP challenge - Simple setup, port 80 must be accessible + # Works for individual domain certificates httpChallenge: entryPoint: web - # Or use DNS challenge (requires API token): + # DNS challenge - For wildcard certificates (advanced) + # Uncomment and comment out httpChallenge to use: # dnsChallenge: # provider: duckdns # resolvers: diff --git a/docker-compose/core.yml b/docker-compose/core.yml index 53fc8b1..eab75f3 100644 --- a/docker-compose/core.yml +++ b/docker-compose/core.yml @@ -18,7 +18,7 @@ services: - TOKEN=${DUCKDNS_TOKEN} # Your DuckDNS token - UPDATE_IP=ipv4 # or ipv6, or both volumes: - - /opt/stacks/core/duckdns:/config + - ./duckdns:/config labels: - "homelab.category=infrastructure" - "homelab.description=Dynamic DNS updater" @@ -40,9 +40,9 @@ services: volumes: - /etc/localtime:/etc/localtime:ro - /var/run/docker.sock:/var/run/docker.sock:ro - - /opt/stacks/core/traefik/traefik.yml:/traefik.yml:ro - - /opt/stacks/core/traefik/dynamic:/dynamic:ro - - /opt/stacks/core/traefik/acme.json:/acme.json + - ./traefik/traefik.yml:/traefik.yml:ro + - ./traefik/dynamic:/dynamic:ro + - ./traefik/acme.json:/acme.json environment: - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN} # If using Cloudflare DNS challenge - DUCKDNS_TOKEN=${DUCKDNS_TOKEN} # If using DuckDNS @@ -73,8 +73,8 @@ services: networks: - traefik-network volumes: - - /opt/stacks/core/authelia/configuration.yml:/config/configuration.yml:ro - - /opt/stacks/core/authelia/users_database.yml:/config/users_database.yml + - ./authelia/configuration.yml:/config/configuration.yml:ro + - ./authelia/users_database.yml:/config/users_database.yml - authelia-data:/data environment: - TZ=${TZ} @@ -115,7 +115,7 @@ services: - "6881:6881" # qBittorrent - "6881:6881/udp" # qBittorrent volumes: - - /opt/stacks/core/gluetun:/gluetun + - ./gluetun:/gluetun environment: - VPN_SERVICE_PROVIDER=surfshark - VPN_TYPE=openvpn diff --git a/docker-compose/dashboards.yml b/docker-compose/dashboards.yml index d1888d7..8d04dae 100644 --- a/docker-compose/dashboards.yml +++ b/docker-compose/dashboards.yml @@ -14,7 +14,7 @@ services: - traefik-network - dockerproxy-network volumes: - - /opt/stacks/homepage/config:/app/config + - ./homepage:/app/config - /var/run/docker.sock:/var/run/docker.sock:ro # For Docker integration - /opt/stacks:/opt/stacks:ro # To discover other stacks environment: @@ -42,9 +42,9 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/homarr/configs:/app/data/configs - - /opt/stacks/homarr/data:/data - - /opt/stacks/homarr/icons:/app/public/icons + - ./homarr/config:/app/config/configs + - ./homarr/data:/data + - ./homarr/icons:/app/public/icons - /var/run/docker.sock:/var/run/docker.sock:ro environment: - TZ=${TZ} diff --git a/docker-compose/homeassistant.yml b/docker-compose/homeassistant.yml index 199330e..d260d25 100644 --- a/docker-compose/homeassistant.yml +++ b/docker-compose/homeassistant.yml @@ -12,7 +12,7 @@ services: restart: unless-stopped network_mode: host # Required for device discovery volumes: - - /opt/stacks/homeassistant/config:/config + - ./$(basename $file .yml)/config:/config - /etc/localtime:/etc/localtime:ro environment: - TZ=${TZ} @@ -33,7 +33,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/esphome/config:/config + - ./$(basename $file .yml)/config:/config - /etc/localtime:/etc/localtime:ro environment: - TZ=${TZ} @@ -84,7 +84,7 @@ services: ports: - "8765:8765" # Optional: direct access volumes: - - /opt/stacks/motioneye/config:/etc/motioneye + - ./$(basename $file .yml)/config:/etc/motioneye - /mnt/surveillance:/var/lib/motioneye # Large video files on separate drive environment: - TZ=${TZ} @@ -133,7 +133,7 @@ services: - "1883:1883" # MQTT - "9001:9001" # Websockets volumes: - - /opt/stacks/mosquitto/config:/mosquitto/config + - ./$(basename $file .yml)/config:/mosquitto/config - /opt/stacks/mosquitto/data:/mosquitto/data - /opt/stacks/mosquitto/log:/mosquitto/log labels: diff --git a/docker-compose/infrastructure.yml b/docker-compose/infrastructure.yml index 6d6f4e1..c96833d 100644 --- a/docker-compose/infrastructure.yml +++ b/docker-compose/infrastructure.yml @@ -45,8 +45,8 @@ services: - "53:53/tcp" # DNS TCP - "53:53/udp" # DNS UDP volumes: - - /opt/stacks/pihole/etc-pihole:/etc/pihole - - /opt/stacks/pihole/etc-dnsmasq.d:/etc/dnsmasq.d + - ./pihole/etc-pihole:/etc/pihole + - ./pihole/etc-dnsmasq.d:/etc/dnsmasq.d environment: - TZ=${TZ:-America/New_York} - WEBPASSWORD=${PIHOLE_PASSWORD:-changeme} @@ -145,7 +145,7 @@ services: pid: host volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - - /opt/stacks/glances/config:/glances/conf + - ./glances/config:/glances/conf environment: - GLANCES_OPT=-w labels: diff --git a/docker-compose/media-extended.yml b/docker-compose/media-extended.yml index 7ee61cb..4fa288f 100644 --- a/docker-compose/media-extended.yml +++ b/docker-compose/media-extended.yml @@ -14,7 +14,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/readarr/config:/config + - ./$(basename $file .yml)/config:/config - /mnt/media/books:/books - /mnt/downloads:/downloads environment: @@ -42,7 +42,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/lidarr/config:/config + - ./$(basename $file .yml)/config:/config - /mnt/media/music:/music - /mnt/downloads:/downloads environment: @@ -70,7 +70,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/lazylibrarian/config:/config + - ./$(basename $file .yml)/config:/config - /mnt/media/books:/books - /mnt/downloads:/downloads environment: @@ -127,7 +127,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/calibre-web/config:/config + - ./$(basename $file .yml)/config:/config - /mnt/media/books:/books environment: - PUID=${PUID:-1000} @@ -155,7 +155,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/jellyseerr/config:/app/config + - ./$(basename $file .yml)/config:/app/config environment: - LOG_LEVEL=info - TZ=${TZ} @@ -198,7 +198,7 @@ services: - "8266:8266" # Server port volumes: - /opt/stacks/tdarr/server:/app/server - - /opt/stacks/tdarr/configs:/app/configs + - ./$(basename $file .yml)/configs:/app/configs - /opt/stacks/tdarr/logs:/app/logs - /mnt/media:/media - /mnt/tdarr-transcode:/temp # Transcode cache on separate drive @@ -228,7 +228,7 @@ services: networks: - media-network volumes: - - /opt/stacks/tdarr/configs:/app/configs + - ./$(basename $file .yml)/configs:/app/configs - /opt/stacks/tdarr/logs:/app/logs - /mnt/media:/media - /mnt/tdarr-transcode:/temp @@ -256,7 +256,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/unmanic/config:/config + - ./$(basename $file .yml)/config:/config - /mnt/media:/library - /mnt/unmanic-cache:/tmp/unmanic # Transcode cache on separate drive environment: diff --git a/docker-compose/media.yml b/docker-compose/media.yml index ba3f1c2..da5e320 100644 --- a/docker-compose/media.yml +++ b/docker-compose/media.yml @@ -16,7 +16,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/plex/config:/config + - ./plex/config:/config - /mnt/media:/media:ro # Large media files on separate drive - plex-transcode:/transcode environment: @@ -64,8 +64,8 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/jellyfin/config:/config - - /opt/stacks/jellyfin/cache:/cache + - ./jellyfin/config:/config + - ./jellyfin/cache:/cache - /mnt/media:/media:ro # Large media files on separate drive environment: - PUID=${PUID:-1000} @@ -95,7 +95,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/sonarr/config:/config + - ./sonarr/config:/config - /mnt/media:/media - /mnt/downloads:/downloads # Large downloads on separate drive environment: @@ -124,7 +124,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/radarr/config:/config + - ./radarr/config:/config - /mnt/media:/media - /mnt/downloads:/downloads # Large downloads on separate drive environment: @@ -153,7 +153,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/prowlarr/config:/config + - ./prowlarr/config:/config environment: - PUID=${PUID:-1000} - PGID=${PGID:-1000} diff --git a/docker-compose/productivity.yml b/docker-compose/productivity.yml index 5d85fe3..6105ca7 100644 --- a/docker-compose/productivity.yml +++ b/docker-compose/productivity.yml @@ -188,7 +188,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/dokuwiki/config:/config + - ./$(basename $file .yml)/config:/config environment: - PUID=${PUID:-1000} - PGID=${PGID:-1000} @@ -214,7 +214,7 @@ services: - traefik-network - bookstack-network volumes: - - /opt/stacks/bookstack/config:/config + - ./$(basename $file .yml)/config:/config environment: - PUID=${PUID:-1000} - PGID=${PGID:-1000} diff --git a/docker-compose/utilities.yml b/docker-compose/utilities.yml index 0ba3b32..3086207 100644 --- a/docker-compose/utilities.yml +++ b/docker-compose/utilities.yml @@ -13,7 +13,7 @@ services: - traefik-network volumes: - /opt/stacks/backrest/data:/data - - /opt/stacks/backrest/config:/config + - ./$(basename $file .yml)/config:/config - /opt/stacks:/opt/stacks:ro # Backup source - /mnt:/mnt:ro # Backup additional drives - backrest-cache:/cache @@ -41,7 +41,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/duplicati/config:/config + - ./$(basename $file .yml)/config:/config - /opt/stacks:/source/stacks:ro - /mnt:/source/mnt:ro - /mnt/backups:/backups @@ -69,7 +69,7 @@ services: - homelab-network - traefik-network volumes: - - /opt/stacks/code-server/config:/config + - ./$(basename $file .yml)/config:/config - /opt/stacks:/opt/stacks # Access to all stacks - /mnt:/mnt:ro # Read-only access to data environment: diff --git a/docs/getting-started.md b/docs/getting-started.md index 7500589..67e6954 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -35,23 +35,33 @@ For most users, the automated setup script handles everything: # Copy these values and add them to your .env file ``` -7. **Generate Authelia Admin Password Hash**: - ```bash - # Replace 'yourpassword' with your desired admin password - docker run --rm authelia/authelia:4.37 authelia crypto hash generate argon2 --password 'yourpassword' - - # Copy the output hash and update /opt/stacks/core/authelia/users_database.yml - # Replace the password field for the admin user - ``` -8. **Configure environment**: +7. **Configure environment**: ```bash cp .env.example .env nano .env # Edit with your settings and paste the Authelia secrets ``` -9. **Deploy core services**: + + **Required variables in .env:** + - `DOMAIN` - Your DuckDNS domain (e.g., yourdomain.duckdns.org) + - `DUCKDNS_TOKEN` - Your DuckDNS token + - `ACME_EMAIL` - Your email for Let's Encrypt certificates + - `AUTHELIA_JWT_SECRET` - Generated in step 6 + - `AUTHELIA_SESSION_SECRET` - Generated in step 6 + - `AUTHELIA_STORAGE_ENCRYPTION_KEY` - Generated in step 6 + - `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN + +8. **Deploy homelab**: ```bash ./scripts/deploy-homelab.sh ``` + + **The deploy script automatically:** + - Creates Docker networks + - Configures Traefik with your email + - Generates Authelia admin password + - Deploys core stack (DuckDNS, Traefik, Authelia, Gluetun) + - Deploys infrastructure stack (Dockge, Pi-hole, monitoring) + - Opens Dockge in your browser **That's it!** Your homelab is ready. Access Dockge at `https://dockge.yourdomain.duckdns.org` @@ -225,3 +235,36 @@ docker compose up -d --build service-name 5. **Use AI assistance** for custom configurations Happy homelabbing! 🚀 + +## Deployment Improvements (Round 4) + +The repository has been enhanced with the following improvements for better user experience: + +### Automated Configuration +- **Email Substitution**: Deploy script automatically configures Traefik with your ACME_EMAIL +- **Password Generation**: Authelia admin password is auto-generated and saved to `/opt/stacks/core/authelia/ADMIN_PASSWORD.txt` +- **Network Creation**: Docker networks are created automatically before deployment + +### Volume Path Standardization +- All compose files now use **relative paths** (e.g., `./service/config`) for portability +- Stacks work correctly when deployed via Dockge or docker compose +- Large shared data still uses absolute paths (`/mnt/media`, `/mnt/downloads`) + +### SSL Certificate Configuration +- **Default**: HTTP challenge (simple setup, works immediately) +- **Optional**: DNS challenge for wildcard certificates (see comments in traefik.yml) +- Certificates are automatically requested and renewed by Traefik + +### What's Automated +✅ Docker network creation +✅ Traefik email configuration +✅ Authelia password generation +✅ Domain configuration in Authelia +✅ Directory structure creation +✅ Service deployment + +### What You Configure +📝 `.env` file with your domain and API keys +📝 DuckDNS token +📝 VPN credentials (if using Gluetun) +📝 Service-specific settings via Dockge diff --git a/scripts/deploy-homelab.sh b/scripts/deploy-homelab.sh index 495c9a6..058252d 100755 --- a/scripts/deploy-homelab.sh +++ b/scripts/deploy-homelab.sh @@ -125,35 +125,55 @@ touch /opt/stacks/core/traefik/acme.json chmod 600 /opt/stacks/core/traefik/acme.json log_success "acme.json created with correct permissions" +# Replace email placeholder in traefik.yml +log_info "Configuring Traefik with email: $ACME_EMAIL..." +sed -i "s/ACME_EMAIL_PLACEHOLDER/${ACME_EMAIL}/g" /opt/stacks/core/traefik/traefik.yml +log_success "Traefik email configured" + # Replace domain placeholder in authelia configuration log_info "Configuring Authelia for domain: $DOMAIN..." sed -i "s/your-domain.duckdns.org/${DOMAIN}/g" /opt/stacks/core/authelia/configuration.yml -# Create Authelia users database with admin credentials from setup script -if [ -f /tmp/authelia_admin_credentials.tmp ]; then - log_info "Configuring Authelia admin user..." - source /tmp/authelia_admin_credentials.tmp +# Generate Authelia admin password if not already configured +if grep -q "CHANGEME" /opt/stacks/core/authelia/users_database.yml 2>/dev/null || [ ! -f /opt/stacks/core/authelia/users_database.yml ]; then + log_info "Generating Authelia admin credentials..." - cat > /opt/stacks/core/authelia/users_database.yml << EOF + # Generate a random password if not provided + ADMIN_PASSWORD="${AUTHELIA_ADMIN_PASSWORD:-$(openssl rand -base64 16)}" + + # Generate password hash using Authelia container + log_info "Generating password hash (this may take a moment)..." + PASSWORD_HASH=$(docker run --rm authelia/authelia:4.37 authelia crypto hash generate argon2 --password "$ADMIN_PASSWORD" | grep 'Digest:' | awk '{print $2}') + + if [ -z "$PASSWORD_HASH" ]; then + log_error "Failed to generate password hash" + log_info "Using template users_database.yml - please configure manually" + else + # Create users_database.yml with generated credentials + cat > /opt/stacks/core/authelia/users_database.yml << EOF ############################################################### # Users Database # ############################################################### users: - ${ADMIN_USER}: - displayname: "${ADMIN_USER}" + admin: + displayname: "Admin User" password: "${PASSWORD_HASH}" - email: ${ADMIN_EMAIL} + email: ${ACME_EMAIL} groups: - admins - - dev + - users EOF - - log_success "Authelia admin user configured: $ADMIN_USER" - rm -f /tmp/authelia_admin_credentials.tmp + + log_success "Authelia admin user configured" + log_info "Admin username: admin" + log_info "Admin password: $ADMIN_PASSWORD" + log_warning "SAVE THIS PASSWORD! Writing to /opt/stacks/core/authelia/ADMIN_PASSWORD.txt" + echo "$ADMIN_PASSWORD" > /opt/stacks/core/authelia/ADMIN_PASSWORD.txt + chmod 600 /opt/stacks/core/authelia/ADMIN_PASSWORD.txt + fi else - log_warning "Admin credentials not found. Using template users_database.yml" - log_info "You will need to manually configure /opt/stacks/core/authelia/users_database.yml" + log_info "Authelia users_database.yml already configured" fi # Deploy core stack