Update all documentation to reflect 60+ services, Dockge, Traefik, Authelia, and new features

- Update README.md with comprehensive overview of all 40+ services
- Add Quick Setup guide with Dockge deployment steps
- List all service stacks by category (Infrastructure, Media, Home Automation, Productivity, etc.)
- Update AI assistant examples for Traefik, Authelia, VPN routing
- Add network architecture with Traefik routing patterns
- Update common operations for Dockge structure
- Add storage strategy (opt/stacks vs /mnt/)
- Update configuration management section
- Add proxying external hosts overview
- Update getting started checklist with all new steps

- Update docker-guidelines.md with Dockge and Traefik sections
- Add comprehensive Dockge structure explanation
- Add Traefik and Authelia integration patterns
- Document when to use SSO vs bypass
- Add VPN routing patterns with Gluetun
- Update storage strategy recommendations

- Completely rewrite getting-started.md with step-by-step Dockge deployment
- Add DuckDNS signup instructions
- Add detailed core infrastructure deployment (DuckDNS, Traefik, Authelia, Dockge)
- Add Homepage configuration steps
- Add API key configuration for widgets
- Add troubleshooting section
- Add security checklist

- Create docs/services-reference.md - comprehensive catalog of all 60+ services
- Organized by category with descriptions
- Access URLs and SSO requirements
- Storage recommendations
- Quick deployment guide
- Summary table by stack

All documentation now accurately reflects the complete homelab setup with Dockge, Traefik, Authelia, Gluetun, Homepage, and all 40+ services across 10 stack files.

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-01-12 01:16:12 +00:00
parent 63f39e51a4
commit 6fec6b501e
4 changed files with 1297 additions and 509 deletions

View File

@@ -1,6 +1,6 @@
# Getting Started Guide
This guide will walk you through setting up your AI-powered homelab from scratch.
This guide will walk you through setting up your AI-powered homelab with Dockge, Traefik, Authelia, and 40+ services from scratch.
## Prerequisites
@@ -11,8 +11,10 @@ Before you begin, ensure you have:
- [ ] Docker Compose V2 installed
- [ ] Git installed
- [ ] At least 8GB RAM (16GB+ recommended)
- [ ] Sufficient disk space (100GB+ recommended)
- [ ] Static IP address for your server
- [ ] Sufficient disk space (100GB+ system, 1TB+ for media recommended)
- [ ] Static IP address for your server (or DHCP reservation)
- [ ] DuckDNS account (free) with a domain
- [ ] Surfshark VPN account (optional, for VPN features)
- [ ] VS Code with GitHub Copilot extension (for AI assistance)
## Step 1: Verify Docker Installation
@@ -45,7 +47,15 @@ git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
```
## Step 3: Configure Environment Variables
## Step 3: Sign Up for DuckDNS
1. Go to https://www.duckdns.org/
2. Sign in with your preferred method
3. Create a domain (e.g., `myhomelab`)
4. Copy your token - you'll need it for `.env`
5. Your domain will be: `myhomelab.duckdns.org`
## Step 4: Configure Environment Variables
```bash
# Copy the example environment file
@@ -59,7 +69,7 @@ id -g # This is your PGID
nano .env
```
**Update these values in `.env`:**
**Critical values to update in `.env`:**
```bash
# Your user/group IDs
PUID=1000 # Replace with your user ID
@@ -71,442 +81,333 @@ TZ=America/New_York
# Your server's IP address
SERVER_IP=192.168.1.100 # Replace with your actual IP
# Directory paths
USERDIR=/home/yourusername/homelab # Update username
MEDIADIR=/mnt/media # Update if different
DOWNLOADDIR=/mnt/downloads # Update if different
# DuckDNS Configuration
DOMAIN=myhomelab.duckdns.org # Your DuckDNS domain
DUCKDNS_TOKEN=your-duckdns-token-here
DUCKDNS_SUBDOMAINS=myhomelab # Without .duckdns.org
# Set secure passwords for services
GRAFANA_ADMIN_PASSWORD=your-secure-password-here
CODE_SERVER_PASSWORD=your-secure-password-here
POSTGRES_PASSWORD=your-secure-password-here
PGADMIN_PASSWORD=your-secure-password-here
JUPYTER_TOKEN=your-secure-token-here
PIHOLE_PASSWORD=your-secure-password-here
# Let's Encrypt Email
ACME_EMAIL=your-email@example.com
# Authelia Secrets (generate with: openssl rand -hex 64)
AUTHELIA_JWT_SECRET=$(openssl rand -hex 64)
AUTHELIA_SESSION_SECRET=$(openssl rand -hex 64)
AUTHELIA_STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 64)
# Surfshark VPN (if using)
SURFSHARK_PRIVATE_KEY=your-wireguard-private-key
SURFSHARK_ADDRESSES=10.14.0.2/16
# Set secure passwords for all services
PIHOLE_PASSWORD=your-secure-password
GRAFANA_ADMIN_PASSWORD=your-secure-password
CODE_SERVER_PASSWORD=your-secure-password
# ... (see .env.example for complete list)
```
**Save and exit** (Ctrl+X, Y, Enter in nano)
## Step 4: Create Docker Networks
## Step 5: Create Dockge Directory Structure
```bash
# Create the main homelab network
docker network create homelab-network
# Create main stacks directory
sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
# Create additional networks for better security
# Create mount points for large data (adjust as needed)
sudo mkdir -p /mnt/media/{movies,tv,music,books,photos}
sudo mkdir -p /mnt/downloads/{complete,incomplete}
sudo mkdir -p /mnt/backups
sudo chown -R $USER:$USER /mnt/media /mnt/downloads /mnt/backups
```
## Step 6: Create Docker Networks
```bash
# Create required external networks
docker network create traefik-network
docker network create homelab-network
docker network create media-network
docker network create monitoring-network
docker network create database-network
docker network create dockerproxy-network
# Verify networks were created
docker network ls
docker network ls | grep -E "traefik|homelab|media|dockerproxy"
```
## Step 5: Create Configuration Directories
## Step 7: Deploy Core Infrastructure (IN ORDER)
### 7.1 DuckDNS (Dynamic DNS)
```bash
# Create the main config directory
mkdir -p config
# Create stack directory
mkdir -p /opt/stacks/duckdns
# Create config directories for services you plan to use
mkdir -p config/{nginx-proxy-manager,pihole,portainer}
mkdir -p config/{plex,sonarr,radarr,prowlarr,qbittorrent,jellyfin}
mkdir -p config/{prometheus,grafana,loki,promtail}
mkdir -p config/{code-server,postgres,redis}
# Copy compose file
cp ~/AI-Homelab/docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
# Set proper permissions
sudo chown -R $(id -u):$(id -g) config/
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/duckdns/.env
# Deploy
cd /opt/stacks/duckdns
docker compose up -d
# Verify it's working
docker compose logs -f
# Should see: "Your IP was updated to X.X.X.X"
```
## Step 6: Copy Configuration Templates (Optional)
For services that need config files:
### 7.2 Traefik (Reverse Proxy with SSL)
```bash
# Prometheus
mkdir -p config/prometheus
cp config-templates/prometheus/prometheus.yml config/prometheus/
# Create stack directory with dynamic configs
mkdir -p /opt/stacks/traefik/dynamic
# Loki
mkdir -p config/loki
cp config-templates/loki/loki-config.yml config/loki/
# Copy compose file
cp ~/AI-Homelab/docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
# Promtail
mkdir -p config/promtail
cp config-templates/promtail/promtail-config.yml config/promtail/
# Copy configuration templates
cp ~/AI-Homelab/config-templates/traefik/traefik.yml /opt/stacks/traefik/
cp ~/AI-Homelab/config-templates/traefik/dynamic/*.yml /opt/stacks/traefik/dynamic/
# Redis
mkdir -p config/redis
cp config-templates/redis/redis.conf config/redis/
# Create acme.json for SSL certificates
touch /opt/stacks/traefik/acme.json
chmod 600 /opt/stacks/traefik/acme.json
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/traefik/.env
# Deploy
cd /opt/stacks/traefik
docker compose up -d
# Check logs
docker compose logs -f
# Should see Traefik starting and certificate resolver configured
```
## Step 7: Start Your First Service (Portainer)
Portainer provides a web UI for managing Docker containers. It's a great first service to deploy.
### 7.3 Authelia (SSO Authentication)
```bash
# Start Portainer
docker compose -f docker-compose/infrastructure.yml up -d portainer
# Create stack directory
mkdir -p /opt/stacks/authelia
# Check if it's running
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
# View logs
docker compose -f docker-compose/infrastructure.yml logs -f portainer
# Copy configuration templates
cp ~/AI-Homelab/config-templates/authelia/*.yml /opt/stacks/authelia/
# Generate password hash for users_database.yml
docker run --rm authelia/authelia:latest authelia crypto hash generate argon2 --password 'yourpassword'
# Copy the hash and edit users_database.yml
# Edit users_database.yml
cd /opt/stacks/authelia
nano users_database.yml
# Replace the password hash with your generated one
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/authelia/.env
# Deploy
docker compose up -d
# Check logs
docker compose logs -f
# Test login at https://auth.yourdomain.duckdns.org
```
**Access Portainer:**
1. Open your browser
2. Navigate to `http://YOUR_SERVER_IP:9000`
3. Create an admin account on first login
4. Select "Get Started" and choose local Docker environment
## Step 8: Deploy Infrastructure Services
### 7.4 Infrastructure Services (Dockge)
```bash
# Start all infrastructure services
docker compose -f docker-compose/infrastructure.yml up -d
# Create stack directory
mkdir -p /opt/stacks/infrastructure
# Check status
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
# Services now running:
# - Nginx Proxy Manager: http://YOUR_SERVER_IP:81
# - Pi-hole: http://YOUR_SERVER_IP:8080/admin
# - Portainer: http://YOUR_SERVER_IP:9000
# - Watchtower: (runs in background, no UI)
# Create necessary subdirectories
mkdir -p /opt/dockge/data
mkdir -p /opt/stacks/pihole/{etc-pihole,etc-dnsmasq.d}
mkdir -p /opt/stacks/glances/config
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/infrastructure/.env
# Deploy Dockge first
cd /opt/stacks/infrastructure
docker compose up -d dockge
# Access Dockge at https://dockge.yourdomain.duckdns.org
# Deploy remaining infrastructure services
docker compose up -d
```
### Configure Nginx Proxy Manager (Optional)
## Step 8: Deploy Additional Stacks
1. Access: `http://YOUR_SERVER_IP:81`
2. Login with default credentials:
- Email: `admin@example.com`
- Password: `changeme`
3. Change password immediately
4. Add proxy hosts for your services
Now use Dockge UI at `https://dockge.yourdomain.duckdns.org` to deploy additional stacks, or continue with command line:
### Configure Pi-hole (Optional)
1. Access: `http://YOUR_SERVER_IP:8080/admin`
2. Login with password from `.env` (PIHOLE_PASSWORD)
3. Configure DNS settings
4. Update your router to use Pi-hole as DNS server
## Step 9: Deploy Media Services (Optional)
If you want a media server setup:
### 8.1 Gluetun + qBittorrent (VPN)
```bash
# Start media services
docker compose -f docker-compose/media.yml up -d
mkdir -p /opt/stacks/gluetun
cp ~/AI-Homelab/docker-compose/gluetun.yml /opt/stacks/gluetun/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/gluetun/.env
# Check status
docker compose -f docker-compose/media.yml ps
cd /opt/stacks/gluetun
docker compose up -d
# Services now running:
# - Plex: http://YOUR_SERVER_IP:32400/web
# - Jellyfin: http://YOUR_SERVER_IP:8096
# - Sonarr: http://YOUR_SERVER_IP:8989
# - Radarr: http://YOUR_SERVER_IP:7878
# - Prowlarr: http://YOUR_SERVER_IP:9696
# - qBittorrent: http://YOUR_SERVER_IP:8081
# Test VPN
docker exec gluetun curl ifconfig.me
# Should show VPN IP
```
### Initial Media Service Setup
**Plex:**
1. Access: `http://YOUR_SERVER_IP:32400/web`
2. Sign in with your Plex account
3. Add your media libraries
**Sonarr/Radarr:**
1. Access the web UI
2. Go to Settings → Profiles → Quality
3. Configure your quality preferences
4. Add Prowlarr as an indexer
5. Add qBittorrent as a download client
## Step 10: Deploy Monitoring Services (Optional)
For system and service monitoring:
### 8.2 Homepage Dashboard
```bash
# Start monitoring services
docker compose -f docker-compose/monitoring.yml up -d
mkdir -p /opt/stacks/homepage/config
cp ~/AI-Homelab/docker-compose/dashboards.yml /opt/stacks/homepage/docker-compose.yml
cp ~/AI-Homelab/config-templates/homepage/* /opt/stacks/homepage/config/
cp ~/AI-Homelab/.env /opt/stacks/homepage/.env
# Check status
docker compose -f docker-compose/monitoring.yml ps
cd /opt/stacks/homepage
docker compose up -d homepage
# Services now running:
# - Prometheus: http://YOUR_SERVER_IP:9090
# - Grafana: http://YOUR_SERVER_IP:3000
# - Node Exporter: http://YOUR_SERVER_IP:9100
# - cAdvisor: http://YOUR_SERVER_IP:8082
# - Uptime Kuma: http://YOUR_SERVER_IP:3001
# Access at https://home.yourdomain.duckdns.org
```
### Configure Grafana
1. Access: `http://YOUR_SERVER_IP:3000`
2. Login with credentials from `.env`:
- Username: `admin`
- Password: `GRAFANA_ADMIN_PASSWORD` from .env
3. Add Prometheus as a data source:
- URL: `http://prometheus:9090`
4. Import dashboards:
- Dashboard ID 1860 for Node Exporter
- Dashboard ID 893 for Docker metrics
### Configure Uptime Kuma
1. Access: `http://YOUR_SERVER_IP:3001`
2. Create an account on first login
3. Add monitors for your services
## Step 11: Deploy Development Services (Optional)
If you need development tools:
### 8.3 Media Stack
```bash
# Start development services
docker compose -f docker-compose/development.yml up -d
mkdir -p /opt/stacks/media
cp ~/AI-Homelab/docker-compose/media.yml /opt/stacks/media/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/media/.env
# Check status
docker compose -f docker-compose/development.yml ps
# Services now running:
# - Code Server: http://YOUR_SERVER_IP:8443
# - PostgreSQL: localhost:5432
# - Redis: localhost:6379
# - pgAdmin: http://YOUR_SERVER_IP:5050
# - Jupyter Lab: http://YOUR_SERVER_IP:8888
# - Node-RED: http://YOUR_SERVER_IP:1880
cd /opt/stacks/media
docker compose up -d
```
## Step 12: Set Up VS Code with GitHub Copilot
### 8.4 Additional Stacks
1. **Install VS Code** on your local machine (if not already installed)
Deploy as needed:
- `media-extended.yml``/opt/stacks/media-extended/`
- `homeassistant.yml``/opt/stacks/homeassistant/`
- `productivity.yml``/opt/stacks/productivity/`
- `utilities.yml``/opt/stacks/utilities/`
- `monitoring.yml``/opt/stacks/monitoring/`
- `development.yml``/opt/stacks/development/`
2. **Install GitHub Copilot extension:**
- Open VS Code
- Go to Extensions (Ctrl+Shift+X)
- Search for "GitHub Copilot"
- Click Install
- Sign in with your GitHub account
## Step 9: Configure Homepage Widgets
3. **Clone your repository in VS Code:**
```bash
# On your local machine
# Note: Replace 'kelinfoxy' with your username if you forked this repository
git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
code .
```
4. **Start using AI assistance:**
- Open Copilot Chat (Ctrl+Shift+I or click the chat icon)
- The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md`
- Ask questions like:
- "Help me add Home Assistant to my homelab"
- "Create a backup script for my Docker volumes"
- "How do I configure GPU support for Plex?"
## Step 13: Verify Everything is Running
Get API keys from each service and add to Homepage config:
```bash
# Check all running containers
docker ps
cd /opt/stacks/homepage/config
nano services.yaml
# Check container health
docker ps --format "table {{.Names}}\t{{.Status}}"
# Get API keys:
# - Sonarr/Radarr/etc: Settings → General → API Key
# - Plex: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/
# - Jellyfin: Dashboard → API Keys
# View resource usage
docker stats --no-stream
# Add to .env:
nano /opt/stacks/homepage/.env
# HOMEPAGE_VAR_SONARR_KEY=...
# HOMEPAGE_VAR_RADARR_KEY=...
# etc.
# Check disk usage
docker system df
# Restart Homepage
cd /opt/stacks/homepage
docker compose restart
```
## Step 14: Set Up Backups
Create a backup script:
## Step 10: Install VS Code and GitHub Copilot
```bash
# Create a backup directory
mkdir -p ~/backups
# Install VS Code (if not already installed)
# Download from https://code.visualstudio.com/
# Create a simple backup script
cat > ~/backup-homelab.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=~/backups
DATE=$(date +%Y%m%d)
# Install GitHub Copilot extension
# In VS Code: Extensions → Search "GitHub Copilot" → Install
# Backup config directories
tar czf $BACKUP_DIR/config-$DATE.tar.gz ~/AI-Homelab/config/
# Open the repository
code ~/AI-Homelab
# Backup .env file
cp ~/AI-Homelab/.env $BACKUP_DIR/.env-$DATE
# Backup Docker volumes (example for Portainer)
docker run --rm \
-v portainer-data:/data \
-v $BACKUP_DIR:/backup \
busybox tar czf /backup/portainer-data-$DATE.tar.gz /data
echo "Backup completed: $DATE"
EOF
# Make it executable
chmod +x ~/backup-homelab.sh
# Test the backup
~/backup-homelab.sh
```
Set up a cron job for automated backups:
```bash
# Open crontab
crontab -e
# Add this line to run backup daily at 2 AM
0 2 * * * /home/yourusername/backup-homelab.sh
```
## Step 15: Configure Firewall (Optional but Recommended)
If using UFW:
```bash
# Allow SSH (if not already allowed)
sudo ufw allow 22/tcp
# Allow web traffic (if exposing services to internet)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Allow specific services from local network only
# Replace 192.168.1.0/24 with your network
sudo ufw allow from 192.168.1.0/24 to any port 9000 proto tcp # Portainer
sudo ufw allow from 192.168.1.0/24 to any port 81 proto tcp # Nginx Proxy Manager
sudo ufw allow from 192.168.1.0/24 to any port 3000 proto tcp # Grafana
# Enable firewall
sudo ufw enable
# Start using AI assistance!
```
## Next Steps
Now that your homelab is running:
1. **Explore Services:**
- Access each service's web UI
- Configure settings as needed
- Set up integrations between services
2. **Add More Services:**
- Ask GitHub Copilot for help adding new services
- Follow the patterns in existing compose files
- Check [awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) for ideas
3. **Optimize:**
- Review logs for errors
- Adjust resource limits
- Set up proper monitoring and alerts
4. **Secure:**
- Change all default passwords
- Set up SSL certificates (use Nginx Proxy Manager)
- Enable 2FA where available
- Keep services updated
5. **Learn:**
- Read the [Docker Guidelines](docker-guidelines.md)
- Experiment with new services
- Use AI assistance to understand and modify configurations
1. Explore Dockge at `https://dockge.yourdomain.duckdns.org`
2. Check Homepage dashboard at `https://home.yourdomain.duckdns.org`
3. Configure services through their web UIs
4. Set up Authelia users in `/opt/stacks/authelia/users_database.yml`
5. Configure Homepage widgets with API keys
6. Use VS Code with Copilot to ask questions and make changes
7. Review [proxying-external-hosts.md](proxying-external-hosts.md) to proxy your Raspberry Pi
## Troubleshooting
### Can't access services
1. **Check if service is running:**
```bash
docker ps
```
2. **Check service logs:**
```bash
docker compose -f docker-compose/file.yml logs service-name
```
3. **Verify network connectivity:**
```bash
ping YOUR_SERVER_IP
```
4. **Check firewall:**
```bash
sudo ufw status
```
### Permission errors
### Can't access services via HTTPS
Check Traefik logs:
```bash
# Fix config directory permissions
sudo chown -R $(id -u):$(id -g) config/
# Verify PUID/PGID in .env match your user
id -u # Should match PUID in .env
id -g # Should match PGID in .env
cd /opt/stacks/traefik
docker compose logs -f
```
### Port already in use
Verify DNS is resolving:
```bash
# Find what's using the port
sudo netstat -tlnp | grep PORT_NUMBER
# Stop the conflicting service or change the port in docker-compose
nslookup dockge.yourdomain.duckdns.org
```
### Out of disk space
Check certificate generation:
```bash
# Check disk usage
df -h
# Clean up Docker resources (removes unused images, containers, networks)
docker system prune -a
# To manage log size, configure log rotation in compose files:
# logging:
# driver: "json-file"
# options:
# max-size: "10m"
# max-file: "3"
docker exec traefik cat /acme.json
```
## Getting Help
### Authelia login not working
- **Documentation:** Check the `docs/` directory for comprehensive guides
- **AI Assistance:** Use GitHub Copilot in VS Code for real-time help
- **Community:** Search for service-specific help in respective communities
- **Issues:** Open an issue on GitHub for problems with this repository
Check Authelia logs:
```bash
cd /opt/stacks/authelia
docker compose logs -f
```
## Success Checklist
Verify password hash in `users_database.yml`
- [ ] Docker and Docker Compose installed
- [ ] Repository cloned
- [ ] `.env` file configured
- [ ] Networks created
- [ ] Config directories created
- [ ] Portainer running and accessible
- [ ] Infrastructure services deployed
- [ ] At least one service category deployed (media/monitoring/dev)
- [ ] VS Code with GitHub Copilot set up
- [ ] Backup strategy in place
- [ ] Firewall configured (if applicable)
- [ ] All services accessible and working
### Service not accessible
Congratulations! Your AI-powered homelab is now running! 🎉
1. Check Traefik dashboard: `https://traefik.yourdomain.duckdns.org`
2. Verify service has correct Traefik labels
3. Check service is on `traefik-network`
4. Review service logs
### Port forwarding
Ensure your router forwards ports 80 and 443 to your server IP.
## Security Checklist
- [ ] All passwords in `.env` are strong and unique
- [ ] Authelia 2FA is enabled for admin accounts
- [ ] `.env` file permissions are 600 (`chmod 600 .env`)
- [ ] acme.json permissions are 600
- [ ] Firewall is configured (only 80, 443 open to internet)
- [ ] Pi-hole is configured as your DNS server
- [ ] Watchtower is monitoring for updates
- [ ] Backrest/Duplicati configured for backups
## Congratulations!
Your AI-powered homelab is now running with:
- ✅ Automatic HTTPS via Traefik + Let's Encrypt
- ✅ SSO protection via Authelia
- ✅ 40+ services ready to deploy
- ✅ Dashboard with service widgets
- ✅ AI assistance via GitHub Copilot
- ✅ Centralized management via Dockge
Continue exploring with VS Code and Copilot to add more services, customize configurations, and proxy external devices!