kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update all documentation to reflect 60+ services, Dockge, Traefik, Authelia, and new features

Browse Source 
- Update README.md with comprehensive overview of all 40+ services
- Add Quick Setup guide with Dockge deployment steps
- List all service stacks by category (Infrastructure, Media, Home Automation, Productivity, etc.)
- Update AI assistant examples for Traefik, Authelia, VPN routing
- Add network architecture with Traefik routing patterns
- Update common operations for Dockge structure
- Add storage strategy (opt/stacks vs /mnt/)
- Update configuration management section
- Add proxying external hosts overview
- Update getting started checklist with all new steps

- Update docker-guidelines.md with Dockge and Traefik sections
- Add comprehensive Dockge structure explanation
- Add Traefik and Authelia integration patterns
- Document when to use SSO vs bypass
- Add VPN routing patterns with Gluetun
- Update storage strategy recommendations

- Completely rewrite getting-started.md with step-by-step Dockge deployment
- Add DuckDNS signup instructions
- Add detailed core infrastructure deployment (DuckDNS, Traefik, Authelia, Dockge)
- Add Homepage configuration steps
- Add API key configuration for widgets
- Add troubleshooting section
- Add security checklist

- Create docs/services-reference.md - comprehensive catalog of all 60+ services
- Organized by category with descriptions
- Access URLs and SSO requirements
- Storage recommendations
- Quick deployment guide
- Summary table by stack

All documentation now accurately reflects the complete homelab setup with Dockge, Traefik, Authelia, Gluetun, Homepage, and all 40+ services across 10 stack files.

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-01-12 01:16:12 +00:00
parent 63f39e51a4
commit 6fec6b501e
4 changed files with 1297 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files

182
docs/docker-guidelines.md
View File

@@ -2,40 +2,184 @@
## Overview
This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab using Dockge, Traefik, and Authelia. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
## Table of Contents
1. [Philosophy](#philosophy)
2. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run)
3. [Service Creation Guidelines](#service-creation-guidelines)
4. [Service Modification Guidelines](#service-modification-guidelines)
5. [Naming Conventions](#naming-conventions)
6. [Network Architecture](#network-architecture)
7. [Volume Management](#volume-management)
8. [Security Best Practices](#security-best-practices)
9. [Monitoring and Logging](#monitoring-and-logging)
10. [Troubleshooting](#troubleshooting)
2. [Dockge Structure](#dockge-structure)
3. [Traefik and Authelia Integration](#traefik-and-authelia-integration)
4. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run)
5. [Service Creation Guidelines](#service-creation-guidelines)
6. [Service Modification Guidelines](#service-modification-guidelines)
7. [Naming Conventions](#naming-conventions)
8. [Network Architecture](#network-architecture)
9. [Volume Management](#volume-management)
10. [Security Best Practices](#security-best-practices)
11. [Monitoring and Logging](#monitoring-and-logging)
12. [Troubleshooting](#troubleshooting)
## Philosophy
### Core Principles
1. **Infrastructure as Code**: All services must be defined in Docker Compose files
2. **Reproducibility**: Any service should be rebuildable from compose files
3. **Documentation**: Every non-obvious configuration must be commented
4. **Consistency**: Use the same patterns across all services
5. **Safety First**: Always test changes in isolation before deploying
1. **Dockge First**: Manage all stacks through Dockge in `/opt/stacks/`
2. **Infrastructure as Code**: All services defined in Docker Compose files
3. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
4. **Reproducibility**: Any service should be rebuildable from compose files
5. **Automatic HTTPS**: All services routed through Traefik with Let's Encrypt
6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps
7. **Documentation**: Every non-obvious configuration must be commented
8. **Consistency**: Use the same patterns across all services
9. **Safety First**: Always test changes in isolation before deploying
### The Stack Mindset
Think of your homelab as an interconnected stack where:
- Services depend on networks
- Networks connect services
- Volumes persist data
- Services depend on networks (especially traefik-network)
- Traefik routes all traffic with automatic SSL
- Authelia protects sensitive services
- VPN (Gluetun) secures downloads
- Changes ripple through the system
Always ask: "How does this change affect other services?"
Always ask: "How does this change affect other services and routing?"
## Dockge Structure
### Directory Organization
All stacks live in `/opt/stacks/stack-name/`:
```
/opt/stacks/
├── traefik/
│ ├── docker-compose.yml
│ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes
│ │ ├── routes.yml
│ │ └── external.yml # External host proxying
│ ├── acme.json # SSL certificates (chmod 600)
│ └── .env
├── authelia/
│ ├── docker-compose.yml
│ ├── configuration.yml # Authelia settings
│ ├── users_database.yml # User accounts
│ └── .env
├── media/
│ ├── docker-compose.yml
│ └── .env
└── ...
```
### Why Dockge?
- **Visual Management**: Web UI at `https://dockge.${DOMAIN}`
- **Direct File Editing**: Edit compose files in-place
- **Stack Organization**: Each service stack is independent
- **AI Compatible**: Files can be managed by AI
- **Git Integration**: Easy to version control
### Storage Strategy
**Small Data** (configs, DBs < 10GB): `/opt/stacks/stack-name/`
```yaml
volumes:
- /opt/stacks/sonarr/config:/config
```
**Large Data** (media, downloads, backups): `/mnt/`
```yaml
volumes:
- /mnt/media/movies:/movies
- /mnt/media/tv:/tv
- /mnt/downloads:/downloads
- /mnt/backups:/backups
```
AI will suggest `/mnt/` when data may exceed 50GB or grow continuously.
## Traefik and Authelia Integration
### Every Service Needs Traefik Labels
Standard pattern for all services:
```yaml
services:
myservice:
image: myimage:latest
container_name: myservice
networks:
- homelab-network
- traefik-network # Required for Traefik
labels:
# Enable Traefik
- "traefik.enable=true"
# Define routing rule
- "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
# Use websecure entrypoint (HTTPS)
- "traefik.http.routers.myservice.entrypoints=websecure"
# Enable Let's Encrypt
- "traefik.http.routers.myservice.tls.certresolver=letsencrypt"
# Add Authelia SSO (if needed)
- "traefik.http.routers.myservice.middlewares=authelia@docker"
# Specify port (if not default 80)
- "traefik.http.services.myservice.loadbalancer.server.port=8080"
```
### When to Use Authelia SSO
**Protect with Authelia**:
- Admin interfaces (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure tools (Portainer, Dockge, Grafana)
- Personal data (Nextcloud, Mealie, wikis)
- Development tools (code-server, GitLab)
- Monitoring dashboards
**Bypass Authelia**:
- Media servers (Plex, Jellyfin) - need app access
- Request services (Jellyseerr) - family-friendly access
- Public services (WordPress, status pages)
- Services with their own auth (Home Assistant)
Configure bypasses in `/opt/stacks/authelia/configuration.yml`:
```yaml
access_control:
rules:
- domain: jellyfin.yourdomain.duckdns.org
policy: bypass
- domain: plex.yourdomain.duckdns.org
policy: bypass
```
### Routing Through VPN (Gluetun)
For services that need VPN (downloads):
```yaml
services:
mydownloader:
image: downloader:latest
container_name: mydownloader
network_mode: "service:gluetun" # Route through VPN
depends_on:
- gluetun
```
Expose ports through Gluetun's compose file:
```yaml
# In gluetun.yml
gluetun:
ports:
- "8080:8080" # mydownloader web UI
```
## Docker Compose vs Docker Run

603
docs/getting-started.md
View File

@@ -1,6 +1,6 @@
# Getting Started Guide
This guide will walk you through setting up your AI-powered homelab from scratch.
This guide will walk you through setting up your AI-powered homelab with Dockge, Traefik, Authelia, and 40+ services from scratch.
## Prerequisites
@@ -11,8 +11,10 @@ Before you begin, ensure you have:
- [ ] Docker Compose V2 installed
- [ ] Git installed
- [ ] At least 8GB RAM (16GB+ recommended)
- [ ] Sufficient disk space (100GB+ recommended)
- [ ] Static IP address for your server
- [ ] Sufficient disk space (100GB+ system, 1TB+ for media recommended)
- [ ] Static IP address for your server (or DHCP reservation)
- [ ] DuckDNS account (free) with a domain
- [ ] Surfshark VPN account (optional, for VPN features)
- [ ] VS Code with GitHub Copilot extension (for AI assistance)
## Step 1: Verify Docker Installation
@@ -45,7 +47,15 @@ git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
```
## Step 3: Configure Environment Variables
## Step 3: Sign Up for DuckDNS
1. Go to https://www.duckdns.org/
2. Sign in with your preferred method
3. Create a domain (e.g., `myhomelab`)
4. Copy your token - you'll need it for `.env`
5. Your domain will be: `myhomelab.duckdns.org`
## Step 4: Configure Environment Variables
```bash
# Copy the example environment file
@@ -59,7 +69,7 @@ id -g # This is your PGID
nano .env
```
**Update these values in `.env`:**
**Critical values to update in `.env`:**
```bash
# Your user/group IDs
PUID=1000 # Replace with your user ID
@@ -71,442 +81,333 @@ TZ=America/New_York
# Your server's IP address
SERVER_IP=192.168.1.100 # Replace with your actual IP
# Directory paths
USERDIR=/home/yourusername/homelab # Update username
MEDIADIR=/mnt/media # Update if different
DOWNLOADDIR=/mnt/downloads # Update if different
# DuckDNS Configuration
DOMAIN=myhomelab.duckdns.org # Your DuckDNS domain
DUCKDNS_TOKEN=your-duckdns-token-here
DUCKDNS_SUBDOMAINS=myhomelab # Without .duckdns.org
# Set secure passwords for services
GRAFANA_ADMIN_PASSWORD=your-secure-password-here
CODE_SERVER_PASSWORD=your-secure-password-here
POSTGRES_PASSWORD=your-secure-password-here
PGADMIN_PASSWORD=your-secure-password-here
JUPYTER_TOKEN=your-secure-token-here
PIHOLE_PASSWORD=your-secure-password-here
# Let's Encrypt Email
ACME_EMAIL=your-email@example.com
# Authelia Secrets (generate with: openssl rand -hex 64)
AUTHELIA_JWT_SECRET=$(openssl rand -hex 64)
AUTHELIA_SESSION_SECRET=$(openssl rand -hex 64)
AUTHELIA_STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 64)
# Surfshark VPN (if using)
SURFSHARK_PRIVATE_KEY=your-wireguard-private-key
SURFSHARK_ADDRESSES=10.14.0.2/16
# Set secure passwords for all services
PIHOLE_PASSWORD=your-secure-password
GRAFANA_ADMIN_PASSWORD=your-secure-password
CODE_SERVER_PASSWORD=your-secure-password
# ... (see .env.example for complete list)
```
**Save and exit** (Ctrl+X, Y, Enter in nano)
## Step 4: Create Docker Networks
## Step 5: Create Dockge Directory Structure
```bash
# Create the main homelab network
docker network create homelab-network
# Create main stacks directory
sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
# Create additional networks for better security
# Create mount points for large data (adjust as needed)
sudo mkdir -p /mnt/media/{movies,tv,music,books,photos}
sudo mkdir -p /mnt/downloads/{complete,incomplete}
sudo mkdir -p /mnt/backups
sudo chown -R $USER:$USER /mnt/media /mnt/downloads /mnt/backups
```
## Step 6: Create Docker Networks
```bash
# Create required external networks
docker network create traefik-network
docker network create homelab-network
docker network create media-network
docker network create monitoring-network
docker network create database-network
docker network create dockerproxy-network
# Verify networks were created
docker network ls
docker network ls | grep -E "traefik|homelab|media|dockerproxy"
```
## Step 5: Create Configuration Directories
## Step 7: Deploy Core Infrastructure (IN ORDER)
### 7.1 DuckDNS (Dynamic DNS)
```bash
# Create the main config directory
mkdir -p config
# Create stack directory
mkdir -p /opt/stacks/duckdns
# Create config directories for services you plan to use
mkdir -p config/{nginx-proxy-manager,pihole,portainer}
mkdir -p config/{plex,sonarr,radarr,prowlarr,qbittorrent,jellyfin}
mkdir -p config/{prometheus,grafana,loki,promtail}
mkdir -p config/{code-server,postgres,redis}
# Copy compose file
cp ~/AI-Homelab/docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
# Set proper permissions
sudo chown -R $(id -u):$(id -g) config/
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/duckdns/.env
# Deploy
cd /opt/stacks/duckdns
docker compose up -d
# Verify it's working
docker compose logs -f
# Should see: "Your IP was updated to X.X.X.X"
```
## Step 6: Copy Configuration Templates (Optional)
For services that need config files:
### 7.2 Traefik (Reverse Proxy with SSL)
```bash
# Prometheus
mkdir -p config/prometheus
cp config-templates/prometheus/prometheus.yml config/prometheus/
# Create stack directory with dynamic configs
mkdir -p /opt/stacks/traefik/dynamic
# Loki
mkdir -p config/loki
cp config-templates/loki/loki-config.yml config/loki/
# Copy compose file
cp ~/AI-Homelab/docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
# Promtail
mkdir -p config/promtail
cp config-templates/promtail/promtail-config.yml config/promtail/
# Copy configuration templates
cp ~/AI-Homelab/config-templates/traefik/traefik.yml /opt/stacks/traefik/
cp ~/AI-Homelab/config-templates/traefik/dynamic/*.yml /opt/stacks/traefik/dynamic/
# Redis
mkdir -p config/redis
cp config-templates/redis/redis.conf config/redis/
# Create acme.json for SSL certificates
touch /opt/stacks/traefik/acme.json
chmod 600 /opt/stacks/traefik/acme.json
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/traefik/.env
# Deploy
cd /opt/stacks/traefik
docker compose up -d
# Check logs
docker compose logs -f
# Should see Traefik starting and certificate resolver configured
```
## Step 7: Start Your First Service (Portainer)
Portainer provides a web UI for managing Docker containers. It's a great first service to deploy.
### 7.3 Authelia (SSO Authentication)
```bash
# Start Portainer
docker compose -f docker-compose/infrastructure.yml up -d portainer
# Create stack directory
mkdir -p /opt/stacks/authelia
# Check if it's running
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
# View logs
docker compose -f docker-compose/infrastructure.yml logs -f portainer
# Copy configuration templates
cp ~/AI-Homelab/config-templates/authelia/*.yml /opt/stacks/authelia/
# Generate password hash for users_database.yml
docker run --rm authelia/authelia:latest authelia crypto hash generate argon2 --password 'yourpassword'
# Copy the hash and edit users_database.yml
# Edit users_database.yml
cd /opt/stacks/authelia
nano users_database.yml
# Replace the password hash with your generated one
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/authelia/.env
# Deploy
docker compose up -d
# Check logs
docker compose logs -f
# Test login at https://auth.yourdomain.duckdns.org
```
**Access Portainer:**
1. Open your browser
2. Navigate to `http://YOUR_SERVER_IP:9000`
3. Create an admin account on first login
4. Select "Get Started" and choose local Docker environment
## Step 8: Deploy Infrastructure Services
### 7.4 Infrastructure Services (Dockge)
```bash
# Start all infrastructure services
docker compose -f docker-compose/infrastructure.yml up -d
# Create stack directory
mkdir -p /opt/stacks/infrastructure
# Check status
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
# Services now running:
# - Nginx Proxy Manager: http://YOUR_SERVER_IP:81
# - Pi-hole: http://YOUR_SERVER_IP:8080/admin
# - Portainer: http://YOUR_SERVER_IP:9000
# - Watchtower: (runs in background, no UI)
# Create necessary subdirectories
mkdir -p /opt/dockge/data
mkdir -p /opt/stacks/pihole/{etc-pihole,etc-dnsmasq.d}
mkdir -p /opt/stacks/glances/config
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/infrastructure/.env
# Deploy Dockge first
cd /opt/stacks/infrastructure
docker compose up -d dockge
# Access Dockge at https://dockge.yourdomain.duckdns.org
# Deploy remaining infrastructure services
docker compose up -d
```
### Configure Nginx Proxy Manager (Optional)
## Step 8: Deploy Additional Stacks
1. Access: `http://YOUR_SERVER_IP:81`
2. Login with default credentials:
- Email: `admin@example.com`
- Password: `changeme`
3. Change password immediately
4. Add proxy hosts for your services
Now use Dockge UI at `https://dockge.yourdomain.duckdns.org` to deploy additional stacks, or continue with command line:
### Configure Pi-hole (Optional)
1. Access: `http://YOUR_SERVER_IP:8080/admin`
2. Login with password from `.env` (PIHOLE_PASSWORD)
3. Configure DNS settings
4. Update your router to use Pi-hole as DNS server
## Step 9: Deploy Media Services (Optional)
If you want a media server setup:
### 8.1 Gluetun + qBittorrent (VPN)
```bash
# Start media services
docker compose -f docker-compose/media.yml up -d
mkdir -p /opt/stacks/gluetun
cp ~/AI-Homelab/docker-compose/gluetun.yml /opt/stacks/gluetun/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/gluetun/.env
# Check status
docker compose -f docker-compose/media.yml ps
cd /opt/stacks/gluetun
docker compose up -d
# Services now running:
# - Plex: http://YOUR_SERVER_IP:32400/web
# - Jellyfin: http://YOUR_SERVER_IP:8096
# - Sonarr: http://YOUR_SERVER_IP:8989
# - Radarr: http://YOUR_SERVER_IP:7878
# - Prowlarr: http://YOUR_SERVER_IP:9696
# - qBittorrent: http://YOUR_SERVER_IP:8081
# Test VPN
docker exec gluetun curl ifconfig.me
# Should show VPN IP
```
### Initial Media Service Setup
**Plex:**
1. Access: `http://YOUR_SERVER_IP:32400/web`
2. Sign in with your Plex account
3. Add your media libraries
**Sonarr/Radarr:**
1. Access the web UI
2. Go to Settings → Profiles → Quality
3. Configure your quality preferences
4. Add Prowlarr as an indexer
5. Add qBittorrent as a download client
## Step 10: Deploy Monitoring Services (Optional)
For system and service monitoring:
### 8.2 Homepage Dashboard
```bash
# Start monitoring services
docker compose -f docker-compose/monitoring.yml up -d
mkdir -p /opt/stacks/homepage/config
cp ~/AI-Homelab/docker-compose/dashboards.yml /opt/stacks/homepage/docker-compose.yml
cp ~/AI-Homelab/config-templates/homepage/* /opt/stacks/homepage/config/
cp ~/AI-Homelab/.env /opt/stacks/homepage/.env
# Check status
docker compose -f docker-compose/monitoring.yml ps
cd /opt/stacks/homepage
docker compose up -d homepage
# Services now running:
# - Prometheus: http://YOUR_SERVER_IP:9090
# - Grafana: http://YOUR_SERVER_IP:3000
# - Node Exporter: http://YOUR_SERVER_IP:9100
# - cAdvisor: http://YOUR_SERVER_IP:8082
# - Uptime Kuma: http://YOUR_SERVER_IP:3001
# Access at https://home.yourdomain.duckdns.org
```
### Configure Grafana
1. Access: `http://YOUR_SERVER_IP:3000`
2. Login with credentials from `.env`:
- Username: `admin`
- Password: `GRAFANA_ADMIN_PASSWORD` from .env
3. Add Prometheus as a data source:
- URL: `http://prometheus:9090`
4. Import dashboards:
- Dashboard ID 1860 for Node Exporter
- Dashboard ID 893 for Docker metrics
### Configure Uptime Kuma
1. Access: `http://YOUR_SERVER_IP:3001`
2. Create an account on first login
3. Add monitors for your services
## Step 11: Deploy Development Services (Optional)
If you need development tools:
### 8.3 Media Stack
```bash
# Start development services
docker compose -f docker-compose/development.yml up -d
mkdir -p /opt/stacks/media
cp ~/AI-Homelab/docker-compose/media.yml /opt/stacks/media/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/media/.env
# Check status
docker compose -f docker-compose/development.yml ps
# Services now running:
# - Code Server: http://YOUR_SERVER_IP:8443
# - PostgreSQL: localhost:5432
# - Redis: localhost:6379
# - pgAdmin: http://YOUR_SERVER_IP:5050
# - Jupyter Lab: http://YOUR_SERVER_IP:8888
# - Node-RED: http://YOUR_SERVER_IP:1880
cd /opt/stacks/media
docker compose up -d
```
## Step 12: Set Up VS Code with GitHub Copilot
### 8.4 Additional Stacks
1. **Install VS Code** on your local machine (if not already installed)
Deploy as needed:
- `media-extended.yml` → `/opt/stacks/media-extended/`
- `homeassistant.yml` → `/opt/stacks/homeassistant/`
- `productivity.yml` → `/opt/stacks/productivity/`
- `utilities.yml` → `/opt/stacks/utilities/`
- `monitoring.yml` → `/opt/stacks/monitoring/`
- `development.yml` → `/opt/stacks/development/`
2. **Install GitHub Copilot extension:**
- Open VS Code
- Go to Extensions (Ctrl+Shift+X)
- Search for "GitHub Copilot"
- Click Install
- Sign in with your GitHub account
## Step 9: Configure Homepage Widgets
3. **Clone your repository in VS Code:**
```bash
# On your local machine
# Note: Replace 'kelinfoxy' with your username if you forked this repository
git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
code .
```
4. **Start using AI assistance:**
- Open Copilot Chat (Ctrl+Shift+I or click the chat icon)
- The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md`
- Ask questions like:
- "Help me add Home Assistant to my homelab"
- "Create a backup script for my Docker volumes"
- "How do I configure GPU support for Plex?"
## Step 13: Verify Everything is Running
Get API keys from each service and add to Homepage config:
```bash
# Check all running containers
docker ps
cd /opt/stacks/homepage/config
nano services.yaml
# Check container health
docker ps --format "table {{.Names}}\t{{.Status}}"
# Get API keys:
# - Sonarr/Radarr/etc: Settings → General → API Key
# - Plex: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/
# - Jellyfin: Dashboard → API Keys
# View resource usage
docker stats --no-stream
# Add to .env:
nano /opt/stacks/homepage/.env
# HOMEPAGE_VAR_SONARR_KEY=...
# HOMEPAGE_VAR_RADARR_KEY=...
# etc.
# Check disk usage
docker system df
# Restart Homepage
cd /opt/stacks/homepage
docker compose restart
```
## Step 14: Set Up Backups
Create a backup script:
## Step 10: Install VS Code and GitHub Copilot
```bash
# Create a backup directory
mkdir -p ~/backups
# Install VS Code (if not already installed)
# Download from https://code.visualstudio.com/
# Create a simple backup script
cat > ~/backup-homelab.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=~/backups
DATE=$(date +%Y%m%d)
# Install GitHub Copilot extension
# In VS Code: Extensions → Search "GitHub Copilot" → Install
# Backup config directories
tar czf $BACKUP_DIR/config-$DATE.tar.gz ~/AI-Homelab/config/
# Open the repository
code ~/AI-Homelab
# Backup .env file
cp ~/AI-Homelab/.env $BACKUP_DIR/.env-$DATE
# Backup Docker volumes (example for Portainer)
docker run --rm \
-v portainer-data:/data \
-v $BACKUP_DIR:/backup \
busybox tar czf /backup/portainer-data-$DATE.tar.gz /data
echo "Backup completed: $DATE"
EOF
# Make it executable
chmod +x ~/backup-homelab.sh
# Test the backup
~/backup-homelab.sh
```
Set up a cron job for automated backups:
```bash
# Open crontab
crontab -e
# Add this line to run backup daily at 2 AM
0 2 * * * /home/yourusername/backup-homelab.sh
```
## Step 15: Configure Firewall (Optional but Recommended)
If using UFW:
```bash
# Allow SSH (if not already allowed)
sudo ufw allow 22/tcp
# Allow web traffic (if exposing services to internet)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Allow specific services from local network only
# Replace 192.168.1.0/24 with your network
sudo ufw allow from 192.168.1.0/24 to any port 9000 proto tcp # Portainer
sudo ufw allow from 192.168.1.0/24 to any port 81 proto tcp # Nginx Proxy Manager
sudo ufw allow from 192.168.1.0/24 to any port 3000 proto tcp # Grafana
# Enable firewall
sudo ufw enable
# Start using AI assistance!
```
## Next Steps
Now that your homelab is running:
1. **Explore Services:**
- Access each service's web UI
- Configure settings as needed
- Set up integrations between services
2. **Add More Services:**
- Ask GitHub Copilot for help adding new services
- Follow the patterns in existing compose files
- Check [awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) for ideas
3. **Optimize:**
- Review logs for errors
- Adjust resource limits
- Set up proper monitoring and alerts
4. **Secure:**
- Change all default passwords
- Set up SSL certificates (use Nginx Proxy Manager)
- Enable 2FA where available
- Keep services updated
5. **Learn:**
- Read the [Docker Guidelines](docker-guidelines.md)
- Experiment with new services
- Use AI assistance to understand and modify configurations
1. Explore Dockge at `https://dockge.yourdomain.duckdns.org`
2. Check Homepage dashboard at `https://home.yourdomain.duckdns.org`
3. Configure services through their web UIs
4. Set up Authelia users in `/opt/stacks/authelia/users_database.yml`
5. Configure Homepage widgets with API keys
6. Use VS Code with Copilot to ask questions and make changes
7. Review [proxying-external-hosts.md](proxying-external-hosts.md) to proxy your Raspberry Pi
## Troubleshooting
### Can't access services
1. **Check if service is running:**
```bash
docker ps
```
2. **Check service logs:**
```bash
docker compose -f docker-compose/file.yml logs service-name
```
3. **Verify network connectivity:**
```bash
ping YOUR_SERVER_IP
```
4. **Check firewall:**
```bash
sudo ufw status
```
### Permission errors
### Can't access services via HTTPS
Check Traefik logs:
```bash
# Fix config directory permissions
sudo chown -R $(id -u):$(id -g) config/
# Verify PUID/PGID in .env match your user
id -u # Should match PUID in .env
id -g # Should match PGID in .env
cd /opt/stacks/traefik
docker compose logs -f
```
### Port already in use
Verify DNS is resolving:
```bash
# Find what's using the port
sudo netstat -tlnp | grep PORT_NUMBER
# Stop the conflicting service or change the port in docker-compose
nslookup dockge.yourdomain.duckdns.org
```
### Out of disk space
Check certificate generation:
```bash
# Check disk usage
df -h
# Clean up Docker resources (removes unused images, containers, networks)
docker system prune -a
# To manage log size, configure log rotation in compose files:
# logging:
# driver: "json-file"
# options:
# max-size: "10m"
# max-file: "3"
docker exec traefik cat /acme.json
```
## Getting Help
### Authelia login not working
- **Documentation:** Check the `docs/` directory for comprehensive guides
- **AI Assistance:** Use GitHub Copilot in VS Code for real-time help
- **Community:** Search for service-specific help in respective communities
- **Issues:** Open an issue on GitHub for problems with this repository
Check Authelia logs:
```bash
cd /opt/stacks/authelia
docker compose logs -f
```
## Success Checklist
Verify password hash in `users_database.yml`
- [ ] Docker and Docker Compose installed
- [ ] Repository cloned
- [ ] `.env` file configured
- [ ] Networks created
- [ ] Config directories created
- [ ] Portainer running and accessible
- [ ] Infrastructure services deployed
- [ ] At least one service category deployed (media/monitoring/dev)
- [ ] VS Code with GitHub Copilot set up
- [ ] Backup strategy in place
- [ ] Firewall configured (if applicable)
- [ ] All services accessible and working
### Service not accessible
Congratulations! Your AI-powered homelab is now running! 🎉
1. Check Traefik dashboard: `https://traefik.yourdomain.duckdns.org`
2. Verify service has correct Traefik labels
3. Check service is on `traefik-network`
4. Review service logs
### Port forwarding
Ensure your router forwards ports 80 and 443 to your server IP.
## Security Checklist
- [ ] All passwords in `.env` are strong and unique
- [ ] Authelia 2FA is enabled for admin accounts
- [ ] `.env` file permissions are 600 (`chmod 600 .env`)
- [ ] acme.json permissions are 600
- [ ] Firewall is configured (only 80, 443 open to internet)
- [ ] Pi-hole is configured as your DNS server
- [ ] Watchtower is monitoring for updates
- [ ] Backrest/Duplicati configured for backups
## Congratulations!
Your AI-powered homelab is now running with:
- ✅ Automatic HTTPS via Traefik + Let's Encrypt
- ✅ SSO protection via Authelia
- ✅ 40+ services ready to deploy
- ✅ Dashboard with service widgets
- ✅ AI assistance via GitHub Copilot
- ✅ Centralized management via Dockge
Continue exploring with VS Code and Copilot to add more services, customize configurations, and proxy external devices!

462
docs/services-reference.md Normal file
View File

@@ -0,0 +1,462 @@
# Complete Services Reference
This document lists all 40+ pre-configured services available in the AI-Homelab repository, organized by category.
## Core Infrastructure (4 services)
### Required - Deploy First
1. **DuckDNS** (`duckdns.yml`)
- Dynamic DNS updater
- Updates your public IP automatically
- Integrates with Let's Encrypt for SSL
- No web UI - runs silently
- Stack: `/opt/stacks/duckdns/`
2. **Traefik** (`traefik.yml`)
- Reverse proxy with automatic SSL
- HTTP to HTTPS redirect
- File-based and Docker label routing
- Dashboard: `https://traefik.${DOMAIN}`
- Stack: `/opt/stacks/traefik/`
3. **Authelia** (`authelia.yml`)
- Single Sign-On (SSO) authentication
- TOTP 2FA support
- File-based or LDAP user database
- Smart bypass rules for media apps
- Login: `https://auth.${DOMAIN}`
- Stack: `/opt/stacks/authelia/`
4. **Gluetun** (`gluetun.yml`)
- VPN client (Surfshark WireGuard)
- Includes qBittorrent
- Control panel: `http://gluetun:8000`
- qBittorrent: `https://qbit.${DOMAIN}`
- Stack: `/opt/stacks/gluetun/`
## Infrastructure Tools (7 services)
From `infrastructure.yml` - Stack: `/opt/stacks/infrastructure/`
5. **Dockge** (PRIMARY management tool)
- Docker Compose stack manager
- Web UI for managing /opt/stacks/
- Direct compose file editing
- Access: `https://dockge.${DOMAIN}`
- SSO: Yes
6. **Portainer** (Secondary)
- Docker container management UI
- Access: `https://portainer.${DOMAIN}`
- SSO: Yes
7. **Pi-hole**
- Network-wide ad blocking
- DNS server
- Access: `https://pihole.${DOMAIN}`
- SSO: Yes
8. **Watchtower**
- Automatic container updates
- Runs 4 AM daily
- No web UI
9. **Dozzle**
- Real-time Docker log viewer
- Access: `https://dozzle.${DOMAIN}`
- SSO: Yes
10. **Glances**
- System and Docker monitoring
- Access: `https://glances.${DOMAIN}`
- SSO: Yes
11. **Docker Proxy**
- Secure Docker socket access
- Backend service
- No web UI
## Dashboards (2 services)
From `dashboards.yml` - Stack: `/opt/stacks/dashboards/`
12. **Homepage** (AI-configurable)
- Application dashboard with Docker integration
- Service widgets for 15+ services
- 11 organized categories
- Access: `https://home.${DOMAIN}`
- SSO: No (landing page)
13. **Homarr**
- Modern alternative dashboard
- Access: `https://homarr.${DOMAIN}`
- SSO: No
## Media Services (6 services)
From `media.yml` - Stack: `/opt/stacks/media/`
14. **Plex**
- Media streaming server
- Hardware transcoding support
- Access: `https://plex.${DOMAIN}`
- SSO: No (app access)
15. **Jellyfin**
- Open-source media server
- Hardware transcoding support
- Access: `https://jellyfin.${DOMAIN}`
- SSO: No (app access)
16. **Sonarr**
- TV show automation
- Access: `https://sonarr.${DOMAIN}`
- SSO: Yes
17. **Radarr**
- Movie automation
- Access: `https://radarr.${DOMAIN}`
- SSO: Yes
18. **Prowlarr**
- Indexer manager
- Integrates with Sonarr, Radarr, etc.
- Access: `https://prowlarr.${DOMAIN}`
- SSO: Yes
19. **qBittorrent**
- Torrent client (routes through Gluetun VPN)
- See gluetun.yml
## Extended Media (10 services)
From `media-extended.yml` - Stack: `/opt/stacks/media-extended/`
20. **Readarr**
- Ebook and audiobook management
- Access: `https://readarr.${DOMAIN}`
- SSO: Yes
21. **Lidarr**
- Music collection manager
- Access: `https://lidarr.${DOMAIN}`
- SSO: Yes
22. **Lazy Librarian**
- Book download automation
- Access: `https://lazylibrarian.${DOMAIN}`
- SSO: Yes
23. **Mylar3**
- Comic book collection manager
- Access: `https://mylar.${DOMAIN}`
- SSO: Yes
24. **Calibre-Web**
- Ebook reader and library management
- Access: `https://calibre.${DOMAIN}`
- SSO: Yes
25. **Jellyseerr**
- Media request management
- Integrates with Plex/Jellyfin
- Access: `https://jellyseerr.${DOMAIN}`
- SSO: No (family access)
26. **FlareSolverr**
- Cloudflare bypass for indexers
- Used by Prowlarr
- No web UI
27. **Tdarr Server**
- Distributed transcoding server
- Access: `https://tdarr.${DOMAIN}`
- SSO: Yes
28. **Tdarr Node**
- Transcoding worker
- No web UI
29. **Unmanic**
- Library optimization and transcoding
- Access: `https://unmanic.${DOMAIN}`
- SSO: Yes
## Home Automation (7 services)
From `homeassistant.yml` - Stack: `/opt/stacks/homeassistant/`
30. **Home Assistant**
- Home automation platform
- Uses host networking
- Access: `https://ha.${DOMAIN}` (or via proxying external host)
- SSO: No (has own auth)
31. **ESPHome**
- ESP8266/ESP32 firmware manager
- Access: `https://esphome.${DOMAIN}`
- SSO: Yes
32. **TasmoAdmin**
- Tasmota device management
- Access: `https://tasmoadmin.${DOMAIN}`
- SSO: Yes
33. **Node-RED**
- Flow-based automation programming
- Access: `https://nodered.${DOMAIN}`
- SSO: Yes
34. **Mosquitto**
- MQTT message broker
- Ports: 1883, 9001
- No web UI
35. **Zigbee2MQTT**
- Zigbee to MQTT bridge
- Access: `https://zigbee2mqtt.${DOMAIN}`
- SSO: Yes
36. **MotionEye**
- Video surveillance system
- Access: `https://motioneye.${DOMAIN}`
- SSO: Yes
## Productivity (8 services + 6 databases)
From `productivity.yml` - Stack: `/opt/stacks/productivity/`
37. **Nextcloud**
- File sync and collaboration platform
- Access: `https://nextcloud.${DOMAIN}`
- SSO: Yes
- Database: nextcloud-db (MariaDB)
38. **Mealie**
- Recipe manager and meal planner
- Access: `https://mealie.${DOMAIN}`
- SSO: No (family access)
39. **WordPress**
- Blog and website platform
- Access: `https://blog.${DOMAIN}`
- SSO: No (public blog)
- Database: wordpress-db (MariaDB)
40. **Gitea**
- Self-hosted Git service
- Access: `https://git.${DOMAIN}`
- SSO: Yes
- Database: gitea-db (PostgreSQL)
41. **DokuWiki**
- File-based wiki (no database)
- Access: `https://wiki.${DOMAIN}`
- SSO: Yes
42. **BookStack**
- Documentation platform
- Access: `https://docs.${DOMAIN}`
- SSO: Yes
- Database: bookstack-db (MariaDB)
43. **MediaWiki**
- Wiki platform
- Access: `https://mediawiki.${DOMAIN}`
- SSO: Yes
- Database: mediawiki-db (MariaDB)
## Utilities (7 services)
From `utilities.yml` - Stack: `/opt/stacks/utilities/`
44. **Backrest**
- Backup management with restic
- Access: `https://backrest.${DOMAIN}`
- SSO: Yes
45. **Duplicati**
- Backup software with encryption
- Access: `https://duplicati.${DOMAIN}`
- SSO: Yes
46. **Uptime Kuma**
- Uptime monitoring and status page
- Access: `https://status.${DOMAIN}`
- SSO: No (public status)
47. **Code Server**
- VS Code in browser
- Full stack access
- Access: `https://code.${DOMAIN}`
- SSO: Yes
48. **Form.io**
- Form builder platform
- Access: `https://forms.${DOMAIN}`
- SSO: Yes
- Database: formio-mongo (MongoDB)
49. **Authelia-Redis**
- Session storage for Authelia
- No web UI
## Monitoring (7 services)
From `monitoring.yml` - Stack: `/opt/stacks/monitoring/`
50. **Prometheus**
- Metrics collection
- Access: `https://prometheus.${DOMAIN}`
- SSO: Yes
51. **Grafana**
- Metrics visualization
- Access: `https://grafana.${DOMAIN}`
- SSO: Yes
52. **Loki**
- Log aggregation
- No web UI (accessed via Grafana)
53. **Promtail**
- Log shipping to Loki
- No web UI
54. **Node Exporter**
- Host metrics exporter
- No web UI
55. **cAdvisor**
- Container metrics
- Access: Port 8080 (internal)
## Development (6 services)
From `development.yml` - Stack: `/opt/stacks/development/`
56. **GitLab CE**
- Git repository with CI/CD
- Access: `https://gitlab.${DOMAIN}`
- SSO: Yes
57. **PostgreSQL**
- SQL database
- Port: 5432
- No web UI
58. **Redis**
- In-memory data store
- Port: 6379
- No web UI
59. **pgAdmin**
- PostgreSQL management UI
- Access: `https://pgadmin.${DOMAIN}`
- SSO: Yes
60. **Jupyter Lab**
- Interactive notebooks
- Access: `https://jupyter.${DOMAIN}`
- SSO: Yes
## Summary by Stack
| Stack | File | Services Count | Description |
|-------|------|----------------|-------------|
| Core Infrastructure | Multiple files | 4 | Traefik, Authelia, DuckDNS, Gluetun |
| Infrastructure | infrastructure.yml | 7 | Dockge, Portainer, Pi-hole, etc. |
| Dashboards | dashboards.yml | 2 | Homepage, Homarr |
| Media | media.yml | 6 | Plex, Jellyfin, *arr apps |
| Media Extended | media-extended.yml | 10 | Books, comics, music, transcoding |
| Home Automation | homeassistant.yml | 7 | HA, ESPHome, Node-RED, MQTT, etc. |
| Productivity | productivity.yml | 14 | Nextcloud, wikis, Git (includes DBs) |
| Utilities | utilities.yml | 7 | Backups, monitoring, Code Server |
| Monitoring | monitoring.yml | 7 | Prometheus, Grafana, Loki |
| Development | development.yml | 6 | GitLab, databases, Jupyter |
**Total: 60+ services (including databases)**
## Access Patterns
### With SSO (Authelia Required)
- Admin tools (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure management (Dockge, Portainer, Grafana)
- Development tools (GitLab, Code Server, pgAdmin)
- Personal data (Nextcloud, wikis, BookStack)
### Without SSO (Direct Access)
- Media streaming (Plex, Jellyfin) - for app access
- Public services (WordPress, Uptime Kuma, Homepage)
- Services with own auth (Home Assistant)
- Family-friendly (Mealie, Jellyseerr)
### Via VPN (Gluetun)
- qBittorrent
- Other download clients (add with network_mode: "service:gluetun")
## Storage Recommendations
### Keep on System Drive (/opt/stacks/)
- All configuration files
- Small databases (< 10GB)
- Application data
### Move to Separate Drive (/mnt/)
- Media files (movies, TV, music, photos) → /mnt/media/
- Downloads → /mnt/downloads/
- Backups → /mnt/backups/
- Surveillance footage → /mnt/surveillance/
- Large databases → /mnt/databases/
- Transcoding cache → /mnt/transcode-cache/
## Quick Deployment Guide
1. **Core (Required)**
```bash
# Deploy in this order:
/opt/stacks/duckdns/
/opt/stacks/traefik/
/opt/stacks/authelia/
/opt/stacks/infrastructure/ (dockge)
```
2. **VPN + Downloads**
```bash
/opt/stacks/gluetun/
```
3. **Dashboard**
```bash
/opt/stacks/homepage/
```
4. **Choose Your Stacks**
- Media: `/opt/stacks/media/` + `/opt/stacks/media-extended/`
- Home Automation: `/opt/stacks/homeassistant/`
- Productivity: `/opt/stacks/productivity/`
- Monitoring: `/opt/stacks/monitoring/`
- Development: `/opt/stacks/development/`
- Utilities: `/opt/stacks/utilities/`
## Configuration Files
All configuration templates available in `config-templates/`:
- `traefik/` - Static and dynamic configs
- `authelia/` - Config and user database
- `homepage/` - Dashboard services and widgets
- `prometheus/` - Scrape configurations
- `loki/` - Log aggregation config
- And more...
## Next Steps
1. Deploy core infrastructure
2. Configure Homepage with API keys
3. Set up Authelia users
4. Deploy service stacks as needed
5. Use VS Code + Copilot for AI assistance
6. Proxy external hosts via Traefik (see docs/proxying-external-hosts.md)
For detailed deployment instructions, see [docs/getting-started.md](../docs/getting-started.md)