kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update all documentation to reflect 60+ services, Dockge, Traefik, Authelia, and new features

Browse Source 
- Update README.md with comprehensive overview of all 40+ services
- Add Quick Setup guide with Dockge deployment steps
- List all service stacks by category (Infrastructure, Media, Home Automation, Productivity, etc.)
- Update AI assistant examples for Traefik, Authelia, VPN routing
- Add network architecture with Traefik routing patterns
- Update common operations for Dockge structure
- Add storage strategy (opt/stacks vs /mnt/)
- Update configuration management section
- Add proxying external hosts overview
- Update getting started checklist with all new steps

- Update docker-guidelines.md with Dockge and Traefik sections
- Add comprehensive Dockge structure explanation
- Add Traefik and Authelia integration patterns
- Document when to use SSO vs bypass
- Add VPN routing patterns with Gluetun
- Update storage strategy recommendations

- Completely rewrite getting-started.md with step-by-step Dockge deployment
- Add DuckDNS signup instructions
- Add detailed core infrastructure deployment (DuckDNS, Traefik, Authelia, Dockge)
- Add Homepage configuration steps
- Add API key configuration for widgets
- Add troubleshooting section
- Add security checklist

- Create docs/services-reference.md - comprehensive catalog of all 60+ services
- Organized by category with descriptions
- Access URLs and SSO requirements
- Storage recommendations
- Quick deployment guide
- Summary table by stack

All documentation now accurately reflects the complete homelab setup with Dockge, Traefik, Authelia, Gluetun, Homepage, and all 40+ services across 10 stack files.

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-01-12 01:16:12 +00:00
parent 63f39e51a4
commit 6fec6b501e
4 changed files with 1297 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files

557
README.md
View File

@@ -4,16 +4,23 @@ AI-Powered Homelab Administration with GitHub Copilot
## Overview
This repository provides a structured approach to managing a homelab infrastructure using Docker Compose, with integrated AI assistance through GitHub Copilot. The AI assistant is specifically trained to help you create, modify, and manage Docker services while maintaining consistency across your entire server stack.
This repository provides a comprehensive, production-ready homelab infrastructure using Docker Compose with Dockge, featuring 40+ pre-configured services. Integrated AI assistance through GitHub Copilot helps you create, modify, and manage Docker services while maintaining consistency across your entire server stack.
The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia for Single Sign-On, Gluetun for VPN routing, and DuckDNS for dynamic DNS - all managed through file-based configurations that the AI can modify.
## Features
- **AI-Powered Management**: GitHub Copilot integration with specialized instructions for Docker service management
- **Docker Compose First**: All persistent services defined in organized compose files
- **Consistent Structure**: Standardized naming conventions and patterns across all services
- **Dockge Structure**: All stacks organized in `/opt/stacks/` for easy management via Dockge
- **40+ Pre-configured Services**: Production-ready compose files across infrastructure, media, home automation, productivity, and monitoring
- **Traefik Reverse Proxy**: Automatic HTTPS with Let's Encrypt via file-based configuration (no web UI needed)
- **Authelia SSO**: Single Sign-On protection for all admin interfaces with smart bypass rules for media apps
- **Gluetun VPN**: Surfshark WireGuard integration for secure downloads
- **Homepage Dashboard**: AI-configurable dashboard with Docker integration and service widgets
- **External Host Proxying**: Proxy external services (Raspberry Pi, routers, NAS) through Traefik
- **Stack-Aware Changes**: AI considers the entire infrastructure when making changes
- **Comprehensive Documentation**: Detailed guidelines and examples for all operations
- **Example Services**: Ready-to-use compose files for common homelab services
- **Comprehensive Documentation**: Detailed guidelines including proxying external hosts
- **File-Based Configuration**: Everything managed via YAML files - no web UI dependencies
## Quick Start
@@ -23,8 +30,10 @@ This repository provides a structured approach to managing a homelab infrastruct
- Docker Compose V2
- Git
- VS Code with GitHub Copilot extension (for AI assistance)
- A domain from DuckDNS (free)
- Surfshark VPN account (optional, for VPN features)
### Initial Setup
### Quick Setup (Dockge Structure)
1. **Clone the repository:**
```bash
@@ -36,45 +45,90 @@ This repository provides a structured approach to managing a homelab infrastruct
2. **Create environment file:**
```bash
cp .env.example .env
# Edit .env with your values
# Edit .env with your values (domain, API keys, passwords)
nano .env
```
3. **Create the main network:**
3. **Create required directories:**
```bash
sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
```
4. **Create Docker networks:**
```bash
docker network create homelab-network
docker network create traefik-network
docker network create media-network
```
4. **Create config directories:**
5. **Deploy core infrastructure (in order):**
```bash
mkdir -p config
# 1. DuckDNS (Dynamic DNS)
mkdir -p /opt/stacks/duckdns && cp docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
cd /opt/stacks/duckdns && docker compose up -d
# 2. Traefik (Reverse Proxy with SSL)
mkdir -p /opt/stacks/traefik/dynamic
cp docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
cp config-templates/traefik/* /opt/stacks/traefik/
cd /opt/stacks/traefik && docker compose up -d
# 3. Authelia (SSO)
mkdir -p /opt/stacks/authelia
cp docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
cp config-templates/authelia/* /opt/stacks/authelia/
cd /opt/stacks/authelia && docker compose up -d
# 4. Dockge (Stack Manager)
mkdir -p /opt/stacks/infrastructure
cp docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
cd /opt/stacks/infrastructure && docker compose up -d dockge
```
5. **Start your first service:**
```bash
# Example: Start Portainer for container management
docker compose -f docker-compose/infrastructure.yml up -d portainer
```
6. **Access Dockge:**
Open `https://dockge.yourdomain.duckdns.org` (use Authelia login)
6. **Access the service:**
Open `http://your-server-ip:9000` in your browser
Now deploy remaining stacks through Dockge's UI!
## Repository Structure
```
AI-Homelab/
├── .github/
│ └── copilot-instructions.md # AI assistant guidelines
│ └── copilot-instructions.md # AI assistant guidelines (Dockge, Traefik, Authelia aware)
├── docker-compose/
│ ├── infrastructure.yml # Core services (proxy, DNS, Portainer)
│ ├── media.yml # Media services (Plex, Sonarr, Radarr)
│ ├── monitoring.yml # Observability (Prometheus, Grafana)
│ ├── development.yml # Dev tools (code-server, databases)
│ └── README.md # Docker Compose documentation
│ ├── traefik.yml # Reverse proxy (deploy first)
│ ├── authelia.yml # SSO authentication
│ ├── duckdns.yml # Dynamic DNS
│ ├── gluetun.yml # VPN client (Surfshark) + qBittorrent
│ ├── infrastructure.yml # Dockge, Portainer, Pi-hole, Watchtower, Dozzle, Glances
│ ├── dashboards.yml # Homepage, Homarr
│ ├── media.yml # Plex, Jellyfin, Sonarr, Radarr, Prowlarr
│ ├── media-extended.yml # Readarr, Lidarr, Lazy Librarian, Mylar3, Calibre-Web,
│ │ # Jellyseerr, FlareSolverr, Tdarr, Unmanic
│ ├── homeassistant.yml # Home Assistant, ESPHome, TasmoAdmin, Node-RED,
│ │ # Mosquitto, Zigbee2MQTT, MotionEye
│ ├── productivity.yml # Nextcloud, Mealie, WordPress, Gitea, DokuWiki,
│ │ # BookStack, MediaWiki (all with databases)
│ ├── utilities.yml # Backrest, Duplicati, Uptime Kuma, Code Server,
│ │ # Form.io, Authelia-Redis
│ ├── monitoring.yml # Prometheus, Grafana, Loki, Promtail, cAdvisor
│ ├── development.yml # GitLab, PostgreSQL, Redis, pgAdmin, Jupyter
│ └── README-dockge.md # Dockge deployment guide
├── config-templates/
│ ├── traefik/ # Traefik static and dynamic configs
│ ├── authelia/ # Authelia config and user database
│ ├── homepage/ # Homepage dashboard configs (with widgets)
│ ├── prometheus/ # Prometheus scrape configs
│ ├── loki/ # Loki log aggregation config
│ └── ... # Other service templates
├── docs/
│ └── docker-guidelines.md # Comprehensive Docker guidelines
├── config/ # Service configurations (gitignored)
├── .env.example # Environment variable template
│ ├── docker-guidelines.md # Comprehensive Docker guidelines
│ ├── getting-started.md # Step-by-step setup guide
│ ├── quick-reference.md # Command reference
│ └── proxying-external-hosts.md # Guide for proxying Raspberry Pi, routers, etc.
├── .env.example # Environment variable template (40+ vars)
├── .gitignore # Git ignore patterns
└── README.md # This file
```
@@ -87,193 +141,384 @@ AI-Homelab/
2. **Open this repository** in VS Code
3. **Start Copilot Chat** and ask questions like:
- "Help me add a new media service to my homelab"
- "Create a docker-compose file for Home Assistant"
- "How do I configure GPU support for Plex?"
- "Check my compose file for port conflicts"
- "Configure Traefik routing for my new service"
- "Add Authelia SSO protection to this service"
- "How do I proxy my Raspberry Pi through Traefik?"
- "Create a Homepage widget for this service"
- "Route this download client through Gluetun VPN"
The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md` to:
- Use `/opt/stacks/` directory structure (Dockge compatible)
- Configure Traefik labels for automatic routing
- Apply Authelia middleware where appropriate
- Suggest `/mnt/` for large data storage
- Add services to Homepage dashboard with widgets
- Maintain consistency with existing services
- Use Docker Compose for all persistent services
- Consider the entire stack when making changes
- Follow naming conventions and best practices
### Example Interactions
**Creating a new service:**
**Adding a new service:**
```
You: "I want to add Home Assistant to my homelab"
You: "Add Tautulli to monitor my Plex server"
Copilot: [Analyzes existing stack, checks for conflicts, creates compose configuration]
- Checks available ports
- Uses consistent naming
- Connects to appropriate networks
- Follows established patterns
Copilot: [Creates compose configuration with]:
- /opt/stacks/tautulli/ directory structure
- Traefik labels for HTTPS access
- Authelia middleware for SSO protection
- Homepage dashboard entry with widget
- Connection to existing Plex service
```
**Modifying a service:**
**Proxying external service:**
```
You: "Add GPU support to my Plex container"
You: "Proxy my Raspberry Pi Home Assistant through Traefik"
Copilot: [Reviews current Plex configuration and updates it]
- Checks if NVIDIA runtime is available
- Updates device mappings
- Adds required environment variables
- Maintains existing configuration
Copilot: [Creates Traefik route configuration]:
- File in /opt/stacks/traefik/dynamic/
- HTTPS with Let's Encrypt
- Authelia bypass (HA has its own auth)
- WebSocket support
- Homepage dashboard entry
```
**Configuring VPN routing:**
```
You: "Route SABnzbd through the VPN"
Copilot: [Updates compose to use Gluetun]:
- network_mode: "service:gluetun"
- Exposes ports through Gluetun
- Maintains Traefik routing
- Updates documentation
```
## Available Service Stacks
### Infrastructure (`infrastructure.yml`)
- **Nginx Proxy Manager**: Web-based reverse proxy with SSL
- **Pi-hole**: Network-wide ad blocking and DNS
- **Portainer**: Docker container management UI
- **Watchtower**: Automatic container updates
### Core Infrastructure (Required)
### Media (`media.yml`)
- **Plex**: Media streaming server
- **Jellyfin**: Open-source media server alternative
- **Sonarr**: TV show automation
- **Radarr**: Movie automation
- **Prowlarr**: Indexer manager
- **qBittorrent**: Torrent client
#### 1. Traefik (`traefik.yml`)
**Reverse proxy with automatic SSL** - Deploy first!
- Automatic HTTPS via Let's Encrypt
- File-based and Docker label routing
- HTTP to HTTPS redirect
- Dashboard at `https://traefik.${DOMAIN}`
#### 2. Authelia (`authelia.yml`)
**Single Sign-On authentication**
- TOTP 2FA support
- LDAP/file-based user database
- Smart bypass rules for media apps
- Login at `https://auth.${DOMAIN}`
#### 3. DuckDNS (`duckdns.yml`)
**Dynamic DNS updater**
- Automatic IP updates
- Integrates with Let's Encrypt
- No web UI - runs silently
#### 4. Gluetun (`gluetun.yml`)
**VPN client (Surfshark WireGuard)**
- Includes qBittorrent
- Download via `https://qbit.${DOMAIN}`
- Easy to route other services through VPN
### Infrastructure Tools (`infrastructure.yml`)
- **Dockge**: Docker Compose stack manager (PRIMARY) - `https://dockge.${DOMAIN}`
- **Portainer**: Docker management UI (secondary) - `https://portainer.${DOMAIN}`
- **Pi-hole**: Network-wide ad blocking - `https://pihole.${DOMAIN}`
- **Watchtower**: Automatic container updates
- **Dozzle**: Real-time Docker logs - `https://dozzle.${DOMAIN}`
- **Glances**: System monitoring - `https://glances.${DOMAIN}`
- **Docker Proxy**: Secure Docker socket access
### Dashboards (`dashboards.yml`)
- **Homepage**: AI-configurable dashboard with widgets - `https://home.${DOMAIN}`
- Docker integration (container status)
- Service widgets (Sonarr, Radarr, Plex, Jellyfin, etc.)
- 11 organized categories
- **Homarr**: Modern alternative dashboard - `https://homarr.${DOMAIN}`
### Media Services (`media.yml`)
- **Plex**: Media streaming server - `https://plex.${DOMAIN}` (no SSO - app access)
- **Jellyfin**: Open-source media server - `https://jellyfin.${DOMAIN}` (no SSO - app access)
- **Sonarr**: TV show automation - `https://sonarr.${DOMAIN}`
- **Radarr**: Movie automation - `https://radarr.${DOMAIN}`
- **Prowlarr**: Indexer manager - `https://prowlarr.${DOMAIN}`
- **qBittorrent**: Torrent client (via VPN) - See gluetun.yml
### Extended Media (`media-extended.yml`)
- **Readarr**: Ebook/audiobook management - `https://readarr.${DOMAIN}`
- **Lidarr**: Music collection manager - `https://lidarr.${DOMAIN}`
- **Lazy Librarian**: Book download automation - `https://lazylibrarian.${DOMAIN}`
- **Mylar3**: Comic book manager - `https://mylar.${DOMAIN}`
- **Calibre-Web**: Ebook reader and server - `https://calibre.${DOMAIN}`
- **Jellyseerr**: Media request management - `https://jellyseerr.${DOMAIN}` (no SSO)
- **FlareSolverr**: Cloudflare bypass (no UI)
- **Tdarr**: Distributed transcoding - `https://tdarr.${DOMAIN}`
- **Unmanic**: Library optimizer - `https://unmanic.${DOMAIN}`
### Home Automation (`homeassistant.yml`)
- **Home Assistant**: Home automation hub - `https://ha.${DOMAIN}` (uses host network)
- **ESPHome**: ESP device manager - `https://esphome.${DOMAIN}`
- **TasmoAdmin**: Tasmota device manager - `https://tasmoadmin.${DOMAIN}`
- **Node-RED**: Flow automation - `https://nodered.${DOMAIN}`
- **Mosquitto**: MQTT broker (no UI)
- **Zigbee2MQTT**: Zigbee bridge - `https://zigbee2mqtt.${DOMAIN}`
- **MotionEye**: Video surveillance - `https://motioneye.${DOMAIN}`
### Productivity (`productivity.yml`)
- **Nextcloud**: File sync & collaboration - `https://nextcloud.${DOMAIN}`
- Includes MariaDB database
- **Mealie**: Recipe manager - `https://mealie.${DOMAIN}` (no SSO)
- **WordPress**: Blog platform - `https://blog.${DOMAIN}` (no SSO - public)
- Includes MariaDB database
- **Gitea**: Self-hosted Git - `https://git.${DOMAIN}`
- Includes PostgreSQL database
- **DokuWiki**: File-based wiki - `https://wiki.${DOMAIN}`
- **BookStack**: Documentation platform - `https://docs.${DOMAIN}`
- Includes MariaDB database
- **MediaWiki**: Wiki platform - `https://mediawiki.${DOMAIN}`
- Includes MariaDB database
### Utilities (`utilities.yml`)
- **Backrest**: Backup manager (restic) - `https://backrest.${DOMAIN}`
- **Duplicati**: Backup software - `https://duplicati.${DOMAIN}`
- **Uptime Kuma**: Status monitoring - `https://status.${DOMAIN}` (no SSO - public)
- **Code Server**: VS Code in browser - `https://code.${DOMAIN}`
- **Form.io**: Form builder - `https://forms.${DOMAIN}`
- Includes MongoDB database
- **Authelia-Redis**: Session storage (no UI)
### Monitoring (`monitoring.yml`)
- **Prometheus**: Metrics collection
- **Grafana**: Metrics visualization
- **Node Exporter**: Host metrics
- **cAdvisor**: Container metrics
- **Uptime Kuma**: Service uptime monitoring
- **Prometheus**: Metrics collection - `https://prometheus.${DOMAIN}`
- **Grafana**: Metrics visualization - `https://grafana.${DOMAIN}`
- **Loki**: Log aggregation
- **Promtail**: Log shipping
- **Node Exporter**: Host metrics
- **cAdvisor**: Container metrics
### Development (`development.yml`)
- **Code Server**: VS Code in browser
- **GitLab CE**: Self-hosted Git repository
- **GitLab CE**: Git with CI/CD - `https://gitlab.${DOMAIN}`
- **PostgreSQL**: SQL database
- **Redis**: In-memory data store
- **pgAdmin**: PostgreSQL UI
- **Jupyter Lab**: Interactive notebooks
- **Node-RED**: Visual automation
- **Redis**: In-memory store
- **pgAdmin**: PostgreSQL UI - `https://pgadmin.${DOMAIN}`
- **Jupyter Lab**: Interactive notebooks - `https://jupyter.${DOMAIN}`
## Common Operations
### Starting Services
### Managing Stacks via Dockge
Access Dockge at `https://dockge.${DOMAIN}` to:
- View all stacks and their status
- Start/stop/restart stacks
- Edit compose files directly
- View logs
- Deploy new stacks
### Command Line Operations
#### Starting Services (Dockge Structure)
Start all services in a compose file:
```bash
docker compose -f docker-compose/infrastructure.yml up -d
# Start entire stack
cd /opt/stacks/media
docker compose up -d
# Start specific services
cd /opt/stacks/media
docker compose up -d sonarr radarr
# Start with rebuild
cd /opt/stacks/infrastructure
docker compose up -d --build
```
Start specific services:
#### Stopping Services
```bash
docker compose -f docker-compose/media.yml up -d plex sonarr radarr
# Stop entire stack
cd /opt/stacks/media
docker compose down
# Stop but keep volumes
cd /opt/stacks/media
docker compose stop
# Stop specific service
cd /opt/stacks/media
docker compose stop plex
```
### Stopping Services
#### Viewing Logs
Stop all services:
```bash
docker compose -f docker-compose/infrastructure.yml down
# Follow logs for entire stack
cd /opt/stacks/media
docker compose logs -f
# Follow logs for specific service
cd /opt/stacks/media
docker compose logs -f plex
# View last 100 lines
cd /opt/stacks/media
docker compose logs --tail=100 plex
# Or use Dozzle web UI at https://dozzle.${DOMAIN}
```
Stop specific service:
#### Updating Services
```bash
docker compose -f docker-compose/media.yml stop plex
```
# Pull latest images
cd /opt/stacks/media
docker compose pull
### Viewing Logs
# Update specific service
cd /opt/stacks/media
docker compose pull plex
docker compose up -d plex
Follow logs for a service:
```bash
docker compose -f docker-compose/media.yml logs -f plex
```
View last 100 lines:
```bash
docker compose -f docker-compose/media.yml logs --tail=100 plex
```
### Updating Services
Pull latest images:
```bash
docker compose -f docker-compose/media.yml pull
```
Update specific service:
```bash
docker compose -f docker-compose/media.yml pull plex
docker compose -f docker-compose/media.yml up -d plex
# Or enable Watchtower for automatic updates
```
### Testing with Docker Run
Test NVIDIA GPU support:
```bash
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
```
Use `docker run` only for temporary testing:
Test a new image:
```bash
# Test NVIDIA GPU support
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
# Test a new image
docker run --rm -it alpine:latest /bin/sh
# Test VPN connection through Gluetun
docker run --rm --network container:gluetun curlimages/curl ifconfig.me
```
## Network Architecture
All services connect to networks for inter-service communication:
Services connect to multiple networks for organization and security:
- **homelab-network**: Main network for all services
- **media-network**: Isolated network for media stack
- **monitoring-network**: Network for observability stack
- **database-network**: Isolated network for databases
- **traefik-network**: For Traefik to reach services (external)
- **homelab-network**: Main network for inter-service communication (external)
- **media-network**: Isolated network for media stack (external)
- **monitoring-network**: Network for observability stack (created per stack)
- **database-network**: Isolated networks for database services (created per stack)
- **dockerproxy-network**: Secure Docker socket access (created in infrastructure)
### Creating Required Networks
Create networks manually:
```bash
# Create external networks (do this once)
docker network create traefik-network
docker network create homelab-network
docker network create media-network
docker network create monitoring-network
docker network create database-network
# Stack-specific networks are created automatically by compose files
```
### Traefik Routing
All services accessed via Traefik with automatic HTTPS:
- Pattern: `https://service.yourdomain.duckdns.org`
- Configured via Docker labels on each service
- SSL certificates automatically managed
- No port exposure needed (except Traefik 80/443)
## Documentation
### Comprehensive Guides
- **[Docker Guidelines](docs/docker-guidelines.md)**: Complete guide to Docker service management
- **[Docker Compose README](docker-compose/README.md)**: Compose-specific documentation
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines
- **[Docker Guidelines](docs/docker-guidelines.md)**: Complete guide to Docker service management with Dockge
- **[Getting Started](docs/getting-started.md)**: Step-by-step setup walkthrough
- **[Quick Reference](docs/quick-reference.md)**: Command reference and troubleshooting
- **[Dockge Deployment](docker-compose/README-dockge.md)**: Dockge-specific deployment guide
- **[Proxying External Hosts](docs/proxying-external-hosts.md)**: Guide for proxying Raspberry Pi, routers, NAS via Traefik
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines (Traefik, Authelia, Dockge aware)
### Key Principles
1. **Docker Compose First**: Always use compose for persistent services
2. **Docker Run for Testing**: Only use `docker run` for temporary containers
3. **Consistency**: Follow established patterns and naming conventions
4. **Stack Awareness**: Consider dependencies and interactions
5. **Documentation**: Comment complex configurations
6. **Security**: Keep secrets in `.env` files, never commit them
1. **Dockge Structure**: All stacks in `/opt/stacks/stack-name/`
2. **Docker Compose First**: Always use compose for persistent services
3. **Docker Run for Testing**: Only use `docker run` for temporary containers
4. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
5. **Traefik for All**: Every service routed through Traefik with automatic SSL
6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps for device access
7. **VPN When Needed**: Route download clients through Gluetun
8. **Large Data Separate**: Use `/mnt/` for media, downloads, large databases
9. **Stack Awareness**: Consider dependencies and interactions
10. **Security**: Keep secrets in `.env` files, never commit them
## Configuration Management
### Environment Variables
All services use variables from `.env`:
All services use variables from `.env` in each stack directory:
- `PUID`/`PGID`: User/group IDs for file permissions
- `TZ`: Timezone for all services
- `DOMAIN`: Your DuckDNS domain (e.g., yourdomain.duckdns.org)
- `SERVER_IP`: Your server's IP address
- Service-specific credentials and paths
- Service-specific credentials and API keys
- Homepage widget API keys (40+ variables)
### Config Files
See `.env.example` for complete list.
Service configurations are stored in `config/service-name/`:
### Storage Strategy
**Small Data** (configs, databases < 10GB): `/opt/stacks/stack-name/`
```yaml
volumes:
- /opt/stacks/sonarr/config:/config
```
config/
├── plex/ # Plex configuration
├── sonarr/ # Sonarr configuration
├── grafana/ # Grafana dashboards
**Large Data** (media, downloads, backups): `/mnt/`
```yaml
volumes:
- /mnt/media:/media
- /mnt/downloads:/downloads
- /mnt/backups:/backups
```
The AI will suggest when to use `/mnt/` based on expected data size.
### Configuration Files
Service configurations stored in stack directories:
```
/opt/stacks/
├── traefik/
│ ├── docker-compose.yml
│ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes
│ │ └── routes.yml
│ └── acme.json # SSL certificates
├── authelia/
│ ├── docker-compose.yml
│ ├── configuration.yml # Authelia settings
│ └── users_database.yml # User accounts
├── homepage/
│ ├── docker-compose.yml
│ └── config/
│ ├── services.yaml # Service definitions
│ ├── docker.yaml # Docker integration
│ ├── settings.yaml # Dashboard settings
│ └── widgets.yaml # Homepage widgets
└── ...
```
**Note**: Config directories are gitignored to prevent committing sensitive data.
Templates available in `config-templates/` directory.
## Security Best Practices
@@ -362,15 +607,51 @@ This project is provided as-is for personal homelab use.
## Getting Started Checklist
- [ ] Install Docker and Docker Compose
- [ ] Install Docker and Docker Compose V2
- [ ] Sign up for DuckDNS (free) and get your domain
- [ ] Get Surfshark VPN credentials (optional, for VPN features)
- [ ] Clone this repository
- [ ] Copy `.env.example` to `.env` and configure
- [ ] Create `homelab-network`: `docker network create homelab-network`
- [ ] Start infrastructure services: `docker compose -f docker-compose/infrastructure.yml up -d`
- [ ] Access Portainer at `http://server-ip:9000`
- [ ] Copy `.env.example` to `.env` and configure all values
- [ ] Create `/opt/stacks` directory: `sudo mkdir -p /opt/stacks && sudo chown $USER:$USER /opt/stacks`
- [ ] Create Docker networks: `docker network create traefik-network homelab-network media-network`
- [ ] Deploy DuckDNS stack
- [ ] Deploy Traefik stack (with config templates)
- [ ] Deploy Authelia stack (with config templates)
- [ ] Deploy infrastructure stack (Dockge)
- [ ] Access Dockge at `https://dockge.${DOMAIN}` and deploy remaining stacks
- [ ] Configure Homepage dashboard (copy templates to /opt/stacks/homepage/config/)
- [ ] Install VS Code with GitHub Copilot extension
- [ ] Open repository in VS Code and start using AI assistance
- [ ] Add more services as needed using AI guidance
## Proxying External Hosts
You can proxy services running on other devices (Raspberry Pi, routers, NAS) through Traefik:
**Example: Raspberry Pi Home Assistant**
```yaml
# In /opt/stacks/traefik/dynamic/external.yml
http:
routers:
ha-pi:
rule: "Host(`ha.yourdomain.duckdns.org`)"
entryPoints:
- websecure
service: ha-pi
tls:
certResolver: letsencrypt
services:
ha-pi:
loadBalancer:
servers:
- url: "http://192.168.1.50:8123"
```
See [docs/proxying-external-hosts.md](docs/proxying-external-hosts.md) for complete guide including:
- Three methods (file provider, Docker labels, hybrid)
- Authelia bypass configuration
- WebSocket support
- Examples for routers, NAS, cameras, Proxmox
## Support

182
docs/docker-guidelines.md
View File

@@ -2,40 +2,184 @@
## Overview
This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab using Dockge, Traefik, and Authelia. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
## Table of Contents
1. [Philosophy](#philosophy)
2. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run)
3. [Service Creation Guidelines](#service-creation-guidelines)
4. [Service Modification Guidelines](#service-modification-guidelines)
5. [Naming Conventions](#naming-conventions)
6. [Network Architecture](#network-architecture)
7. [Volume Management](#volume-management)
8. [Security Best Practices](#security-best-practices)
9. [Monitoring and Logging](#monitoring-and-logging)
10. [Troubleshooting](#troubleshooting)
2. [Dockge Structure](#dockge-structure)
3. [Traefik and Authelia Integration](#traefik-and-authelia-integration)
4. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run)
5. [Service Creation Guidelines](#service-creation-guidelines)
6. [Service Modification Guidelines](#service-modification-guidelines)
7. [Naming Conventions](#naming-conventions)
8. [Network Architecture](#network-architecture)
9. [Volume Management](#volume-management)
10. [Security Best Practices](#security-best-practices)
11. [Monitoring and Logging](#monitoring-and-logging)
12. [Troubleshooting](#troubleshooting)
## Philosophy
### Core Principles
1. **Infrastructure as Code**: All services must be defined in Docker Compose files
2. **Reproducibility**: Any service should be rebuildable from compose files
3. **Documentation**: Every non-obvious configuration must be commented
4. **Consistency**: Use the same patterns across all services
5. **Safety First**: Always test changes in isolation before deploying
1. **Dockge First**: Manage all stacks through Dockge in `/opt/stacks/`
2. **Infrastructure as Code**: All services defined in Docker Compose files
3. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
4. **Reproducibility**: Any service should be rebuildable from compose files
5. **Automatic HTTPS**: All services routed through Traefik with Let's Encrypt
6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps
7. **Documentation**: Every non-obvious configuration must be commented
8. **Consistency**: Use the same patterns across all services
9. **Safety First**: Always test changes in isolation before deploying
### The Stack Mindset
Think of your homelab as an interconnected stack where:
- Services depend on networks
- Networks connect services
- Volumes persist data
- Services depend on networks (especially traefik-network)
- Traefik routes all traffic with automatic SSL
- Authelia protects sensitive services
- VPN (Gluetun) secures downloads
- Changes ripple through the system
Always ask: "How does this change affect other services?"
Always ask: "How does this change affect other services and routing?"
## Dockge Structure
### Directory Organization
All stacks live in `/opt/stacks/stack-name/`:
```
/opt/stacks/
├── traefik/
│ ├── docker-compose.yml
│ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes
│ │ ├── routes.yml
│ │ └── external.yml # External host proxying
│ ├── acme.json # SSL certificates (chmod 600)
│ └── .env
├── authelia/
│ ├── docker-compose.yml
│ ├── configuration.yml # Authelia settings
│ ├── users_database.yml # User accounts
│ └── .env
├── media/
│ ├── docker-compose.yml
│ └── .env
└── ...
```
### Why Dockge?
- **Visual Management**: Web UI at `https://dockge.${DOMAIN}`
- **Direct File Editing**: Edit compose files in-place
- **Stack Organization**: Each service stack is independent
- **AI Compatible**: Files can be managed by AI
- **Git Integration**: Easy to version control
### Storage Strategy
**Small Data** (configs, DBs < 10GB): `/opt/stacks/stack-name/`
```yaml
volumes:
- /opt/stacks/sonarr/config:/config
```
**Large Data** (media, downloads, backups): `/mnt/`
```yaml
volumes:
- /mnt/media/movies:/movies
- /mnt/media/tv:/tv
- /mnt/downloads:/downloads
- /mnt/backups:/backups
```
AI will suggest `/mnt/` when data may exceed 50GB or grow continuously.
## Traefik and Authelia Integration
### Every Service Needs Traefik Labels
Standard pattern for all services:
```yaml
services:
myservice:
image: myimage:latest
container_name: myservice
networks:
- homelab-network
- traefik-network # Required for Traefik
labels:
# Enable Traefik
- "traefik.enable=true"
# Define routing rule
- "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
# Use websecure entrypoint (HTTPS)
- "traefik.http.routers.myservice.entrypoints=websecure"
# Enable Let's Encrypt
- "traefik.http.routers.myservice.tls.certresolver=letsencrypt"
# Add Authelia SSO (if needed)
- "traefik.http.routers.myservice.middlewares=authelia@docker"
# Specify port (if not default 80)
- "traefik.http.services.myservice.loadbalancer.server.port=8080"
```
### When to Use Authelia SSO
**Protect with Authelia**:
- Admin interfaces (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure tools (Portainer, Dockge, Grafana)
- Personal data (Nextcloud, Mealie, wikis)
- Development tools (code-server, GitLab)
- Monitoring dashboards
**Bypass Authelia**:
- Media servers (Plex, Jellyfin) - need app access
- Request services (Jellyseerr) - family-friendly access
- Public services (WordPress, status pages)
- Services with their own auth (Home Assistant)
Configure bypasses in `/opt/stacks/authelia/configuration.yml`:
```yaml
access_control:
rules:
- domain: jellyfin.yourdomain.duckdns.org
policy: bypass
- domain: plex.yourdomain.duckdns.org
policy: bypass
```
### Routing Through VPN (Gluetun)
For services that need VPN (downloads):
```yaml
services:
mydownloader:
image: downloader:latest
container_name: mydownloader
network_mode: "service:gluetun" # Route through VPN
depends_on:
- gluetun
```
Expose ports through Gluetun's compose file:
```yaml
# In gluetun.yml
gluetun:
ports:
- "8080:8080" # mydownloader web UI
```
## Docker Compose vs Docker Run

595
docs/getting-started.md
View File

@@ -1,6 +1,6 @@
# Getting Started Guide
This guide will walk you through setting up your AI-powered homelab from scratch.
This guide will walk you through setting up your AI-powered homelab with Dockge, Traefik, Authelia, and 40+ services from scratch.
## Prerequisites
@@ -11,8 +11,10 @@ Before you begin, ensure you have:
- [ ] Docker Compose V2 installed
- [ ] Git installed
- [ ] At least 8GB RAM (16GB+ recommended)
- [ ] Sufficient disk space (100GB+ recommended)
- [ ] Static IP address for your server
- [ ] Sufficient disk space (100GB+ system, 1TB+ for media recommended)
- [ ] Static IP address for your server (or DHCP reservation)
- [ ] DuckDNS account (free) with a domain
- [ ] Surfshark VPN account (optional, for VPN features)
- [ ] VS Code with GitHub Copilot extension (for AI assistance)
## Step 1: Verify Docker Installation
@@ -45,7 +47,15 @@ git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
```
## Step 3: Configure Environment Variables
## Step 3: Sign Up for DuckDNS
1. Go to https://www.duckdns.org/
2. Sign in with your preferred method
3. Create a domain (e.g., `myhomelab`)
4. Copy your token - you'll need it for `.env`
5. Your domain will be: `myhomelab.duckdns.org`
## Step 4: Configure Environment Variables
```bash
# Copy the example environment file
@@ -59,7 +69,7 @@ id -g # This is your PGID
nano .env
```
**Update these values in `.env`:**
**Critical values to update in `.env`:**
```bash
# Your user/group IDs
PUID=1000 # Replace with your user ID
@@ -71,442 +81,333 @@ TZ=America/New_York
# Your server's IP address
SERVER_IP=192.168.1.100 # Replace with your actual IP
# Directory paths
USERDIR=/home/yourusername/homelab # Update username
MEDIADIR=/mnt/media # Update if different
DOWNLOADDIR=/mnt/downloads # Update if different
# DuckDNS Configuration
DOMAIN=myhomelab.duckdns.org # Your DuckDNS domain
DUCKDNS_TOKEN=your-duckdns-token-here
DUCKDNS_SUBDOMAINS=myhomelab # Without .duckdns.org
# Set secure passwords for services
GRAFANA_ADMIN_PASSWORD=your-secure-password-here
CODE_SERVER_PASSWORD=your-secure-password-here
POSTGRES_PASSWORD=your-secure-password-here
PGADMIN_PASSWORD=your-secure-password-here
JUPYTER_TOKEN=your-secure-token-here
PIHOLE_PASSWORD=your-secure-password-here
# Let's Encrypt Email
ACME_EMAIL=your-email@example.com
# Authelia Secrets (generate with: openssl rand -hex 64)
AUTHELIA_JWT_SECRET=$(openssl rand -hex 64)
AUTHELIA_SESSION_SECRET=$(openssl rand -hex 64)
AUTHELIA_STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 64)
# Surfshark VPN (if using)
SURFSHARK_PRIVATE_KEY=your-wireguard-private-key
SURFSHARK_ADDRESSES=10.14.0.2/16
# Set secure passwords for all services
PIHOLE_PASSWORD=your-secure-password
GRAFANA_ADMIN_PASSWORD=your-secure-password
CODE_SERVER_PASSWORD=your-secure-password
# ... (see .env.example for complete list)
```
**Save and exit** (Ctrl+X, Y, Enter in nano)
## Step 4: Create Docker Networks
## Step 5: Create Dockge Directory Structure
```bash
# Create the main homelab network
docker network create homelab-network
# Create main stacks directory
sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
# Create additional networks for better security
# Create mount points for large data (adjust as needed)
sudo mkdir -p /mnt/media/{movies,tv,music,books,photos}
sudo mkdir -p /mnt/downloads/{complete,incomplete}
sudo mkdir -p /mnt/backups
sudo chown -R $USER:$USER /mnt/media /mnt/downloads /mnt/backups
```
## Step 6: Create Docker Networks
```bash
# Create required external networks
docker network create traefik-network
docker network create homelab-network
docker network create media-network
docker network create monitoring-network
docker network create database-network
docker network create dockerproxy-network
# Verify networks were created
docker network ls
docker network ls | grep -E "traefik|homelab|media|dockerproxy"
```
## Step 5: Create Configuration Directories
## Step 7: Deploy Core Infrastructure (IN ORDER)
### 7.1 DuckDNS (Dynamic DNS)
```bash
# Create the main config directory
mkdir -p config
# Create stack directory
mkdir -p /opt/stacks/duckdns
# Create config directories for services you plan to use
mkdir -p config/{nginx-proxy-manager,pihole,portainer}
mkdir -p config/{plex,sonarr,radarr,prowlarr,qbittorrent,jellyfin}
mkdir -p config/{prometheus,grafana,loki,promtail}
mkdir -p config/{code-server,postgres,redis}
# Copy compose file
cp ~/AI-Homelab/docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
# Set proper permissions
sudo chown -R $(id -u):$(id -g) config/
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/duckdns/.env
# Deploy
cd /opt/stacks/duckdns
docker compose up -d
# Verify it's working
docker compose logs -f
# Should see: "Your IP was updated to X.X.X.X"
```
## Step 6: Copy Configuration Templates (Optional)
For services that need config files:
### 7.2 Traefik (Reverse Proxy with SSL)
```bash
# Prometheus
mkdir -p config/prometheus
cp config-templates/prometheus/prometheus.yml config/prometheus/
# Create stack directory with dynamic configs
mkdir -p /opt/stacks/traefik/dynamic
# Loki
mkdir -p config/loki
cp config-templates/loki/loki-config.yml config/loki/
# Copy compose file
cp ~/AI-Homelab/docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
# Promtail
mkdir -p config/promtail
cp config-templates/promtail/promtail-config.yml config/promtail/
# Copy configuration templates
cp ~/AI-Homelab/config-templates/traefik/traefik.yml /opt/stacks/traefik/
cp ~/AI-Homelab/config-templates/traefik/dynamic/*.yml /opt/stacks/traefik/dynamic/
# Redis
mkdir -p config/redis
cp config-templates/redis/redis.conf config/redis/
# Create acme.json for SSL certificates
touch /opt/stacks/traefik/acme.json
chmod 600 /opt/stacks/traefik/acme.json
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/traefik/.env
# Deploy
cd /opt/stacks/traefik
docker compose up -d
# Check logs
docker compose logs -f
# Should see Traefik starting and certificate resolver configured
```
## Step 7: Start Your First Service (Portainer)
Portainer provides a web UI for managing Docker containers. It's a great first service to deploy.
### 7.3 Authelia (SSO Authentication)
```bash
# Start Portainer
docker compose -f docker-compose/infrastructure.yml up -d portainer
# Create stack directory
mkdir -p /opt/stacks/authelia
# Check if it's running
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
# View logs
docker compose -f docker-compose/infrastructure.yml logs -f portainer
# Copy configuration templates
cp ~/AI-Homelab/config-templates/authelia/*.yml /opt/stacks/authelia/
# Generate password hash for users_database.yml
docker run --rm authelia/authelia:latest authelia crypto hash generate argon2 --password 'yourpassword'
# Copy the hash and edit users_database.yml
# Edit users_database.yml
cd /opt/stacks/authelia
nano users_database.yml
# Replace the password hash with your generated one
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/authelia/.env
# Deploy
docker compose up -d
# Check logs
docker compose logs -f
# Test login at https://auth.yourdomain.duckdns.org
```
**Access Portainer:**
1. Open your browser
2. Navigate to `http://YOUR_SERVER_IP:9000`
3. Create an admin account on first login
4. Select "Get Started" and choose local Docker environment
## Step 8: Deploy Infrastructure Services
### 7.4 Infrastructure Services (Dockge)
```bash
# Start all infrastructure services
docker compose -f docker-compose/infrastructure.yml up -d
# Create stack directory
mkdir -p /opt/stacks/infrastructure
# Check status
docker compose -f docker-compose/infrastructure.yml ps
# Copy compose file
cp ~/AI-Homelab/docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
# Services now running:
# - Nginx Proxy Manager: http://YOUR_SERVER_IP:81
# - Pi-hole: http://YOUR_SERVER_IP:8080/admin
# - Portainer: http://YOUR_SERVER_IP:9000
# - Watchtower: (runs in background, no UI)
# Create necessary subdirectories
mkdir -p /opt/dockge/data
mkdir -p /opt/stacks/pihole/{etc-pihole,etc-dnsmasq.d}
mkdir -p /opt/stacks/glances/config
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/infrastructure/.env
# Deploy Dockge first
cd /opt/stacks/infrastructure
docker compose up -d dockge
# Access Dockge at https://dockge.yourdomain.duckdns.org
# Deploy remaining infrastructure services
docker compose up -d
```
### Configure Nginx Proxy Manager (Optional)
## Step 8: Deploy Additional Stacks
1. Access: `http://YOUR_SERVER_IP:81`
2. Login with default credentials:
- Email: `admin@example.com`
- Password: `changeme`
3. Change password immediately
4. Add proxy hosts for your services
Now use Dockge UI at `https://dockge.yourdomain.duckdns.org` to deploy additional stacks, or continue with command line:
### Configure Pi-hole (Optional)
1. Access: `http://YOUR_SERVER_IP:8080/admin`
2. Login with password from `.env` (PIHOLE_PASSWORD)
3. Configure DNS settings
4. Update your router to use Pi-hole as DNS server
## Step 9: Deploy Media Services (Optional)
If you want a media server setup:
### 8.1 Gluetun + qBittorrent (VPN)
```bash
# Start media services
docker compose -f docker-compose/media.yml up -d
mkdir -p /opt/stacks/gluetun
cp ~/AI-Homelab/docker-compose/gluetun.yml /opt/stacks/gluetun/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/gluetun/.env
# Check status
docker compose -f docker-compose/media.yml ps
cd /opt/stacks/gluetun
docker compose up -d
# Services now running:
# - Plex: http://YOUR_SERVER_IP:32400/web
# - Jellyfin: http://YOUR_SERVER_IP:8096
# - Sonarr: http://YOUR_SERVER_IP:8989
# - Radarr: http://YOUR_SERVER_IP:7878
# - Prowlarr: http://YOUR_SERVER_IP:9696
# - qBittorrent: http://YOUR_SERVER_IP:8081
# Test VPN
docker exec gluetun curl ifconfig.me
# Should show VPN IP
```
### Initial Media Service Setup
**Plex:**
1. Access: `http://YOUR_SERVER_IP:32400/web`
2. Sign in with your Plex account
3. Add your media libraries
**Sonarr/Radarr:**
1. Access the web UI
2. Go to Settings → Profiles → Quality
3. Configure your quality preferences
4. Add Prowlarr as an indexer
5. Add qBittorrent as a download client
## Step 10: Deploy Monitoring Services (Optional)
For system and service monitoring:
### 8.2 Homepage Dashboard
```bash
# Start monitoring services
docker compose -f docker-compose/monitoring.yml up -d
mkdir -p /opt/stacks/homepage/config
cp ~/AI-Homelab/docker-compose/dashboards.yml /opt/stacks/homepage/docker-compose.yml
cp ~/AI-Homelab/config-templates/homepage/* /opt/stacks/homepage/config/
cp ~/AI-Homelab/.env /opt/stacks/homepage/.env
# Check status
docker compose -f docker-compose/monitoring.yml ps
cd /opt/stacks/homepage
docker compose up -d homepage
# Services now running:
# - Prometheus: http://YOUR_SERVER_IP:9090
# - Grafana: http://YOUR_SERVER_IP:3000
# - Node Exporter: http://YOUR_SERVER_IP:9100
# - cAdvisor: http://YOUR_SERVER_IP:8082
# - Uptime Kuma: http://YOUR_SERVER_IP:3001
# Access at https://home.yourdomain.duckdns.org
```
### Configure Grafana
1. Access: `http://YOUR_SERVER_IP:3000`
2. Login with credentials from `.env`:
- Username: `admin`
- Password: `GRAFANA_ADMIN_PASSWORD` from .env
3. Add Prometheus as a data source:
- URL: `http://prometheus:9090`
4. Import dashboards:
- Dashboard ID 1860 for Node Exporter
- Dashboard ID 893 for Docker metrics
### Configure Uptime Kuma
1. Access: `http://YOUR_SERVER_IP:3001`
2. Create an account on first login
3. Add monitors for your services
## Step 11: Deploy Development Services (Optional)
If you need development tools:
### 8.3 Media Stack
```bash
# Start development services
docker compose -f docker-compose/development.yml up -d
mkdir -p /opt/stacks/media
cp ~/AI-Homelab/docker-compose/media.yml /opt/stacks/media/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/media/.env
# Check status
docker compose -f docker-compose/development.yml ps
# Services now running:
# - Code Server: http://YOUR_SERVER_IP:8443
# - PostgreSQL: localhost:5432
# - Redis: localhost:6379
# - pgAdmin: http://YOUR_SERVER_IP:5050
# - Jupyter Lab: http://YOUR_SERVER_IP:8888
# - Node-RED: http://YOUR_SERVER_IP:1880
cd /opt/stacks/media
docker compose up -d
```
## Step 12: Set Up VS Code with GitHub Copilot
### 8.4 Additional Stacks
1. **Install VS Code** on your local machine (if not already installed)
Deploy as needed:
- `media-extended.yml` → `/opt/stacks/media-extended/`
- `homeassistant.yml` → `/opt/stacks/homeassistant/`
- `productivity.yml` → `/opt/stacks/productivity/`
- `utilities.yml` → `/opt/stacks/utilities/`
- `monitoring.yml` → `/opt/stacks/monitoring/`
- `development.yml` → `/opt/stacks/development/`
2. **Install GitHub Copilot extension:**
- Open VS Code
- Go to Extensions (Ctrl+Shift+X)
- Search for "GitHub Copilot"
- Click Install
- Sign in with your GitHub account
## Step 9: Configure Homepage Widgets
Get API keys from each service and add to Homepage config:
3. **Clone your repository in VS Code:**
```bash
# On your local machine
# Note: Replace 'kelinfoxy' with your username if you forked this repository
git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab
code .
cd /opt/stacks/homepage/config
nano services.yaml
# Get API keys:
# - Sonarr/Radarr/etc: Settings → General → API Key
# - Plex: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/
# - Jellyfin: Dashboard → API Keys
# Add to .env:
nano /opt/stacks/homepage/.env
# HOMEPAGE_VAR_SONARR_KEY=...
# HOMEPAGE_VAR_RADARR_KEY=...
# etc.
# Restart Homepage
cd /opt/stacks/homepage
docker compose restart
```
4. **Start using AI assistance:**
- Open Copilot Chat (Ctrl+Shift+I or click the chat icon)
- The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md`
- Ask questions like:
- "Help me add Home Assistant to my homelab"
- "Create a backup script for my Docker volumes"
- "How do I configure GPU support for Plex?"
## Step 13: Verify Everything is Running
## Step 10: Install VS Code and GitHub Copilot
```bash
# Check all running containers
docker ps
# Install VS Code (if not already installed)
# Download from https://code.visualstudio.com/
# Check container health
docker ps --format "table {{.Names}}\t{{.Status}}"
# Install GitHub Copilot extension
# In VS Code: Extensions → Search "GitHub Copilot" → Install
# View resource usage
docker stats --no-stream
# Open the repository
code ~/AI-Homelab
# Check disk usage
docker system df
```
## Step 14: Set Up Backups
Create a backup script:
```bash
# Create a backup directory
mkdir -p ~/backups
# Create a simple backup script
cat > ~/backup-homelab.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=~/backups
DATE=$(date +%Y%m%d)
# Backup config directories
tar czf $BACKUP_DIR/config-$DATE.tar.gz ~/AI-Homelab/config/
# Backup .env file
cp ~/AI-Homelab/.env $BACKUP_DIR/.env-$DATE
# Backup Docker volumes (example for Portainer)
docker run --rm \
-v portainer-data:/data \
-v $BACKUP_DIR:/backup \
busybox tar czf /backup/portainer-data-$DATE.tar.gz /data
echo "Backup completed: $DATE"
EOF
# Make it executable
chmod +x ~/backup-homelab.sh
# Test the backup
~/backup-homelab.sh
```
Set up a cron job for automated backups:
```bash
# Open crontab
crontab -e
# Add this line to run backup daily at 2 AM
0 2 * * * /home/yourusername/backup-homelab.sh
```
## Step 15: Configure Firewall (Optional but Recommended)
If using UFW:
```bash
# Allow SSH (if not already allowed)
sudo ufw allow 22/tcp
# Allow web traffic (if exposing services to internet)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Allow specific services from local network only
# Replace 192.168.1.0/24 with your network
sudo ufw allow from 192.168.1.0/24 to any port 9000 proto tcp # Portainer
sudo ufw allow from 192.168.1.0/24 to any port 81 proto tcp # Nginx Proxy Manager
sudo ufw allow from 192.168.1.0/24 to any port 3000 proto tcp # Grafana
# Enable firewall
sudo ufw enable
# Start using AI assistance!
```
## Next Steps
Now that your homelab is running:
1. **Explore Services:**
- Access each service's web UI
- Configure settings as needed
- Set up integrations between services
2. **Add More Services:**
- Ask GitHub Copilot for help adding new services
- Follow the patterns in existing compose files
- Check [awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) for ideas
3. **Optimize:**
- Review logs for errors
- Adjust resource limits
- Set up proper monitoring and alerts
4. **Secure:**
- Change all default passwords
- Set up SSL certificates (use Nginx Proxy Manager)
- Enable 2FA where available
- Keep services updated
5. **Learn:**
- Read the [Docker Guidelines](docker-guidelines.md)
- Experiment with new services
- Use AI assistance to understand and modify configurations
1. Explore Dockge at `https://dockge.yourdomain.duckdns.org`
2. Check Homepage dashboard at `https://home.yourdomain.duckdns.org`
3. Configure services through their web UIs
4. Set up Authelia users in `/opt/stacks/authelia/users_database.yml`
5. Configure Homepage widgets with API keys
6. Use VS Code with Copilot to ask questions and make changes
7. Review [proxying-external-hosts.md](proxying-external-hosts.md) to proxy your Raspberry Pi
## Troubleshooting
### Can't access services
### Can't access services via HTTPS
1. **Check if service is running:**
Check Traefik logs:
```bash
docker ps
cd /opt/stacks/traefik
docker compose logs -f
```
2. **Check service logs:**
Verify DNS is resolving:
```bash
docker compose -f docker-compose/file.yml logs service-name
nslookup dockge.yourdomain.duckdns.org
```
3. **Verify network connectivity:**
Check certificate generation:
```bash
ping YOUR_SERVER_IP
docker exec traefik cat /acme.json
```
4. **Check firewall:**
### Authelia login not working
Check Authelia logs:
```bash
sudo ufw status
cd /opt/stacks/authelia
docker compose logs -f
```
### Permission errors
Verify password hash in `users_database.yml`
```bash
# Fix config directory permissions
sudo chown -R $(id -u):$(id -g) config/
### Service not accessible
# Verify PUID/PGID in .env match your user
id -u # Should match PUID in .env
id -g # Should match PGID in .env
```
1. Check Traefik dashboard: `https://traefik.yourdomain.duckdns.org`
2. Verify service has correct Traefik labels
3. Check service is on `traefik-network`
4. Review service logs
### Port already in use
### Port forwarding
```bash
# Find what's using the port
sudo netstat -tlnp | grep PORT_NUMBER
Ensure your router forwards ports 80 and 443 to your server IP.
# Stop the conflicting service or change the port in docker-compose
```
## Security Checklist
### Out of disk space
- [ ] All passwords in `.env` are strong and unique
- [ ] Authelia 2FA is enabled for admin accounts
- [ ] `.env` file permissions are 600 (`chmod 600 .env`)
- [ ] acme.json permissions are 600
- [ ] Firewall is configured (only 80, 443 open to internet)
- [ ] Pi-hole is configured as your DNS server
- [ ] Watchtower is monitoring for updates
- [ ] Backrest/Duplicati configured for backups
```bash
# Check disk usage
df -h
## Congratulations!
# Clean up Docker resources (removes unused images, containers, networks)
docker system prune -a
Your AI-powered homelab is now running with:
- ✅ Automatic HTTPS via Traefik + Let's Encrypt
- ✅ SSO protection via Authelia
- ✅ 40+ services ready to deploy
- ✅ Dashboard with service widgets
- ✅ AI assistance via GitHub Copilot
- ✅ Centralized management via Dockge
# To manage log size, configure log rotation in compose files:
# logging:
# driver: "json-file"
# options:
# max-size: "10m"
# max-file: "3"
```
## Getting Help
- **Documentation:** Check the `docs/` directory for comprehensive guides
- **AI Assistance:** Use GitHub Copilot in VS Code for real-time help
- **Community:** Search for service-specific help in respective communities
- **Issues:** Open an issue on GitHub for problems with this repository
## Success Checklist
- [ ] Docker and Docker Compose installed
- [ ] Repository cloned
- [ ] `.env` file configured
- [ ] Networks created
- [ ] Config directories created
- [ ] Portainer running and accessible
- [ ] Infrastructure services deployed
- [ ] At least one service category deployed (media/monitoring/dev)
- [ ] VS Code with GitHub Copilot set up
- [ ] Backup strategy in place
- [ ] Firewall configured (if applicable)
- [ ] All services accessible and working
Congratulations! Your AI-powered homelab is now running! 🎉
Continue exploring with VS Code and Copilot to add more services, customize configurations, and proxy external devices!

462
docs/services-reference.md Normal file
View File

@@ -0,0 +1,462 @@
# Complete Services Reference
This document lists all 40+ pre-configured services available in the AI-Homelab repository, organized by category.
## Core Infrastructure (4 services)
### Required - Deploy First
1. **DuckDNS** (`duckdns.yml`)
- Dynamic DNS updater
- Updates your public IP automatically
- Integrates with Let's Encrypt for SSL
- No web UI - runs silently
- Stack: `/opt/stacks/duckdns/`
2. **Traefik** (`traefik.yml`)
- Reverse proxy with automatic SSL
- HTTP to HTTPS redirect
- File-based and Docker label routing
- Dashboard: `https://traefik.${DOMAIN}`
- Stack: `/opt/stacks/traefik/`
3. **Authelia** (`authelia.yml`)
- Single Sign-On (SSO) authentication
- TOTP 2FA support
- File-based or LDAP user database
- Smart bypass rules for media apps
- Login: `https://auth.${DOMAIN}`
- Stack: `/opt/stacks/authelia/`
4. **Gluetun** (`gluetun.yml`)
- VPN client (Surfshark WireGuard)
- Includes qBittorrent
- Control panel: `http://gluetun:8000`
- qBittorrent: `https://qbit.${DOMAIN}`
- Stack: `/opt/stacks/gluetun/`
## Infrastructure Tools (7 services)
From `infrastructure.yml` - Stack: `/opt/stacks/infrastructure/`
5. **Dockge** (PRIMARY management tool)
- Docker Compose stack manager
- Web UI for managing /opt/stacks/
- Direct compose file editing
- Access: `https://dockge.${DOMAIN}`
- SSO: Yes
6. **Portainer** (Secondary)
- Docker container management UI
- Access: `https://portainer.${DOMAIN}`
- SSO: Yes
7. **Pi-hole**
- Network-wide ad blocking
- DNS server
- Access: `https://pihole.${DOMAIN}`
- SSO: Yes
8. **Watchtower**
- Automatic container updates
- Runs 4 AM daily
- No web UI
9. **Dozzle**
- Real-time Docker log viewer
- Access: `https://dozzle.${DOMAIN}`
- SSO: Yes
10. **Glances**
- System and Docker monitoring
- Access: `https://glances.${DOMAIN}`
- SSO: Yes
11. **Docker Proxy**
- Secure Docker socket access
- Backend service
- No web UI
## Dashboards (2 services)
From `dashboards.yml` - Stack: `/opt/stacks/dashboards/`
12. **Homepage** (AI-configurable)
- Application dashboard with Docker integration
- Service widgets for 15+ services
- 11 organized categories
- Access: `https://home.${DOMAIN}`
- SSO: No (landing page)
13. **Homarr**
- Modern alternative dashboard
- Access: `https://homarr.${DOMAIN}`
- SSO: No
## Media Services (6 services)
From `media.yml` - Stack: `/opt/stacks/media/`
14. **Plex**
- Media streaming server
- Hardware transcoding support
- Access: `https://plex.${DOMAIN}`
- SSO: No (app access)
15. **Jellyfin**
- Open-source media server
- Hardware transcoding support
- Access: `https://jellyfin.${DOMAIN}`
- SSO: No (app access)
16. **Sonarr**
- TV show automation
- Access: `https://sonarr.${DOMAIN}`
- SSO: Yes
17. **Radarr**
- Movie automation
- Access: `https://radarr.${DOMAIN}`
- SSO: Yes
18. **Prowlarr**
- Indexer manager
- Integrates with Sonarr, Radarr, etc.
- Access: `https://prowlarr.${DOMAIN}`
- SSO: Yes
19. **qBittorrent**
- Torrent client (routes through Gluetun VPN)
- See gluetun.yml
## Extended Media (10 services)
From `media-extended.yml` - Stack: `/opt/stacks/media-extended/`
20. **Readarr**
- Ebook and audiobook management
- Access: `https://readarr.${DOMAIN}`
- SSO: Yes
21. **Lidarr**
- Music collection manager
- Access: `https://lidarr.${DOMAIN}`
- SSO: Yes
22. **Lazy Librarian**
- Book download automation
- Access: `https://lazylibrarian.${DOMAIN}`
- SSO: Yes
23. **Mylar3**
- Comic book collection manager
- Access: `https://mylar.${DOMAIN}`
- SSO: Yes
24. **Calibre-Web**
- Ebook reader and library management
- Access: `https://calibre.${DOMAIN}`
- SSO: Yes
25. **Jellyseerr**
- Media request management
- Integrates with Plex/Jellyfin
- Access: `https://jellyseerr.${DOMAIN}`
- SSO: No (family access)
26. **FlareSolverr**
- Cloudflare bypass for indexers
- Used by Prowlarr
- No web UI
27. **Tdarr Server**
- Distributed transcoding server
- Access: `https://tdarr.${DOMAIN}`
- SSO: Yes
28. **Tdarr Node**
- Transcoding worker
- No web UI
29. **Unmanic**
- Library optimization and transcoding
- Access: `https://unmanic.${DOMAIN}`
- SSO: Yes
## Home Automation (7 services)
From `homeassistant.yml` - Stack: `/opt/stacks/homeassistant/`
30. **Home Assistant**
- Home automation platform
- Uses host networking
- Access: `https://ha.${DOMAIN}` (or via proxying external host)
- SSO: No (has own auth)
31. **ESPHome**
- ESP8266/ESP32 firmware manager
- Access: `https://esphome.${DOMAIN}`
- SSO: Yes
32. **TasmoAdmin**
- Tasmota device management
- Access: `https://tasmoadmin.${DOMAIN}`
- SSO: Yes
33. **Node-RED**
- Flow-based automation programming
- Access: `https://nodered.${DOMAIN}`
- SSO: Yes
34. **Mosquitto**
- MQTT message broker
- Ports: 1883, 9001
- No web UI
35. **Zigbee2MQTT**
- Zigbee to MQTT bridge
- Access: `https://zigbee2mqtt.${DOMAIN}`
- SSO: Yes
36. **MotionEye**
- Video surveillance system
- Access: `https://motioneye.${DOMAIN}`
- SSO: Yes
## Productivity (8 services + 6 databases)
From `productivity.yml` - Stack: `/opt/stacks/productivity/`
37. **Nextcloud**
- File sync and collaboration platform
- Access: `https://nextcloud.${DOMAIN}`
- SSO: Yes
- Database: nextcloud-db (MariaDB)
38. **Mealie**
- Recipe manager and meal planner
- Access: `https://mealie.${DOMAIN}`
- SSO: No (family access)
39. **WordPress**
- Blog and website platform
- Access: `https://blog.${DOMAIN}`
- SSO: No (public blog)
- Database: wordpress-db (MariaDB)
40. **Gitea**
- Self-hosted Git service
- Access: `https://git.${DOMAIN}`
- SSO: Yes
- Database: gitea-db (PostgreSQL)
41. **DokuWiki**
- File-based wiki (no database)
- Access: `https://wiki.${DOMAIN}`
- SSO: Yes
42. **BookStack**
- Documentation platform
- Access: `https://docs.${DOMAIN}`
- SSO: Yes
- Database: bookstack-db (MariaDB)
43. **MediaWiki**
- Wiki platform
- Access: `https://mediawiki.${DOMAIN}`
- SSO: Yes
- Database: mediawiki-db (MariaDB)
## Utilities (7 services)
From `utilities.yml` - Stack: `/opt/stacks/utilities/`
44. **Backrest**
- Backup management with restic
- Access: `https://backrest.${DOMAIN}`
- SSO: Yes
45. **Duplicati**
- Backup software with encryption
- Access: `https://duplicati.${DOMAIN}`
- SSO: Yes
46. **Uptime Kuma**
- Uptime monitoring and status page
- Access: `https://status.${DOMAIN}`
- SSO: No (public status)
47. **Code Server**
- VS Code in browser
- Full stack access
- Access: `https://code.${DOMAIN}`
- SSO: Yes
48. **Form.io**
- Form builder platform
- Access: `https://forms.${DOMAIN}`
- SSO: Yes
- Database: formio-mongo (MongoDB)
49. **Authelia-Redis**
- Session storage for Authelia
- No web UI
## Monitoring (7 services)
From `monitoring.yml` - Stack: `/opt/stacks/monitoring/`
50. **Prometheus**
- Metrics collection
- Access: `https://prometheus.${DOMAIN}`
- SSO: Yes
51. **Grafana**
- Metrics visualization
- Access: `https://grafana.${DOMAIN}`
- SSO: Yes
52. **Loki**
- Log aggregation
- No web UI (accessed via Grafana)
53. **Promtail**
- Log shipping to Loki
- No web UI
54. **Node Exporter**
- Host metrics exporter
- No web UI
55. **cAdvisor**
- Container metrics
- Access: Port 8080 (internal)
## Development (6 services)
From `development.yml` - Stack: `/opt/stacks/development/`
56. **GitLab CE**
- Git repository with CI/CD
- Access: `https://gitlab.${DOMAIN}`
- SSO: Yes
57. **PostgreSQL**
- SQL database
- Port: 5432
- No web UI
58. **Redis**
- In-memory data store
- Port: 6379
- No web UI
59. **pgAdmin**
- PostgreSQL management UI
- Access: `https://pgadmin.${DOMAIN}`
- SSO: Yes
60. **Jupyter Lab**
- Interactive notebooks
- Access: `https://jupyter.${DOMAIN}`
- SSO: Yes
## Summary by Stack
| Stack | File | Services Count | Description |
|-------|------|----------------|-------------|
| Core Infrastructure | Multiple files | 4 | Traefik, Authelia, DuckDNS, Gluetun |
| Infrastructure | infrastructure.yml | 7 | Dockge, Portainer, Pi-hole, etc. |
| Dashboards | dashboards.yml | 2 | Homepage, Homarr |
| Media | media.yml | 6 | Plex, Jellyfin, *arr apps |
| Media Extended | media-extended.yml | 10 | Books, comics, music, transcoding |
| Home Automation | homeassistant.yml | 7 | HA, ESPHome, Node-RED, MQTT, etc. |
| Productivity | productivity.yml | 14 | Nextcloud, wikis, Git (includes DBs) |
| Utilities | utilities.yml | 7 | Backups, monitoring, Code Server |
| Monitoring | monitoring.yml | 7 | Prometheus, Grafana, Loki |
| Development | development.yml | 6 | GitLab, databases, Jupyter |
**Total: 60+ services (including databases)**
## Access Patterns
### With SSO (Authelia Required)
- Admin tools (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure management (Dockge, Portainer, Grafana)
- Development tools (GitLab, Code Server, pgAdmin)
- Personal data (Nextcloud, wikis, BookStack)
### Without SSO (Direct Access)
- Media streaming (Plex, Jellyfin) - for app access
- Public services (WordPress, Uptime Kuma, Homepage)
- Services with own auth (Home Assistant)
- Family-friendly (Mealie, Jellyseerr)
### Via VPN (Gluetun)
- qBittorrent
- Other download clients (add with network_mode: "service:gluetun")
## Storage Recommendations
### Keep on System Drive (/opt/stacks/)
- All configuration files
- Small databases (< 10GB)
- Application data
### Move to Separate Drive (/mnt/)
- Media files (movies, TV, music, photos) → /mnt/media/
- Downloads → /mnt/downloads/
- Backups → /mnt/backups/
- Surveillance footage → /mnt/surveillance/
- Large databases → /mnt/databases/
- Transcoding cache → /mnt/transcode-cache/
## Quick Deployment Guide
1. **Core (Required)**
```bash
# Deploy in this order:
/opt/stacks/duckdns/
/opt/stacks/traefik/
/opt/stacks/authelia/
/opt/stacks/infrastructure/ (dockge)
```
2. **VPN + Downloads**
```bash
/opt/stacks/gluetun/
```
3. **Dashboard**
```bash
/opt/stacks/homepage/
```
4. **Choose Your Stacks**
- Media: `/opt/stacks/media/` + `/opt/stacks/media-extended/`
- Home Automation: `/opt/stacks/homeassistant/`
- Productivity: `/opt/stacks/productivity/`
- Monitoring: `/opt/stacks/monitoring/`
- Development: `/opt/stacks/development/`
- Utilities: `/opt/stacks/utilities/`
## Configuration Files
All configuration templates available in `config-templates/`:
- `traefik/` - Static and dynamic configs
- `authelia/` - Config and user database
- `homepage/` - Dashboard services and widgets
- `prometheus/` - Scrape configurations
- `loki/` - Log aggregation config
- And more...
## Next Steps
1. Deploy core infrastructure
2. Configure Homepage with API keys
3. Set up Authelia users
4. Deploy service stacks as needed
5. Use VS Code + Copilot for AI assistance
6. Proxy external hosts via Traefik (see docs/proxying-external-hosts.md)
For detailed deployment instructions, see [docs/getting-started.md](../docs/getting-started.md)