kelin/EZ-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update all documentation to reflect 60+ services, Dockge, Traefik, Authelia, and new features

Browse Source 
- Update README.md with comprehensive overview of all 40+ services
- Add Quick Setup guide with Dockge deployment steps
- List all service stacks by category (Infrastructure, Media, Home Automation, Productivity, etc.)
- Update AI assistant examples for Traefik, Authelia, VPN routing
- Add network architecture with Traefik routing patterns
- Update common operations for Dockge structure
- Add storage strategy (opt/stacks vs /mnt/)
- Update configuration management section
- Add proxying external hosts overview
- Update getting started checklist with all new steps

- Update docker-guidelines.md with Dockge and Traefik sections
- Add comprehensive Dockge structure explanation
- Add Traefik and Authelia integration patterns
- Document when to use SSO vs bypass
- Add VPN routing patterns with Gluetun
- Update storage strategy recommendations

- Completely rewrite getting-started.md with step-by-step Dockge deployment
- Add DuckDNS signup instructions
- Add detailed core infrastructure deployment (DuckDNS, Traefik, Authelia, Dockge)
- Add Homepage configuration steps
- Add API key configuration for widgets
- Add troubleshooting section
- Add security checklist

- Create docs/services-reference.md - comprehensive catalog of all 60+ services
- Organized by category with descriptions
- Access URLs and SSO requirements
- Storage recommendations
- Quick deployment guide
- Summary table by stack

All documentation now accurately reflects the complete homelab setup with Dockge, Traefik, Authelia, Gluetun, Homepage, and all 40+ services across 10 stack files.

Co-authored-by: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-01-12 01:16:12 +00:00
parent 63f39e51a4
commit 6fec6b501e
4 changed files with 1297 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files

557
README.md
View File

@@ -4,16 +4,23 @@ AI-Powered Homelab Administration with GitHub Copilot
## Overview ## Overview
This repository provides a structured approach to managing a homelab infrastructure using Docker Compose, with integrated AI assistance through GitHub Copilot. The AI assistant is specifically trained to help you create, modify, and manage Docker services while maintaining consistency across your entire server stack. This repository provides a comprehensive, production-ready homelab infrastructure using Docker Compose with Dockge, featuring 40+ pre-configured services. Integrated AI assistance through GitHub Copilot helps you create, modify, and manage Docker services while maintaining consistency across your entire server stack.
The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia for Single Sign-On, Gluetun for VPN routing, and DuckDNS for dynamic DNS - all managed through file-based configurations that the AI can modify.
## Features ## Features
- **AI-Powered Management**: GitHub Copilot integration with specialized instructions for Docker service management - **AI-Powered Management**: GitHub Copilot integration with specialized instructions for Docker service management
- **Docker Compose First**: All persistent services defined in organized compose files - **Dockge Structure**: All stacks organized in `/opt/stacks/` for easy management via Dockge
- **Consistent Structure**: Standardized naming conventions and patterns across all services - **40+ Pre-configured Services**: Production-ready compose files across infrastructure, media, home automation, productivity, and monitoring
- **Traefik Reverse Proxy**: Automatic HTTPS with Let's Encrypt via file-based configuration (no web UI needed)
- **Authelia SSO**: Single Sign-On protection for all admin interfaces with smart bypass rules for media apps
- **Gluetun VPN**: Surfshark WireGuard integration for secure downloads
- **Homepage Dashboard**: AI-configurable dashboard with Docker integration and service widgets
- **External Host Proxying**: Proxy external services (Raspberry Pi, routers, NAS) through Traefik
- **Stack-Aware Changes**: AI considers the entire infrastructure when making changes - **Stack-Aware Changes**: AI considers the entire infrastructure when making changes
- **Comprehensive Documentation**: Detailed guidelines and examples for all operations - **Comprehensive Documentation**: Detailed guidelines including proxying external hosts
- **Example Services**: Ready-to-use compose files for common homelab services - **File-Based Configuration**: Everything managed via YAML files - no web UI dependencies
## Quick Start ## Quick Start
@@ -23,8 +30,10 @@ This repository provides a structured approach to managing a homelab infrastruct
- Docker Compose V2 - Docker Compose V2
- Git - Git
- VS Code with GitHub Copilot extension (for AI assistance) - VS Code with GitHub Copilot extension (for AI assistance)
- A domain from DuckDNS (free)
- Surfshark VPN account (optional, for VPN features)
### Initial Setup ### Quick Setup (Dockge Structure)
1. **Clone the repository:** 1. **Clone the repository:**
```bash ```bash
@@ -36,45 +45,90 @@ This repository provides a structured approach to managing a homelab infrastruct
2. **Create environment file:** 2. **Create environment file:**
```bash ```bash
cp .env.example .env cp .env.example .env
# Edit .env with your values # Edit .env with your values (domain, API keys, passwords)
nano .env nano .env
``` ```
3. **Create the main network:** 3. **Create required directories:**
```bash
sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
```
4. **Create Docker networks:**
```bash ```bash
docker network create homelab-network docker network create homelab-network
docker network create traefik-network
docker network create media-network
``` ```
4. **Create config directories:** 5. **Deploy core infrastructure (in order):**
```bash ```bash
mkdir -p config # 1. DuckDNS (Dynamic DNS)
mkdir -p /opt/stacks/duckdns && cp docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
cd /opt/stacks/duckdns && docker compose up -d
# 2. Traefik (Reverse Proxy with SSL)
mkdir -p /opt/stacks/traefik/dynamic
cp docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
cp config-templates/traefik/* /opt/stacks/traefik/
cd /opt/stacks/traefik && docker compose up -d
# 3. Authelia (SSO)
mkdir -p /opt/stacks/authelia
cp docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
cp config-templates/authelia/* /opt/stacks/authelia/
cd /opt/stacks/authelia && docker compose up -d
# 4. Dockge (Stack Manager)
mkdir -p /opt/stacks/infrastructure
cp docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
cd /opt/stacks/infrastructure && docker compose up -d dockge
``` ```
5. **Start your first service:** 6. **Access Dockge:**
```bash Open `https://dockge.yourdomain.duckdns.org` (use Authelia login)
# Example: Start Portainer for container management
docker compose -f docker-compose/infrastructure.yml up -d portainer
```
6. **Access the service:** Now deploy remaining stacks through Dockge's UI!
Open `http://your-server-ip:9000` in your browser
## Repository Structure ## Repository Structure
``` ```
AI-Homelab/ AI-Homelab/
├── .github/ ├── .github/
│ └── copilot-instructions.md # AI assistant guidelines │ └── copilot-instructions.md # AI assistant guidelines (Dockge, Traefik, Authelia aware)
├── docker-compose/ ├── docker-compose/
│ ├── infrastructure.yml # Core services (proxy, DNS, Portainer) │ ├── traefik.yml # Reverse proxy (deploy first)
│ ├── media.yml # Media services (Plex, Sonarr, Radarr) │ ├── authelia.yml # SSO authentication
│ ├── monitoring.yml # Observability (Prometheus, Grafana) │ ├── duckdns.yml # Dynamic DNS
│ ├── development.yml # Dev tools (code-server, databases) │ ├── gluetun.yml # VPN client (Surfshark) + qBittorrent
│ └── README.md # Docker Compose documentation │ ├── infrastructure.yml # Dockge, Portainer, Pi-hole, Watchtower, Dozzle, Glances
│ ├── dashboards.yml # Homepage, Homarr
│ ├── media.yml # Plex, Jellyfin, Sonarr, Radarr, Prowlarr
│ ├── media-extended.yml # Readarr, Lidarr, Lazy Librarian, Mylar3, Calibre-Web,
│ │ # Jellyseerr, FlareSolverr, Tdarr, Unmanic
│ ├── homeassistant.yml # Home Assistant, ESPHome, TasmoAdmin, Node-RED,
│ │ # Mosquitto, Zigbee2MQTT, MotionEye
│ ├── productivity.yml # Nextcloud, Mealie, WordPress, Gitea, DokuWiki,
│ │ # BookStack, MediaWiki (all with databases)
│ ├── utilities.yml # Backrest, Duplicati, Uptime Kuma, Code Server,
│ │ # Form.io, Authelia-Redis
│ ├── monitoring.yml # Prometheus, Grafana, Loki, Promtail, cAdvisor
│ ├── development.yml # GitLab, PostgreSQL, Redis, pgAdmin, Jupyter
│ └── README-dockge.md # Dockge deployment guide
├── config-templates/
│ ├── traefik/ # Traefik static and dynamic configs
│ ├── authelia/ # Authelia config and user database
│ ├── homepage/ # Homepage dashboard configs (with widgets)
│ ├── prometheus/ # Prometheus scrape configs
│ ├── loki/ # Loki log aggregation config
│ └── ... # Other service templates
├── docs/ ├── docs/
│ └── docker-guidelines.md # Comprehensive Docker guidelines │ ├── docker-guidelines.md # Comprehensive Docker guidelines
├── config/ # Service configurations (gitignored) │ ├── getting-started.md # Step-by-step setup guide
├── .env.example # Environment variable template │ ├── quick-reference.md # Command reference
│ └── proxying-external-hosts.md # Guide for proxying Raspberry Pi, routers, etc.
├── .env.example # Environment variable template (40+ vars)
├── .gitignore # Git ignore patterns ├── .gitignore # Git ignore patterns
└── README.md # This file └── README.md # This file
``` ```
@@ -87,193 +141,384 @@ AI-Homelab/
2. **Open this repository** in VS Code 2. **Open this repository** in VS Code
3. **Start Copilot Chat** and ask questions like: 3. **Start Copilot Chat** and ask questions like:
- "Help me add a new media service to my homelab" - "Help me add a new media service to my homelab"
- "Create a docker-compose file for Home Assistant" - "Configure Traefik routing for my new service"
- "How do I configure GPU support for Plex?" - "Add Authelia SSO protection to this service"
- "Check my compose file for port conflicts" - "How do I proxy my Raspberry Pi through Traefik?"
- "Create a Homepage widget for this service"
- "Route this download client through Gluetun VPN"
The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md` to: The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md` to:
- Use `/opt/stacks/` directory structure (Dockge compatible)
- Configure Traefik labels for automatic routing
- Apply Authelia middleware where appropriate
- Suggest `/mnt/` for large data storage
- Add services to Homepage dashboard with widgets
- Maintain consistency with existing services - Maintain consistency with existing services
- Use Docker Compose for all persistent services
- Consider the entire stack when making changes - Consider the entire stack when making changes
- Follow naming conventions and best practices
### Example Interactions ### Example Interactions
**Creating a new service:** **Adding a new service:**
``` ```
You: "I want to add Home Assistant to my homelab" You: "Add Tautulli to monitor my Plex server"
Copilot: [Analyzes existing stack, checks for conflicts, creates compose configuration] Copilot: [Creates compose configuration with]:
- Checks available ports - /opt/stacks/tautulli/ directory structure
- Uses consistent naming - Traefik labels for HTTPS access
- Connects to appropriate networks - Authelia middleware for SSO protection
- Follows established patterns - Homepage dashboard entry with widget
- Connection to existing Plex service
``` ```
**Modifying a service:** **Proxying external service:**
``` ```
You: "Add GPU support to my Plex container" You: "Proxy my Raspberry Pi Home Assistant through Traefik"
Copilot: [Reviews current Plex configuration and updates it] Copilot: [Creates Traefik route configuration]:
- Checks if NVIDIA runtime is available - File in /opt/stacks/traefik/dynamic/
- Updates device mappings - HTTPS with Let's Encrypt
- Adds required environment variables - Authelia bypass (HA has its own auth)
- Maintains existing configuration - WebSocket support
- Homepage dashboard entry
```
**Configuring VPN routing:**
```
You: "Route SABnzbd through the VPN"
Copilot: [Updates compose to use Gluetun]:
- network_mode: "service:gluetun"
- Exposes ports through Gluetun
- Maintains Traefik routing
- Updates documentation
``` ```
## Available Service Stacks ## Available Service Stacks
### Infrastructure (`infrastructure.yml`) ### Core Infrastructure (Required)
- **Nginx Proxy Manager**: Web-based reverse proxy with SSL
- **Pi-hole**: Network-wide ad blocking and DNS
- **Portainer**: Docker container management UI
- **Watchtower**: Automatic container updates
### Media (`media.yml`) #### 1. Traefik (`traefik.yml`)
- **Plex**: Media streaming server **Reverse proxy with automatic SSL** - Deploy first!
- **Jellyfin**: Open-source media server alternative - Automatic HTTPS via Let's Encrypt
- **Sonarr**: TV show automation - File-based and Docker label routing
- **Radarr**: Movie automation - HTTP to HTTPS redirect
- **Prowlarr**: Indexer manager - Dashboard at `https://traefik.${DOMAIN}`
- **qBittorrent**: Torrent client
#### 2. Authelia (`authelia.yml`)
**Single Sign-On authentication**
- TOTP 2FA support
- LDAP/file-based user database
- Smart bypass rules for media apps
- Login at `https://auth.${DOMAIN}`
#### 3. DuckDNS (`duckdns.yml`)
**Dynamic DNS updater**
- Automatic IP updates
- Integrates with Let's Encrypt
- No web UI - runs silently
#### 4. Gluetun (`gluetun.yml`)
**VPN client (Surfshark WireGuard)**
- Includes qBittorrent
- Download via `https://qbit.${DOMAIN}`
- Easy to route other services through VPN
### Infrastructure Tools (`infrastructure.yml`)
- **Dockge**: Docker Compose stack manager (PRIMARY) - `https://dockge.${DOMAIN}`
- **Portainer**: Docker management UI (secondary) - `https://portainer.${DOMAIN}`
- **Pi-hole**: Network-wide ad blocking - `https://pihole.${DOMAIN}`
- **Watchtower**: Automatic container updates
- **Dozzle**: Real-time Docker logs - `https://dozzle.${DOMAIN}`
- **Glances**: System monitoring - `https://glances.${DOMAIN}`
- **Docker Proxy**: Secure Docker socket access
### Dashboards (`dashboards.yml`)
- **Homepage**: AI-configurable dashboard with widgets - `https://home.${DOMAIN}`
- Docker integration (container status)
- Service widgets (Sonarr, Radarr, Plex, Jellyfin, etc.)
- 11 organized categories
- **Homarr**: Modern alternative dashboard - `https://homarr.${DOMAIN}`
### Media Services (`media.yml`)
- **Plex**: Media streaming server - `https://plex.${DOMAIN}` (no SSO - app access)
- **Jellyfin**: Open-source media server - `https://jellyfin.${DOMAIN}` (no SSO - app access)
- **Sonarr**: TV show automation - `https://sonarr.${DOMAIN}`
- **Radarr**: Movie automation - `https://radarr.${DOMAIN}`
- **Prowlarr**: Indexer manager - `https://prowlarr.${DOMAIN}`
- **qBittorrent**: Torrent client (via VPN) - See gluetun.yml
### Extended Media (`media-extended.yml`)
- **Readarr**: Ebook/audiobook management - `https://readarr.${DOMAIN}`
- **Lidarr**: Music collection manager - `https://lidarr.${DOMAIN}`
- **Lazy Librarian**: Book download automation - `https://lazylibrarian.${DOMAIN}`
- **Mylar3**: Comic book manager - `https://mylar.${DOMAIN}`
- **Calibre-Web**: Ebook reader and server - `https://calibre.${DOMAIN}`
- **Jellyseerr**: Media request management - `https://jellyseerr.${DOMAIN}` (no SSO)
- **FlareSolverr**: Cloudflare bypass (no UI)
- **Tdarr**: Distributed transcoding - `https://tdarr.${DOMAIN}`
- **Unmanic**: Library optimizer - `https://unmanic.${DOMAIN}`
### Home Automation (`homeassistant.yml`)
- **Home Assistant**: Home automation hub - `https://ha.${DOMAIN}` (uses host network)
- **ESPHome**: ESP device manager - `https://esphome.${DOMAIN}`
- **TasmoAdmin**: Tasmota device manager - `https://tasmoadmin.${DOMAIN}`
- **Node-RED**: Flow automation - `https://nodered.${DOMAIN}`
- **Mosquitto**: MQTT broker (no UI)
- **Zigbee2MQTT**: Zigbee bridge - `https://zigbee2mqtt.${DOMAIN}`
- **MotionEye**: Video surveillance - `https://motioneye.${DOMAIN}`
### Productivity (`productivity.yml`)
- **Nextcloud**: File sync & collaboration - `https://nextcloud.${DOMAIN}`
- Includes MariaDB database
- **Mealie**: Recipe manager - `https://mealie.${DOMAIN}` (no SSO)
- **WordPress**: Blog platform - `https://blog.${DOMAIN}` (no SSO - public)
- Includes MariaDB database
- **Gitea**: Self-hosted Git - `https://git.${DOMAIN}`
- Includes PostgreSQL database
- **DokuWiki**: File-based wiki - `https://wiki.${DOMAIN}`
- **BookStack**: Documentation platform - `https://docs.${DOMAIN}`
- Includes MariaDB database
- **MediaWiki**: Wiki platform - `https://mediawiki.${DOMAIN}`
- Includes MariaDB database
### Utilities (`utilities.yml`)
- **Backrest**: Backup manager (restic) - `https://backrest.${DOMAIN}`
- **Duplicati**: Backup software - `https://duplicati.${DOMAIN}`
- **Uptime Kuma**: Status monitoring - `https://status.${DOMAIN}` (no SSO - public)
- **Code Server**: VS Code in browser - `https://code.${DOMAIN}`
- **Form.io**: Form builder - `https://forms.${DOMAIN}`
- Includes MongoDB database
- **Authelia-Redis**: Session storage (no UI)
### Monitoring (`monitoring.yml`) ### Monitoring (`monitoring.yml`)
- **Prometheus**: Metrics collection
- **Grafana**: Metrics visualization - **Prometheus**: Metrics collection - `https://prometheus.${DOMAIN}`
- **Node Exporter**: Host metrics - **Grafana**: Metrics visualization - `https://grafana.${DOMAIN}`
- **cAdvisor**: Container metrics
- **Uptime Kuma**: Service uptime monitoring
- **Loki**: Log aggregation - **Loki**: Log aggregation
- **Promtail**: Log shipping - **Promtail**: Log shipping
- **Node Exporter**: Host metrics
- **cAdvisor**: Container metrics
### Development (`development.yml`) ### Development (`development.yml`)
- **Code Server**: VS Code in browser
- **GitLab CE**: Self-hosted Git repository - **GitLab CE**: Git with CI/CD - `https://gitlab.${DOMAIN}`
- **PostgreSQL**: SQL database - **PostgreSQL**: SQL database
- **Redis**: In-memory data store - **Redis**: In-memory store
- **pgAdmin**: PostgreSQL UI - **pgAdmin**: PostgreSQL UI - `https://pgadmin.${DOMAIN}`
- **Jupyter Lab**: Interactive notebooks - **Jupyter Lab**: Interactive notebooks - `https://jupyter.${DOMAIN}`
- **Node-RED**: Visual automation
## Common Operations ## Common Operations
### Starting Services ### Managing Stacks via Dockge
Access Dockge at `https://dockge.${DOMAIN}` to:
- View all stacks and their status
- Start/stop/restart stacks
- Edit compose files directly
- View logs
- Deploy new stacks
### Command Line Operations
#### Starting Services (Dockge Structure)
Start all services in a compose file:
```bash ```bash
docker compose -f docker-compose/infrastructure.yml up -d # Start entire stack
cd /opt/stacks/media
docker compose up -d
# Start specific services
cd /opt/stacks/media
docker compose up -d sonarr radarr
# Start with rebuild
cd /opt/stacks/infrastructure
docker compose up -d --build
``` ```
Start specific services: #### Stopping Services
```bash ```bash
docker compose -f docker-compose/media.yml up -d plex sonarr radarr # Stop entire stack
cd /opt/stacks/media
docker compose down
# Stop but keep volumes
cd /opt/stacks/media
docker compose stop
# Stop specific service
cd /opt/stacks/media
docker compose stop plex
``` ```
### Stopping Services #### Viewing Logs
Stop all services:
```bash ```bash
docker compose -f docker-compose/infrastructure.yml down # Follow logs for entire stack
cd /opt/stacks/media
docker compose logs -f
# Follow logs for specific service
cd /opt/stacks/media
docker compose logs -f plex
# View last 100 lines
cd /opt/stacks/media
docker compose logs --tail=100 plex
# Or use Dozzle web UI at https://dozzle.${DOMAIN}
``` ```
Stop specific service: #### Updating Services
```bash ```bash
docker compose -f docker-compose/media.yml stop plex # Pull latest images
``` cd /opt/stacks/media
docker compose pull
### Viewing Logs # Update specific service
cd /opt/stacks/media
docker compose pull plex
docker compose up -d plex
Follow logs for a service: # Or enable Watchtower for automatic updates
```bash
docker compose -f docker-compose/media.yml logs -f plex
```
View last 100 lines:
```bash
docker compose -f docker-compose/media.yml logs --tail=100 plex
```
### Updating Services
Pull latest images:
```bash
docker compose -f docker-compose/media.yml pull
```
Update specific service:
```bash
docker compose -f docker-compose/media.yml pull plex
docker compose -f docker-compose/media.yml up -d plex
``` ```
### Testing with Docker Run ### Testing with Docker Run
Test NVIDIA GPU support: Use `docker run` only for temporary testing:
```bash
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
```
Test a new image:
```bash ```bash
# Test NVIDIA GPU support
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
# Test a new image
docker run --rm -it alpine:latest /bin/sh docker run --rm -it alpine:latest /bin/sh
# Test VPN connection through Gluetun
docker run --rm --network container:gluetun curlimages/curl ifconfig.me
``` ```
## Network Architecture ## Network Architecture
All services connect to networks for inter-service communication: Services connect to multiple networks for organization and security:
- **homelab-network**: Main network for all services - **traefik-network**: For Traefik to reach services (external)
- **media-network**: Isolated network for media stack - **homelab-network**: Main network for inter-service communication (external)
- **monitoring-network**: Network for observability stack - **media-network**: Isolated network for media stack (external)
- **database-network**: Isolated network for databases - **monitoring-network**: Network for observability stack (created per stack)
- **database-network**: Isolated networks for database services (created per stack)
- **dockerproxy-network**: Secure Docker socket access (created in infrastructure)
### Creating Required Networks
Create networks manually:
```bash ```bash
# Create external networks (do this once)
docker network create traefik-network
docker network create homelab-network docker network create homelab-network
docker network create media-network docker network create media-network
docker network create monitoring-network
docker network create database-network # Stack-specific networks are created automatically by compose files
``` ```
### Traefik Routing
All services accessed via Traefik with automatic HTTPS:
- Pattern: `https://service.yourdomain.duckdns.org`
- Configured via Docker labels on each service
- SSL certificates automatically managed
- No port exposure needed (except Traefik 80/443)
## Documentation ## Documentation
### Comprehensive Guides ### Comprehensive Guides
- **[Docker Guidelines](docs/docker-guidelines.md)**: Complete guide to Docker service management - **[Docker Guidelines](docs/docker-guidelines.md)**: Complete guide to Docker service management with Dockge
- **[Docker Compose README](docker-compose/README.md)**: Compose-specific documentation - **[Getting Started](docs/getting-started.md)**: Step-by-step setup walkthrough
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines - **[Quick Reference](docs/quick-reference.md)**: Command reference and troubleshooting
- **[Dockge Deployment](docker-compose/README-dockge.md)**: Dockge-specific deployment guide
- **[Proxying External Hosts](docs/proxying-external-hosts.md)**: Guide for proxying Raspberry Pi, routers, NAS via Traefik
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines (Traefik, Authelia, Dockge aware)
### Key Principles ### Key Principles
1. **Docker Compose First**: Always use compose for persistent services 1. **Dockge Structure**: All stacks in `/opt/stacks/stack-name/`
2. **Docker Run for Testing**: Only use `docker run` for temporary containers 2. **Docker Compose First**: Always use compose for persistent services
3. **Consistency**: Follow established patterns and naming conventions 3. **Docker Run for Testing**: Only use `docker run` for temporary containers
4. **Stack Awareness**: Consider dependencies and interactions 4. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
5. **Documentation**: Comment complex configurations 5. **Traefik for All**: Every service routed through Traefik with automatic SSL
6. **Security**: Keep secrets in `.env` files, never commit them 6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps for device access
7. **VPN When Needed**: Route download clients through Gluetun
8. **Large Data Separate**: Use `/mnt/` for media, downloads, large databases
9. **Stack Awareness**: Consider dependencies and interactions
10. **Security**: Keep secrets in `.env` files, never commit them
## Configuration Management ## Configuration Management
### Environment Variables ### Environment Variables
All services use variables from `.env`: All services use variables from `.env` in each stack directory:
- `PUID`/`PGID`: User/group IDs for file permissions - `PUID`/`PGID`: User/group IDs for file permissions
- `TZ`: Timezone for all services - `TZ`: Timezone for all services
- `DOMAIN`: Your DuckDNS domain (e.g., yourdomain.duckdns.org)
- `SERVER_IP`: Your server's IP address - `SERVER_IP`: Your server's IP address
- Service-specific credentials and paths - Service-specific credentials and API keys
- Homepage widget API keys (40+ variables)
### Config Files See `.env.example` for complete list.
Service configurations are stored in `config/service-name/`: ### Storage Strategy
**Small Data** (configs, databases < 10GB): `/opt/stacks/stack-name/`
```yaml
volumes:
- /opt/stacks/sonarr/config:/config
``` ```
config/
├── plex/ # Plex configuration **Large Data** (media, downloads, backups): `/mnt/`
├── sonarr/ # Sonarr configuration ```yaml
├── grafana/ # Grafana dashboards volumes:
- /mnt/media:/media
- /mnt/downloads:/downloads
- /mnt/backups:/backups
```
The AI will suggest when to use `/mnt/` based on expected data size.
### Configuration Files
Service configurations stored in stack directories:
```
/opt/stacks/
├── traefik/
│ ├── docker-compose.yml
│ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes
│ │ └── routes.yml
│ └── acme.json # SSL certificates
├── authelia/
│ ├── docker-compose.yml
│ ├── configuration.yml # Authelia settings
│ └── users_database.yml # User accounts
├── homepage/
│ ├── docker-compose.yml
│ └── config/
│ ├── services.yaml # Service definitions
│ ├── docker.yaml # Docker integration
│ ├── settings.yaml # Dashboard settings
│ └── widgets.yaml # Homepage widgets
└── ... └── ...
``` ```
**Note**: Config directories are gitignored to prevent committing sensitive data. Templates available in `config-templates/` directory.
## Security Best Practices ## Security Best Practices
@@ -362,15 +607,51 @@ This project is provided as-is for personal homelab use.
## Getting Started Checklist ## Getting Started Checklist
- [ ] Install Docker and Docker Compose - [ ] Install Docker and Docker Compose V2
- [ ] Sign up for DuckDNS (free) and get your domain
- [ ] Get Surfshark VPN credentials (optional, for VPN features)
- [ ] Clone this repository - [ ] Clone this repository
- [ ] Copy `.env.example` to `.env` and configure - [ ] Copy `.env.example` to `.env` and configure all values
- [ ] Create `homelab-network`: `docker network create homelab-network` - [ ] Create `/opt/stacks` directory: `sudo mkdir -p /opt/stacks && sudo chown $USER:$USER /opt/stacks`
- [ ] Start infrastructure services: `docker compose -f docker-compose/infrastructure.yml up -d` - [ ] Create Docker networks: `docker network create traefik-network homelab-network media-network`
- [ ] Access Portainer at `http://server-ip:9000` - [ ] Deploy DuckDNS stack
- [ ] Deploy Traefik stack (with config templates)
- [ ] Deploy Authelia stack (with config templates)
- [ ] Deploy infrastructure stack (Dockge)
- [ ] Access Dockge at `https://dockge.${DOMAIN}` and deploy remaining stacks
- [ ] Configure Homepage dashboard (copy templates to /opt/stacks/homepage/config/)
- [ ] Install VS Code with GitHub Copilot extension - [ ] Install VS Code with GitHub Copilot extension
- [ ] Open repository in VS Code and start using AI assistance - [ ] Open repository in VS Code and start using AI assistance
- [ ] Add more services as needed using AI guidance
## Proxying External Hosts
You can proxy services running on other devices (Raspberry Pi, routers, NAS) through Traefik:
**Example: Raspberry Pi Home Assistant**
```yaml
# In /opt/stacks/traefik/dynamic/external.yml
http:
routers:
ha-pi:
rule: "Host(`ha.yourdomain.duckdns.org`)"
entryPoints:
- websecure
service: ha-pi
tls:
certResolver: letsencrypt
services:
ha-pi:
loadBalancer:
servers:
- url: "http://192.168.1.50:8123"
```
See [docs/proxying-external-hosts.md](docs/proxying-external-hosts.md) for complete guide including:
- Three methods (file provider, Docker labels, hybrid)
- Authelia bypass configuration
- WebSocket support
- Examples for routers, NAS, cameras, Proxmox
## Support ## Support

182
docs/docker-guidelines.md
View File

@@ -2,40 +2,184 @@
## Overview ## Overview
This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure. This document provides comprehensive guidelines for managing Docker services in your AI-powered homelab using Dockge, Traefik, and Authelia. These guidelines ensure consistency, maintainability, and reliability across your entire infrastructure.
## Table of Contents ## Table of Contents
1. [Philosophy](#philosophy) 1. [Philosophy](#philosophy)
2. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run) 2. [Dockge Structure](#dockge-structure)
3. [Service Creation Guidelines](#service-creation-guidelines) 3. [Traefik and Authelia Integration](#traefik-and-authelia-integration)
4. [Service Modification Guidelines](#service-modification-guidelines) 4. [Docker Compose vs Docker Run](#docker-compose-vs-docker-run)
5. [Naming Conventions](#naming-conventions) 5. [Service Creation Guidelines](#service-creation-guidelines)
6. [Network Architecture](#network-architecture) 6. [Service Modification Guidelines](#service-modification-guidelines)
7. [Volume Management](#volume-management) 7. [Naming Conventions](#naming-conventions)
8. [Security Best Practices](#security-best-practices) 8. [Network Architecture](#network-architecture)
9. [Monitoring and Logging](#monitoring-and-logging) 9. [Volume Management](#volume-management)
10. [Troubleshooting](#troubleshooting) 10. [Security Best Practices](#security-best-practices)
11. [Monitoring and Logging](#monitoring-and-logging)
12. [Troubleshooting](#troubleshooting)
## Philosophy ## Philosophy
### Core Principles ### Core Principles
1. **Infrastructure as Code**: All services must be defined in Docker Compose files 1. **Dockge First**: Manage all stacks through Dockge in `/opt/stacks/`
2. **Reproducibility**: Any service should be rebuildable from compose files 2. **Infrastructure as Code**: All services defined in Docker Compose files
3. **Documentation**: Every non-obvious configuration must be commented 3. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
4. **Consistency**: Use the same patterns across all services 4. **Reproducibility**: Any service should be rebuildable from compose files
5. **Safety First**: Always test changes in isolation before deploying 5. **Automatic HTTPS**: All services routed through Traefik with Let's Encrypt
6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps
7. **Documentation**: Every non-obvious configuration must be commented
8. **Consistency**: Use the same patterns across all services
9. **Safety First**: Always test changes in isolation before deploying
### The Stack Mindset ### The Stack Mindset
Think of your homelab as an interconnected stack where: Think of your homelab as an interconnected stack where:
- Services depend on networks - Services depend on networks (especially traefik-network)
- Networks connect services - Traefik routes all traffic with automatic SSL
- Volumes persist data - Authelia protects sensitive services
- VPN (Gluetun) secures downloads
- Changes ripple through the system - Changes ripple through the system
Always ask: "How does this change affect other services?" Always ask: "How does this change affect other services and routing?"
## Dockge Structure
### Directory Organization
All stacks live in `/opt/stacks/stack-name/`:
```
/opt/stacks/
├── traefik/
│ ├── docker-compose.yml
│ ├── traefik.yml # Static config
│ ├── dynamic/ # Dynamic routes
│ │ ├── routes.yml
│ │ └── external.yml # External host proxying
│ ├── acme.json # SSL certificates (chmod 600)
│ └── .env
├── authelia/
│ ├── docker-compose.yml
│ ├── configuration.yml # Authelia settings
│ ├── users_database.yml # User accounts
│ └── .env
├── media/
│ ├── docker-compose.yml
│ └── .env
└── ...
```
### Why Dockge?
- **Visual Management**: Web UI at `https://dockge.${DOMAIN}`
- **Direct File Editing**: Edit compose files in-place
- **Stack Organization**: Each service stack is independent
- **AI Compatible**: Files can be managed by AI
- **Git Integration**: Easy to version control
### Storage Strategy
**Small Data** (configs, DBs < 10GB): `/opt/stacks/stack-name/`
```yaml
volumes:
- /opt/stacks/sonarr/config:/config
```
**Large Data** (media, downloads, backups): `/mnt/`
```yaml
volumes:
- /mnt/media/movies:/movies
- /mnt/media/tv:/tv
- /mnt/downloads:/downloads
- /mnt/backups:/backups
```
AI will suggest `/mnt/` when data may exceed 50GB or grow continuously.
## Traefik and Authelia Integration
### Every Service Needs Traefik Labels
Standard pattern for all services:
```yaml
services:
myservice:
image: myimage:latest
container_name: myservice
networks:
- homelab-network
- traefik-network # Required for Traefik
labels:
# Enable Traefik
- "traefik.enable=true"
# Define routing rule
- "traefik.http.routers.myservice.rule=Host(`myservice.${DOMAIN}`)"
# Use websecure entrypoint (HTTPS)
- "traefik.http.routers.myservice.entrypoints=websecure"
# Enable Let's Encrypt
- "traefik.http.routers.myservice.tls.certresolver=letsencrypt"
# Add Authelia SSO (if needed)
- "traefik.http.routers.myservice.middlewares=authelia@docker"
# Specify port (if not default 80)
- "traefik.http.services.myservice.loadbalancer.server.port=8080"
```
### When to Use Authelia SSO
**Protect with Authelia**:
- Admin interfaces (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure tools (Portainer, Dockge, Grafana)
- Personal data (Nextcloud, Mealie, wikis)
- Development tools (code-server, GitLab)
- Monitoring dashboards
**Bypass Authelia**:
- Media servers (Plex, Jellyfin) - need app access
- Request services (Jellyseerr) - family-friendly access
- Public services (WordPress, status pages)
- Services with their own auth (Home Assistant)
Configure bypasses in `/opt/stacks/authelia/configuration.yml`:
```yaml
access_control:
rules:
- domain: jellyfin.yourdomain.duckdns.org
policy: bypass
- domain: plex.yourdomain.duckdns.org
policy: bypass
```
### Routing Through VPN (Gluetun)
For services that need VPN (downloads):
```yaml
services:
mydownloader:
image: downloader:latest
container_name: mydownloader
network_mode: "service:gluetun" # Route through VPN
depends_on:
- gluetun
```
Expose ports through Gluetun's compose file:
```yaml
# In gluetun.yml
gluetun:
ports:
- "8080:8080" # mydownloader web UI
```
## Docker Compose vs Docker Run ## Docker Compose vs Docker Run

595
docs/getting-started.md
View File

@@ -1,6 +1,6 @@
# Getting Started Guide # Getting Started Guide
This guide will walk you through setting up your AI-powered homelab from scratch. This guide will walk you through setting up your AI-powered homelab with Dockge, Traefik, Authelia, and 40+ services from scratch.
## Prerequisites ## Prerequisites
@@ -11,8 +11,10 @@ Before you begin, ensure you have:
- [ ] Docker Compose V2 installed - [ ] Docker Compose V2 installed
- [ ] Git installed - [ ] Git installed
- [ ] At least 8GB RAM (16GB+ recommended) - [ ] At least 8GB RAM (16GB+ recommended)
- [ ] Sufficient disk space (100GB+ recommended) - [ ] Sufficient disk space (100GB+ system, 1TB+ for media recommended)
- [ ] Static IP address for your server - [ ] Static IP address for your server (or DHCP reservation)
- [ ] DuckDNS account (free) with a domain
- [ ] Surfshark VPN account (optional, for VPN features)
- [ ] VS Code with GitHub Copilot extension (for AI assistance) - [ ] VS Code with GitHub Copilot extension (for AI assistance)
## Step 1: Verify Docker Installation ## Step 1: Verify Docker Installation
@@ -45,7 +47,15 @@ git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab cd AI-Homelab
``` ```
## Step 3: Configure Environment Variables ## Step 3: Sign Up for DuckDNS
1. Go to https://www.duckdns.org/
2. Sign in with your preferred method
3. Create a domain (e.g., `myhomelab`)
4. Copy your token - you'll need it for `.env`
5. Your domain will be: `myhomelab.duckdns.org`
## Step 4: Configure Environment Variables
```bash ```bash
# Copy the example environment file # Copy the example environment file
@@ -59,7 +69,7 @@ id -g # This is your PGID
nano .env nano .env
``` ```
**Update these values in `.env`:** **Critical values to update in `.env`:**
```bash ```bash
# Your user/group IDs # Your user/group IDs
PUID=1000 # Replace with your user ID PUID=1000 # Replace with your user ID
@@ -71,442 +81,333 @@ TZ=America/New_York
# Your server's IP address # Your server's IP address
SERVER_IP=192.168.1.100 # Replace with your actual IP SERVER_IP=192.168.1.100 # Replace with your actual IP
# Directory paths # DuckDNS Configuration
USERDIR=/home/yourusername/homelab # Update username DOMAIN=myhomelab.duckdns.org # Your DuckDNS domain
MEDIADIR=/mnt/media # Update if different DUCKDNS_TOKEN=your-duckdns-token-here
DOWNLOADDIR=/mnt/downloads # Update if different DUCKDNS_SUBDOMAINS=myhomelab # Without .duckdns.org
# Set secure passwords for services # Let's Encrypt Email
GRAFANA_ADMIN_PASSWORD=your-secure-password-here ACME_EMAIL=your-email@example.com
CODE_SERVER_PASSWORD=your-secure-password-here
POSTGRES_PASSWORD=your-secure-password-here # Authelia Secrets (generate with: openssl rand -hex 64)
PGADMIN_PASSWORD=your-secure-password-here AUTHELIA_JWT_SECRET=$(openssl rand -hex 64)
JUPYTER_TOKEN=your-secure-token-here AUTHELIA_SESSION_SECRET=$(openssl rand -hex 64)
PIHOLE_PASSWORD=your-secure-password-here AUTHELIA_STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 64)
# Surfshark VPN (if using)
SURFSHARK_PRIVATE_KEY=your-wireguard-private-key
SURFSHARK_ADDRESSES=10.14.0.2/16
# Set secure passwords for all services
PIHOLE_PASSWORD=your-secure-password
GRAFANA_ADMIN_PASSWORD=your-secure-password
CODE_SERVER_PASSWORD=your-secure-password
# ... (see .env.example for complete list)
``` ```
**Save and exit** (Ctrl+X, Y, Enter in nano) **Save and exit** (Ctrl+X, Y, Enter in nano)
## Step 4: Create Docker Networks ## Step 5: Create Dockge Directory Structure
```bash ```bash
# Create the main homelab network # Create main stacks directory
docker network create homelab-network sudo mkdir -p /opt/stacks
sudo chown -R $USER:$USER /opt/stacks
# Create additional networks for better security # Create mount points for large data (adjust as needed)
sudo mkdir -p /mnt/media/{movies,tv,music,books,photos}
sudo mkdir -p /mnt/downloads/{complete,incomplete}
sudo mkdir -p /mnt/backups
sudo chown -R $USER:$USER /mnt/media /mnt/downloads /mnt/backups
```
## Step 6: Create Docker Networks
```bash
# Create required external networks
docker network create traefik-network
docker network create homelab-network
docker network create media-network docker network create media-network
docker network create monitoring-network docker network create dockerproxy-network
docker network create database-network
# Verify networks were created # Verify networks were created
docker network ls docker network ls | grep -E "traefik|homelab|media|dockerproxy"
``` ```
## Step 5: Create Configuration Directories ## Step 7: Deploy Core Infrastructure (IN ORDER)
### 7.1 DuckDNS (Dynamic DNS)
```bash ```bash
# Create the main config directory # Create stack directory
mkdir -p config mkdir -p /opt/stacks/duckdns
# Create config directories for services you plan to use # Copy compose file
mkdir -p config/{nginx-proxy-manager,pihole,portainer} cp ~/AI-Homelab/docker-compose/duckdns.yml /opt/stacks/duckdns/docker-compose.yml
mkdir -p config/{plex,sonarr,radarr,prowlarr,qbittorrent,jellyfin}
mkdir -p config/{prometheus,grafana,loki,promtail}
mkdir -p config/{code-server,postgres,redis}
# Set proper permissions # Copy .env
sudo chown -R $(id -u):$(id -g) config/ cp ~/AI-Homelab/.env /opt/stacks/duckdns/.env
# Deploy
cd /opt/stacks/duckdns
docker compose up -d
# Verify it's working
docker compose logs -f
# Should see: "Your IP was updated to X.X.X.X"
``` ```
## Step 6: Copy Configuration Templates (Optional) ### 7.2 Traefik (Reverse Proxy with SSL)
For services that need config files:
```bash ```bash
# Prometheus # Create stack directory with dynamic configs
mkdir -p config/prometheus mkdir -p /opt/stacks/traefik/dynamic
cp config-templates/prometheus/prometheus.yml config/prometheus/
# Loki # Copy compose file
mkdir -p config/loki cp ~/AI-Homelab/docker-compose/traefik.yml /opt/stacks/traefik/docker-compose.yml
cp config-templates/loki/loki-config.yml config/loki/
# Promtail # Copy configuration templates
mkdir -p config/promtail cp ~/AI-Homelab/config-templates/traefik/traefik.yml /opt/stacks/traefik/
cp config-templates/promtail/promtail-config.yml config/promtail/ cp ~/AI-Homelab/config-templates/traefik/dynamic/*.yml /opt/stacks/traefik/dynamic/
# Redis # Create acme.json for SSL certificates
mkdir -p config/redis touch /opt/stacks/traefik/acme.json
cp config-templates/redis/redis.conf config/redis/ chmod 600 /opt/stacks/traefik/acme.json
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/traefik/.env
# Deploy
cd /opt/stacks/traefik
docker compose up -d
# Check logs
docker compose logs -f
# Should see Traefik starting and certificate resolver configured
``` ```
## Step 7: Start Your First Service (Portainer) ### 7.3 Authelia (SSO Authentication)
Portainer provides a web UI for managing Docker containers. It's a great first service to deploy.
```bash ```bash
# Start Portainer # Create stack directory
docker compose -f docker-compose/infrastructure.yml up -d portainer mkdir -p /opt/stacks/authelia
# Check if it's running # Copy compose file
docker compose -f docker-compose/infrastructure.yml ps cp ~/AI-Homelab/docker-compose/authelia.yml /opt/stacks/authelia/docker-compose.yml
# View logs # Copy configuration templates
docker compose -f docker-compose/infrastructure.yml logs -f portainer cp ~/AI-Homelab/config-templates/authelia/*.yml /opt/stacks/authelia/
# Generate password hash for users_database.yml
docker run --rm authelia/authelia:latest authelia crypto hash generate argon2 --password 'yourpassword'
# Copy the hash and edit users_database.yml
# Edit users_database.yml
cd /opt/stacks/authelia
nano users_database.yml
# Replace the password hash with your generated one
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/authelia/.env
# Deploy
docker compose up -d
# Check logs
docker compose logs -f
# Test login at https://auth.yourdomain.duckdns.org
``` ```
**Access Portainer:** ### 7.4 Infrastructure Services (Dockge)
1. Open your browser
2. Navigate to `http://YOUR_SERVER_IP:9000`
3. Create an admin account on first login
4. Select "Get Started" and choose local Docker environment
## Step 8: Deploy Infrastructure Services
```bash ```bash
# Start all infrastructure services # Create stack directory
docker compose -f docker-compose/infrastructure.yml up -d mkdir -p /opt/stacks/infrastructure
# Check status # Copy compose file
docker compose -f docker-compose/infrastructure.yml ps cp ~/AI-Homelab/docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
# Services now running: # Create necessary subdirectories
# - Nginx Proxy Manager: http://YOUR_SERVER_IP:81 mkdir -p /opt/dockge/data
# - Pi-hole: http://YOUR_SERVER_IP:8080/admin mkdir -p /opt/stacks/pihole/{etc-pihole,etc-dnsmasq.d}
# - Portainer: http://YOUR_SERVER_IP:9000 mkdir -p /opt/stacks/glances/config
# - Watchtower: (runs in background, no UI)
# Copy .env
cp ~/AI-Homelab/.env /opt/stacks/infrastructure/.env
# Deploy Dockge first
cd /opt/stacks/infrastructure
docker compose up -d dockge
# Access Dockge at https://dockge.yourdomain.duckdns.org
# Deploy remaining infrastructure services
docker compose up -d
``` ```
### Configure Nginx Proxy Manager (Optional) ## Step 8: Deploy Additional Stacks
1. Access: `http://YOUR_SERVER_IP:81` Now use Dockge UI at `https://dockge.yourdomain.duckdns.org` to deploy additional stacks, or continue with command line:
2. Login with default credentials:
- Email: `admin@example.com`
- Password: `changeme`
3. Change password immediately
4. Add proxy hosts for your services
### Configure Pi-hole (Optional) ### 8.1 Gluetun + qBittorrent (VPN)
1. Access: `http://YOUR_SERVER_IP:8080/admin`
2. Login with password from `.env` (PIHOLE_PASSWORD)
3. Configure DNS settings
4. Update your router to use Pi-hole as DNS server
## Step 9: Deploy Media Services (Optional)
If you want a media server setup:
```bash ```bash
# Start media services mkdir -p /opt/stacks/gluetun
docker compose -f docker-compose/media.yml up -d cp ~/AI-Homelab/docker-compose/gluetun.yml /opt/stacks/gluetun/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/gluetun/.env
# Check status cd /opt/stacks/gluetun
docker compose -f docker-compose/media.yml ps docker compose up -d
# Services now running: # Test VPN
# - Plex: http://YOUR_SERVER_IP:32400/web docker exec gluetun curl ifconfig.me
# - Jellyfin: http://YOUR_SERVER_IP:8096 # Should show VPN IP
# - Sonarr: http://YOUR_SERVER_IP:8989
# - Radarr: http://YOUR_SERVER_IP:7878
# - Prowlarr: http://YOUR_SERVER_IP:9696
# - qBittorrent: http://YOUR_SERVER_IP:8081
``` ```
### Initial Media Service Setup ### 8.2 Homepage Dashboard
**Plex:**
1. Access: `http://YOUR_SERVER_IP:32400/web`
2. Sign in with your Plex account
3. Add your media libraries
**Sonarr/Radarr:**
1. Access the web UI
2. Go to Settings → Profiles → Quality
3. Configure your quality preferences
4. Add Prowlarr as an indexer
5. Add qBittorrent as a download client
## Step 10: Deploy Monitoring Services (Optional)
For system and service monitoring:
```bash ```bash
# Start monitoring services mkdir -p /opt/stacks/homepage/config
docker compose -f docker-compose/monitoring.yml up -d cp ~/AI-Homelab/docker-compose/dashboards.yml /opt/stacks/homepage/docker-compose.yml
cp ~/AI-Homelab/config-templates/homepage/* /opt/stacks/homepage/config/
cp ~/AI-Homelab/.env /opt/stacks/homepage/.env
# Check status cd /opt/stacks/homepage
docker compose -f docker-compose/monitoring.yml ps docker compose up -d homepage
# Services now running: # Access at https://home.yourdomain.duckdns.org
# - Prometheus: http://YOUR_SERVER_IP:9090
# - Grafana: http://YOUR_SERVER_IP:3000
# - Node Exporter: http://YOUR_SERVER_IP:9100
# - cAdvisor: http://YOUR_SERVER_IP:8082
# - Uptime Kuma: http://YOUR_SERVER_IP:3001
``` ```
### Configure Grafana ### 8.3 Media Stack
1. Access: `http://YOUR_SERVER_IP:3000`
2. Login with credentials from `.env`:
- Username: `admin`
- Password: `GRAFANA_ADMIN_PASSWORD` from .env
3. Add Prometheus as a data source:
- URL: `http://prometheus:9090`
4. Import dashboards:
- Dashboard ID 1860 for Node Exporter
- Dashboard ID 893 for Docker metrics
### Configure Uptime Kuma
1. Access: `http://YOUR_SERVER_IP:3001`
2. Create an account on first login
3. Add monitors for your services
## Step 11: Deploy Development Services (Optional)
If you need development tools:
```bash ```bash
# Start development services mkdir -p /opt/stacks/media
docker compose -f docker-compose/development.yml up -d cp ~/AI-Homelab/docker-compose/media.yml /opt/stacks/media/docker-compose.yml
cp ~/AI-Homelab/.env /opt/stacks/media/.env
# Check status cd /opt/stacks/media
docker compose -f docker-compose/development.yml ps docker compose up -d
# Services now running:
# - Code Server: http://YOUR_SERVER_IP:8443
# - PostgreSQL: localhost:5432
# - Redis: localhost:6379
# - pgAdmin: http://YOUR_SERVER_IP:5050
# - Jupyter Lab: http://YOUR_SERVER_IP:8888
# - Node-RED: http://YOUR_SERVER_IP:1880
``` ```
## Step 12: Set Up VS Code with GitHub Copilot ### 8.4 Additional Stacks
1. **Install VS Code** on your local machine (if not already installed) Deploy as needed:
- `media-extended.yml` → `/opt/stacks/media-extended/`
- `homeassistant.yml` → `/opt/stacks/homeassistant/`
- `productivity.yml` → `/opt/stacks/productivity/`
- `utilities.yml` → `/opt/stacks/utilities/`
- `monitoring.yml` → `/opt/stacks/monitoring/`
- `development.yml` → `/opt/stacks/development/`
2. **Install GitHub Copilot extension:** ## Step 9: Configure Homepage Widgets
- Open VS Code
- Go to Extensions (Ctrl+Shift+X) Get API keys from each service and add to Homepage config:
- Search for "GitHub Copilot"
- Click Install
- Sign in with your GitHub account
3. **Clone your repository in VS Code:**
```bash ```bash
# On your local machine cd /opt/stacks/homepage/config
# Note: Replace 'kelinfoxy' with your username if you forked this repository nano services.yaml
git clone https://github.com/kelinfoxy/AI-Homelab.git
cd AI-Homelab # Get API keys:
code . # - Sonarr/Radarr/etc: Settings → General → API Key
# - Plex: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/
# - Jellyfin: Dashboard → API Keys
# Add to .env:
nano /opt/stacks/homepage/.env
# HOMEPAGE_VAR_SONARR_KEY=...
# HOMEPAGE_VAR_RADARR_KEY=...
# etc.
# Restart Homepage
cd /opt/stacks/homepage
docker compose restart
``` ```
4. **Start using AI assistance:** ## Step 10: Install VS Code and GitHub Copilot
- Open Copilot Chat (Ctrl+Shift+I or click the chat icon)
- The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md`
- Ask questions like:
- "Help me add Home Assistant to my homelab"
- "Create a backup script for my Docker volumes"
- "How do I configure GPU support for Plex?"
## Step 13: Verify Everything is Running
```bash ```bash
# Check all running containers # Install VS Code (if not already installed)
docker ps # Download from https://code.visualstudio.com/
# Check container health # Install GitHub Copilot extension
docker ps --format "table {{.Names}}\t{{.Status}}" # In VS Code: Extensions → Search "GitHub Copilot" → Install
# View resource usage # Open the repository
docker stats --no-stream code ~/AI-Homelab
# Check disk usage # Start using AI assistance!
docker system df
```
## Step 14: Set Up Backups
Create a backup script:
```bash
# Create a backup directory
mkdir -p ~/backups
# Create a simple backup script
cat > ~/backup-homelab.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=~/backups
DATE=$(date +%Y%m%d)
# Backup config directories
tar czf $BACKUP_DIR/config-$DATE.tar.gz ~/AI-Homelab/config/
# Backup .env file
cp ~/AI-Homelab/.env $BACKUP_DIR/.env-$DATE
# Backup Docker volumes (example for Portainer)
docker run --rm \
-v portainer-data:/data \
-v $BACKUP_DIR:/backup \
busybox tar czf /backup/portainer-data-$DATE.tar.gz /data
echo "Backup completed: $DATE"
EOF
# Make it executable
chmod +x ~/backup-homelab.sh
# Test the backup
~/backup-homelab.sh
```
Set up a cron job for automated backups:
```bash
# Open crontab
crontab -e
# Add this line to run backup daily at 2 AM
0 2 * * * /home/yourusername/backup-homelab.sh
```
## Step 15: Configure Firewall (Optional but Recommended)
If using UFW:
```bash
# Allow SSH (if not already allowed)
sudo ufw allow 22/tcp
# Allow web traffic (if exposing services to internet)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Allow specific services from local network only
# Replace 192.168.1.0/24 with your network
sudo ufw allow from 192.168.1.0/24 to any port 9000 proto tcp # Portainer
sudo ufw allow from 192.168.1.0/24 to any port 81 proto tcp # Nginx Proxy Manager
sudo ufw allow from 192.168.1.0/24 to any port 3000 proto tcp # Grafana
# Enable firewall
sudo ufw enable
``` ```
## Next Steps ## Next Steps
Now that your homelab is running: 1. Explore Dockge at `https://dockge.yourdomain.duckdns.org`
2. Check Homepage dashboard at `https://home.yourdomain.duckdns.org`
1. **Explore Services:** 3. Configure services through their web UIs
- Access each service's web UI 4. Set up Authelia users in `/opt/stacks/authelia/users_database.yml`
- Configure settings as needed 5. Configure Homepage widgets with API keys
- Set up integrations between services 6. Use VS Code with Copilot to ask questions and make changes
7. Review [proxying-external-hosts.md](proxying-external-hosts.md) to proxy your Raspberry Pi
2. **Add More Services:**
- Ask GitHub Copilot for help adding new services
- Follow the patterns in existing compose files
- Check [awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) for ideas
3. **Optimize:**
- Review logs for errors
- Adjust resource limits
- Set up proper monitoring and alerts
4. **Secure:**
- Change all default passwords
- Set up SSL certificates (use Nginx Proxy Manager)
- Enable 2FA where available
- Keep services updated
5. **Learn:**
- Read the [Docker Guidelines](docker-guidelines.md)
- Experiment with new services
- Use AI assistance to understand and modify configurations
## Troubleshooting ## Troubleshooting
### Can't access services ### Can't access services via HTTPS
1. **Check if service is running:** Check Traefik logs:
```bash ```bash
docker ps cd /opt/stacks/traefik
docker compose logs -f
``` ```
2. **Check service logs:** Verify DNS is resolving:
```bash ```bash
docker compose -f docker-compose/file.yml logs service-name nslookup dockge.yourdomain.duckdns.org
``` ```
3. **Verify network connectivity:** Check certificate generation:
```bash ```bash
ping YOUR_SERVER_IP docker exec traefik cat /acme.json
``` ```
4. **Check firewall:** ### Authelia login not working
Check Authelia logs:
```bash ```bash
sudo ufw status cd /opt/stacks/authelia
docker compose logs -f
``` ```
### Permission errors Verify password hash in `users_database.yml`
```bash ### Service not accessible
# Fix config directory permissions
sudo chown -R $(id -u):$(id -g) config/
# Verify PUID/PGID in .env match your user 1. Check Traefik dashboard: `https://traefik.yourdomain.duckdns.org`
id -u # Should match PUID in .env 2. Verify service has correct Traefik labels
id -g # Should match PGID in .env 3. Check service is on `traefik-network`
``` 4. Review service logs
### Port already in use ### Port forwarding
```bash Ensure your router forwards ports 80 and 443 to your server IP.
# Find what's using the port
sudo netstat -tlnp | grep PORT_NUMBER
# Stop the conflicting service or change the port in docker-compose ## Security Checklist
```
### Out of disk space - [ ] All passwords in `.env` are strong and unique
- [ ] Authelia 2FA is enabled for admin accounts
- [ ] `.env` file permissions are 600 (`chmod 600 .env`)
- [ ] acme.json permissions are 600
- [ ] Firewall is configured (only 80, 443 open to internet)
- [ ] Pi-hole is configured as your DNS server
- [ ] Watchtower is monitoring for updates
- [ ] Backrest/Duplicati configured for backups
```bash ## Congratulations!
# Check disk usage
df -h
# Clean up Docker resources (removes unused images, containers, networks) Your AI-powered homelab is now running with:
docker system prune -a - ✅ Automatic HTTPS via Traefik + Let's Encrypt
- ✅ SSO protection via Authelia
- ✅ 40+ services ready to deploy
- ✅ Dashboard with service widgets
- ✅ AI assistance via GitHub Copilot
- ✅ Centralized management via Dockge
# To manage log size, configure log rotation in compose files: Continue exploring with VS Code and Copilot to add more services, customize configurations, and proxy external devices!
# logging:
# driver: "json-file"
# options:
# max-size: "10m"
# max-file: "3"
```
## Getting Help
- **Documentation:** Check the `docs/` directory for comprehensive guides
- **AI Assistance:** Use GitHub Copilot in VS Code for real-time help
- **Community:** Search for service-specific help in respective communities
- **Issues:** Open an issue on GitHub for problems with this repository
## Success Checklist
- [ ] Docker and Docker Compose installed
- [ ] Repository cloned
- [ ] `.env` file configured
- [ ] Networks created
- [ ] Config directories created
- [ ] Portainer running and accessible
- [ ] Infrastructure services deployed
- [ ] At least one service category deployed (media/monitoring/dev)
- [ ] VS Code with GitHub Copilot set up
- [ ] Backup strategy in place
- [ ] Firewall configured (if applicable)
- [ ] All services accessible and working
Congratulations! Your AI-powered homelab is now running! 🎉

462
docs/services-reference.md Normal file
View File

@@ -0,0 +1,462 @@
# Complete Services Reference
This document lists all 40+ pre-configured services available in the AI-Homelab repository, organized by category.
## Core Infrastructure (4 services)
### Required - Deploy First
1. **DuckDNS** (`duckdns.yml`)
- Dynamic DNS updater
- Updates your public IP automatically
- Integrates with Let's Encrypt for SSL
- No web UI - runs silently
- Stack: `/opt/stacks/duckdns/`
2. **Traefik** (`traefik.yml`)
- Reverse proxy with automatic SSL
- HTTP to HTTPS redirect
- File-based and Docker label routing
- Dashboard: `https://traefik.${DOMAIN}`
- Stack: `/opt/stacks/traefik/`
3. **Authelia** (`authelia.yml`)
- Single Sign-On (SSO) authentication
- TOTP 2FA support
- File-based or LDAP user database
- Smart bypass rules for media apps
- Login: `https://auth.${DOMAIN}`
- Stack: `/opt/stacks/authelia/`
4. **Gluetun** (`gluetun.yml`)
- VPN client (Surfshark WireGuard)
- Includes qBittorrent
- Control panel: `http://gluetun:8000`
- qBittorrent: `https://qbit.${DOMAIN}`
- Stack: `/opt/stacks/gluetun/`
## Infrastructure Tools (7 services)
From `infrastructure.yml` - Stack: `/opt/stacks/infrastructure/`
5. **Dockge** (PRIMARY management tool)
- Docker Compose stack manager
- Web UI for managing /opt/stacks/
- Direct compose file editing
- Access: `https://dockge.${DOMAIN}`
- SSO: Yes
6. **Portainer** (Secondary)
- Docker container management UI
- Access: `https://portainer.${DOMAIN}`
- SSO: Yes
7. **Pi-hole**
- Network-wide ad blocking
- DNS server
- Access: `https://pihole.${DOMAIN}`
- SSO: Yes
8. **Watchtower**
- Automatic container updates
- Runs 4 AM daily
- No web UI
9. **Dozzle**
- Real-time Docker log viewer
- Access: `https://dozzle.${DOMAIN}`
- SSO: Yes
10. **Glances**
- System and Docker monitoring
- Access: `https://glances.${DOMAIN}`
- SSO: Yes
11. **Docker Proxy**
- Secure Docker socket access
- Backend service
- No web UI
## Dashboards (2 services)
From `dashboards.yml` - Stack: `/opt/stacks/dashboards/`
12. **Homepage** (AI-configurable)
- Application dashboard with Docker integration
- Service widgets for 15+ services
- 11 organized categories
- Access: `https://home.${DOMAIN}`
- SSO: No (landing page)
13. **Homarr**
- Modern alternative dashboard
- Access: `https://homarr.${DOMAIN}`
- SSO: No
## Media Services (6 services)
From `media.yml` - Stack: `/opt/stacks/media/`
14. **Plex**
- Media streaming server
- Hardware transcoding support
- Access: `https://plex.${DOMAIN}`
- SSO: No (app access)
15. **Jellyfin**
- Open-source media server
- Hardware transcoding support
- Access: `https://jellyfin.${DOMAIN}`
- SSO: No (app access)
16. **Sonarr**
- TV show automation
- Access: `https://sonarr.${DOMAIN}`
- SSO: Yes
17. **Radarr**
- Movie automation
- Access: `https://radarr.${DOMAIN}`
- SSO: Yes
18. **Prowlarr**
- Indexer manager
- Integrates with Sonarr, Radarr, etc.
- Access: `https://prowlarr.${DOMAIN}`
- SSO: Yes
19. **qBittorrent**
- Torrent client (routes through Gluetun VPN)
- See gluetun.yml
## Extended Media (10 services)
From `media-extended.yml` - Stack: `/opt/stacks/media-extended/`
20. **Readarr**
- Ebook and audiobook management
- Access: `https://readarr.${DOMAIN}`
- SSO: Yes
21. **Lidarr**
- Music collection manager
- Access: `https://lidarr.${DOMAIN}`
- SSO: Yes
22. **Lazy Librarian**
- Book download automation
- Access: `https://lazylibrarian.${DOMAIN}`
- SSO: Yes
23. **Mylar3**
- Comic book collection manager
- Access: `https://mylar.${DOMAIN}`
- SSO: Yes
24. **Calibre-Web**
- Ebook reader and library management
- Access: `https://calibre.${DOMAIN}`
- SSO: Yes
25. **Jellyseerr**
- Media request management
- Integrates with Plex/Jellyfin
- Access: `https://jellyseerr.${DOMAIN}`
- SSO: No (family access)
26. **FlareSolverr**
- Cloudflare bypass for indexers
- Used by Prowlarr
- No web UI
27. **Tdarr Server**
- Distributed transcoding server
- Access: `https://tdarr.${DOMAIN}`
- SSO: Yes
28. **Tdarr Node**
- Transcoding worker
- No web UI
29. **Unmanic**
- Library optimization and transcoding
- Access: `https://unmanic.${DOMAIN}`
- SSO: Yes
## Home Automation (7 services)
From `homeassistant.yml` - Stack: `/opt/stacks/homeassistant/`
30. **Home Assistant**
- Home automation platform
- Uses host networking
- Access: `https://ha.${DOMAIN}` (or via proxying external host)
- SSO: No (has own auth)
31. **ESPHome**
- ESP8266/ESP32 firmware manager
- Access: `https://esphome.${DOMAIN}`
- SSO: Yes
32. **TasmoAdmin**
- Tasmota device management
- Access: `https://tasmoadmin.${DOMAIN}`
- SSO: Yes
33. **Node-RED**
- Flow-based automation programming
- Access: `https://nodered.${DOMAIN}`
- SSO: Yes
34. **Mosquitto**
- MQTT message broker
- Ports: 1883, 9001
- No web UI
35. **Zigbee2MQTT**
- Zigbee to MQTT bridge
- Access: `https://zigbee2mqtt.${DOMAIN}`
- SSO: Yes
36. **MotionEye**
- Video surveillance system
- Access: `https://motioneye.${DOMAIN}`
- SSO: Yes
## Productivity (8 services + 6 databases)
From `productivity.yml` - Stack: `/opt/stacks/productivity/`
37. **Nextcloud**
- File sync and collaboration platform
- Access: `https://nextcloud.${DOMAIN}`
- SSO: Yes
- Database: nextcloud-db (MariaDB)
38. **Mealie**
- Recipe manager and meal planner
- Access: `https://mealie.${DOMAIN}`
- SSO: No (family access)
39. **WordPress**
- Blog and website platform
- Access: `https://blog.${DOMAIN}`
- SSO: No (public blog)
- Database: wordpress-db (MariaDB)
40. **Gitea**
- Self-hosted Git service
- Access: `https://git.${DOMAIN}`
- SSO: Yes
- Database: gitea-db (PostgreSQL)
41. **DokuWiki**
- File-based wiki (no database)
- Access: `https://wiki.${DOMAIN}`
- SSO: Yes
42. **BookStack**
- Documentation platform
- Access: `https://docs.${DOMAIN}`
- SSO: Yes
- Database: bookstack-db (MariaDB)
43. **MediaWiki**
- Wiki platform
- Access: `https://mediawiki.${DOMAIN}`
- SSO: Yes
- Database: mediawiki-db (MariaDB)
## Utilities (7 services)
From `utilities.yml` - Stack: `/opt/stacks/utilities/`
44. **Backrest**
- Backup management with restic
- Access: `https://backrest.${DOMAIN}`
- SSO: Yes
45. **Duplicati**
- Backup software with encryption
- Access: `https://duplicati.${DOMAIN}`
- SSO: Yes
46. **Uptime Kuma**
- Uptime monitoring and status page
- Access: `https://status.${DOMAIN}`
- SSO: No (public status)
47. **Code Server**
- VS Code in browser
- Full stack access
- Access: `https://code.${DOMAIN}`
- SSO: Yes
48. **Form.io**
- Form builder platform
- Access: `https://forms.${DOMAIN}`
- SSO: Yes
- Database: formio-mongo (MongoDB)
49. **Authelia-Redis**
- Session storage for Authelia
- No web UI
## Monitoring (7 services)
From `monitoring.yml` - Stack: `/opt/stacks/monitoring/`
50. **Prometheus**
- Metrics collection
- Access: `https://prometheus.${DOMAIN}`
- SSO: Yes
51. **Grafana**
- Metrics visualization
- Access: `https://grafana.${DOMAIN}`
- SSO: Yes
52. **Loki**
- Log aggregation
- No web UI (accessed via Grafana)
53. **Promtail**
- Log shipping to Loki
- No web UI
54. **Node Exporter**
- Host metrics exporter
- No web UI
55. **cAdvisor**
- Container metrics
- Access: Port 8080 (internal)
## Development (6 services)
From `development.yml` - Stack: `/opt/stacks/development/`
56. **GitLab CE**
- Git repository with CI/CD
- Access: `https://gitlab.${DOMAIN}`
- SSO: Yes
57. **PostgreSQL**
- SQL database
- Port: 5432
- No web UI
58. **Redis**
- In-memory data store
- Port: 6379
- No web UI
59. **pgAdmin**
- PostgreSQL management UI
- Access: `https://pgadmin.${DOMAIN}`
- SSO: Yes
60. **Jupyter Lab**
- Interactive notebooks
- Access: `https://jupyter.${DOMAIN}`
- SSO: Yes
## Summary by Stack
| Stack | File | Services Count | Description |
|-------|------|----------------|-------------|
| Core Infrastructure | Multiple files | 4 | Traefik, Authelia, DuckDNS, Gluetun |
| Infrastructure | infrastructure.yml | 7 | Dockge, Portainer, Pi-hole, etc. |
| Dashboards | dashboards.yml | 2 | Homepage, Homarr |
| Media | media.yml | 6 | Plex, Jellyfin, *arr apps |
| Media Extended | media-extended.yml | 10 | Books, comics, music, transcoding |
| Home Automation | homeassistant.yml | 7 | HA, ESPHome, Node-RED, MQTT, etc. |
| Productivity | productivity.yml | 14 | Nextcloud, wikis, Git (includes DBs) |
| Utilities | utilities.yml | 7 | Backups, monitoring, Code Server |
| Monitoring | monitoring.yml | 7 | Prometheus, Grafana, Loki |
| Development | development.yml | 6 | GitLab, databases, Jupyter |
**Total: 60+ services (including databases)**
## Access Patterns
### With SSO (Authelia Required)
- Admin tools (Sonarr, Radarr, Prowlarr, etc.)
- Infrastructure management (Dockge, Portainer, Grafana)
- Development tools (GitLab, Code Server, pgAdmin)
- Personal data (Nextcloud, wikis, BookStack)
### Without SSO (Direct Access)
- Media streaming (Plex, Jellyfin) - for app access
- Public services (WordPress, Uptime Kuma, Homepage)
- Services with own auth (Home Assistant)
- Family-friendly (Mealie, Jellyseerr)
### Via VPN (Gluetun)
- qBittorrent
- Other download clients (add with network_mode: "service:gluetun")
## Storage Recommendations
### Keep on System Drive (/opt/stacks/)
- All configuration files
- Small databases (< 10GB)
- Application data
### Move to Separate Drive (/mnt/)
- Media files (movies, TV, music, photos) → /mnt/media/
- Downloads → /mnt/downloads/
- Backups → /mnt/backups/
- Surveillance footage → /mnt/surveillance/
- Large databases → /mnt/databases/
- Transcoding cache → /mnt/transcode-cache/
## Quick Deployment Guide
1. **Core (Required)**
```bash
# Deploy in this order:
/opt/stacks/duckdns/
/opt/stacks/traefik/
/opt/stacks/authelia/
/opt/stacks/infrastructure/ (dockge)
```
2. **VPN + Downloads**
```bash
/opt/stacks/gluetun/
```
3. **Dashboard**
```bash
/opt/stacks/homepage/
```
4. **Choose Your Stacks**
- Media: `/opt/stacks/media/` + `/opt/stacks/media-extended/`
- Home Automation: `/opt/stacks/homeassistant/`
- Productivity: `/opt/stacks/productivity/`
- Monitoring: `/opt/stacks/monitoring/`
- Development: `/opt/stacks/development/`
- Utilities: `/opt/stacks/utilities/`
## Configuration Files
All configuration templates available in `config-templates/`:
- `traefik/` - Static and dynamic configs
- `authelia/` - Config and user database
- `homepage/` - Dashboard services and widgets
- `prometheus/` - Scrape configurations
- `loki/` - Log aggregation config
- And more...
## Next Steps
1. Deploy core infrastructure
2. Configure Homepage with API keys
3. Set up Authelia users
4. Deploy service stacks as needed
5. Use VS Code + Copilot for AI assistance
6. Proxy external hosts via Traefik (see docs/proxying-external-hosts.md)
For detailed deployment instructions, see [docs/getting-started.md](../docs/getting-started.md)