Complete documentation refactoring with 57 service docs
- Refactored README.md, getting-started.md, quick-reference.md - Enhanced setup-homelab.sh with 9-step automated process - Created services-overview.md with all stacks - Added comprehensive documentation for 57 services in docs/service-docs/ - All services include: overview, configuration, resources, educational content - Coverage: Core, Infrastructure, Dashboards, Media, Media-Extended, Home Assistant, Productivity, Utilities, Monitoring, Development stacks - Educational focus with links to tutorials, videos, and guides
This commit is contained in:
661
README.md
661
README.md
@@ -1,12 +1,22 @@
|
||||
# AI-Homelab
|
||||
|
||||
AI-Powered Homelab Administration with GitHub Copilot
|
||||
Leverage Github Copilot in VS Code as a complete homelab management interface.
|
||||
|
||||
## Overview
|
||||
|
||||
This repository provides a comprehensive, production-ready homelab infrastructure using Docker Compose with Dockge, featuring 40+ pre-configured services. Integrated AI assistance through GitHub Copilot helps you create, modify, and manage Docker services while maintaining consistency across your entire server stack.
|
||||
|
||||
The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia for Single Sign-On, Gluetun for VPN routing, and DuckDNS for dynamic DNS - all managed through file-based configurations that the AI can modify.
|
||||
The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia for Single Sign-On, Gluetun for VPN routing, and DuckDNS for dynamic DNS - all managed through file-based configurations that the AI can manage.
|
||||
|
||||
## Designed to be noob friendly
|
||||
|
||||
* simple install process
|
||||
* Just tell the AI what you want
|
||||
* doesn't require setting up a raid
|
||||
* you can start with the hardware you have and add raid as your homelab grows
|
||||
* The AI can guide you through advanced concepts
|
||||
* setting up a raid and transfering your data
|
||||
* adding a 2nd or 3rd server to your homelab
|
||||
|
||||
## Features
|
||||
|
||||
@@ -22,7 +32,16 @@ The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia f
|
||||
- **Comprehensive Documentation**: Detailed guidelines including proxying external hosts
|
||||
- **File-Based Configuration**: Everything managed via YAML files - no web UI dependencies
|
||||
|
||||
## Quick Start
|
||||
## Documentation
|
||||
|
||||
- **[Getting Started](docs/getting-started.md)**: Step-by-step setup guide
|
||||
- **[Services Reference](docs/services-reference.md)**: Complete list of all 60+ pre-configured services
|
||||
- **[Docker Guidelines](docs/docker-guidelines.md)**: Comprehensive guide to Docker service management with Dockge
|
||||
- **[Quick Reference](docs/quick-reference.md)**: Command reference and troubleshooting
|
||||
- **[Proxying External Hosts](docs/proxying-external-hosts.md)**: Guide for proxying Raspberry Pi, routers, NAS via Traefik
|
||||
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines (Traefik, Authelia, Dockge aware)
|
||||
|
||||
# Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
@@ -34,592 +53,83 @@ The infrastructure uses Traefik for reverse proxy with automatic SSL, Authelia f
|
||||
- Surfshark VPN account (optional, for VPN features)
|
||||
- Sufficient disk space: 120GB+ system drive (NVMe or SSD highly recommended), 2TB+ for media & additional disks for services like Nextcloud that require lots of space
|
||||
|
||||
### Quick Setup (Dockge Structure)
|
||||
## Quick Setup (Dockge Structure)
|
||||
|
||||
1. **Clone the repository:**
|
||||
1. **Clone the repository into your home folder:**
|
||||
```bash
|
||||
# Note: Replace 'kelinfoxy' with your username if you forked this repository
|
||||
cd ~
|
||||
git clone https://github.com/kelinfoxy/AI-Homelab.git
|
||||
cd AI-Homelab
|
||||
```
|
||||
|
||||
2. **(Optional) Run first-run setup script:**
|
||||
|
||||
**For fresh Debian installations only**, this automated script will:
|
||||
- Update system and install Docker Engine + Compose V2
|
||||
- Configure user groups (sudo, docker) and SSH access
|
||||
- Detect NVIDIA GPU and provide driver installation guidance
|
||||
- Create directory structure and Docker networks
|
||||
For fresh Debian installations only, this automated script will install Docker Engine + Compose V2, configure user groups, detect NVIDIA GPU, and create directory structure.
|
||||
|
||||
```bash
|
||||
sudo ./scripts/setup-homelab.sh
|
||||
```
|
||||
|
||||
After completion, log out and log back in for group changes to take effect.
|
||||
|
||||
**Skip this step if Docker is already installed and configured.**
|
||||
|
||||
3. **Create and configure environment file:**
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env # Edit with your domain, API keys, and passwords
|
||||
```
|
||||
|
||||
**Required variables:**
|
||||
- `DOMAIN` - Your DuckDNS domain (e.g., yourhomelab.duckdns.org)
|
||||
- `DUCKDNS_TOKEN` - Token from DuckDNS.org
|
||||
- `TZ` - Your timezone (e.g., America/New_York)
|
||||
- Authelia user credentials
|
||||
- API keys for services you plan to use
|
||||
> Alternativly you can ssh in from VS Code using the Remote-ssh plugin and edit in a nice editor
|
||||
|
||||
Required variables: DOMAIN, DUCKDNS_TOKEN, TZ, Authelia user credentials, API keys for services you plan to use.
|
||||
> See [Getting Started](docs/getting-started.md) for more details
|
||||
4. **Run deployment script:**
|
||||
|
||||
This automated script will:
|
||||
- Create required directories
|
||||
- Verify Docker networks exist
|
||||
- Deploy core infrastructure (DuckDNS, Traefik, Authelia, Gluetun)
|
||||
- Deploy infrastructure stack with Dockge
|
||||
- Wait for Dockge to be ready
|
||||
- Automatically open Dockge in your browser
|
||||
This automated script will create required directories, verify Docker networks exist, deploy core stack (DuckDNS, Traefik, Authelia, Gluetun), deploy the infrastructure stack and open Dockge in your browser when ready.
|
||||
|
||||
```bash
|
||||
./scripts/deploy-homelab.sh
|
||||
```
|
||||
|
||||
The script will automatically open `https://dockge.yourdomain.duckdns.org` when ready.
|
||||
|
||||
**Manual deployment alternative:**
|
||||
```bash
|
||||
# Deploy core stack
|
||||
mkdir -p /opt/stacks/core
|
||||
cp docker-compose/core.yml /opt/stacks/core/docker-compose.yml
|
||||
cp -r config-templates/traefik /opt/stacks/core/
|
||||
cp -r config-templates/authelia /opt/stacks/core/
|
||||
cp .env /opt/stacks/core/
|
||||
cd /opt/stacks/core && docker compose up -d
|
||||
|
||||
# Deploy infrastructure stack
|
||||
mkdir -p /opt/stacks/infrastructure
|
||||
cp docker-compose/infrastructure.yml /opt/stacks/infrastructure/docker-compose.yml
|
||||
cp .env /opt/stacks/infrastructure/
|
||||
cd /opt/stacks/infrastructure && docker compose up -d
|
||||
|
||||
# Manually open: https://dockge.yourdomain.duckdns.org
|
||||
```
|
||||
|
||||
5. **Deploy additional stacks through Dockge:**
|
||||
|
||||
Log in to Dockge with your Authelia credentials and deploy additional stacks:
|
||||
- `dashboards.yml` - Homepage and Homarr dashboards
|
||||
- `media.yml` - Plex, Jellyfin, Sonarr, Radarr, Prowlarr, qBittorrent
|
||||
- `media-extended.yml` - Readarr, Lidarr, Lazy Librarian, Mylar3, Calibre-Web
|
||||
- `homeassistant.yml` - Home Assistant, ESPHome, Node-RED, and accessories
|
||||
- `productivity.yml` - Nextcloud, Gitea, WordPress, wikis
|
||||
- `monitoring.yml` - Grafana, Prometheus, Loki
|
||||
- `utilities.yml` - Backups, code editors, password manager
|
||||
Log in to Dockge with your Authelia credentials and deploy additional stacks: dashboards.yml, media.yml, media-extended.yml, homeassistant.yml, productivity.yml, monitoring.yml, utilities.yml.
|
||||
|
||||
## Repository Structure
|
||||
6. **Configure VS Code to control the server via Github Copilot**
|
||||
|
||||
Log into VS Code, install and configure the Github Copilot extension with your api key.
|
||||
Use the Copilot chat window to manage your homelab
|
||||
|
||||
> Tip: If you have a paid account use the free models to perform simple tasks like starting/stopping a service, and premium models to do more advanced tasks.
|
||||
|
||||
```
|
||||
AI-Homelab/
|
||||
├── .github/
|
||||
│ └── copilot-instructions.md # AI assistant guidelines (Dockge, Traefik, Authelia aware)
|
||||
├── docker-compose/
|
||||
│ ├── traefik.yml # Reverse proxy (deploy first)
|
||||
│ ├── authelia.yml # SSO authentication
|
||||
│ ├── duckdns.yml # Dynamic DNS
|
||||
│ ├── gluetun.yml # VPN client (Surfshark) + qBittorrent
|
||||
│ ├── infrastructure.yml # Dockge, Portainer, Pi-hole, Watchtower, Dozzle, Glances
|
||||
│ ├── dashboards.yml # Homepage, Homarr
|
||||
│ ├── media.yml # Plex, Jellyfin, Sonarr, Radarr, Prowlarr
|
||||
│ ├── media-extended.yml # Readarr, Lidarr, Lazy Librarian, Mylar3, Calibre-Web,
|
||||
│ │ # Jellyseerr, FlareSolverr, Tdarr, Unmanic
|
||||
│ ├── homeassistant.yml # Home Assistant, ESPHome, TasmoAdmin, Node-RED,
|
||||
│ │ # Mosquitto, Zigbee2MQTT, MotionEye
|
||||
│ ├── productivity.yml # Nextcloud, Mealie, WordPress, Gitea, DokuWiki,
|
||||
│ │ # BookStack, MediaWiki (all with databases)
|
||||
│ ├── utilities.yml # Backrest, Duplicati, Uptime Kuma, Code Server,
|
||||
│ │ # Form.io, Authelia-Redis
|
||||
│ ├── monitoring.yml # Prometheus, Grafana, Loki, Promtail, cAdvisor
|
||||
│ ├── development.yml # GitLab, PostgreSQL, Redis, pgAdmin, Jupyter
|
||||
│ └── README-dockge.md # Dockge deployment guide
|
||||
├── config-templates/
|
||||
│ ├── traefik/ # Traefik static and dynamic configs
|
||||
│ ├── authelia/ # Authelia config and user database
|
||||
│ ├── homepage/ # Homepage dashboard configs (with widgets)
|
||||
│ ├── prometheus/ # Prometheus scrape configs
|
||||
│ ├── loki/ # Loki log aggregation config
|
||||
│ └── ... # Other service templates
|
||||
├── docs/
|
||||
│ ├── docker-guidelines.md # Comprehensive Docker guidelines
|
||||
│ ├── getting-started.md # Step-by-step setup guide
|
||||
│ ├── quick-reference.md # Command reference
|
||||
│ └── proxying-external-hosts.md # Guide for proxying Raspberry Pi, routers, etc.
|
||||
├── .env.example # Environment variable template (40+ vars)
|
||||
├── .gitignore # Git ignore patterns
|
||||
└── README.md # This file
|
||||
```
|
||||
# #
|
||||
|
||||
## Using the AI Assistant
|
||||
# AI Capabilities and Examples #
|
||||
|
||||
### In VS Code
|
||||
Ask the AI to modify anything about the AI-Homelab folder to suit your purposes.
|
||||
|
||||
Want to change /opt/stacks to something else? Just tell the AI what you want.
|
||||
Prefer Portainer over Dockge? Ask the AI to refactor the entire AI-Homelab folder to convert to Portainer as default instead of Dockge.
|
||||
Don't like the selection of included services? Tell the AI exactly what services you want and what you don't.
|
||||
Don't like how the services are arranged in the stacks?
|
||||
Want to replace one service with a different service?
|
||||
|
||||
> Just tell the AI what you want.
|
||||
|
||||
1. **Install GitHub Copilot** extension in VS Code
|
||||
2. **Open this repository** in VS Code
|
||||
3. **Start Copilot Chat** and ask questions like:
|
||||
- "Help me add a new media service to my homelab"
|
||||
- "Configure Traefik routing for my new service"
|
||||
- "Add Authelia SSO protection to this service"
|
||||
- "How do I proxy my Raspberry Pi through Traefik?"
|
||||
- "Create a Homepage widget for this service"
|
||||
- "Help me reorganize or customize my Homepage"
|
||||
- "Route this download client through Gluetun VPN"
|
||||
|
||||
The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md` to:
|
||||
- Use `/opt/stacks/` directory structure (Dockge compatible)
|
||||
- Configure Traefik labels for automatic routing
|
||||
- Apply Authelia middleware where appropriate
|
||||
- Suggest `/mnt/` for large data storage
|
||||
- Add services to Homepage dashboard with widgets
|
||||
- Maintain consistency with existing services
|
||||
- Consider the entire stack when making changes
|
||||
|
||||
### Example Interactions
|
||||
|
||||
**Adding a new service:**
|
||||
```
|
||||
You: "Add Tautulli to monitor my Plex server"
|
||||
|
||||
Copilot: [Creates compose configuration with]:
|
||||
- /opt/stacks/tautulli/ directory structure
|
||||
- Traefik labels for HTTPS access
|
||||
- Authelia middleware for SSO protection
|
||||
- Homepage dashboard entry with widget
|
||||
- Connection to existing Plex service
|
||||
```
|
||||
|
||||
**Proxying external service:**
|
||||
```
|
||||
You: "Proxy my Raspberry Pi Home Assistant through Traefik"
|
||||
|
||||
Copilot: [Creates Traefik route configuration]:
|
||||
- File in /opt/stacks/traefik/dynamic/
|
||||
- HTTPS with Let's Encrypt
|
||||
- Authelia bypass (HA has its own auth)
|
||||
- WebSocket support
|
||||
- Homepage dashboard entry
|
||||
```
|
||||
|
||||
**Configuring VPN routing:**
|
||||
```
|
||||
You: "Route SABnzbd through the VPN"
|
||||
|
||||
Copilot: [Updates compose to use Gluetun]:
|
||||
- network_mode: "service:gluetun"
|
||||
- Exposes ports through Gluetun
|
||||
- Maintains Traefik routing
|
||||
- Updates documentation
|
||||
```
|
||||
|
||||
## Available Service Stacks
|
||||
|
||||
### Core Infrastructure (Required)
|
||||
|
||||
#### 1. Traefik (`traefik.yml`)
|
||||
**Reverse proxy with automatic SSL** - Deploy first!
|
||||
- Automatic HTTPS via Let's Encrypt
|
||||
- File-based and Docker label routing
|
||||
- HTTP to HTTPS redirect
|
||||
- Dashboard at `https://traefik.${DOMAIN}`
|
||||
|
||||
#### 2. Authelia (`authelia.yml`)
|
||||
**Single Sign-On authentication**
|
||||
- TOTP 2FA support
|
||||
- LDAP/file-based user database
|
||||
- Smart bypass rules for media apps
|
||||
- Login at `https://auth.${DOMAIN}`
|
||||
|
||||
#### 3. DuckDNS (`duckdns.yml`)
|
||||
**Dynamic DNS updater**
|
||||
- Automatic IP updates
|
||||
- Integrates with Let's Encrypt
|
||||
- No web UI - runs silently
|
||||
|
||||
#### 4. Gluetun (`gluetun.yml`)
|
||||
**VPN client (Surfshark WireGuard)**
|
||||
- Includes qBittorrent
|
||||
- Download via `https://qbit.${DOMAIN}`
|
||||
- Easy to route other services through VPN
|
||||
|
||||
### Infrastructure Tools (`infrastructure.yml`)
|
||||
|
||||
- **Dockge**: Docker Compose stack manager (PRIMARY) - `https://dockge.${DOMAIN}`
|
||||
- **Portainer**: Docker management UI (secondary) - `https://portainer.${DOMAIN}`
|
||||
- **Pi-hole**: Network-wide ad blocking - `https://pihole.${DOMAIN}`
|
||||
- **Watchtower**: Automatic container updates
|
||||
- **Dozzle**: Real-time Docker logs - `https://dozzle.${DOMAIN}`
|
||||
- **Glances**: System monitoring - `https://glances.${DOMAIN}`
|
||||
- **Docker Proxy**: Secure Docker socket access
|
||||
|
||||
### Dashboards (`dashboards.yml`)
|
||||
|
||||
- **Homepage**: AI-configurable dashboard with widgets - `https://home.${DOMAIN}`
|
||||
- Docker integration (container status)
|
||||
- Service widgets (Sonarr, Radarr, Plex, Jellyfin, etc.)
|
||||
- 11 organized categories
|
||||
- **Homarr**: Modern alternative dashboard - `https://homarr.${DOMAIN}`
|
||||
|
||||
### Media Services (`media.yml`)
|
||||
|
||||
- **Plex**: Media streaming server - `https://plex.${DOMAIN}` (no SSO - app access)
|
||||
- **Jellyfin**: Open-source media server - `https://jellyfin.${DOMAIN}` (no SSO - app access)
|
||||
- **Sonarr**: TV show automation - `https://sonarr.${DOMAIN}`
|
||||
- **Radarr**: Movie automation - `https://radarr.${DOMAIN}`
|
||||
- **Prowlarr**: Indexer manager - `https://prowlarr.${DOMAIN}`
|
||||
- **qBittorrent**: Torrent client (via VPN) - See gluetun.yml
|
||||
|
||||
### Extended Media (`media-extended.yml`)
|
||||
|
||||
- **Readarr**: Ebook/audiobook management - `https://readarr.${DOMAIN}`
|
||||
- **Lidarr**: Music collection manager - `https://lidarr.${DOMAIN}`
|
||||
- **Lazy Librarian**: Book download automation - `https://lazylibrarian.${DOMAIN}`
|
||||
- **Mylar3**: Comic book manager - `https://mylar.${DOMAIN}`
|
||||
- **Calibre-Web**: Ebook reader and server - `https://calibre.${DOMAIN}`
|
||||
- **Jellyseerr**: Media request management - `https://jellyseerr.${DOMAIN}` (no SSO)
|
||||
- **FlareSolverr**: Cloudflare bypass (no UI)
|
||||
- **Tdarr**: Distributed transcoding - `https://tdarr.${DOMAIN}`
|
||||
- **Unmanic**: Library optimizer - `https://unmanic.${DOMAIN}`
|
||||
|
||||
### Home Automation (`homeassistant.yml`)
|
||||
|
||||
- **Home Assistant**: Home automation hub - `https://ha.${DOMAIN}` (uses host network)
|
||||
- **ESPHome**: ESP device manager - `https://esphome.${DOMAIN}`
|
||||
- **TasmoAdmin**: Tasmota device manager - `https://tasmoadmin.${DOMAIN}`
|
||||
- **Node-RED**: Flow automation - `https://nodered.${DOMAIN}`
|
||||
- **Mosquitto**: MQTT broker (no UI)
|
||||
- **Zigbee2MQTT**: Zigbee bridge - `https://zigbee2mqtt.${DOMAIN}`
|
||||
- **MotionEye**: Video surveillance - `https://motioneye.${DOMAIN}`
|
||||
|
||||
### Productivity (`productivity.yml`)
|
||||
|
||||
- **Nextcloud**: File sync & collaboration - `https://nextcloud.${DOMAIN}`
|
||||
- Includes MariaDB database
|
||||
- **Mealie**: Recipe manager - `https://mealie.${DOMAIN}` (no SSO)
|
||||
- **WordPress**: Blog platform - `https://blog.${DOMAIN}` (no SSO - public)
|
||||
- Includes MariaDB database
|
||||
- **Gitea**: Self-hosted Git - `https://git.${DOMAIN}`
|
||||
- Includes PostgreSQL database
|
||||
- **DokuWiki**: File-based wiki - `https://wiki.${DOMAIN}`
|
||||
- **BookStack**: Documentation platform - `https://docs.${DOMAIN}`
|
||||
- Includes MariaDB database
|
||||
- **MediaWiki**: Wiki platform - `https://mediawiki.${DOMAIN}`
|
||||
- Includes MariaDB database
|
||||
|
||||
### Utilities (`utilities.yml`)
|
||||
|
||||
- **Backrest**: Backup manager (restic) - `https://backrest.${DOMAIN}`
|
||||
- **Duplicati**: Backup software - `https://duplicati.${DOMAIN}`
|
||||
- **Uptime Kuma**: Status monitoring - `https://status.${DOMAIN}` (no SSO - public)
|
||||
- **Code Server**: VS Code in browser - `https://code.${DOMAIN}`
|
||||
- **Form.io**: Form builder - `https://forms.${DOMAIN}`
|
||||
- Includes MongoDB database
|
||||
- **Authelia-Redis**: Session storage (no UI)
|
||||
|
||||
### Monitoring (`monitoring.yml`)
|
||||
|
||||
- **Prometheus**: Metrics collection - `https://prometheus.${DOMAIN}`
|
||||
- **Grafana**: Metrics visualization - `https://grafana.${DOMAIN}`
|
||||
- **Loki**: Log aggregation
|
||||
- **Promtail**: Log shipping
|
||||
- **Node Exporter**: Host metrics
|
||||
- **cAdvisor**: Container metrics
|
||||
|
||||
### Development (`development.yml`)
|
||||
|
||||
- **GitLab CE**: Git with CI/CD - `https://gitlab.${DOMAIN}`
|
||||
- **PostgreSQL**: SQL database
|
||||
- **Redis**: In-memory store
|
||||
- **pgAdmin**: PostgreSQL UI - `https://pgadmin.${DOMAIN}`
|
||||
- **Jupyter Lab**: Interactive notebooks - `https://jupyter.${DOMAIN}`
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Managing Stacks via Dockge
|
||||
|
||||
Access Dockge at `https://dockge.${DOMAIN}` to:
|
||||
- View all stacks and their status
|
||||
- Start/stop/restart stacks
|
||||
- Edit compose files directly
|
||||
- View logs
|
||||
- Deploy new stacks
|
||||
|
||||
### Command Line Operations
|
||||
|
||||
#### Starting Services (Dockge Structure)
|
||||
|
||||
```bash
|
||||
# Start entire stack
|
||||
cd /opt/stacks/media
|
||||
docker compose up -d
|
||||
|
||||
# Start specific services
|
||||
cd /opt/stacks/media
|
||||
docker compose up -d sonarr radarr
|
||||
|
||||
# Start with rebuild
|
||||
cd /opt/stacks/infrastructure
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
#### Stopping Services
|
||||
|
||||
```bash
|
||||
# Stop entire stack
|
||||
cd /opt/stacks/media
|
||||
docker compose down
|
||||
|
||||
# Stop but keep volumes
|
||||
cd /opt/stacks/media
|
||||
docker compose stop
|
||||
|
||||
# Stop specific service
|
||||
cd /opt/stacks/media
|
||||
docker compose stop plex
|
||||
```
|
||||
|
||||
#### Viewing Logs
|
||||
|
||||
```bash
|
||||
# Follow logs for entire stack
|
||||
cd /opt/stacks/media
|
||||
docker compose logs -f
|
||||
|
||||
# Follow logs for specific service
|
||||
cd /opt/stacks/media
|
||||
docker compose logs -f plex
|
||||
|
||||
# View last 100 lines
|
||||
cd /opt/stacks/media
|
||||
docker compose logs --tail=100 plex
|
||||
|
||||
# Or use Dozzle web UI at https://dozzle.${DOMAIN}
|
||||
```
|
||||
|
||||
#### Updating Services
|
||||
|
||||
```bash
|
||||
# Pull latest images
|
||||
cd /opt/stacks/media
|
||||
docker compose pull
|
||||
|
||||
# Update specific service
|
||||
cd /opt/stacks/media
|
||||
docker compose pull plex
|
||||
docker compose up -d plex
|
||||
|
||||
# Or enable Watchtower for automatic updates
|
||||
```
|
||||
|
||||
### Testing with Docker Run
|
||||
|
||||
Use `docker run` only for temporary testing:
|
||||
|
||||
```bash
|
||||
# Test NVIDIA GPU support
|
||||
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
|
||||
|
||||
# Test a new image
|
||||
docker run --rm -it alpine:latest /bin/sh
|
||||
|
||||
# Test VPN connection through Gluetun
|
||||
docker run --rm --network container:gluetun curlimages/curl ifconfig.me
|
||||
```
|
||||
|
||||
## Network Architecture
|
||||
|
||||
Services connect to multiple networks for organization and security:
|
||||
|
||||
- **traefik-network**: For Traefik to reach services (external)
|
||||
- **homelab-network**: Main network for inter-service communication (external)
|
||||
- **media-network**: Isolated network for media stack (external)
|
||||
- **monitoring-network**: Network for observability stack (created per stack)
|
||||
- **database-network**: Isolated networks for database services (created per stack)
|
||||
- **dockerproxy-network**: Secure Docker socket access (created in infrastructure)
|
||||
|
||||
### Creating Required Networks
|
||||
|
||||
```bash
|
||||
# Create external networks (do this once)
|
||||
docker network create traefik-network
|
||||
docker network create homelab-network
|
||||
docker network create media-network
|
||||
|
||||
# Stack-specific networks are created automatically by compose files
|
||||
```
|
||||
|
||||
### Traefik Routing
|
||||
|
||||
All services accessed via Traefik with automatic HTTPS:
|
||||
- Pattern: `https://service.yourdomain.duckdns.org`
|
||||
- Configured via Docker labels on each service
|
||||
- SSL certificates automatically managed
|
||||
- No port exposure needed (except Traefik 80/443)
|
||||
|
||||
## Documentation
|
||||
|
||||
### Comprehensive Guides
|
||||
|
||||
- **[Docker Guidelines](docs/docker-guidelines.md)**: Complete guide to Docker service management with Dockge
|
||||
- **[Getting Started](docs/getting-started.md)**: Step-by-step setup walkthrough
|
||||
- **[Quick Reference](docs/quick-reference.md)**: Command reference and troubleshooting
|
||||
- **[Dockge Deployment](docker-compose/README-dockge.md)**: Dockge-specific deployment guide
|
||||
- **[Proxying External Hosts](docs/proxying-external-hosts.md)**: Guide for proxying Raspberry Pi, routers, NAS via Traefik
|
||||
- **[Copilot Instructions](.github/copilot-instructions.md)**: AI assistant guidelines (Traefik, Authelia, Dockge aware)
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **Dockge Structure**: All stacks in `/opt/stacks/stack-name/`
|
||||
2. **Docker Compose First**: Always use compose for persistent services
|
||||
3. **Docker Run for Testing**: Only use `docker run` for temporary containers
|
||||
4. **File-Based Configuration**: Traefik labels and Authelia YAML (AI-manageable)
|
||||
5. **Traefik for All**: Every service routed through Traefik with automatic SSL
|
||||
6. **Smart SSO**: Authelia protects admin interfaces, bypasses media apps for device access
|
||||
7. **VPN When Needed**: Route download clients through Gluetun
|
||||
8. **Large Data Separate**: Use `/mnt/` for media, downloads, large databases
|
||||
9. **Stack Awareness**: Consider dependencies and interactions
|
||||
10. **Security**: Keep secrets in `.env` files, never commit them
|
||||
|
||||
## Configuration Management
|
||||
|
||||
### Environment Variables
|
||||
|
||||
All services use variables from `.env` in each stack directory:
|
||||
- `PUID`/`PGID`: User/group IDs for file permissions
|
||||
- `TZ`: Timezone for all services
|
||||
- `DOMAIN`: Your DuckDNS domain (e.g., yourdomain.duckdns.org)
|
||||
- `SERVER_IP`: Your server's IP address
|
||||
- Service-specific credentials and API keys
|
||||
- Homepage widget API keys (40+ variables)
|
||||
|
||||
See `.env.example` for complete list.
|
||||
|
||||
### Storage Strategy
|
||||
|
||||
**Small Data** (configs, databases < 10GB): `/opt/stacks/stack-name/`
|
||||
```yaml
|
||||
volumes:
|
||||
- /opt/stacks/sonarr/config:/config
|
||||
```
|
||||
|
||||
**Large Data** (media, downloads, backups): `/mnt/`
|
||||
```yaml
|
||||
volumes:
|
||||
- /mnt/media:/media
|
||||
- /mnt/downloads:/downloads
|
||||
- /mnt/backups:/backups
|
||||
```
|
||||
|
||||
The AI will suggest when to use `/mnt/` based on expected data size.
|
||||
|
||||
### Configuration Files
|
||||
|
||||
Service configurations stored in stack directories:
|
||||
```
|
||||
/opt/stacks/
|
||||
├── traefik/
|
||||
│ ├── docker-compose.yml
|
||||
│ ├── traefik.yml # Static config
|
||||
│ ├── dynamic/ # Dynamic routes
|
||||
│ │ └── routes.yml
|
||||
│ └── acme.json # SSL certificates
|
||||
├── authelia/
|
||||
│ ├── docker-compose.yml
|
||||
│ ├── configuration.yml # Authelia settings
|
||||
│ └── users_database.yml # User accounts
|
||||
├── homepage/
|
||||
│ ├── docker-compose.yml
|
||||
│ └── config/
|
||||
│ ├── services.yaml # Service definitions
|
||||
│ ├── docker.yaml # Docker integration
|
||||
│ ├── settings.yaml # Dashboard settings
|
||||
│ └── widgets.yaml # Homepage widgets
|
||||
└── ...
|
||||
```
|
||||
|
||||
Templates available in `config-templates/` directory.
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Pin Image Versions**: Never use `:latest` in production
|
||||
2. **Use Environment Variables**: Store secrets in `.env` (gitignored)
|
||||
3. **Run as Non-Root**: Set PUID/PGID to match your user
|
||||
4. **Limit Exposure**: Bind ports to localhost when possible
|
||||
5. **Regular Updates**: Keep images updated via Watchtower
|
||||
6. **Scan Images**: Use `docker scan` to check for vulnerabilities
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
|
||||
1. Check logs: `docker compose -f file.yml logs service-name`
|
||||
2. Validate config: `docker compose -f file.yml config`
|
||||
3. Check port conflicts: `sudo netstat -tlnp | grep PORT`
|
||||
4. Verify network exists: `docker network ls`
|
||||
|
||||
### Permission Issues
|
||||
|
||||
1. Check PUID/PGID match your user: `id -u` and `id -g`
|
||||
2. Fix ownership: `sudo chown -R 1000:1000 ./config/service-name`
|
||||
|
||||
### Network Issues
|
||||
|
||||
1. Verify network exists: `docker network inspect homelab-network`
|
||||
2. Test connectivity: `docker compose exec service1 ping service2`
|
||||
|
||||
### Getting Help
|
||||
|
||||
- Review the [Docker Guidelines](docs/docker-guidelines.md)
|
||||
- Ask GitHub Copilot in VS Code
|
||||
- Check service-specific documentation
|
||||
- Review Docker logs for error messages
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### What to Backup
|
||||
|
||||
1. **Docker Compose files** (version controlled in git)
|
||||
2. **Config directories**: `./config/*`
|
||||
3. **Named volumes**: `docker volume ls`
|
||||
4. **Environment file**: `.env` (securely, not in git)
|
||||
|
||||
### Backup Named Volumes
|
||||
|
||||
```bash
|
||||
# Backup a volume
|
||||
docker run --rm \
|
||||
-v volume-name:/data \
|
||||
-v $(pwd)/backups:/backup \
|
||||
busybox tar czf /backup/volume-backup.tar.gz /data
|
||||
```
|
||||
|
||||
### Restore Named Volumes
|
||||
|
||||
```bash
|
||||
# Restore a volume
|
||||
docker run --rm \
|
||||
-v volume-name:/data \
|
||||
-v $(pwd)/backups:/backup \
|
||||
busybox tar xzf /backup/volume-backup.tar.gz -C /
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please:
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Follow existing patterns and conventions
|
||||
4. Test your changes
|
||||
5. Submit a pull request
|
||||
- "Disable SSO for Wordpress"
|
||||
|
||||
The AI assistant automatically follows the guidelines in `.github/copilot-instructions.md`
|
||||
* to use `/opt/stacks/` directory structure,
|
||||
* configure Traefik labels,
|
||||
* apply Authelia middleware where appropriate,
|
||||
* suggest `/mnt/` for large data storage,
|
||||
* add services to Homepage dashboard with widgets,
|
||||
* maintain consistency with existing services,
|
||||
* and consider the entire stack when making changes.
|
||||
|
||||
## License
|
||||
|
||||
@@ -632,57 +142,8 @@ This project is provided as-is for personal homelab use.
|
||||
- GitHub Copilot for AI assistance capabilities
|
||||
- All the open-source projects used in example compose files
|
||||
|
||||
## Getting Started Checklist
|
||||
|
||||
- [ ] Install Docker and Docker Compose V2
|
||||
- [ ] Sign up for DuckDNS (free) and get your domain
|
||||
- [ ] Get Surfshark VPN credentials (optional, for VPN features)
|
||||
- [ ] Clone this repository
|
||||
- [ ] Copy `.env.example` to `.env` and configure all values
|
||||
- [ ] Create `/opt/stacks` directory: `sudo mkdir -p /opt/stacks && sudo chown $USER:$USER /opt/stacks`
|
||||
- [ ] Create Docker networks: `docker network create traefik-network homelab-network media-network`
|
||||
- [ ] Deploy DuckDNS stack
|
||||
- [ ] Deploy Traefik stack (with config templates)
|
||||
- [ ] Deploy Authelia stack (with config templates)
|
||||
- [ ] Deploy infrastructure stack (Dockge)
|
||||
- [ ] Access Dockge at `https://dockge.${DOMAIN}` and deploy remaining stacks
|
||||
- [ ] Configure Homepage dashboard (copy templates to /opt/stacks/homepage/config/)
|
||||
- [ ] Install VS Code with GitHub Copilot extension
|
||||
- [ ] Open repository in VS Code and start using AI assistance
|
||||
|
||||
## Proxying External Hosts
|
||||
|
||||
You can proxy services running on other devices (Raspberry Pi, routers, NAS) through Traefik:
|
||||
|
||||
**Example: Raspberry Pi Home Assistant**
|
||||
```yaml
|
||||
# In /opt/stacks/traefik/dynamic/external.yml
|
||||
http:
|
||||
routers:
|
||||
ha-pi:
|
||||
rule: "Host(`ha.yourdomain.duckdns.org`)"
|
||||
entryPoints:
|
||||
- websecure
|
||||
service: ha-pi
|
||||
tls:
|
||||
certResolver: letsencrypt
|
||||
|
||||
services:
|
||||
ha-pi:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://192.168.1.50:8123"
|
||||
```
|
||||
|
||||
See [docs/proxying-external-hosts.md](docs/proxying-external-hosts.md) for complete guide including:
|
||||
- Three methods (file provider, Docker labels, hybrid)
|
||||
- Authelia bypass configuration
|
||||
- WebSocket support
|
||||
- Examples for routers, NAS, cameras, Proxmox
|
||||
|
||||
## Support
|
||||
|
||||
For issues, questions, or suggestions:
|
||||
- Open an issue on GitHub
|
||||
- Consult the comprehensive [documentation](docs/docker-guidelines.md)
|
||||
For issues, or questions:
|
||||
- Use GitHub Copilot in VS Code for real-time assistance
|
||||
- Consult the comprehensive documentation
|
||||
Reference in New Issue
Block a user