From 47ffc28f0bcad9654ff00f20b16635243a3a5f62 Mon Sep 17 00:00:00 2001 From: kelinfoxy <67766943+kelinfoxy@users.noreply.github.com> Date: Wed, 21 Jan 2026 14:10:03 -0500 Subject: [PATCH] Documentation update --- .github/copilot-instructions.md | 4 +- README.md | 2 +- docs/README.md | 2 +- docs/ai-vscode-setup.md | 101 +++++++++ docs/automated-setup.md | 111 ++++++++++ docs/getting-started.md | 362 ++------------------------------ docs/how-it-works.md | 206 ++++++++++++++++++ docs/post-setup-next-steps.md | 100 +++++++++ docs/services-overview.md | 2 +- docs/ssl-certificates.md | 146 +++++++++++++ 10 files changed, 686 insertions(+), 350 deletions(-) create mode 100644 docs/ai-vscode-setup.md create mode 100644 docs/automated-setup.md create mode 100644 docs/how-it-works.md create mode 100644 docs/post-setup-next-steps.md create mode 100644 docs/ssl-certificates.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 8edd865..5c0f42b 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,6 +1,6 @@ # AI Homelab Management Assistant -You are an AI assistant for the **AI-Homelab** project - a production-ready Docker homelab infrastructure managed through GitHub Copilot in VS Code. This system deploys 70+ services with automated SSL, SSO authentication, and VPN routing using a file-based, AI-manageable architecture. +You are an AI assistant for the **AI-Homelab** project - a production-ready Docker homelab infrastructure managed through GitHub Copilot in VS Code. This system deploys 50+ services with automated SSL, SSO authentication, and VPN routing using a file-based, AI-manageable architecture. ## Project Architecture @@ -303,7 +303,7 @@ labels: - **[Getting Started](../docs/getting-started.md)**: Step-by-step deployment guide - **[Docker Guidelines](../docs/docker-guidelines.md)**: Comprehensive service management patterns (1000+ lines) -- **[Services Reference](../docs/services-overview.md)**: All 70+ pre-configured services +- **[Services Reference](../docs/services-overview.md)**: All 50+ pre-configured services - **[Proxying External Hosts](../docs/proxying-external-hosts.md)**: Traefik file provider patterns for non-Docker services - **[Quick Reference](../docs/quick-reference.md)**: Command cheat sheet and troubleshooting diff --git a/README.md b/README.md index 7efce9f..9a076b4 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ [![Traefik](https://img.shields.io/badge/Traefik-24.0.0-24A1C6)](https://traefik.io) [![Authelia](https://img.shields.io/badge/Authelia-4.38.0-113155)](https://www.authelia.com) -> **Production-ready homelab infrastructure** with automated SSL, SSO authentication, and VPN routing. Deploy 70+ services through a file-based, AI-manageable architecture using Dockge for visual management. +> **Production-ready homelab infrastructure** with automated SSL, SSO authentication, and VPN routing. Deploy 50+ services through a file-based, AI-manageable architecture using Dockge for visual management. ## 🚀 Quick Start diff --git a/docs/README.md b/docs/README.md index 6896cae..b187f97 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,6 @@ # AI-Homelab Documentation -Welcome to the AI-Homelab documentation! This is your comprehensive guide to deploying and managing a production-ready homelab infrastructure with 70+ pre-configured services. +Welcome to the AI-Homelab documentation! This is your comprehensive guide to deploying and managing a production-ready homelab infrastructure with 50+ pre-configured services. ## 📚 Documentation Structure diff --git a/docs/ai-vscode-setup.md b/docs/ai-vscode-setup.md new file mode 100644 index 0000000..0efc5dd --- /dev/null +++ b/docs/ai-vscode-setup.md @@ -0,0 +1,101 @@ +# AI-Assisted VS Code Setup + +This guide will help you set up VS Code with GitHub Copilot to manage your AI-Homelab using AI assistance. + +## Prerequisites + +- VS Code installed on your local machine +- GitHub Copilot extension installed +- SSH access to your homelab server +- Basic familiarity with VS Code + +## Step 1: Install Required Extensions + +1. Open VS Code +2. Go to Extensions (Ctrl+Shift+X) +3. Search for and install: + - **GitHub Copilot** (by GitHub) + - **Remote SSH** (by Microsoft) - for connecting to your server + - **Docker** (by Microsoft) - for Docker support + - **YAML** (by Red Hat) - for editing compose files + +## Step 2: Connect to Your Homelab Server + +1. In VS Code, open the Command Palette (Ctrl+Shift+P) +2. Type "Remote-SSH: Connect to Host..." +3. Enter your server's SSH details: `ssh user@your-server-ip` +4. Authenticate with your password or SSH key + +## Step 3: Open the AI-Homelab Repository + +1. Once connected to your server, open the terminal in VS Code (Ctrl+`) +2. Navigate to your repository: + ```bash + cd ~/AI-Homelab + ``` +3. Open the folder in VS Code: `File > Open Folder` and select `/home/your-user/AI-Homelab` + +## Step 4: Enable GitHub Copilot + +1. Make sure you're signed into GitHub in VS Code +2. GitHub Copilot should activate automatically +3. You can test it by opening a file and typing a comment or code + +## Step 5: Use AI Assistance for Homelab Management + +The AI assistant is configured with comprehensive knowledge of your homelab architecture. You can ask it to: + +### Common Tasks +- **Add new services**: "Add a new service to my media stack" +- **Modify configurations**: "Change the port for my Plex service" +- **Troubleshoot issues**: "Why isn't my service starting?" +- **Update services**: "Update all services to latest versions" +- **Configure routing**: "Add Traefik routing for my new service" + +### How to Interact +1. Open any relevant file (docker-compose.yml, configuration files) +2. Use comments to describe what you want: `# TODO: Add new service here` +3. Or use the chat interface: Ask questions in natural language +4. The AI will suggest edits, create new files, or run commands + +### Example Prompts +- "Create a compose file for a new media service" +- "Help me configure Authelia for a new user" +- "Add VPN routing to my download service" +- "Set up monitoring for my new application" + +## Step 6: Best Practices + +- **Always backup** before making changes +- **Test in isolation** - deploy single services first +- **Use the AI** for complex configurations +- **Read the documentation** linked in responses +- **Validate YAML** before deploying: `docker compose config` + +## Troubleshooting + +### Copilot Not Working +- Check your GitHub subscription includes Copilot +- Ensure you're signed into GitHub in VS Code +- Try reloading VS Code window + +### SSH Connection Issues +- Verify SSH keys are set up correctly +- Check firewall settings on your server +- Ensure SSH service is running + +### AI Not Understanding Context +- Open the relevant files first +- Provide specific file paths +- Include error messages when troubleshooting + +## Next Steps + +Once set up, you can manage your entire homelab through VS Code: +- Deploy new services +- Modify configurations +- Monitor logs +- Troubleshoot issues +- Scale your infrastructure + +The AI assistant follows the same patterns and conventions as your existing setup, ensuring consistency and reliability. \ No newline at end of file diff --git a/docs/automated-setup.md b/docs/automated-setup.md new file mode 100644 index 0000000..a321d61 --- /dev/null +++ b/docs/automated-setup.md @@ -0,0 +1,111 @@ +# Automated Setup (Recommended) + +For most users, the automated setup script handles everything from system preparation to deployment. + +## Prerequisites +- **Fresh Debian/Ubuntu server** (or existing system) +- **Root/sudo access** +- **Internet connection** +- **VS Code with GitHub Copilot** (for AI assistance) + +## Simple Setup + +1. **Connect to your server** via SSH + >Tip: Use VS Code on your local machine to ssh + in to your server for the easiest install! + +2. **Install git if needed** + ```bash + sudo apt update && sudo apt upgrade -y && sudo apt install git +3. **Clone the repository**: + ```bash + git clone https://github.com/kelinfoxy/AI-Homelab.git + cd AI-Homelab +4. **Configure environment**: + ```bash + cp .env.example .env + nano .env # Edit with your domain and tokens + ``` + **Required variables in .env:** + - `DOMAIN` - Your DuckDNS domain (e.g., yourdomain.duckdns.org) + - `DUCKDNS_TOKEN` - Your DuckDNS token from [duckdns.org](https://www.duckdns.org/) + - `ACME_EMAIL` - Your email for Let's Encrypt certificates + - `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN + + **Note:** The `.env` file stays in the repository folder (`~/AI-Homelab/.env`). The deploy script copies it to stack directories automatically. Authelia secrets (JWT, session, encryption key) are auto-generated by the setup script - leave them with default values for now. + +5. **Run the setup script:** + ```bash + sudo ./scripts/setup-homelab.sh + ``` + + The script will: + - Update system packages + - Install Docker Engine + Compose V2 (if needed) + - Configure user groups (docker, sudo) + - Set up firewall (UFW) + - Enable SSH server + - **Generate Authelia secrets** (JWT, session, encryption key) + - **Prompt for admin username, password, and email** + - **Generate argon2id password hash** (30-60 seconds) + - Create `/opt/stacks/` directory structure + - Set up Docker networks (homelab, traefik, dockerproxy, media) + - Detect NVIDIA GPU and offer driver installation + + **Important:** If NVIDIA drivers were installed, reboot your system now before continuing. + +6. **Deploy homelab**: + ```bash + sudo ./scripts/deploy-homelab.sh + ``` + + **The deploy script automatically:** + - Creates Docker networks + - Configures Traefik with your email and domain + - **Obtains wildcard SSL certificate** (*.yourdomain.duckdns.org) via DNS challenge + - Deploys core stack (DuckDNS, Traefik, Authelia, Gluetun) + - Deploys infrastructure stack (Dockge, Pi-hole, monitoring) + - Deploys dashboards stack (Homepage, Homarr) + - Opens Dockge in your browser + + **Note:** Certificate generation may take 2-5 minutes. All services will use the wildcard certificate automatically. + + **Login credentials:** + - Username: `admin` (default username - or the custom username you specified during setup) + - Password: The secure password you created when prompted by the setup script + +**That's it!** Your homelab is ready. +**Access Dockge at `https://dockge.yourdomain.duckdns.org`** + +## What the Setup Script Does + +The `setup-homelab.sh` script is a comprehensive first-run configuration tool: + +**System Preparation:** +- ✅ Pre-flight checks (internet connectivity, disk space 50GB+) +- ✅ Updates system packages +- ✅ Installs required packages (git, curl, etc.) +- ✅ Installs Docker Engine + Compose V2 (if not present) +- ✅ Configures user permissions (docker, sudo groups) +- ✅ Sets up firewall (UFW with SSH, HTTP, HTTPS) +- ✅ Enables SSH server + +**Authelia Configuration (Interactive):** +- ✅ Generates three cryptographic secrets (JWT, session, encryption) +- ✅ Prompts for admin username (default: admin) +- ✅ Prompts for secure password with confirmation +- ✅ Prompts for admin email address +- ✅ Generates argon2id password hash using Docker (30-60s process) +- ✅ Validates Docker is available before password operations +- ✅ Saves credentials securely for deployment script + +**Infrastructure Setup:** +- ✅ Creates directory structure (`/opt/stacks/`) +- ✅ Sets up Docker networks (homelab, traefik, dockerproxy, media) +- ✅ Detects NVIDIA GPU and offers driver installation + +**Safety Features:** +- Skips completed steps (safe to re-run) +- Timeout handling (60s for Docker operations) +- Comprehensive error messages with troubleshooting hints +- Exit on critical failures with clear next steps \ No newline at end of file diff --git a/docs/getting-started.md b/docs/getting-started.md index ea06b65..a13607a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,6 +1,6 @@ # Getting Started Guide -Welcome to your AI-powered homelab! This guide will walk you through setting up your production-ready infrastructure with Dockge, Traefik, Authelia, and 70+ services. +Welcome to your AI-powered homelab! This guide will walk you through setting up your production-ready infrastructure with Dockge, Traefik, Authelia, and 50+ services. ## Getting Started Checklist - [ ] Clone this repository to your home folder @@ -13,359 +13,31 @@ Welcome to your AI-powered homelab! This guide will walk you through setting up - [ ] (optional) Deploy additional stacks as needed via Dockge ([services overview](services-overview.md)) - [ ] Configure and use VS Code with Github Copilot to manage the server ([AI management](.github/copilot-instructions.md)) -## Quick Setup (Recommended) +## Setup Options -For most users, the automated setup script handles everything: +Choose the setup method that works best for you: -### Prerequisites -- **Fresh Debian/Ubuntu server** (or existing system) -- **Root/sudo access** -- **Internet connection** -- **VS Code with GitHub Copilot** (for AI assistance) +### 🚀 Automated Setup (Recommended) +For most users, the automated scripts handle everything. See [Automated Setup Guide](automated-setup.md) for step-by-step instructions. -### Simple Setup +### 🔧 Manual Setup +If you prefer manual control or the automated script fails, see the [Manual Setup Guide](manual-setup.md) for detailed instructions. -1. **Connect to your server** via SSH - >Tip: Use VS Code on your local machine to ssh - in to your server for the easiest install! +### 🤖 AI-Assisted Setup +Learn how to use VS Code with GitHub Copilot for AI-powered homelab management. See [AI VS Code Setup](ai-vscode-setup.md). -2. **Install git if needed** - ```bash - sudo apt update && sudo apt upgrade -y && sudo apt install git -3. **Clone the rep**: - ```bash - git clone https://github.com/kelinfoxy/AI-Homelab.git - cd AI-Homelab -4. **Configure environment**: - ```bash - cp .env.example .env - nano .env # Edit with your domain and tokens - ``` - **Required variables in .env:** - - `DOMAIN` - Your DuckDNS domain (e.g., yourdomain.duckdns.org) - - `DUCKDNS_TOKEN` - Your DuckDNS token from [duckdns.org](https://www.duckdns.org/) - - `ACME_EMAIL` - Your email for Let's Encrypt certificates - - `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN - - **Note:** The `.env` file stays in the repository folder (`~/AI-Homelab/.env`). The deploy script copies it to stack directories automatically. Authelia secrets (JWT, session, encryption key) are auto-generated by the setup script - leave them with default values for now. +## How It All Works -5. **Run the setup script:** - ```bash - sudo ./scripts/setup-homelab.sh - ``` - - The script will: - - Update system packages - - Install Docker Engine + Compose V2 (if needed) - - Configure user groups (docker, sudo) - - Set up firewall (UFW) - - Enable SSH server - - **Generate Authelia secrets** (JWT, session, encryption key) - - **Prompt for admin username, password, and email** - - **Generate argon2id password hash** (30-60 seconds) - - Create `/opt/stacks/` directory structure - - Set up Docker networks (homelab, traefik, dockerproxy, media) - - Detect NVIDIA GPU and offer driver installation - - **Important:** If NVIDIA drivers were installed, reboot your system now before continuing. +Before diving in, understand how your homelab infrastructure works together. See [How Your AI Homelab Works](how-it-works.md) for a comprehensive overview. -6. **Deploy homelab**: - ```bash - sudo ./scripts/deploy-homelab.sh - ``` - - **The deploy script automatically:** - - Creates Docker networks - - Configures Traefik with your email and domain - - **Obtains wildcard SSL certificate** (*.yourdomain.duckdns.org) via DNS challenge - - Deploys core stack (DuckDNS, Traefik, Authelia, Gluetun) - - Deploys infrastructure stack (Dockge, Pi-hole, monitoring) - - Deploys dashboards stack (Homepage, Homarr) - - Opens Dockge in your browser - - **Note:** Certificate generation may take 2-5 minutes. All services will use the wildcard certificate automatically. - - **Login credentials:** - - Username: `admin` (default username - or the custom username you specified during setup) - - Password: The secure password you created when prompted by the setup script +## SSL Certificates -**That's it!** Your homelab is ready. -**Access Dockge at `https://dockge.yourdomain.duckdns.org`** +Your homelab uses Let's Encrypt for automatic HTTPS certificates. See [SSL Certificates Guide](ssl-certificates.md) for details on certificate management and troubleshooting. -## What the Setup Script Does +## What Comes Next -The `setup-homelab.sh` script is a comprehensive first-run configuration tool: +After setup, learn what to do with your running homelab. See [Post-Setup Next Steps](post-setup-next-steps.md) for accessing services, customization, and maintenance. -**System Preparation:** -- ✅ Pre-flight checks (internet connectivity, disk space 50GB+) -- ✅ Updates system packages -- ✅ Installs required packages (git, curl, etc.) -- ✅ Installs Docker Engine + Compose V2 (if not present) -- ✅ Configures user permissions (docker, sudo groups) -- ✅ Sets up firewall (UFW with SSH, HTTP, HTTPS) -- ✅ Enables SSH server +## On-Demand Remote Services -**Authelia Configuration (Interactive):** -- ✅ Generates three cryptographic secrets (JWT, session, encryption) -- ✅ Prompts for admin username (default: admin) -- ✅ Prompts for secure password with confirmation -- ✅ Prompts for admin email address -- ✅ Generates argon2id password hash using Docker (30-60s process) -- ✅ Validates Docker is available before password operations -- ✅ Saves credentials securely for deployment script - -**Infrastructure Setup:** -- ✅ Creates directory structure (`/opt/stacks/`) -- ✅ Sets up Docker networks (homelab, traefik, dockerproxy, media) -- ✅ Detects NVIDIA GPU and offers driver installation - -**Safety Features:** -- Skips completed steps (safe to re-run) -- Timeout handling (60s for Docker operations) -- Comprehensive error messages with troubleshooting hints -- Exit on critical failures with clear next steps - -## Manual Setup (Alternative) - -If you prefer manual control or the automated script fails, see the [Manual Setup Guide](manual-setup.md) for step-by-step instructions on installing Docker, configuring services, and deploying the homelab manually. - -# Notes about SSL Certificates from LetsEncrypt with DuckDNS - -## How SSL Certificates Work in Your Homelab - -Your homelab uses **Let's Encrypt** to automatically provide free SSL certificates for all your services. This ensures secure HTTPS connections without manual certificate management. - -### The Certificate Flow - -1. **Domain Registration**: DuckDNS provides your dynamic domain (e.g., `yourname.duckdns.org`) -2. **Certificate Request**: Traefik requests a wildcard certificate (`*.yourname.duckdns.org`) -3. **Domain Validation**: Let's Encrypt validates you own the domain via DNS challenge -4. **Certificate Issuance**: Free SSL certificate is issued and stored -5. **Automatic Renewal**: Certificates renew automatically before expiration - -### DuckDNS + Let's Encrypt Integration - -**DuckDNS** handles dynamic DNS updates, while **Let's Encrypt** provides certificates: - -- **DuckDNS**: Updates your public IP → domain mapping every 5 minutes -- **Let's Encrypt**: Issues trusted SSL certificates via ACME protocol -- **DNS Challenge**: Proves domain ownership by setting TXT records - -### Wildcard Certificates Explained - -Your setup uses a **wildcard certificate** (`*.yourdomain.duckdns.org`) that covers: -- `dockge.yourdomain.duckdns.org` -- `plex.yourdomain.duckdns.org` -- `jellyfin.yourdomain.duckdns.org` -- Any other subdomain automatically - -**Why wildcard?** One certificate covers all services - no need for individual certificates per service. - -### Certificate Storage & Management - -- **Location**: `/opt/stacks/core/traefik/acme.json` -- **Permissions**: Must be `600` (read/write for owner only) -- **Backup**: Always backup this file - contains your certificates -- **Renewal**: Automatic, 30 days before expiration - -# Testing vs Production Certificates - -### Staging Server (For Testing) -```yaml -# In traefik.yml, add this line for testing: -caServer: https://acme-staging-v02.api.letsencrypt.org/directory -``` - -**Staging Benefits:** -- Unlimited certificates (no rate limits) -- Fast issuance for testing -- **Not trusted by browsers** (shows "Not Secure") - -**When to use staging:** -- Setting up new environments -- Testing configurations -- Learning/development - -### Production Server (For Live Use) -```yaml -# Remove or comment out caServer line for production -# certificatesResolvers: -# letsencrypt: -# acme: -# # No caServer = production -``` - -# Production Limits: - ->**This is why you want to use staging certificates for testing purposes!!!** -**Always use staging certificates if you are running the setup & deploy scripts repeatedly** - -- **50 certificates per domain per week** -- **5 duplicate certificates per week** -- **Trusted by all browsers** - -## Certificate Troubleshooting - -### Check Certificate Status -```bash -# Count certificates in storage -python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')" -``` - -### Common Issues & Solutions - -**"Certificate not trusted" or "Not Secure" warnings:** -- **Staging certificates**: Expected - use production for live sites -- **DNS propagation**: Wait 5-10 minutes after setup -- **Browser cache**: Clear browser cache and try incognito mode - -**Certificate request fails:** -- Check Traefik logs: `docker logs traefik | grep -i certificate` -- Verify DuckDNS token is correct in `.env` -- Ensure ports 80/443 are open and forwarded -- Wait 1+ hours between certificate requests - -**Rate limit exceeded:** -- Switch to staging server for testing -- Wait 1 week for production limits to reset -- Check status at: https://letsencrypt.org/docs/rate-limits/ - -### DNS Challenge Process - -When requesting certificates, Traefik: -1. Asks DuckDNS to set TXT record: `_acme-challenge.yourdomain.duckdns.org` -2. Let's Encrypt checks the TXT record to verify ownership -3. If valid, certificate is issued -4. TXT record is cleaned up automatically - -**Note:** DuckDNS allows only ONE TXT record at a time. Multiple Traefik instances will conflict. - -### Certificate Validation Commands - -```bash -# Test certificate validity -echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername dockge.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer -dates - -# Check if certificate covers wildcards -echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername any-subdomain.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -text | grep "Subject Alternative Name" -``` - -## Best Practices - -### For Production -- Use production Let's Encrypt server -- Backup `acme.json` regularly -- Monitor certificate expiration (Traefik dashboard) -- Keep DuckDNS token secure - -### For Development/Testing -- Use staging server to avoid rate limits -- Test with different subdomains -- Reset environments safely (preserve `acme.json` if possible) - -### Security Notes -- Certificates are stored encrypted in `acme.json` -- Private keys never leave your server -- HTTPS provides encryption in transit -- Consider additional security headers in Traefik - -## Certificate Lifecycle - -- **Validity**: 90 days -- **Renewal**: Automatic, 30 days before expiration -- **Storage**: Persistent across container restarts -- **Backup**: Include in your homelab backup strategy - - -# Post-Setup Next Steps - -### Access Your Services -- **Homepage**: `https://home.yourdomain.duckdns.org` - - Great place to start - - After configuring your services come back and add widgets to the services with the api keys (optional) - - Or ask the AI to find the api keys and add the widgets - -- **Dockge**: `https://dockge.yourdomain.duckdns.org` - - Deploy & Manage the stacks & services - -- **Authelia**: `https://auth.yourdomain.duckdns.org` - - Configure 2FA (optional) - -- **Traefik**: `https://traefik.yourdomain.duckdns.org` - - View/Edit your routing rules - - Tip: Let the AI manage the routing - -- **VS Code**: `https://code.yourdomin.duckdns.org` - - Install GitHub Copilot Chat extension - - Open the AI-Homelab repository - - Use AI assistance for: - - Adding new services - - Configuring Traefik routing - - Managing Docker stacks - -- ## Monitoring services - - Use Dockge to easily view live container logs - - Configure Uptime Kuma to provide uptime tracking with dashboards - -## Debloat or add custom service - -Tell the AI what service you want to install -give it a docker based github repository or docker hub image. -Use your imagination, the copilot instructions are configured with best practices and a framework to add new services. - -### To remove a stack - ```bash - cd /opt/stacks/stack-name - docker compose down - cd .. - sudo rm -rf stack-name - ``` -### To remove the volumes/resources for the stack - ```bash - # Stop stack and remove everything - cd /opt/stacks/stack-name - docker compose down -v --remove-orphans - - # Remove unused Docker resources - docker system prune -a --volumes - ``` - - -## Troubleshooting - -### Script Issues -- **Permission denied**: Run with `sudo` -- **Docker not found**: Log out/in or run `newgrp docker` -- **Network conflicts**: Check existing networks with `docker network ls` - -### Service Issues -- **Can't access services**: Check Traefik dashboard at `https://traefik.yourdomain.duckdns.org` -- **SSL certificate errors**: Wait 2-5 minutes for wildcard certificate to be obtained from Let's Encrypt - - Check status: `python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')"` - - View logs: `docker exec traefik tail -50 /var/log/traefik/traefik.log | grep certificate` -- **Authelia login fails**: Check user database configuration at `/opt/stacks/core/authelia/users_database.yml` -- **"Not secure" warnings**: Clear browser cache or wait for DNS propagation (up to 5 minutes) -- **Check logs**: Use Dozzle web interface at `https://dozzle.yourdomain.duckdns.org` or run `docker logs ` - -### Common Fixes -```bash -# Restart Docker -sudo systemctl restart docker - -# Check service logs -cd /opt/stacks/stack-name -docker compose logs -f - -# Rebuild service -docker compose up -d --build service-name -``` - -## Next Steps - -1. **Explore services** through Dockge -2. **Set up backups** with Backrest (default Restic-based solution) -3. **Set up monitoring** with Grafana/Prometheus -4. **Add external services** via Traefik proxying -5. **Use AI assistance** for custom configurations - -Happy homelabbing! 🚀 +For advanced users: Learn how to set up lazy-loading services on remote servers (like Raspberry Pi) that start automatically when accessed. See [On-Demand Remote Services](Ondemand-Remote-Services.md). diff --git a/docs/how-it-works.md b/docs/how-it-works.md new file mode 100644 index 0000000..9fc2ca8 --- /dev/null +++ b/docs/how-it-works.md @@ -0,0 +1,206 @@ +# How Your AI Homelab Works + +Welcome to your AI-powered homelab! This guide explains how all the components work together to create a production-ready, self-managing infrastructure. Don't worry if it seems complex at first - the AI assistant handles most of the technical details for you. + +## Quick Overview + +Your homelab is a **Docker-based infrastructure** that automatically: +- Provides secure HTTPS access to all services +- Manages user authentication and authorization +- Routes traffic intelligently +- Updates itself +- Backs up your data +- Monitors system health + +Everything runs in **containers** - like lightweight virtual machines - that are orchestrated by **Docker Compose** and managed through the **Dockge** web interface. + +## Core Components + +### 🏠 **Homepage Dashboard** (`https://home.yourdomain.duckdns.org`) +Your central hub for accessing all services. Think of it as the "start menu" for your homelab. +- **What it does**: Shows all your deployed services with quick links +- **AI Integration**: The AI can automatically add new services and configure widgets +- **Customization**: Add weather, system stats, and service-specific widgets + +### 🐳 **Dockge** (`https://dockge.yourdomain.duckdns.org`) +Your primary management interface for deploying and managing services. +- **What it does**: Web-based Docker Compose manager +- **Stacks**: Groups services into logical units (media, monitoring, productivity) +- **One-Click Deploy**: Upload compose files and deploy instantly + +### 🔐 **Authelia** (`https://auth.yourdomain.duckdns.org`) +Your security gatekeeper that protects sensitive services. +- **What it does**: Single sign-on (SSO) authentication +- **Security**: Two-factor authentication, session management +- **Smart Bypass**: Automatically bypasses auth for media apps (Plex, Jellyfin) + +### 🌐 **Traefik** (`https://traefik.yourdomain.duckdns.org`) +Your intelligent traffic director and SSL certificate manager. +- **What it does**: Reverse proxy that routes web traffic to the right services +- **SSL**: Automatically obtains and renews free HTTPS certificates +- **Labels**: Services "advertise" themselves to Traefik via Docker labels + +### 🦆 **DuckDNS** +Your dynamic DNS service that gives your homelab a consistent domain name. +- **What it does**: Updates `yourdomain.duckdns.org` to point to your home IP +- **Integration**: Works with Traefik to get wildcard SSL certificates + +### 🛡️ **Gluetun (VPN)** +Your download traffic protector. +- **What it does**: Routes torrent and download traffic through VPN +- **Security**: Prevents ISP throttling and hides your IP for downloads +- **Integration**: Download services connect through Gluetun's network + +## How Services Get Added + +### The AI Way (Recommended) +1. **Tell the AI**: "Add Plex to my media stack" +2. **AI Creates**: Docker Compose file with proper configuration +3. **AI Configures**: Traefik routing, Authelia protection, resource limits +4. **AI Deploys**: Service goes live with HTTPS and SSO +5. **AI Updates**: Homepage dashboard automatically + +### Manual Way +1. **Find Service**: Choose from 50+ pre-configured services +2. **Upload to Dockge**: Use the web interface +3. **Configure**: Set environment variables and volumes +4. **Deploy**: Click deploy and wait +5. **Access**: Service is immediately available at `https://servicename.yourdomain.duckdns.org` + +## Network Architecture + +### Internal Networks +- **`traefik-network`**: All web-facing services connect here +- **`homelab-network`**: Internal service communication +- **`media-network`**: Media services (Plex, Jellyfin, etc.) +- **VPN Networks**: Download services route through Gluetun + +### External Access +- **Port 80/443**: Only Traefik exposes these to the internet +- **Domain**: `*.yourdomain.duckdns.org` points to your home +- **SSL**: Wildcard certificate covers all subdomains automatically + +## Storage Strategy + +### Configuration Files +- **Location**: `/opt/stacks/stack-name/config/` +- **Purpose**: Service settings, databases, user data +- **Backup**: Included in automatic backups + +### Media & Large Data +- **Location**: `/mnt/media/`, `/mnt/downloads/` +- **Purpose**: Movies, TV shows, music, downloads +- **Performance**: Direct mounted drives for speed + +### Secrets & Environment +- **Location**: `.env` files in each stack directory +- **Security**: Never committed to git +- **Management**: AI can help update variables + +## AI Features + +### VS Code Integration +- **Copilot Chat**: Natural language commands for infrastructure management +- **File Editing**: AI modifies Docker Compose files, configuration YAML +- **Troubleshooting**: AI analyzes logs and suggests fixes +- **Documentation**: AI keeps docs synchronized with deployed services + +### OpenWebUI (Future) +- **Web Interface**: Chat with AI directly in your browser +- **API Tools**: AI can interact with your services' APIs +- **Workflows**: Automated service management and monitoring +- **Status**: Currently in development phase + +## Lazy Loading (Sablier) + +Some services start **on-demand** to save resources: +- **How it works**: Service starts when you first access it +- **Benefits**: Saves RAM and CPU when services aren't in use +- **Configuration**: AI manages the lazy loading rules + +## Monitoring & Maintenance + +### Built-in Monitoring +- **Grafana/Prometheus**: System metrics and dashboards +- **Uptime Kuma**: Service uptime monitoring +- **Dozzle**: Live container log viewing +- **Node Exporter**: Hardware monitoring + +### Automatic Updates +- **Watchtower**: Updates Docker images automatically +- **Backrest**: Scheduled backups using Restic +- **Certificate Renewal**: SSL certificates renew automatically + +## Security Model + +### Defense in Depth +1. **Network Level**: Firewall blocks unauthorized access +2. **SSL/TLS**: All traffic encrypted with valid certificates +3. **Authentication**: Authelia protects admin interfaces +4. **Authorization**: User roles and permissions +5. **Container Security**: Services run as non-root users + +### VPN for Downloads +- **Purpose**: Hide IP address for torrenting +- **Implementation**: Download containers route through VPN +- **Provider**: Surfshark (configurable) + +## Scaling & Customization + +### Adding Services +- **Pre-built**: 50+ services ready to deploy +- **Custom**: AI can create configurations for any Docker service +- **External**: Proxy services on other devices (Raspberry Pi, NAS) + +### Resource Management +- **Limits**: CPU, memory, and I/O limits prevent resource exhaustion +- **Reservations**: Guaranteed minimum resources +- **GPU Support**: Automatic NVIDIA GPU detection and configuration + +## Troubleshooting Philosophy + +### Logs First +Every service provides detailed logs. The AI can help analyze them. + +### Isolation Testing +Deploy services one at a time to identify conflicts. + +### Configuration Validation +AI validates Docker Compose syntax before deployment. + +### Rollback Ready +Previous configurations are preserved for quick recovery. + +## Getting Help + +### Documentation Links +- **[Automated Setup](automated-setup.md)**: Step-by-step deployment +- **[SSL Certificates](ssl-certificates.md)**: HTTPS configuration details +- **[Post-Setup](post-setup-next-steps.md)**: What to do after deployment +- **[AI VS Code Setup](ai-vscode-setup.md)**: Configure AI assistance +- **[Services Overview](../docs/services-overview.md)**: All available services +- **[Docker Guidelines](../docs/docker-guidelines.md)**: Technical details + +### AI Assistance +- **In VS Code**: Use GitHub Copilot Chat for instant help +- **Examples**: + - "Add a new service to my homelab" + - "Fix SSL certificate issues" + - "Configure backup for my data" + - "Set up monitoring dashboard" + +### Community Resources +- **GitHub Issues**: Report bugs or request features +- **Discussions**: Ask questions and share configurations +- **Wiki**: Community-contributed guides and tutorials + +## Architecture Summary + +Your homelab follows these principles: +- **Infrastructure as Code**: Everything defined in files +- **GitOps**: Version control for all configurations +- **Security First**: SSO protection by default +- **AI-Assisted**: Intelligent management and troubleshooting +- **Production Ready**: Monitoring, backups, and high availability + +The result is a powerful, secure, and easy-to-manage homelab that grows with your needs! \ No newline at end of file diff --git a/docs/post-setup-next-steps.md b/docs/post-setup-next-steps.md new file mode 100644 index 0000000..d16bd4a --- /dev/null +++ b/docs/post-setup-next-steps.md @@ -0,0 +1,100 @@ +# Post-Setup Next Steps + +Congratulations! Your AI-powered homelab is now running. Here's what to do next. + +## Access Your Services + +- **Homepage**: `https://home.yourdomain.duckdns.org` + - Great place to start exploring your services + - After configuring your services, come back and add widgets with API keys (optional) + - Or ask the AI to find the API keys and add the widgets + +- **Dockge**: `https://dockge.yourdomain.duckdns.org` + - Deploy & Manage the stacks & services + - Your primary management interface + +- **Authelia**: `https://auth.yourdomain.duckdns.org` + - Configure 2FA for enhanced security (optional) + +- **Traefik**: `https://traefik.yourdomain.duckdns.org` + - View/Edit your routing rules + - Tip: Let the AI manage the routing for you + +- **VS Code**: `https://code.yourdomain.duckdns.org` + - Install GitHub Copilot Chat extension + - Open the AI-Homelab repository + - Use AI assistance for: + - Adding new services + - Configuring Traefik routing + - Managing Docker stacks + +## Monitoring Services + +- Use Dockge to easily view live container logs +- Configure Uptime Kuma to provide uptime tracking with dashboards +- Check Grafana for system metrics and monitoring + +## Customize Your Homelab + +### Add Custom Services + +Tell the AI what service you want to install - give it a Docker-based GitHub repository or Docker Hub image. Use your imagination, the Copilot instructions are configured with best practices and a framework to add new services. + +### Remove Unwanted Services + +To remove a stack: +```bash +cd /opt/stacks/stack-name +docker compose down +cd .. +sudo rm -rf stack-name +``` + +To remove the volumes/resources for the stack: +```bash +# Stop stack and remove everything +cd /opt/stacks/stack-name +docker compose down -v --remove-orphans + +# Remove unused Docker resources +docker system prune -a --volumes +``` + +## Troubleshooting + +### Script Issues +- **Permission denied**: Run with `sudo` +- **Docker not found**: Log out/in or run `newgrp docker` +- **Network conflicts**: Check existing networks with `docker network ls` + +### Service Issues +- **Can't access services**: Check Traefik dashboard at `https://traefik.yourdomain.duckdns.org` +- **SSL certificate errors**: Wait 2-5 minutes for wildcard certificate to be obtained from Let's Encrypt + - Check status: `python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')"` + - View logs: `docker exec traefik tail -50 /var/log/traefik/traefik.log | grep certificate` +- **Authelia login fails**: Check user database configuration at `/opt/stacks/core/authelia/users_database.yml` +- **"Not secure" warnings**: Clear browser cache or wait for DNS propagation (up to 5 minutes) +- **Check logs**: Use Dozzle web interface at `https://dozzle.yourdomain.duckdns.org` or run `docker logs ` + +### Common Fixes +```bash +# Restart Docker +sudo systemctl restart docker + +# Check service logs +cd /opt/stacks/stack-name +docker compose logs -f + +# Rebuild service +docker compose up -d --build service-name +``` + +## Next Steps + +1. **Explore services** through Dockge +2. **Set up backups** with Backrest (default Restic-based solution) +3. **Set up monitoring** with Grafana/Prometheus +4. **Add external services** via Traefik proxying +5. **Use AI assistance** for custom configurations + +Happy homelabbing! 🚀 \ No newline at end of file diff --git a/docs/services-overview.md b/docs/services-overview.md index d126750..62d13ed 100644 --- a/docs/services-overview.md +++ b/docs/services-overview.md @@ -1,6 +1,6 @@ # Services Overview -This document provides a comprehensive overview of all 70+ pre-configured services available in the AI-Homelab repository. +This document provides a comprehensive overview of all 50+ pre-configured services available in the AI-Homelab repository. ## Services Overview diff --git a/docs/ssl-certificates.md b/docs/ssl-certificates.md new file mode 100644 index 0000000..eabe2ee --- /dev/null +++ b/docs/ssl-certificates.md @@ -0,0 +1,146 @@ +# SSL Certificates with Let's Encrypt and DuckDNS + +Your homelab uses **Let's Encrypt** to automatically provide free SSL certificates for all your services. This ensures secure HTTPS connections without manual certificate management. + +## How SSL Certificates Work in Your Homelab + +### The Certificate Flow + +1. **Domain Registration**: DuckDNS provides your dynamic domain (e.g., `yourname.duckdns.org`) +2. **Certificate Request**: Traefik requests a wildcard certificate (`*.yourname.duckdns.org`) +3. **Domain Validation**: Let's Encrypt validates you own the domain via DNS challenge +4. **Certificate Issuance**: Free SSL certificate is issued and stored +5. **Automatic Renewal**: Certificates renew automatically before expiration + +### DuckDNS + Let's Encrypt Integration + +**DuckDNS** handles dynamic DNS updates, while **Let's Encrypt** provides certificates: + +- **DuckDNS**: Updates your public IP → domain mapping every 5 minutes +- **Let's Encrypt**: Issues trusted SSL certificates via ACME protocol +- **DNS Challenge**: Proves domain ownership by setting TXT records + +### Wildcard Certificates Explained + +Your setup uses a **wildcard certificate** (`*.yourdomain.duckdns.org`) that covers: +- `dockge.yourdomain.duckdns.org` +- `plex.yourdomain.duckdns.org` +- `jellyfin.yourdomain.duckdns.org` +- Any other subdomain automatically + +**Why wildcard?** One certificate covers all services - no need for individual certificates per service. + +### Certificate Storage & Management + +- **Location**: `/opt/stacks/core/traefik/acme.json` +- **Permissions**: Must be `600` (read/write for owner only) +- **Backup**: Always backup this file - contains your certificates +- **Renewal**: Automatic, 30 days before expiration + +## Testing vs Production Certificates + +### Staging Server (For Testing) +```yaml +# In traefik.yml, add this line for testing: +caServer: https://acme-staging-v02.api.letsencrypt.org/directory +``` + +**Staging Benefits:** +- Unlimited certificates (no rate limits) +- Fast issuance for testing +- **Not trusted by browsers** (shows "Not Secure") + +**When to use staging:** +- Setting up new environments +- Testing configurations +- Learning/development + +### Production Server (For Live Use) +```yaml +# Remove or comment out caServer line for production +# certificatesResolvers: +# letsencrypt: +# acme: +# # No caServer = production +``` + +**Production Limits:** + +>This is why you want to use staging certificates for testing purposes!!! +>Always use staging certificates if you are running the setup & deploy scripts repeatedly + +- **50 certificates per domain per week** +- **5 duplicate certificates per week** +- **Trusted by all browsers** + +## Certificate Troubleshooting + +### Check Certificate Status +```bash +# Count certificates in storage +python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')}" +``` + +### Common Issues & Solutions + +**"Certificate not trusted" or "Not Secure" warnings:** +- **Staging certificates**: Expected - use production for live sites +- **DNS propagation**: Wait 5-10 minutes after setup +- **Browser cache**: Clear browser cache and try incognito mode + +**Certificate request fails:** +- Check Traefik logs: `docker logs traefik | grep -i certificate` +- Verify DuckDNS token is correct in `.env` +- Ensure ports 80/443 are open and forwarded +- Wait 1+ hours between certificate requests + +**Rate limit exceeded:** +- Switch to staging server for testing +- Wait 1 week for production limits to reset +- Check status at: https://letsencrypt.org/docs/rate-limits/ + +### DNS Challenge Process + +When requesting certificates, Traefik: +1. Asks DuckDNS to set TXT record: `_acme-challenge.yourdomain.duckdns.org` +2. Let's Encrypt checks the TXT record to verify ownership +3. If valid, certificate is issued +4. TXT record is cleaned up automatically + +**Note:** DuckDNS allows only ONE TXT record at a time. Multiple Traefik instances will conflict. + +### Certificate Validation Commands + +```bash +# Test certificate validity +echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername dockge.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer -dates + +# Check if certificate covers wildcards +echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername any-subdomain.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -text | grep "Subject Alternative Name" +``` + +## Best Practices + +### For Production +- Use production Let's Encrypt server +- Backup `acme.json` regularly +- Monitor certificate expiration (Traefik dashboard) +- Keep DuckDNS token secure + +### For Development/Testing +- Use staging server to avoid rate limits +- Test with different subdomains +- Reset environments safely (preserve `acme.json` if possible) + +### Security Notes +- Certificates are stored encrypted in `acme.json` +- Private keys never leave your server +- HTTPS provides encryption in transit +- Consider additional security headers in Traefik + +## Certificate Lifecycle + +- **Validity**: 90 days +- **Renewal**: Automatic, 30 days before expiration +- **Storage**: Persistent across container restarts +- **Backup**: Include in your homelab backup strategy \ No newline at end of file