EZ-Homelab/docs/getting-started.md

# Getting Started Guide

Welcome to your AI-powered homelab! This guide will walk you through setting up your production-ready infrastructure with Dockge, Traefik, Authelia, and 60+ services.

## Getting Started Checklist
- [ ] Clone this repository to your home folder
- [ ] Configure `.env` file with your domain and tokens
- [ ] Run setup script (generates Authelia secrets and admin user)
- [ ] Log out and back in for Docker group permissions
- [ ] Run deployment script (deploys all core, infrastructure & dashboard services)
- [ ] Access Dockge web UI
- [ ] Set up 2FA with Authelia
- [ ] (optional) Deploy additional stacks as needed via Dockge
- [ ] Configure and use VS Code with Github Copilot to manage the server

## Quick Setup (Recommended)

For most users, the automated setup script handles everything:

### Prerequisites
- **Fresh Debian/Ubuntu server** (or existing system)
- **Root/sudo access**
- **Internet connection**
- **VS Code with GitHub Copilot** (for AI assistance)

### Simple Setup

1. **Connect to your server** via SSH
   >Tip: Use VS Code on your local machine to ssh  
   in to your server for the easiest install!

2. **Install git if needed**
   ```bash
      sudo apt update && sudo apt upgrade -y && sudo apt install git
3. **Clone the rep**:
   ```bash
   git clone https://github.com/kelinfoxy/AI-Homelab.git
   cd AI-Homeb
4. **Configure environment**:
   ```bash
   cp .env.example .env
   nano .env  # Edit with your domain and tokens
   ```
   **Required variables in .env:**
   - `DOMAIN` - Your DuckDNS domain (e.g., yourdomain.duckdns.org)
   - `DUCKDNS_TOKEN` - Your DuckDNS token from [duckdns.org](https://www.duckdns.org/)
   - `ACME_EMAIL` - Your email for Let's Encrypt certificates
   - `SURFSHARK_USERNAME` and `SURFSHARK_PASSWORD` - If using VPN
   
   **Note:** The `.env` file stays in the repository folder (`~/AI-Homelab/.env`). The deploy script copies it to stack directories automatically. Authelia secrets (JWT, session, encryption key) are auto-generated by the setup script - leave them with default values for now.

5. **Run the setup script:**
   ```bash
   sudo ./scripts/setup-homelab.sh
   ```
   
   The script will:
   - Update system packages
   - Install Docker Engine + Compose V2 (if needed)
   - Configure user groups (docker, sudo)
   - Set up firewall (UFW)
   - Enable SSH server
   - **Generate Authelia secrets** (JWT, session, encryption key)
   - **Prompt for admin username, password, and email**
   - **Generate argon2id password hash** (30-60 seconds)
   - Create `/opt/stacks/` directory structure
   - Set up Docker networks (homelab, traefik, dockerproxy, media)
   - Detect NVIDIA GPU and offer driver installation
   
   **Important:** If NVIDIA drivers were installed, reboot your system now before continuing.

6. **Deploy homelab**:
   ```bash
   sudo ./scripts/deploy-homelab.sh
   ```
   
   **The deploy script automatically:**
   - Creates Docker networks
   - Configures Traefik with your email and domain
   - **Obtains wildcard SSL certificate** (*.yourdomain.duckdns.org) via DNS challenge
   - Deploys core stack (DuckDNS, Traefik, Authelia, Gluetun)
   - Deploys infrastructure stack (Dockge, Pi-hole, monitoring)
   - Deploys dashboards stack (Homepage, Homarr)
   - Opens Dockge in your browser
   
   **Note:** Certificate generation may take 2-5 minutes. All services will use the wildcard certificate automatically.
   
   **Login credentials:**
   - Username: `admin` (default username - or the custom username you specified during setup)
   - Password: The secure password you created when prompted by the setup script

**That's it!** Your homelab is ready.  
**Access Dockge at `https://dockge.yourdomain.duckdns.org`**

## What the Setup Script Does

The `setup-homelab.sh` script is a comprehensive first-run configuration tool:

**System Preparation:**
- ✅ Pre-flight checks (internet connectivity, disk space 50GB+)
- ✅ Updates system packages
- ✅ Installs required packages (git, curl, etc.)
- ✅ Installs Docker Engine + Compose V2 (if not present)
- ✅ Configures user permissions (docker, sudo groups)
- ✅ Sets up firewall (UFW with SSH, HTTP, HTTPS)
- ✅ Enables SSH server

**Authelia Configuration (Interactive):**
- ✅ Generates three cryptographic secrets (JWT, session, encryption)
- ✅ Prompts for admin username (default: admin)
- ✅ Prompts for secure password with confirmation
- ✅ Prompts for admin email address
- ✅ Generates argon2id password hash using Docker (30-60s process)
- ✅ Validates Docker is available before password operations
- ✅ Saves credentials securely for deployment script

**Infrastructure Setup:**
- ✅ Creates directory structure (`/opt/stacks/`)
- ✅ Sets up Docker networks (homelab, traefik, dockerproxy, media)
- ✅ Detects NVIDIA GPU and offers driver installation

**Safety Features:**
- Skips completed steps (safe to re-run)
- Timeout handling (60s for Docker operations)
- Comprehensive error messages with troubleshooting hints
- Exit on critical failures with clear next steps

## Manual Setup (Alternative)

If you prefer manual control or the automated script fails, see the [Manual Setup Guide](manual-setup.md) for step-by-step instructions on installing Docker, configuring services, and deploying the homelab manually.

# Notes about SSL Certificates from LetsEncrypt with DuckDNS

## How SSL Certificates Work in Your Homelab

Your homelab uses **Let's Encrypt** to automatically provide free SSL certificates for all your services. This ensures secure HTTPS connections without manual certificate management.

### The Certificate Flow

1. **Domain Registration**: DuckDNS provides your dynamic domain (e.g., `yourname.duckdns.org`)
2. **Certificate Request**: Traefik requests a wildcard certificate (`*.yourname.duckdns.org`)
3. **Domain Validation**: Let's Encrypt validates you own the domain via DNS challenge
4. **Certificate Issuance**: Free SSL certificate is issued and stored
5. **Automatic Renewal**: Certificates renew automatically before expiration

### DuckDNS + Let's Encrypt Integration

**DuckDNS** handles dynamic DNS updates, while **Let's Encrypt** provides certificates:

- **DuckDNS**: Updates your public IP → domain mapping every 5 minutes
- **Let's Encrypt**: Issues trusted SSL certificates via ACME protocol
- **DNS Challenge**: Proves domain ownership by setting TXT records

### Wildcard Certificates Explained

Your setup uses a **wildcard certificate** (`*.yourdomain.duckdns.org`) that covers:
- `dockge.yourdomain.duckdns.org`
- `plex.yourdomain.duckdns.org`
- `jellyfin.yourdomain.duckdns.org`
- Any other subdomain automatically

**Why wildcard?** One certificate covers all services - no need for individual certificates per service.

### Certificate Storage & Management

- **Location**: `/opt/stacks/core/traefik/acme.json`
- **Permissions**: Must be `600` (read/write for owner only)
- **Backup**: Always backup this file - contains your certificates
- **Renewal**: Automatic, 30 days before expiration

# Testing vs Production Certificates

### Staging Server (For Testing)
```yaml
# In traefik.yml, add this line for testing:
caServer: https://acme-staging-v02.api.letsencrypt.org/directory
```

**Staging Benefits:**
- Unlimited certificates (no rate limits)
- Fast issuance for testing
- **Not trusted by browsers** (shows "Not Secure")

**When to use staging:**
- Setting up new environments
- Testing configurations
- Learning/development

### Production Server (For Live Use)
```yaml
# Remove or comment out caServer line for production
# certificatesResolvers:
#   letsencrypt:
#     acme:
#       # No caServer = production
```

# Production Limits: 

>**This is why you want to use staging certificates for testing purposes!!!**  
**Always use staging certificates if you are running the setup & deploy scripts repeatedly**

- **50 certificates per domain per week**
- **5 duplicate certificates per week**
- **Trusted by all browsers**

## Certificate Troubleshooting

### Check Certificate Status
```bash
# Count certificates in storage
python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')"
```

### Common Issues & Solutions

**"Certificate not trusted" or "Not Secure" warnings:**
- **Staging certificates**: Expected - use production for live sites
- **DNS propagation**: Wait 5-10 minutes after setup
- **Browser cache**: Clear browser cache and try incognito mode

**Certificate request fails:**
- Check Traefik logs: `docker logs traefik | grep -i certificate`
- Verify DuckDNS token is correct in `.env`
- Ensure ports 80/443 are open and forwarded
- Wait 1+ hours between certificate requests

**Rate limit exceeded:**
- Switch to staging server for testing
- Wait 1 week for production limits to reset
- Check status at: https://letsencrypt.org/docs/rate-limits/

### DNS Challenge Process

When requesting certificates, Traefik:
1. Asks DuckDNS to set TXT record: `_acme-challenge.yourdomain.duckdns.org`
2. Let's Encrypt checks the TXT record to verify ownership
3. If valid, certificate is issued
4. TXT record is cleaned up automatically

**Note:** DuckDNS allows only ONE TXT record at a time. Multiple Traefik instances will conflict.

### Certificate Validation Commands

```bash
# Test certificate validity
echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername dockge.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -subject -issuer -dates

# Check if certificate covers wildcards
echo | openssl s_client -connect yourdomain.duckdns.org:443 -servername any-subdomain.yourdomain.duckdns.org 2>/dev/null | openssl x509 -noout -text | grep "Subject Alternative Name"
```

## Best Practices

### For Production
- Use production Let's Encrypt server
- Backup `acme.json` regularly
- Monitor certificate expiration (Traefik dashboard)
- Keep DuckDNS token secure

### For Development/Testing
- Use staging server to avoid rate limits
- Test with different subdomains
- Reset environments safely (preserve `acme.json` if possible)

### Security Notes
- Certificates are stored encrypted in `acme.json`
- Private keys never leave your server
- HTTPS provides encryption in transit
- Consider additional security headers in Traefik

## Certificate Lifecycle

- **Validity**: 90 days
- **Renewal**: Automatic, 30 days before expiration
- **Storage**: Persistent across container restarts
- **Backup**: Include in your homelab backup strategy


# Post-Setup Next Steps

### Access Your Services
- **Homepage**: `https://home.yourdomain.duckdns.org`
  - Great place to start
  - After configuring your services come back and add widgets to the services with the api keys (optional)
  - Or ask the AI to find the api keys and add the widgets

- **Dockge**: `https://dockge.yourdomain.duckdns.org`
  - Deploy & Manage the stacks & services

- **Authelia**: `https://auth.yourdomain.duckdns.org`
  - Configure 2FA (optional)

- **Traefik**: `https://traefik.yourdomain.duckdns.org`
  - View/Edit your routing rules
  - Tip: Let the AI manage the routing

- **VS Code**: `https://code.yourdomin.duckdns.org`
   - Install GitHub Copilot Chat extension
   - Open the AI-Homelab repository
   - Use AI assistance for:
      - Adding new services
      - Configuring Traefik routing
      - Managing Docker stacks

## Debloat or add custom service

Tell the AI what service you want to install or give it a docker based github repository or docker hub image.
Use your imagination, the copilot instructions are configured with best practices and a framework to add new services.

## Troubleshooting

### Script Issues
- **Permission denied**: Run with `sudo`
- **Docker not found**: Log out/in or run `newgrp docker`
- **Network conflicts**: Check existing networks with `docker network ls`

### Service Issues
- **Can't access services**: Check Traefik dashboard at `https://traefik.yourdomain.duckdns.org`
- **SSL certificate errors**: Wait 2-5 minutes for wildcard certificate to be obtained from Let's Encrypt
  - Check status: `python3 -c "import json; d=json.load(open('/opt/stacks/core/traefik/acme.json')); print(f'Certificates: {len(d[\"letsencrypt\"][\"Certificates\"])}')"`
  - View logs: `docker exec traefik tail -50 /var/log/traefik/traefik.log | grep certificate`
- **Authelia login fails**: Check user database configuration at `/opt/stacks/core/authelia/users_database.yml`
- **"Not secure" warnings**: Clear browser cache or wait for DNS propagation (up to 5 minutes)
- **Check logs**: Use Dozzle web interface at `https://dozzle.yourdomain.duckdns.org` or run `docker logs <container-name>`

### Common Fixes
```bash
# Restart Docker
sudo systemctl restart docker

# Check service logs
cd /opt/stacks/stack-name
docker compose logs -f

# Rebuild service
docker compose up -d --build service-name
```

## Managing Docker Stacks

### Removing a Stack

**Important:** Simply deleting the folder is NOT enough! You must properly stop the containers first.

#### Safe Stack Removal Process:

1. **Navigate to the stack directory:**
   ```bash
   cd /opt/stacks/stack-name
   ```

2. **Stop and remove containers:**
   ```bash
   docker compose down
   ```

3. **Optional: Remove volumes (WARNING: This deletes all data!):**
   ```bash
   docker compose down -v  # Removes named volumes
   ```

4. **Optional: Remove images:**
   ```bash
   docker compose down --rmi all  # Removes all images used by the stack
   ```

5. **Delete the stack folder:**
   ```bash
   cd /opt/stacks
   sudo rm -rf stack-name
   ```

#### Complete Cleanup (Including Orphaned Resources):

```bash
# Stop stack and remove everything
cd /opt/stacks/stack-name
docker compose down -v --remove-orphans

# Remove unused Docker resources
docker system prune -a --volumes

# Remove specific images if needed
docker rmi image-name:tag
```

### Before Removing a Stack:

- **Backup important data** from volumes or bind mounts
- **Check dependencies** - other services might rely on this stack
- **Review Traefik routes** - remove any custom routing rules
- **Check network usage** - stacks might create custom networks

### What Happens If You Just Delete the Folder?

- **Containers keep running** - they'll continue consuming resources
- **Data remains** - volumes and bind mounts are preserved
- **Networks persist** - custom networks aren't automatically cleaned up
- **Zombie processes** - orphaned containers may cause conflicts

**Always use `docker compose down` before deleting the folder!**

### Restoring a Removed Stack:

If you accidentally removed a stack and want it back:

1. **Check if it's in the repository:**
   ```bash
   cd ~/AI-Homelab/docker-compose
   ls -la stack-name.yml
   ```

2. **Redeploy from Dockge** or run:
   ```bash
   cd ~/AI-Homelab/docker-compose
   cp stack-name.yml /opt/stacks/
   cd /opt/stacks/stack-name
   docker compose up -d
   ```

## Next Steps

1. **Explore services** through Dockge
2. **Configure backups** with Backrest/Duplicati
3. **Set up monitoring** with Grafana/Prometheus
4. **Add external services** via Traefik proxying
5. **Use AI assistance** for custom configurations

Happy homelabbing! 🚀