Complete documentation refactoring with 57 service docs

- Refactored README.md, getting-started.md, quick-reference.md - Enhanced setup-homelab.sh with 9-step automated process - Created services-overview.md with all stacks - Added comprehensive documentation for 57 services in docs/service-docs/ - All services include: overview, configuration, resources, educational content - Coverage: Core, Infrastructure, Dashboards, Media, Media-Extended, Home Assistant, Productivity, Utilities, Monitoring, Development stacks - Educational focus with links to tutorials, videos, and guides
2026-01-12 18:03:12 -05:00
parent 906e6008ea
commit cba45556c7
69 changed files with 24404 additions and 1522 deletions
--- a/docs/service-docs/watchtower.md
+++ b/docs/service-docs/watchtower.md
@@ -0,0 +1,579 @@
+# Watchtower - Automated Container Updates
+
+## Table of Contents
+- [Overview](#overview)
+- [What is Watchtower?](#what-is-watchtower)
+- [Why Use Watchtower?](#why-use-watchtower)
+- [How It Works](#how-it-works)
+- [Configuration in AI-Homelab](#configuration-in-ai-homelab)
+- [Official Resources](#official-resources)
+- [Educational Resources](#educational-resources)
+- [Docker Configuration](#docker-configuration)
+- [Controlling Updates](#controlling-updates)
+- [Advanced Topics](#advanced-topics)
+- [Troubleshooting](#troubleshooting)
+
+## Overview
+
+**Category:** Infrastructure Automation  
+**Docker Image:** [containrrr/watchtower](https://hub.docker.com/r/containrrr/watchtower)  
+**Default Stack:** `infrastructure.yml`  
+**Web UI:** None (runs as automated service)  
+**Default Behavior:** Checks for updates daily at 4 AM
+
+## What is Watchtower?
+
+Watchtower is an automated Docker container update service that monitors running containers and automatically updates them when new images are available. It pulls new images, stops old containers, and starts new ones while preserving volumes and configuration.
+
+### Key Features
+- **Automatic Updates:** Keeps containers up-to-date automatically
+- **Schedule Control:** Cron-based update scheduling
+- **Selective Monitoring:** Choose which containers to update
+- **Notifications:** Email, Slack, Discord, etc. on updates
+- **Update Strategies:** Rolling updates, one-time runs
+- **Cleanup:** Automatically removes old images
+- **Dry Run Mode:** Test without actually updating
+- **Label-Based Control:** Fine-grained update control per container
+- **Zero Downtime:** Seamless container recreation
+
+## Why Use Watchtower?
+
+1. **Security:** Automatic security patches
+2. **Convenience:** No manual image updates needed
+3. **Consistency:** All containers stay updated
+4. **Time Savings:** Automates tedious update process
+5. **Minimal Downtime:** Fast container recreation
+6. **Safe Updates:** Preserves volumes and configuration
+7. **Rollback Support:** Keep old images for fallback
+8. **Notification:** Get notified of updates
+
+## How It Works
+
+```
+Watchtower (scheduled check)
+     ↓
+Check Docker Hub for new images
+     ↓
+Compare with local image digests
+     ↓
+If new version found:
+     ├─ Pull new image
+     ├─ Stop old container
+     ├─ Remove old container
+     ├─ Create new container (same config)
+     ├─ Start new container
+     └─ Remove old image (optional)
+```
+
+### Update Process
+
+1. **Scheduled Check:** Watchtower wakes up (e.g., 4 AM daily)
+2. **Scan Containers:** Finds all monitored containers
+3. **Check Registries:** Queries Docker Hub/registries for new images
+4. **Compare Digests:** Checks if image hash changed
+5. **Pull New Image:** Downloads latest version
+6. **Recreate Container:** Stops old, starts new with same config
+7. **Cleanup:** Removes old images (if configured)
+8. **Notify:** Sends notification (if configured)
+
+### What Gets Preserved
+
+✅ **Preserved:**
+- Volumes and data
+- Networks
+- Environment variables
+- Labels
+- Port mappings
+- Restart policy
+
+❌ **Not Preserved:**
+- Running processes inside container
+- Temporary files in container filesystem
+- In-memory data
+
+## Configuration in AI-Homelab
+
+### Directory Structure
+
+```
+# Watchtower doesn't need persistent storage
+# All configuration via environment variables
+```
+
+### Environment Variables
+
+```bash
+# Schedule (cron format)
+WATCHTOWER_SCHEDULE=0 0 4 * * *  # Daily at 4 AM
+
+# Cleanup old images
+WATCHTOWER_CLEANUP=true
+
+# Include stopped containers
+WATCHTOWER_INCLUDE_STOPPED=false
+
+# Include restarting containers
+WATCHTOWER_INCLUDE_RESTARTING=false
+
+# Debug logging
+WATCHTOWER_DEBUG=false
+
+# Notifications (optional)
+WATCHTOWER_NOTIFICATIONS=email
+WATCHTOWER_NOTIFICATION_EMAIL_FROM=watchtower@yourdomain.com
+WATCHTOWER_NOTIFICATION_EMAIL_TO=admin@yourdomain.com
+WATCHTOWER_NOTIFICATION_EMAIL_SERVER=smtp.gmail.com
+WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=587
+WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=your-email@gmail.com
+WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=your-app-password
+```
+
+## Official Resources
+
+- **GitHub:** https://github.com/containrrr/watchtower
+- **Docker Hub:** https://hub.docker.com/r/containrrr/watchtower
+- **Documentation:** https://containrrr.dev/watchtower/
+- **Arguments Reference:** https://containrrr.dev/watchtower/arguments/
+
+## Educational Resources
+
+### Videos
+- [Watchtower - Auto Update Docker Containers (Techno Tim)](https://www.youtube.com/watch?v=5lP_pdjcVMo)
+- [Keep Docker Containers Updated Automatically](https://www.youtube.com/watch?v=SZ-wprcMYGY)
+- [Watchtower Setup Tutorial (DB Tech)](https://www.youtube.com/watch?v=Ejtzf-Y8Vac)
+
+### Articles & Guides
+- [Watchtower Official Documentation](https://containrrr.dev/watchtower/)
+- [Docker Update Strategies](https://containrrr.dev/watchtower/arguments/)
+- [Notification Configuration](https://containrrr.dev/watchtower/notifications/)
+- [Cron Schedule Examples](https://crontab.guru/)
+
+### Concepts to Learn
+- **Container Updates:** How Docker updates work
+- **Image Digests:** SHA256 hashes for images
+- **Cron Scheduling:** Time-based task automation
+- **Rolling Updates:** Zero-downtime deployment
+- **Image Tags vs Digests:** Differences and implications
+- **Semantic Versioning:** Understanding version numbers
+- **Docker Labels:** Metadata for containers
+
+## Docker Configuration
+
+### Complete Service Definition
+
+```yaml
+watchtower:
+  image: containrrr/watchtower:latest
+  container_name: watchtower
+  restart: unless-stopped
+  volumes:
+    - /var/run/docker.sock:/var/run/docker.sock
+  environment:
+    - WATCHTOWER_SCHEDULE=0 0 4 * * *  # 4 AM daily
+    - WATCHTOWER_CLEANUP=true
+    - WATCHTOWER_INCLUDE_STOPPED=false
+    - WATCHTOWER_INCLUDE_RESTARTING=true
+    - WATCHTOWER_ROLLING_RESTART=true
+    - WATCHTOWER_TIMEOUT=30s
+    - TZ=America/New_York
+    # Optional: Notifications
+    # - WATCHTOWER_NOTIFICATIONS=email
+    # - WATCHTOWER_NOTIFICATION_EMAIL_FROM=watchtower@yourdomain.com
+    # - WATCHTOWER_NOTIFICATION_EMAIL_TO=admin@yourdomain.com
+```
+
+### Schedule Formats
+
+```bash
+# Cron format: second minute hour day month weekday
+# * * * * * *  = every second
+# 0 * * * * *  = every minute
+# 0 0 * * * *  = every hour
+# 0 0 4 * * *  = daily at 4 AM
+# 0 0 4 * * 0  = weekly on Sunday at 4 AM
+# 0 0 4 1 * *  = monthly on 1st at 4 AM
+
+# Common schedules:
+WATCHTOWER_SCHEDULE="0 0 4 * * *"        # Daily 4 AM
+WATCHTOWER_SCHEDULE="0 0 4 * * 0"        # Weekly Sunday 4 AM
+WATCHTOWER_SCHEDULE="0 0 2 1,15 * *"     # 1st and 15th at 2 AM
+WATCHTOWER_SCHEDULE="0 */6 * * * *"      # Every 6 hours
+```
+
+Use https://crontab.guru/ to test cron expressions.
+
+## Controlling Updates
+
+### Label-Based Control
+
+**Enable/Disable per Container:**
+
+```yaml
+myservice:
+  image: myapp:latest
+  labels:
+    # Enable updates (default)
+    - "com.centurylinklabs.watchtower.enable=true"
+    
+    # OR disable updates
+    - "com.centurylinklabs.watchtower.enable=false"
+```
+
+### Monitor Specific Containers
+
+Run Watchtower with container names:
+```bash
+docker run -d \
+  --name watchtower \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  containrrr/watchtower \
+  container1 container2 container3
+```
+
+### Exclude Containers
+
+```yaml
+watchtower:
+  image: containrrr/watchtower
+  environment:
+    - WATCHTOWER_LABEL_ENABLE=true  # Only update labeled containers
+```
+
+Then add label to containers you want updated:
+```yaml
+myservice:
+  labels:
+    - "com.centurylinklabs.watchtower.enable=true"
+```
+
+### Update Scope
+
+**Scope Options:**
+```yaml
+environment:
+  # Only update containers with specific scope
+  - WATCHTOWER_SCOPE=myscope
+
+# Then label containers:
+myservice:
+  labels:
+    - "com.centurylinklabs.watchtower.scope=myscope"
+```
+
+## Advanced Topics
+
+### Notification Configuration
+
+#### Email Notifications
+
+```yaml
+environment:
+  - WATCHTOWER_NOTIFICATIONS=email
+  - WATCHTOWER_NOTIFICATION_EMAIL_FROM=watchtower@yourdomain.com
+  - WATCHTOWER_NOTIFICATION_EMAIL_TO=admin@yourdomain.com
+  - WATCHTOWER_NOTIFICATION_EMAIL_SERVER=smtp.gmail.com
+  - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT=587
+  - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_USER=your-email@gmail.com
+  - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PASSWORD=your-app-password
+  - WATCHTOWER_NOTIFICATION_EMAIL_SERVER_TLS_SKIP_VERIFY=false
+```
+
+#### Slack Notifications
+
+```yaml
+environment:
+  - WATCHTOWER_NOTIFICATIONS=slack
+  - WATCHTOWER_NOTIFICATION_SLACK_HOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK/URL
+  - WATCHTOWER_NOTIFICATION_SLACK_IDENTIFIER=watchtower
+```
+
+#### Discord Notifications
+
+```yaml
+environment:
+  - WATCHTOWER_NOTIFICATIONS=discord
+  - WATCHTOWER_NOTIFICATION_URL=https://discord.com/api/webhooks/YOUR/WEBHOOK
+```
+
+### One-Time Run
+
+Run once and exit (useful for testing):
+```bash
+docker run --rm \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  containrrr/watchtower \
+  --run-once
+```
+
+### Dry Run Mode
+
+Test without actually updating:
+```bash
+docker run --rm \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  containrrr/watchtower \
+  --run-once \
+  --debug \
+  --dry-run
+```
+
+### Rolling Restart
+
+Update containers one at a time (safer):
+```yaml
+environment:
+  - WATCHTOWER_ROLLING_RESTART=true
+```
+
+### Monitor Only Mode
+
+Check for updates but don't apply:
+```yaml
+environment:
+  - WATCHTOWER_MONITOR_ONLY=true
+  - WATCHTOWER_NOTIFICATIONS=email  # Get notified of available updates
+```
+
+### Custom Update Commands
+
+Run commands after update:
+```yaml
+myservice:
+  labels:
+    - "com.centurylinklabs.watchtower.lifecycle.post-update=/scripts/post-update.sh"
+```
+
+### Private Registry
+
+Update containers from private registries:
+```bash
+# Docker Hub
+docker login
+
+# Private registry
+docker login registry.example.com
+
+# Watchtower will use stored credentials
+```
+
+Or configure explicitly:
+```yaml
+environment:
+  - REPO_USER=username
+  - REPO_PASS=password
+```
+
+## Troubleshooting
+
+### Check Watchtower Status
+
+```bash
+# View logs
+docker logs watchtower
+
+# Follow logs in real-time
+docker logs -f watchtower
+
+# Check last run
+docker logs watchtower | grep "Session done"
+```
+
+### Force Manual Update
+
+```bash
+# Run Watchtower once
+docker run --rm \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  containrrr/watchtower \
+  --run-once \
+  --debug
+```
+
+### Container Not Updating
+
+```bash
+# Check if container is monitored
+docker logs watchtower | grep container-name
+
+# Verify image has updates
+docker pull image:tag
+
+# Check for update labels
+docker inspect container-name | grep watchtower
+
+# Force update specific container
+docker run --rm \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  containrrr/watchtower \
+  --run-once \
+  container-name
+```
+
+### Updates Breaking Services
+
+```bash
+# Stop Watchtower
+docker stop watchtower
+
+# Rollback manually
+docker pull image:old-tag
+docker compose up -d --force-recreate service-name
+
+# Or restore from backup
+# If you kept old images (no cleanup):
+docker images | grep image-name
+docker tag image:old-sha image:latest
+docker compose up -d service-name
+```
+
+### High Resource Usage
+
+```bash
+# Check Watchtower stats
+docker stats watchtower
+
+# Increase check interval
+# Change from hourly to daily:
+WATCHTOWER_SCHEDULE="0 0 4 * * *"
+
+# Enable cleanup to free disk space
+WATCHTOWER_CLEANUP=true
+```
+
+### Notification Not Working
+
+```bash
+# Test email manually
+docker exec watchtower wget --spider https://smtp.gmail.com
+
+# Check credentials
+docker logs watchtower | grep -i notification
+
+# Verify SMTP settings
+# For Gmail, use App Password, not account password
+```
+
+## Best Practices
+
+### Recommended Configuration
+
+```yaml
+watchtower:
+  image: containrrr/watchtower:latest
+  container_name: watchtower
+  restart: unless-stopped
+  volumes:
+    - /var/run/docker.sock:/var/run/docker.sock
+  environment:
+    # Daily at 4 AM
+    - WATCHTOWER_SCHEDULE=0 0 4 * * *
+    
+    # Clean up old images
+    - WATCHTOWER_CLEANUP=true
+    
+    # Rolling restart (safer)
+    - WATCHTOWER_ROLLING_RESTART=true
+    
+    # 30 second timeout
+    - WATCHTOWER_TIMEOUT=30s
+    
+    # Enable notifications
+    - WATCHTOWER_NOTIFICATIONS=email
+    - WATCHTOWER_NOTIFICATION_EMAIL_FROM=watchtower@yourdomain.com
+    - WATCHTOWER_NOTIFICATION_EMAIL_TO=admin@yourdomain.com
+    
+    - TZ=America/New_York
+```
+
+### Update Strategy
+
+1. **Start Conservative:**
+   - Weekly updates initially
+   - Monitor for issues
+   - Gradually increase frequency
+
+2. **Use Labels:**
+   - Disable updates for critical services
+   - Update testing services first
+   - Separate production from testing
+
+3. **Enable Notifications:**
+   - Know when updates happen
+   - Track update history
+   - Alert on failures
+
+4. **Test Regularly:**
+   - Dry run before enabling
+   - Manual one-time runs
+   - Verify services after updates
+
+5. **Backup Strategy:**
+   - Keep old images (disable cleanup initially)
+   - Backup volumes before major updates
+   - Document rollback procedures
+
+### Selective Updates
+
+```yaml
+# Core infrastructure: Manual updates only
+traefik:
+  labels:
+    - "com.centurylinklabs.watchtower.enable=false"
+
+authelia:
+  labels:
+    - "com.centurylinklabs.watchtower.enable=false"
+
+# Media services: Auto-update
+plex:
+  labels:
+    - "com.centurylinklabs.watchtower.enable=true"
+
+sonarr:
+  labels:
+    - "com.centurylinklabs.watchtower.enable=true"
+```
+
+## Security Considerations
+
+1. **Docker Socket Access:** Watchtower needs full Docker access (security risk)
+2. **Automatic Updates:** Can break things unexpectedly
+3. **Test Environment:** Test updates in dev before production
+4. **Backup First:** Always backup critical services
+5. **Monitor Logs:** Watch for failed updates
+6. **Rollback Plan:** Know how to revert updates
+7. **Pin Versions:** Use specific tags for critical services
+8. **Update Windows:** Schedule during low-usage times
+9. **Notification:** Always enable notifications
+10. **Review Updates:** Check changelog before auto-updating
+
+## Summary
+
+Watchtower automates container updates, providing:
+- Automatic security patches
+- Consistent update schedule
+- Minimal manual intervention
+- Clean, simple configuration
+- Notification support
+- Label-based control
+
+**Use Watchtower for:**
+- Non-critical services
+- Media apps (Plex, Sonarr, etc.)
+- Utilities and tools
+- Development environments
+
+**Avoid for:**
+- Core infrastructure (Traefik, Authelia)
+- Databases (require careful updates)
+- Custom applications
+- Services requiring migration steps
+
+**Remember:**
+- Start with conservative schedule
+- Enable cleanup after testing
+- Use labels for fine control
+- Monitor notifications
+- Have rollback plan
+- Test in dry-run mode first
+- Backup before major updates