Wiki v1.0

Added a wiki

config-templates/dokuwiki/data/pages/services/infrastructure/watchtower.txt

@@ -0,0 +1,404 @@
+====== Watchtower ======
+
+Watchtower is an automated container update service that monitors running Docker containers and automatically updates them when new images are available. It ensures your homelab services stay up-to-date with the latest security patches and features.
+
+===== Overview =====
+
+**Purpose:** Automated container updates
+**Deployment:** Infrastructure stack (currently disabled)
+**Monitoring:** Passive background service
+**Update Strategy:** Rolling updates with health checks
+
+===== Key Features =====
+
+**Automated Updates:**
+  * **Image Monitoring**: Checks for new image versions
+  * **Scheduled Updates**: Configurable update intervals
+  * **Rolling Updates**: Updates containers one by one
+  * **Health Checks**: Waits for container health before proceeding
+
+**Update Control:**
+  * **Include/Exclude**: Control which containers to update
+  * **Update Notifications**: Email/Slack notifications
+  * **Rollback Support**: Revert to previous versions
+  * **Dry Run Mode**: Test updates without applying
+
+**Safety Features:**
+  * **Health Monitoring**: Ensures containers start successfully
+  * **Timeout Control**: Prevents hanging updates
+  * **Resource Limits**: Controls update resource usage
+  * **Backup Integration**: Coordinates with backup services
+
+===== Configuration =====
+
+**Container Configuration:**
+```yaml
+services:
+  watchtower:
+    image: containrrr/watchtower:latest
+    container_name: watchtower
+    restart: unless-stopped
+    environment:
+      - WATCHTOWER_CLEANUP=true
+      - WATCHTOWER_POLL_INTERVAL=3600
+      - WATCHTOWER_TIMEOUT=30s
+      - WATCHTOWER_NOTIFICATIONS=shoutrrr
+      - WATCHTOWER_NOTIFICATION_URL=discord://token@webhook
+      - WATCHTOWER_INCLUDE_STOPPED=false
+      - WATCHTOWER_REVIVE_STOPPED=false
+      - WATCHTOWER_REMOVE_VOLUMES=false
+      - WATCHTOWER_LABEL_ENABLE=true
+      - WATCHTOWER_MONITOR_ONLY=false
+      - WATCHTOWER_RUN_ONCE=false
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    command: --interval 3600 --cleanup --label-enable
+    deploy:
+      resources:
+        limits:
+          cpus: '0.2'
+          memory: 64M
+        reservations:
+          cpus: '0.01'
+          memory: 16M
+```
+
+**Environment Variables:**
+```bash
+# Update interval (seconds)
+WATCHTOWER_POLL_INTERVAL=3600
+
+# Update timeout
+WATCHTOWER_TIMEOUT=30s
+
+# Cleanup old images
+WATCHTOWER_CLEANUP=true
+
+# Notification settings
+WATCHTOWER_NOTIFICATIONS=shoutrrr
+WATCHTOWER_NOTIFICATION_URL=discord://token@webhook
+
+# Container control
+WATCHTOWER_INCLUDE_STOPPED=false
+WATCHTOWER_REVIVE_STOPPED=false
+WATCHTOWER_REMOVE_VOLUMES=false
+
+# Label-based control
+WATCHTOWER_LABEL_ENABLE=true
+
+# Monitoring mode
+WATCHTOWER_MONITOR_ONLY=false
+
+# One-time run
+WATCHTOWER_RUN_ONCE=false
+```
+
+===== Update Process =====
+
+**Monitoring Phase:**
+  1. **Image Check**: Queries Docker registry for new versions
+  2. **Version Comparison**: Compares current vs latest versions
+  3. **Update Decision**: Determines if update is needed
+  4. **Schedule Planning**: Plans update timing
+
+**Update Execution:**
+  1. **Container Stop**: Gracefully stops current container
+  2. **Image Pull**: Downloads new image version
+  3. **Container Start**: Starts container with new image
+  4. **Health Check**: Verifies container health
+  5. **Cleanup**: Removes old images (if enabled)
+
+**Post-Update:**
+  * **Notification**: Sends update notifications
+  * **Logging**: Records update details
+  * **Monitoring**: Continues monitoring for next updates
+  * **Error Handling**: Handles update failures
+
+===== Container Selection =====
+
+**Label-Based Control:**
+```yaml
+# Enable updates for specific containers
+labels:
+  - "com.centurylinklabs.watchtower.enable=true"
+
+# Disable updates for specific containers
+labels:
+  - "com.centurylinklabs.watchtower.enable=false"
+```
+
+**Include/Exclude Patterns:**
+```bash
+# Include only specific containers
+command: --include=traefik,authelia,dockge
+
+# Exclude specific containers
+command: --exclude=plex,jellyfin
+
+# Use regex patterns
+command: --include="^media-.*"
+```
+
+**Scope Control:**
+  * **All Containers**: Update all running containers
+  * **Specific Services**: Update only selected services
+  * **Stack-Based**: Update containers in specific stacks
+  * **Label-Based**: Use Docker labels for control
+
+===== Notification System =====
+
+**Supported Notifications:**
+  * **Email**: SMTP email notifications
+  * **Slack**: Slack channel notifications
+  * **Discord**: Discord webhook notifications
+  * **Gotify**: Gotify push notifications
+  * **Telegram**: Telegram bot notifications
+
+**Notification Configuration:**
+```yaml
+environment:
+  - WATCHTOWER_NOTIFICATIONS=shoutrrr
+  - WATCHTOWER_NOTIFICATION_URL=slack://token@channel
+  # Or for Discord
+  - WATCHTOWER_NOTIFICATION_URL=discord://token@webhook
+  # Or for email
+  - WATCHTOWER_NOTIFICATION_URL=smtp://user:pass@host:port
+```
+
+**Notification Content:**
+  * **Update Started**: Container update beginning
+  * **Update Completed**: Successful update confirmation
+  * **Update Failed**: Error details and troubleshooting
+  * **Rollback Performed**: Automatic rollback notifications
+
+===== Safety & Reliability =====
+
+**Health Checks:**
+```yaml
+# Wait for health checks
+command: --interval 3600 --cleanup --label-enable --enable-healthchecks
+```
+
+**Timeout Management:**
+```yaml
+# Set update timeouts
+environment:
+  - WATCHTOWER_TIMEOUT=60s
+```
+
+**Rollback Capability:**
+```yaml
+# Enable automatic rollback on failure
+command: --rollback-on-failure
+```
+
+**Resource Protection:**
+  * **CPU Limits**: Prevent update resource exhaustion
+  * **Memory Limits**: Control memory usage during updates
+  * **Network Limits**: Manage download bandwidth
+  * **Concurrent Updates**: Limit simultaneous updates
+
+===== Scheduling =====
+
+**Update Intervals:**
+```bash
+# Check every hour
+command: --interval 3600
+
+# Check every 24 hours
+command: --interval 86400
+
+# Check at specific times
+command: --schedule "0 0 4 * * *"  # Daily at 4 AM
+```
+
+**Maintenance Windows:**
+  * **Off-hours Updates**: Schedule updates during low-usage times
+  * **Weekend Updates**: Perform updates on weekends
+  * **Manual Control**: Trigger updates manually when needed
+  * **Holiday Scheduling**: Avoid updates during holidays
+
+===== Troubleshooting =====
+
+**Update Failures:**
+```bash
+# Check Watchtower logs
+docker logs watchtower
+
+# Manual update test
+docker pull image:latest
+docker stop container
+docker rm container
+docker run -d --name container image:latest
+```
+
+**Permission Issues:**
+  * **Docker Socket**: Verify socket access permissions
+  * **Registry Access**: Check Docker registry authentication
+  * **Network Issues**: Verify internet connectivity
+  * **Disk Space**: Ensure sufficient space for image downloads
+
+**Notification Problems:**
+  * **Webhook URLs**: Verify notification endpoint URLs
+  * **Authentication**: Check API tokens and credentials
+  * **Network Access**: Ensure outbound connectivity
+  * **Rate Limits**: Check service rate limiting
+
+**Performance Issues:**
+  * **Resource Usage**: Monitor CPU/memory during updates
+  * **Update Frequency**: Adjust polling intervals
+  * **Concurrent Updates**: Limit simultaneous container updates
+  * **Network Bandwidth**: Control download speeds
+
+**Troubleshooting Steps:**
+  1. **Check logs**: `docker logs watchtower`
+  2. **Test manually**: Perform manual container updates
+  3. **Verify configuration**: Check environment variables
+  4. **Test notifications**: Send test notifications
+  5. **Restart service**: `docker restart watchtower`
+
+===== Advanced Configuration =====
+
+**Custom Update Logic:**
+```bash
+# Use custom update script
+command: --script /path/to/update-script.sh
+```
+
+**Lifecycle Hooks:**
+```yaml
+# Pre/post update hooks
+command: --pre-check /path/to/pre-check.sh --post-check /path/to/post-check.sh
+```
+
+**Advanced Filtering:**
+```bash
+# Complex filtering rules
+command: --filter-by-label=com.example.version=latest --filter-by-label=com.example.tier=frontend
+```
+
+**Monitoring Integration:**
+```yaml
+# Export metrics
+command: --metrics
+environment:
+  - WATCHTOWER_METRICS_PORT=8080
+```
+
+===== Security Considerations =====
+
+**Access Control:**
+  * **Docker Socket Security**: Read-only socket access
+  * **Registry Authentication**: Secure registry credentials
+  * **Network Security**: Secure update traffic
+  * **Audit Logging**: Track all update activities
+
+**Update Security:**
+  * **Image Verification**: Verify image authenticity
+  * **Vulnerability Scanning**: Check for security issues
+  * **Trusted Sources**: Only update from trusted registries
+  * **Rollback Security**: Secure rollback procedures
+
+===== Integration with Backup =====
+
+**Backup Coordination:**
+```yaml
+# Coordinate with backup services
+command: --pre-check /scripts/backup-check.sh --post-check /scripts/backup-verify.sh
+```
+
+**Backup Scripts:**
+```bash
+#!/bin/bash
+# Pre-update backup
+docker exec backup-service backup-now
+
+# Post-update verification
+docker exec backup-service verify-backup
+```
+
+**Automated Backup:**
+  * **Pre-update Backup**: Backup before each update
+  * **Post-update Verification**: Verify backup integrity
+  * **Rollback Backup**: Maintain rollback capability
+  * **Retention Policy**: Manage backup retention
+
+===== Best Practices =====
+
+**Update Strategy:**
+  * **Staged Updates**: Update non-critical services first
+  * **Monitoring**: Monitor updates closely initially
+  * **Testing**: Test updates in development first
+  * **Documentation**: Document update procedures
+
+**Safety Measures:**
+  * **Health Checks**: Always enable health checks
+  * **Timeouts**: Set appropriate update timeouts
+  * **Notifications**: Enable comprehensive notifications
+  * **Rollback**: Have rollback procedures ready
+
+**Performance:**
+  * **Resource Limits**: Appropriate CPU/memory limits
+  * **Update Windows**: Schedule during low-usage times
+  * **Concurrent Limits**: Limit simultaneous updates
+  * **Network Management**: Control bandwidth usage
+
+**Monitoring:**
+  * **Update Tracking**: Monitor update success/failure
+  * **Performance Impact**: Track update performance impact
+  * **Error Analysis**: Analyze update failure patterns
+  * **Success Metrics**: Track update success rates
+
+===== Use Cases =====
+
+**Production Environments:**
+  * **Security Updates**: Automatic security patch deployment
+  * **Feature Updates**: Deploy new features automatically
+  * **Compliance**: Maintain compliance with update policies
+  * **Stability**: Ensure service stability through updates
+
+**Development Environments:**
+  * **Testing Updates**: Test update procedures safely
+  * **CI/CD Integration**: Integrate with development pipelines
+  * **Version Control**: Manage container versions
+  * **Rollback Testing**: Test rollback capabilities
+
+**Homelab Management:**
+  * **Convenience**: Hands-off update management
+  * **Security**: Maintain security through updates
+  * **Stability**: Prevent version drift issues
+  * **Monitoring**: Track update status and health
+
+**Enterprise Deployments:**
+  * **Policy Compliance**: Enforce update policies
+  * **Change Management**: Manage change through updates
+  * **Audit Trails**: Maintain update audit logs
+  * **Reporting**: Generate update compliance reports
+
+===== Manual Update Process =====
+
+**When Watchtower is Disabled:**
+```bash
+# Manual update procedure
+# 1. Identify containers to update
+docker ps --format "table {{.Names}}\t{{.Image}}"
+
+# 2. Check for updates
+docker pull image:latest
+
+# 3. Backup current state
+docker tag current-image backup-image
+
+# 4. Stop and update container
+docker stop container
+docker rm container
+docker run -d --name container image:latest
+
+# 5. Verify update
+docker logs container
+docker ps | grep container
+```
+
+Watchtower provides automated container updates with safety features and monitoring, ensuring your homelab services remain current and secure.
+
+**Next:** Learn about [[services:infrastructure:code-server|Code Server]] or explore [[architecture:backup|Backup Architecture]].