298 lines
8.3 KiB
Plaintext
298 lines
8.3 KiB
Plaintext
====== System Architecture ======
|
|
|
|
The AI-Homelab is built on a production-ready, scalable architecture designed for reliability, security, and ease of management.
|
|
|
|
===== Core Principles =====
|
|
|
|
**Infrastructure as Code:**
|
|
* All services defined in Docker Compose files
|
|
* Configuration managed through YAML files
|
|
* Version control with Git
|
|
* Reproducible deployments
|
|
|
|
**Security First:**
|
|
* SSO protection for admin interfaces
|
|
* Automatic HTTPS with Let's Encrypt
|
|
* VPN routing for downloads
|
|
* Network isolation and segmentation
|
|
|
|
**Scalability:**
|
|
* Resource limits prevent exhaustion
|
|
* Lazy loading reduces resource usage
|
|
* Modular service architecture
|
|
* Easy addition of new services
|
|
|
|
**Observability:**
|
|
* Comprehensive logging
|
|
* Metrics collection
|
|
* Health monitoring
|
|
* Alerting capabilities
|
|
|
|
===== Network Architecture =====
|
|
|
|
```
|
|
Internet
|
|
↓
|
|
[Router] Port Forwarding (80, 443)
|
|
↓
|
|
[DuckDNS] Dynamic DNS Updates
|
|
↓
|
|
[Traefik] Reverse Proxy + SSL Termination
|
|
↓
|
|
[Authelia] SSO Authentication
|
|
↓
|
|
[Docker Services] Isolated Containers
|
|
```
|
|
|
|
**Network Layers:**
|
|
|
|
**External Access:**
|
|
* **DuckDNS**: Dynamic DNS service
|
|
* **Port Forwarding**: 80/443 to Traefik
|
|
* **SSL Termination**: Wildcard certificate
|
|
|
|
**Reverse Proxy:**
|
|
* **Traefik**: Routes traffic to services
|
|
* **Authelia**: SSO middleware
|
|
* **Load Balancing**: Service discovery
|
|
|
|
**Service Networks:**
|
|
* **traefik-network**: All web services
|
|
* **homelab-network**: Internal communication
|
|
* **media-network**: Media services
|
|
* **isolated networks**: Security segmentation
|
|
|
|
===== Service Architecture =====
|
|
|
|
**Core Stack (Essential Infrastructure):**
|
|
* **DuckDNS**: DNS updates every 5 minutes
|
|
* **Traefik**: HTTP routing and SSL
|
|
* **Authelia**: Authentication and authorization
|
|
* **Gluetun**: VPN client for downloads
|
|
* **Sablier**: Lazy loading service
|
|
|
|
**Infrastructure Stack:**
|
|
* **Dockge**: Primary management interface
|
|
* **Pi-hole**: Network-wide DNS and ad blocking
|
|
* **Dozzle**: Live Docker log viewer
|
|
* **Glances**: System resource monitoring
|
|
|
|
**Service Categories:**
|
|
|
|
**Media Services:**
|
|
* **Jellyfin/Plex**: Media servers with transcoding
|
|
* **qBittorrent**: Torrent client (VPN routed)
|
|
* **Sonarr/Radarr**: Download automation
|
|
* **Prowlarr**: Indexer management
|
|
|
|
**Productivity Services:**
|
|
* **Nextcloud**: File synchronization
|
|
* **Gitea**: Git service and CI/CD
|
|
* **BookStack**: Documentation platform
|
|
* **WordPress**: Blogging platform
|
|
|
|
**Monitoring & Observability:**
|
|
* **Grafana**: Dashboard and visualization
|
|
* **Prometheus**: Metrics collection
|
|
* **Uptime Kuma**: Status monitoring
|
|
* **Loki**: Log aggregation
|
|
|
|
===== Storage Architecture =====
|
|
|
|
**Configuration Storage:**
|
|
```
|
|
/opt/stacks/
|
|
├── core/ # Core infrastructure
|
|
├── infrastructure/ # Management tools
|
|
├── media/ # Media services
|
|
├── productivity/ # Office tools
|
|
└── monitoring/ # Observability
|
|
```
|
|
|
|
**Data Storage Strategy:**
|
|
|
|
**Small Data (< 50GB):**
|
|
* **Location**: `/opt/stacks/stack-name/config/`
|
|
* **Type**: Bind mounts
|
|
* **Backup**: Included in configuration backups
|
|
|
|
**Large Data (> 50GB):**
|
|
* **Location**: `/mnt/media/`, `/mnt/downloads/`, `/mnt/backups/`
|
|
* **Type**: External mounts
|
|
* **Backup**: Separate backup strategies
|
|
|
|
**Database Storage:**
|
|
* **Type**: Named Docker volumes
|
|
* **Location**: Docker managed
|
|
* **Backup**: Volume snapshots
|
|
|
|
===== Security Architecture =====
|
|
|
|
**Authentication & Authorization:**
|
|
|
|
**Authelia SSO:**
|
|
* **Protocol**: SAML, OpenID Connect
|
|
* **Storage**: File-based user database
|
|
* **2FA**: TOTP, WebAuthn support
|
|
* **Policies**: Domain-based access control
|
|
|
|
**Service Authentication:**
|
|
* **Admin Services**: Authelia protected
|
|
* **Media Services**: Bypass for app compatibility
|
|
* **APIs**: Token-based authentication
|
|
|
|
**Network Security:**
|
|
* **Firewall**: UFW with minimal ports
|
|
* **SSL/TLS**: End-to-end encryption
|
|
* **VPN**: Download traffic protection
|
|
* **Isolation**: Docker network segmentation
|
|
|
|
===== Deployment Architecture =====
|
|
|
|
**Two-Phase Deployment:**
|
|
|
|
**Phase 1: Setup**
|
|
```bash
|
|
sudo ./scripts/setup-homelab.sh
|
|
```
|
|
* System preparation
|
|
* Docker installation
|
|
* Authelia configuration
|
|
* Infrastructure setup
|
|
|
|
**Phase 2: Deployment**
|
|
```bash
|
|
sudo ./scripts/deploy-homelab.sh
|
|
```
|
|
* Core stack deployment
|
|
* SSL certificate generation
|
|
* Infrastructure services
|
|
* Health verification
|
|
|
|
**Service Deployment:**
|
|
* **Dockge**: Web-based stack management
|
|
* **Manual**: Docker Compose commands
|
|
* **Automated**: CI/CD pipelines
|
|
|
|
===== Resource Management =====
|
|
|
|
**Resource Limits:**
|
|
```yaml
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: '2.0'
|
|
memory: 4G
|
|
reservations:
|
|
cpus: '0.5'
|
|
memory: 1G
|
|
```
|
|
|
|
**Resource Allocation Strategy:**
|
|
* **Core Services**: Minimal resources (0.1-0.5 CPU, 64MB-256MB RAM)
|
|
* **Web Services**: Moderate resources (1-2 CPU, 1-4GB RAM)
|
|
* **Media Services**: High resources (2-4 CPU, 4-8GB RAM)
|
|
* **Background Services**: Variable based on workload
|
|
|
|
**Lazy Loading:**
|
|
* **Sablier**: On-demand service startup
|
|
* **Resource Savings**: 50-80% reduction in idle usage
|
|
* **Automatic Scaling**: Services start when accessed
|
|
|
|
===== Monitoring Architecture =====
|
|
|
|
**Metrics Collection:**
|
|
* **Prometheus**: Time-series metrics
|
|
* **Node Exporter**: System metrics
|
|
* **cAdvisor**: Container metrics
|
|
* **Custom Exporters**: Service-specific metrics
|
|
|
|
**Logging:**
|
|
* **Dozzle**: Real-time log viewer
|
|
* **Loki**: Log aggregation
|
|
* **Promtail**: Log shipping
|
|
* **Structured Logging**: JSON format
|
|
|
|
**Alerting:**
|
|
* **Uptime Kuma**: Service availability
|
|
* **Grafana**: Threshold-based alerts
|
|
* **Email/SMS**: Notification channels
|
|
|
|
===== Backup Architecture =====
|
|
|
|
**Backup Strategy:**
|
|
* **Backrest**: Primary backup solution (Restic)
|
|
* **Duplicati**: Alternative encrypted backups
|
|
* **Automated**: Scheduled backups
|
|
* **Encrypted**: AES-256 encryption
|
|
|
|
**Backup Types:**
|
|
* **Configuration**: `/opt/stacks/` directories
|
|
* **User Data**: Service volumes and mounts
|
|
* **SSL Certificates**: `/opt/stacks/core/traefik/acme.json`
|
|
* **Databases**: Volume snapshots
|
|
|
|
**Recovery:**
|
|
* **Point-in-time**: Versioned backups
|
|
* **Bare metal**: Complete system recovery
|
|
* **Service-level**: Individual service restoration
|
|
|
|
===== High Availability =====
|
|
|
|
**Redundancy:**
|
|
* **Load Balancing**: Traefik distributes traffic
|
|
* **Health Checks**: Automatic service monitoring
|
|
* **Failover**: Automatic service restart
|
|
* **Backups**: Multiple backup locations
|
|
|
|
**Scalability:**
|
|
* **Horizontal**: Multiple service instances
|
|
* **Vertical**: Resource scaling
|
|
* **Storage**: Distributed storage options
|
|
* **Network**: High-bandwidth connections
|
|
|
|
===== Development Architecture =====
|
|
|
|
**AI Integration:**
|
|
* **GitHub Copilot**: Intelligent assistance
|
|
* **Copilot Instructions**: Context-aware guidance
|
|
* **Automated Configuration**: AI-generated compose files
|
|
* **Documentation**: AI-maintained wiki
|
|
|
|
**Version Control:**
|
|
* **Git**: Source code management
|
|
* **Branches**: Feature development
|
|
* **Tags**: Release versioning
|
|
* **CI/CD**: Automated testing and deployment
|
|
|
|
===== Performance Optimization =====
|
|
|
|
**Caching:**
|
|
* **Browser Caching**: Static asset optimization
|
|
* **Database Caching**: Query result caching
|
|
* **CDN**: Content delivery networks
|
|
* **Reverse Proxy**: Traefik caching
|
|
|
|
**Optimization Techniques:**
|
|
* **Compression**: Gzip/Brotli compression
|
|
* **Minification**: Asset optimization
|
|
* **Lazy Loading**: On-demand resource loading
|
|
* **Connection Pooling**: Database optimization
|
|
|
|
===== Compliance & Governance =====
|
|
|
|
**Security Standards:**
|
|
* **SSL/TLS**: Industry standard encryption
|
|
* **Access Control**: Least privilege principle
|
|
* **Audit Logging**: Comprehensive activity logs
|
|
* **Regular Updates**: Security patch management
|
|
|
|
**Data Protection:**
|
|
* **Encryption**: Data at rest and in transit
|
|
* **Backup Encryption**: Secure offsite storage
|
|
* **Privacy**: Minimal data collection
|
|
* **Retention**: Configurable data lifecycle
|
|
|
|
This architecture provides a solid foundation for a production-ready homelab that can scale with your needs while maintaining security and reliability.
|
|
|
|
**Next:** Learn about [[architecture:networking|Network Architecture]] or explore [[services:start|Available Services]]. |