Merge remote changes and resolve conflict in copilot-instructions.md
This commit is contained in:
kelinfoxy
@@ -1,9 +1,11 @@
|
||||
# AI-Homelab Setup Scripts
|
||||
|
||||
This directory contains two scripts for automated AI-Homelab deployment:
|
||||
This directory contains scripts for automated AI-Homelab deployment and management:
|
||||
|
||||
1. **setup-homelab.sh** - System preparation
|
||||
2. **deploy-homelab.sh** - Core infrastructure deployment
|
||||
3. **reset-test-environment.sh** - Safe test environment cleanup
|
||||
4. **reset-ondemand-services.sh** - Reload services for Sablier lazy loading
|
||||
|
||||
## setup-homelab.sh
|
||||
|
||||
@@ -186,3 +188,135 @@ cd /opt/stacks/infrastructure && docker compose up -d
|
||||
- Waits up to 60 seconds for Dockge to become ready
|
||||
- Automatically copies .env to stack directories
|
||||
- Safe to run multiple times (idempotent)
|
||||
|
||||
---
|
||||
|
||||
## reset-test-environment.sh
|
||||
|
||||
Safe cleanup script for testing environments. Completely removes all deployed services, data, and configurations while preserving the underlying system setup. Intended for development and testing scenarios only.
|
||||
|
||||
### What It Does
|
||||
|
||||
1. **Stop All Stacks** - Gracefully stops dashboards, infrastructure, and core stacks
|
||||
2. **Preserve SSL Certificates** - Backs up `acme.json` to the repository folder for reuse
|
||||
3. **Remove Docker Volumes** - Deletes all homelab-related named volumes (data will be lost)
|
||||
4. **Clean Stack Directories** - Removes `/opt/stacks/core`, `/opt/stacks/infrastructure`, `/opt/stacks/dashboards`
|
||||
5. **Clear Dockge Data** - Removes Dockge's persistent data directory
|
||||
6. **Clean Temporary Files** - Removes temporary files and setup artifacts
|
||||
7. **Remove Networks** - Deletes homelab-network, traefik-network, dockerproxy-network, media-network
|
||||
8. **Prune Resources** - Runs Docker system prune to clean up unused resources
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
cd ~/AI-Homelab
|
||||
|
||||
# Make the script executable (if needed)
|
||||
chmod +x scripts/reset-test-environment.sh
|
||||
|
||||
# Run with sudo (required for system cleanup)
|
||||
sudo ./scripts/reset-test-environment.sh
|
||||
```
|
||||
|
||||
### Safety Features
|
||||
|
||||
- **Confirmation Required** - Must type "yes" to confirm reset
|
||||
- **Root Check** - Ensures running with sudo but not as root user
|
||||
- **Colored Output** - Clear visual feedback for each step
|
||||
- **Error Handling** - Continues with warnings if some operations fail
|
||||
- **Preserves System** - Docker, packages, user groups, and firewall settings remain intact
|
||||
|
||||
### After Running
|
||||
|
||||
The system will be returned to a clean state ready for re-deployment:
|
||||
|
||||
1. Ensure `.env` file is properly configured
|
||||
2. Run: `sudo ./scripts/setup-homelab.sh` (if needed)
|
||||
3. Run: `./scripts/deploy-homelab.sh`
|
||||
|
||||
### Requirements
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- Root access (via sudo)
|
||||
- Existing AI-Homelab deployment
|
||||
|
||||
### Warnings
|
||||
|
||||
- **DATA LOSS** - All application data, databases, and configurations will be permanently deleted
|
||||
- **SSL Certificates** - Preserved in repository folder but must be manually restored if needed
|
||||
- **Production Use** - This script is for testing only - DO NOT use in production environments
|
||||
|
||||
### Notes
|
||||
|
||||
- Preserves Docker installation and system packages
|
||||
- Maintains user group memberships and firewall rules
|
||||
- SSL certificates are backed up to `~/AI-Homelab/acme.json`
|
||||
- Safe to run multiple times
|
||||
- Provides clear next steps after completion
|
||||
|
||||
---
|
||||
|
||||
## reset-ondemand-services.sh
|
||||
|
||||
Service management script for Sablier lazy loading. Restarts stacks to reload configuration changes and stops web services so Sablier can control them on-demand, while keeping databases running.
|
||||
|
||||
### What It Does
|
||||
|
||||
1. **Restart Stacks** - Brings down and back up various service stacks to reload compose file changes
|
||||
2. **Stop Web Services** - Stops containers with `sablier.enable=true` label so Sablier can start them on-demand
|
||||
3. **Preserve Databases** - Leaves database containers running for data persistence
|
||||
|
||||
### Supported Stacks
|
||||
|
||||
The script manages the following stacks:
|
||||
- arr-stack (Sonarr, Radarr, Prowlarr)
|
||||
- backrest (backup management)
|
||||
- bitwarden (password manager)
|
||||
- bookstack (documentation)
|
||||
- code-server (VS Code server)
|
||||
- dokuwiki (wiki)
|
||||
- dozzle (log viewer)
|
||||
- duplicati (alternative backup)
|
||||
- formio (form builder)
|
||||
- gitea (git server)
|
||||
- glances (system monitor)
|
||||
- mealie (recipe manager)
|
||||
- mediawiki (wiki)
|
||||
- nextcloud (cloud storage)
|
||||
- tdarr (media processing)
|
||||
- unmanic (media optimization)
|
||||
- wordpress (blog/CMS)
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
cd ~/AI-Homelab
|
||||
|
||||
# Make the script executable (if needed)
|
||||
chmod +x scripts/reset-ondemand-services.sh
|
||||
|
||||
# Run as regular user (docker group membership required)
|
||||
./scripts/reset-ondemand-services.sh
|
||||
```
|
||||
|
||||
### When to Use
|
||||
|
||||
- After modifying compose files for Sablier lazy loading configuration
|
||||
- When services need to reload configuration changes
|
||||
- To ensure Sablier has control over web service startup
|
||||
- During initial setup of lazy loading for multiple services
|
||||
|
||||
### Requirements
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- User must be in docker group
|
||||
- Sablier must be running in core stack
|
||||
- Service stacks must be deployed
|
||||
|
||||
### Notes
|
||||
|
||||
- Handles different compose file naming conventions (.yml vs .yaml)
|
||||
- Stops only services with Sablier labels enabled
|
||||
- Databases remain running to preserve data
|
||||
- Safe to run multiple times
|
||||
- Provides clear feedback on operations performed
|
||||
|
||||
@@ -1,202 +0,0 @@
|
||||
#!/bin/bash
|
||||
# AI-Homelab Resource Limits Application Script
|
||||
# Applies researched resource limits to all Docker Compose stacks
|
||||
# Run as: sudo ./apply-resource-limits.sh
|
||||
|
||||
set -e
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
log_info() {
|
||||
echo -e "${BLUE}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[SUCCESS]${NC} $1"
|
||||
}
|
||||
|
||||
log_warning() {
|
||||
echo -e "${YELLOW}[WARNING]${NC} $1"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# Check if running as root
|
||||
if [ "$EUID" -ne 0 ]; then
|
||||
log_error "Please run as root (sudo ./apply-resource-limits.sh)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get actual user
|
||||
ACTUAL_USER="${SUDO_USER:-$USER}"
|
||||
|
||||
log_info "Applying researched resource limits to all stacks..."
|
||||
echo ""
|
||||
|
||||
# Function to add resource limits to a service in docker-compose.yml
|
||||
add_resource_limits() {
|
||||
local compose_file="$1"
|
||||
local service_name="$2"
|
||||
local template="$3"
|
||||
|
||||
# Define resource limits based on template
|
||||
case $template in
|
||||
"lightweight")
|
||||
limits="cpus: '0.25'\n memory: 128M\n pids: 256"
|
||||
reservations="cpus: '0.10'\n memory: 64M"
|
||||
;;
|
||||
"web")
|
||||
limits="cpus: '0.50'\n memory: 256M\n pids: 512"
|
||||
reservations="cpus: '0.25'\n memory: 128M"
|
||||
;;
|
||||
"database")
|
||||
limits="cpus: '1.0'\n memory: 1G\n pids: 1024"
|
||||
reservations="cpus: '0.50'\n memory: 512M"
|
||||
;;
|
||||
"media")
|
||||
limits="cpus: '2.0'\n memory: 2G\n pids: 2048"
|
||||
reservations="cpus: '1.0'\n memory: 1G"
|
||||
;;
|
||||
"downloader")
|
||||
limits="cpus: '1.0'\n memory: 512M\n pids: 1024"
|
||||
reservations="cpus: '0.50'\n memory: 256M"
|
||||
;;
|
||||
"heavy")
|
||||
limits="cpus: '1.5'\n memory: 1G\n pids: 2048"
|
||||
reservations="cpus: '0.75'\n memory: 512M"
|
||||
;;
|
||||
"monitoring")
|
||||
limits="cpus: '0.75'\n memory: 512M\n pids: 1024"
|
||||
reservations="cpus: '0.25'\n memory: 256M"
|
||||
;;
|
||||
*)
|
||||
log_warning "Unknown template: $template for $service_name"
|
||||
return
|
||||
;;
|
||||
esac
|
||||
|
||||
# Check if service already has deploy.resources
|
||||
if grep -A 10 " $service_name:" "$compose_file" | grep -q "deploy:"; then
|
||||
log_warning "$service_name in $compose_file already has deploy section - skipping"
|
||||
return
|
||||
fi
|
||||
|
||||
# Find the service definition and add deploy.resources after the image line
|
||||
if grep -q "^ $service_name:" "$compose_file"; then
|
||||
# Create a temporary file with the deploy section
|
||||
local deploy_section=" deploy:
|
||||
resources:
|
||||
limits:
|
||||
$limits
|
||||
reservations:
|
||||
$reservations"
|
||||
|
||||
# Use awk to insert the deploy section after the image line
|
||||
awk -v service="$service_name" -v deploy="$deploy_section" '
|
||||
/^ '"$service_name"':/ { in_service=1 }
|
||||
in_service && /^ image:/ {
|
||||
print $0
|
||||
print deploy
|
||||
in_service=0
|
||||
next
|
||||
}
|
||||
{ print }
|
||||
' "$compose_file" > "${compose_file}.tmp" && mv "${compose_file}.tmp" "$compose_file"
|
||||
|
||||
log_success "Added $template limits to $service_name in $(basename "$compose_file")"
|
||||
else
|
||||
log_warning "Service $service_name not found in $compose_file"
|
||||
fi
|
||||
}
|
||||
|
||||
# Process each stack
|
||||
STACKS_DIR="/opt/stacks"
|
||||
|
||||
# Core stack (already has some limits)
|
||||
log_info "Processing core stack..."
|
||||
if [ -f "$STACKS_DIR/core/docker-compose.yml" ]; then
|
||||
# DuckDNS is already done, check if others need limits
|
||||
if ! grep -A 5 " authelia:" "$STACKS_DIR/core/docker-compose.yml" | grep -q "deploy:"; then
|
||||
add_resource_limits "$STACKS_DIR/core/docker-compose.yml" "authelia" "lightweight"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Infrastructure stack
|
||||
log_info "Processing infrastructure stack..."
|
||||
if [ -f "$STACKS_DIR/infrastructure/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/infrastructure/docker-compose.yml" "pihole" "lightweight"
|
||||
add_resource_limits "$STACKS_DIR/infrastructure/docker-compose.yml" "dockge" "web"
|
||||
add_resource_limits "$STACKS_DIR/infrastructure/docker-compose.yml" "glances" "web"
|
||||
fi
|
||||
|
||||
# Dashboard stack
|
||||
log_info "Processing dashboard stack..."
|
||||
if [ -f "$STACKS_DIR/dashboards/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/dashboards/docker-compose.yml" "homepage" "web"
|
||||
add_resource_limits "$STACKS_DIR/dashboards/docker-compose.yml" "homarr" "web"
|
||||
fi
|
||||
|
||||
# Media stack
|
||||
log_info "Processing media stack..."
|
||||
if [ -f "$STACKS_DIR/media/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/media/docker-compose.yml" "jellyfin" "media"
|
||||
add_resource_limits "$STACKS_DIR/media/docker-compose.yml" "calibre-web" "web"
|
||||
fi
|
||||
|
||||
# Downloaders stack
|
||||
log_info "Processing downloaders stack..."
|
||||
if [ -f "$STACKS_DIR/downloaders/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/downloaders/docker-compose.yml" "qbittorrent" "downloader"
|
||||
fi
|
||||
|
||||
# Home Assistant stack
|
||||
log_info "Processing home assistant stack..."
|
||||
if [ -f "$STACKS_DIR/homeassistant/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/homeassistant/docker-compose.yml" "homeassistant" "heavy"
|
||||
add_resource_limits "$STACKS_DIR/homeassistant/docker-compose.yml" "esphome" "web"
|
||||
add_resource_limits "$STACKS_DIR/homeassistant/docker-compose.yml" "nodered" "web"
|
||||
fi
|
||||
|
||||
# Productivity stack
|
||||
log_info "Processing productivity stack..."
|
||||
if [ -f "$STACKS_DIR/productivity/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/productivity/docker-compose.yml" "nextcloud" "heavy"
|
||||
add_resource_limits "$STACKS_DIR/productivity/docker-compose.yml" "gitea" "web"
|
||||
fi
|
||||
|
||||
# Monitoring stack
|
||||
log_info "Processing monitoring stack..."
|
||||
if [ -f "$STACKS_DIR/monitoring/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/monitoring/docker-compose.yml" "prometheus" "monitoring"
|
||||
add_resource_limits "$STACKS_DIR/monitoring/docker-compose.yml" "grafana" "web"
|
||||
add_resource_limits "$STACKS_DIR/monitoring/docker-compose.yml" "loki" "monitoring"
|
||||
add_resource_limits "$STACKS_DIR/monitoring/docker-compose.yml" "uptime-kuma" "web"
|
||||
fi
|
||||
|
||||
# Development stack
|
||||
log_info "Processing development stack..."
|
||||
if [ -f "$STACKS_DIR/development/docker-compose.yml" ]; then
|
||||
add_resource_limits "$STACKS_DIR/development/docker-compose.yml" "code-server" "heavy"
|
||||
fi
|
||||
|
||||
# Fix ownership
|
||||
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$STACKS_DIR"
|
||||
|
||||
echo ""
|
||||
log_success "Resource limits application complete!"
|
||||
echo ""
|
||||
log_info "Next steps:"
|
||||
echo " 1. Review the applied limits: docker compose config"
|
||||
echo " 2. Deploy updated stacks: docker compose up -d"
|
||||
echo " 3. Monitor usage: docker stats"
|
||||
echo " 4. Adjust limits as needed based on real usage"
|
||||
echo ""
|
||||
log_info "Note: These are conservative defaults based on typical usage patterns."
|
||||
log_info "Monitor actual resource usage and adjust limits accordingly."
|
||||
40
scripts/reset-ondemand-services.sh
Normal file
40
scripts/reset-ondemand-services.sh
Normal file
@@ -0,0 +1,40 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to bring stacks down and back up (to reload compose file changes)
|
||||
# then stop the webui containers so Sablier can control them
|
||||
# leaving the databases running
|
||||
|
||||
stacks=(
|
||||
"arr-stack"
|
||||
"backrest"
|
||||
"bitwarden"
|
||||
"bookstack"
|
||||
"code-server"
|
||||
"dokuwiki"
|
||||
"dozzle"
|
||||
"duplicati"
|
||||
"formio"
|
||||
"gitea"
|
||||
"glances"
|
||||
"mealie"
|
||||
"mediawiki"
|
||||
"nextcloud"
|
||||
"tdarr"
|
||||
"unmanic"
|
||||
"wordpress"
|
||||
)
|
||||
|
||||
for stack in "${stacks[@]}"; do
|
||||
if [[ "$stack" == "duplicati" || "$stack" == "formio" || "$stack" == "tdarr" ]]; then
|
||||
file="/opt/stacks/$stack/docker-compose.yml"
|
||||
else
|
||||
file="/opt/stacks/$stack/docker-compose.yaml"
|
||||
fi
|
||||
echo "Starting $stack..."
|
||||
docker compose -f "$file" stop && docker compose -f "$file" up -d
|
||||
done
|
||||
|
||||
echo "Stopping web services to enable on-demand starting by Sablier..."
|
||||
docker ps --filter "label=sablier.enable=true" --format "{{.Names}}" | xargs -r docker stop
|
||||
|
||||
echo "All stacks started (DBs running). Web services stopped and will be started on-demand by Sablier."
|
||||
Reference in New Issue
Block a user