Commit Graph

102 Commits

Author SHA1 Message Date
EZ-Homelab
3e221bb3f2 feat: enhance reset-test-environment.sh for complete cleanup
- Stop and remove ALL containers (not just specific stacks)
- Remove ALL Docker images, volumes, and networks
- Completely remove /opt/stacks and /opt/dockge directories
- Updated warnings to reflect thorough cleanup
- Maintains safety checks and user confirmation
2026-01-23 18:22:43 -05:00
EZ-Homelab
3701b11a73 feat: Update Sablier configs for debian hostname and reorganize .env.example
- Change Traefik configs to use ${SERVER_HOSTNAME} placeholder (defaults to debian)
- Update ez-homelab.sh to replace SERVER_HOSTNAME in config templates
- Set Sablier session duration to 5m for testing (increase to 30m for production)
- Add SERVER_HOSTNAME prompt and saving in setup script
- Reorganize .env.example with better structure and SMTP variables
- Add production guidance comments to docker-compose files
- Intentional SMTP variable redundancy for service flexibility
2026-01-23 15:25:25 -05:00
EZ-Homelab
3b01467477 Fix Docker detection logic to avoid unnecessary system setup
- Script now properly detects if Docker is installed and user is in docker group
- Prevents forcing logout/login when Docker is already properly configured
- Only runs system_setup when actually needed
2026-01-22 21:32:44 -05:00
EZ-Homelab
d4e6ce682b fix: Skip Docker/Docker Compose installation when already installed
- Update ez-homelab.sh Step 3: Check if Docker is installed before attempting installation
- Update ez-homelab.sh Step 4: Check if Docker Compose is installed before attempting installation
- Update setup-homelab.sh Step 3: Improve Docker check to verify service status and start if needed
- Both scripts now skip installation and notify when components are already available
- Maintains backward compatibility and proper service management
2026-01-22 20:07:31 -05:00
EZ-Homelab
06ce59e02a feat: Add automatic environment variable replacement in deployment script
- Add replace_env_vars() function to automatically scan and replace  patterns
- Function checks all config files (.yml, .yaml, .conf, .json) for variables
- Replaces variables found in .env file with their values
- Warns about variables referenced in templates but missing from .env
- Applied to both core deployment and stack setup for Dockge
- Maintains backward compatibility with existing hardcoded replacements
- Makes deployment script maintenance-free for new services
2026-01-22 19:55:31 -05:00
EZ-Homelab
9cc0e93c79 feat: Add comprehensive restart policy documentation and Sablier lazy loading support
- Add SERVER_HOSTNAME env var for Sablier group naming
- Update default hostname from 'jarvis' to 'debian' for generic repo compatibility
- Add restart policy documentation to all docker-compose files
- Add Sablier labels to lazy-loaded services (jellyfin, dozzle, glances, code-server, homarr, dokuwiki)
- Update sablier.yml template to use debian- prefixes
- Enhance deploy script to auto-detect hostname and update configurations
- Ensure all YAML files remain syntactically valid
2026-01-22 19:49:24 -05:00
EZ-Homelab
4140a32adb fix: Update script to replace ${DEFAULT_EMAIL} in authelia users database
- Add sed replacement for ${DEFAULT_EMAIL} with actual admin email
- Ensures email addresses are properly configured during deployment
2026-01-22 19:05:18 -05:00
EZ-Homelab
71d9a1e152 feat: Complete EZ-Homelab deployment system overhaul
- Add unified ez-homelab.sh script with guided menu interface
- Create dedicated Dockge stack in /opt/dockge for clean isolation
- Move dockerproxy from core to infrastructure stack
- Fix Authelia configuration with proper variable placeholders
- Update all compose files to use  variables
- Enhance script with comprehensive variable replacement
- Fix sed delimiter conflicts and middleware issues
- Add proper step numbering and error handling
- Prepare all stacks for Dockge management
- Update README with new deployment instructions
2026-01-22 18:56:20 -05:00
EZ-Homelab
89760895f2 Refactor docker-compose structure to folder-based organization
- Remove redundant .yml files from main docker-compose folder
- Update deploy script to use folder-based structure for all stacks
- Update documentation to reflect new folder-based organization
- Standardize all stacks to use docker-compose.yml in individual folders

This eliminates confusion between file-based and folder-based structures,
making the repository more maintainable and consistent.
2026-01-22 16:44:44 -05:00
EZ-Homelab
535ae5017b Fix deploy script: source .env after copying it
- Move .env sourcing after cp to avoid 'No such file' error
- Ensure secrets are available for placeholder replacement
2026-01-22 16:14:24 -05:00
EZ-Homelab
12baa5cdf3 Fix EZ-Homelab scripts for successful fresh installs
- Fix Authelia config template paths (/data/ -> /config/)
- Add secret placeholder replacement in deploy script
- Add user database placeholder replacement
- Add automatic dashboard stack deployment (Homepage, Homarr)
- Remove incorrect Portainer reference from infrastructure log
- Update deployment steps and final summary
2026-01-22 16:08:39 -05:00
kelinfoxy
30e0481685 copilot Instructions improvements
and documentation updates
2026-01-21 18:52:40 -05:00
kelinfoxy
b03ffd092c Add Dokuwiki deployment to deploy-homelab.sh for immediate availability 2026-01-20 20:21:00 -05:00
kelinfoxy
ae35eafdb6 Merge remote changes and resolve conflict in copilot-instructions.md 2026-01-20 19:40:20 -05:00
kelinfoxy
16b7e1f1a7 Documentation Reorganization
Major upgrade to the documentation.
2026-01-20 19:01:21 -05:00
kelinfoxy
21ee910267 Merge remote updates, accepting remote versions for conflicted files 2026-01-17 20:22:10 -05:00
00a3111ce4 Refine setup script prompts and cleanup deploy script credential files
- Skip password confirmation prompt if DEFAULT_PASSWORD is already set and valid
- Remove redundant placeholder values from is_placeholder function
- Clean up temporary and permanent credential files after deployment to avoid redundancy with .env
2026-01-16 19:51:18 -05:00
39324c9c9d Fix get_env_value to strip comments and trim spaces properly 2026-01-16 18:48:23 -05:00
8d3175918c Add prompts for DEFAULT_USER, DEFAULT_EMAIL, DEFAULT_PASSWORD if placeholders; use defaults for Authelia credentials 2026-01-16 18:41:22 -05:00
882491c418 Fix ADMIN_USER assignment and add prompt for username/email if placeholders; trim spaces in get_env_value 2026-01-16 18:37:02 -05:00
10f5a38487 Add validation for DUCKDNS_TOKEN and DUCKDNS_SUBDOMAINS in setup script 2026-01-16 18:28:27 -05:00
0df343a9e0 Add 'your-username' to placeholder checks 2026-01-16 18:25:50 -05:00
86acc36e30 Remove newlines from variables before sed to prevent multi-line sed commands 2026-01-16 17:33:28 -05:00
40bd255727 Escape | characters in variables before using sed to prevent unterminated s command 2026-01-16 17:31:43 -05:00
0cbc6128bd Fix sed commands to use # delimiter to avoid issues with special characters in variables 2026-01-16 17:30:27 -05:00
7e7488a4c5 Fix missing generate_secret function in setup script 2026-01-16 17:28:50 -05:00
5561355e25 Modify setup script for minimal interaction: use valid .env values, prompt for Authelia credentials with options 2026-01-16 17:02:20 -05:00
655df5c159 Apply researched resource limits to all Docker Compose stacks
- Add deploy.resources sections to all services based on service type
- Lightweight services: 0.25 CPU, 128M RAM (Traefik, Authelia, Pi-hole)
- Web services: 0.50 CPU, 256M RAM (Dashboards, simple web apps)
- Media services: 2.0 CPU, 2G RAM (Jellyfin, transcoding)
- Database services: 1.0 CPU, 1G RAM (PostgreSQL, caching)
- Heavy apps: 1.5 CPU, 1G RAM (Nextcloud, Home Assistant)
- Monitoring: 0.75 CPU, 512M RAM (Prometheus, Grafana)
- Include CPU, memory, and process limits with reservations
- Create comprehensive resource limits template documentation
2026-01-15 21:30:11 -05:00
a697bc265c Add SSL certificate preservation across test environment resets
- Add acme.json to .gitignore to prevent accidental commits
- Modify reset script to preserve certificates in repo folder before cleanup
- Modify setup script to restore certificates to correct location
- Update step numbering in reset script (now 7 steps)
- Maintain proper file permissions and ownership for certificates
2026-01-15 20:25:07 -05:00
ac27a073e3 Fix setup script to ensure Docker Compose is installed 2026-01-15 15:28:06 -05:00
ef5868b481 Update deploy-homelab.sh script 2026-01-15 03:32:21 -05:00
258e8eec94 Refactor scripts for improved maintainability
- setup-homelab.sh: Fixed syntax errors, placeholder detection, and hardcoded paths
- deploy-homelab.sh: Refactored from inline code to function-based structure
- Both scripts now use consistent function organization for better readability
- Enhanced credential handling and error checking
- All scripts validated for syntax correctness
2026-01-14 18:10:23 -05:00
650700ed0a Re-enable Watchtower with correct Docker API version
Fixes:
- docker-compose/infrastructure.yml:
  - Uncommented Watchtower service
  - Updated image from 1.7.1 to latest
  - Changed DOCKER_API_VERSION from 1.44 to 1.52 (current Docker version)
  - Added default empty value for WATCHTOWER_NOTIFICATION_URL

- scripts/deploy-homelab.sh:
  - Removed "temporarily disabled" note
  - Added Watchtower to infrastructure stack list

- docs/services-overview.md:
  - Updated infrastructure stack count from 7 to 8
  - Added Watchtower to service list

Watchtower now runs successfully with scheduled updates at 4 AM daily
2026-01-14 02:25:20 -05:00
3e53cc3225 Remove automatic deployment prompt from setup script
Changes:
- scripts/setup-homelab.sh: Remove interactive deployment prompt
  - Users must now run deploy script manually
  - Simplifies both scripts (no sudo workarounds needed)
  - Clearer two-step process: setup then deploy

- Documentation updates:
  - README.md: Updated step 3-4 with manual deployment
  - docs/getting-started.md: Removed step 6 (log out), clarified steps
  - docs/manual-setup.md: Added sudo to deploy command
  - docs/troubleshooting/COMMON-ISSUES.md: Added sudo to all deploy commands

Rationale:
- Automatic deployment via 'su -' cannot work with sudo requirement
- Manual two-step process is clearer and more reliable
- Setup focuses on configuration, deploy focuses on services
2026-01-14 02:04:56 -05:00
ff454d35c6 Update deploy script to use media-management.yml instead of media-extended.yml 2026-01-14 01:41:23 -05:00
e6c8f25275 Fix password hash extraction bug in deploy script and sudo issue in setup script
- deploy-homelab.sh: Fix password hash extraction from Docker output
  - Changed from 'grep || tail' fallback to 'sed | grep' pipeline
  - Properly strips 'Digest: ' prefix before extracting hash
  - Prevents corrupted hash format that caused Authelia crash loop

- setup-homelab.sh: Fix automatic deployment call
  - Added 'sudo' when running deploy script from setup
  - Prevents 'Please run as root' error during automatic deployment
2026-01-14 01:23:44 -05:00
d12706fda2 feat: persist Authelia credentials to .env file
- setup-homelab.sh: Save AUTHELIA_ADMIN_* credentials to .env file
- deploy-homelab.sh: Check .env file as fallback if temp files don't exist
- .env.example: Document auto-generated Authelia admin variables

This ensures credentials survive reboots (e.g., when NVIDIA drivers are installed)
and the deploy script can find them even when run manually after reboot.
2026-01-14 00:10:38 -05:00
56604b77e9 fix: store Authelia credentials in persistent location
- setup-homelab.sh: Store temp files in /opt/stacks/.setup-temp instead of /tmp
- deploy-homelab.sh: Read credentials from new persistent location
- reset-test-environment.sh: Clean up new temp directory

This fixes the issue where credentials were inaccessible when deploy script
runs via 'su -' (login shell) from setup script, as /tmp files created by
root are not accessible across the su boundary.
2026-01-14 00:03:34 -05:00
9f122af4b5 feat: implement task list updates
- getting-started.md: Moved checklist before Simple Setup, removed Round 4 section
- authelia-customization.md: Updated Authentik reference to alternatives stack
- services-overview.md: Added clickable links to all stack compose files
- setup-homelab.sh: Added prompt to run deployment script after setup (defaults to yes)
- traefik.yml: Changed default to DNS challenge for wildcard certificates (DuckDNS)

All documentation now reflects wildcard certificate usage with DNS challenge.
2026-01-13 23:14:25 -05:00
487f645652 Round 9: Homepage variable replacement and additional stack deployment
Features added:
 Homepage config variable replacement - Fixed HOMEPAGE_VAR_DOMAIN substitution
  - Homepage doesn't support environment variables in configs
  - Deploy script now uses sed to replace {{HOMEPAGE_VAR_DOMAIN}} with actual domain
  - All homepage/*.yaml files processed after template copy

 Additional stacks deployment to Dockge
  - 7 additional stacks now copied to /opt/stacks/: media, media-extended,
    homeassistant, productivity, monitoring, utilities, alternatives
  - Stacks are NOT started automatically - user deploys via Dockge UI as needed
  - Optional image pre-pull with user prompt (defaults to no)
  - Significantly improves first-time Dockge experience

 Watchtower temporarily disabled
  - Documented Docker API v1.44 compatibility issue with Docker 29.x
  - Added clear instructions for re-enabling when issue is resolved
  - Infrastructure stack now deploys 6 services (was 7)

Deployment workflow:
1. Core stack (4 services) - DuckDNS, Traefik, Authelia, Gluetun
2. Infrastructure stack (6 services) - Dockge, Pi-hole, Dozzle, Glances, Docker Proxy
3. Dashboards stack (2 services) - Homepage (configured), Homarr
4. Additional stacks (7 stacks copied, not started)

Tested: All 11 active containers healthy, all stacks visible in Dockge
2026-01-13 21:36:38 -05:00
cf061f35d2 Fix: Resolve password hash corruption in Authelia users_database.yml
Critical fix for argon2 password hash preservation:
- Root cause: Bash variable expansion of $ characters in argon2id hashes
- Solution: Write hash directly from Docker output to file, bypass bash variables entirely
- setup-homelab.sh: Stream Docker output directly to /tmp/authelia_password_hash.tmp
- deploy-homelab.sh: Read hash file in Python to avoid any bash expansion
- Result: Password hash correctly preserved with full $argon2id$v=19$m=... format

Other changes:
- Added DOCKER_API_VERSION=1.44 env var for watchtower (API compatibility)
- Watchtower still has issues with Docker 29.1.4 - keeping version pinned for investigation

Tested on Debian 12 with Docker 29.1.4:
 All 11 critical containers healthy
 Authelia authentication working correctly
 Password hash preserved through entire deployment workflow
⚠️  Watchtower restart loop (non-critical, under investigation)
2026-01-13 21:02:49 -05:00
659d580d14 Round 8: Attempt to fix sed escaping for password hash
Issue: sed with | delimiter still has problems with $ in argon2 hash
Attempted fix: Escape special characters before sed replacement

Note: Manual sed with double quotes works, suggesting escaping strategy
may need refinement. Need to test if this resolves the issue.
2026-01-13 20:15:21 -05:00
ee8a359542 Fix password hash corruption in users_database.yml
Issue: Heredoc variable expansion was mangling password hashes containing $ characters
Solution: Use quoted heredoc ('EOF') with placeholders, then sed replace

The unquoted heredoc was interpreting $ in the argon2 hash as shell variable
expansion, corrupting the hash format.
2026-01-13 20:06:43 -05:00
8b5ba494dd Round 7 Prep: Add safe cleanup procedures to prevent system crashes
CRITICAL: Previous rounds caused system crashes during cleanup operations

New Safe Reset Script:
- Gracefully stops all containers before cleanup
- Waits for proper shutdown sequences
- Removes Docker volumes only after containers stopped
- Prevents filesystem corruption from aggressive rm operations
- Includes confirmation prompts for safety

Deploy Script Improvements:
- Stops existing containers before config file operations
- Removes dangerous auto-cleanup of Docker volumes
- Adds safety checks before directory removal
- Warns about existing databases instead of auto-removing

Dangerous Operations Removed:
- No more rm -rf while containers running
- No more automatic volume deletion
- No more blind directory removal
- No more container restart during volume operations

Testing Guidelines:
- Always use reset-test-environment.sh for cleanup
- Never run cleanup while containers active
- Monitor system health during operations
- Proper shutdown sequence documented

This prevents the BIOS-level crashes experienced in previous rounds.
2026-01-13 20:02:04 -05:00
12df3a1ae2 Round 6: Fix deployment script reliability and credential handling
- Add pre-flight validation checks (internet, disk space, Docker availability)
- Fix Authelia password hash extraction (handle 'Digest:' prefix format)
- Improve credential flow between setup and deploy scripts
- Save plain password for user reference in ADMIN_PASSWORD.txt
- Add cleanup for directory/file conflicts on re-runs
- Add automatic Authelia database cleanup for encryption key mismatches
- Add error recovery guidance with cleanup trap
- Display credentials prominently after deployment
- Update step numbering (now 10 steps with pre-flight)
- Update documentation to Round 6

Tested on fresh Debian 12 installation - both scripts now complete successfully.
2026-01-13 19:57:45 -05:00
ac0e39d091 Round 5 improvements: complete automation and documentation fixes
- Fix password file ownership (user can now read without sudo)
- Add dashboards stack to automated deployment (Step 5/6)
- Add SSL certificate notes to deploy script output
- Clarify .env file location in documentation (stays in repo folder)
- Update README and getting-started.md with accurate deployment steps
- Add Watchtower notification URL documentation
- Improve user feedback with admin credentials and dashboard URLs
- Remove dashboards from 'Next Steps' since it's now automated

User experience improvements:
- Password file readable by user immediately
- Homepage and Homarr deployed automatically
- Clear guidance on .env file management
- Better SSL certificate expectations
2026-01-13 18:43:10 -05:00
f0a3907002 Round 4 improvements: automated config, relative paths, simplified deployment
- Automate Traefik email substitution in deploy script
- Auto-generate Authelia admin password (saved to ADMIN_PASSWORD.txt)
- Standardize all volume paths to use relative paths (./service/config)
- Switch Traefik to HTTP challenge by default (DNS challenge optional)
- Update documentation with improved setup instructions
- Enhance troubleshooting guide
- Update AGENT_INSTRUCTIONS with new conventions
- Simplify .env.example with clearer guidance

These changes reduce manual configuration steps and improve deployment reliability.
2026-01-13 18:30:06 -05:00
f92424ed6d Fix critical deployment issues for Round 4
- Add DOCKER_API_VERSION=1.44 to Watchtower (fixes crash loop)
- Add dockerproxy-network creation to deploy script (fixes dashboard deployment)
- Add explicit acme.json file creation with 600 permissions (fixes SSL cert acquisition)
- Fix setup script to correctly resolve user home directory when run with sudo

These fixes resolve all critical blockers discovered in Round 3 testing.
2026-01-13 17:36:47 -05:00
bbcc4c19c9 Update Homepage dashboard and deployment scripts
- Homepage: Reorganize services by stack instead of by category
- Homepage: Add comprehensive Available to Install sections for all stacks
- Homepage: Update config templates with {{HOMEPAGE_VAR_DOMAIN}} placeholder
- Homepage: Change layout from row to column style
- Scripts: Add sudo requirement to deploy-homelab.sh
- Scripts: Replace NVIDIA driver installation with official installer method
- Scripts: Add build prerequisites and nouveau blacklisting
- Docs: Add AI Automation Guidelines section to docker-guidelines.md
- Docs: Document Homepage auto-update requirements and workflow
- Config: Add bookmarks.yaml template for Homepage
- Config: Add alternatives.yml compose file (Portainer, Authentik)
- Config: Update .env.example and authelia configuration
2026-01-13 00:04:43 -05:00
kelinfoxy
cba45556c7 Complete documentation refactoring with 57 service docs
- Refactored README.md, getting-started.md, quick-reference.md
- Enhanced setup-homelab.sh with 9-step automated process
- Created services-overview.md with all stacks
- Added comprehensive documentation for 57 services in docs/service-docs/
- All services include: overview, configuration, resources, educational content
- Coverage: Core, Infrastructure, Dashboards, Media, Media-Extended, Home Assistant, Productivity, Utilities, Monitoring, Development stacks
- Educational focus with links to tutorials, videos, and guides
2026-01-12 18:03:12 -05:00