Initial Docker Compose infrastructure backup

2025-12-27 13:58:19 +00:00
parent 4efaa045c7
commit 13c2599776
4 changed files with 1419 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,352 @@
+# Docker Compose Infrastructure Documentation
+
+## Overview
+
+This infrastructure manages a comprehensive self-hosted environment including media management (Jellyfin, *arr stack), document management (Paperless-ngx), AI services (Open WebUI, LiteLLM), photo management (Immich), genealogy tools (GrampsWeb), and various utilities.
+
+## Architecture
+
+### Network Segmentation
+
+- **media_net**: Media services and *arr applications
+- **db_net**: Database services (internal only)
+- **web_net**: Web-accessible services
+- **internal_net**: Internal service communication
+
+### Service Categories
+
+#### Management & Infrastructure
+- **Glances**: System monitoring dashboard
+- **Portainer**: Container management UI
+- **Watchtower**: Automatic container updates
+- **Homepage**: Service dashboard
+- **Docker Proxy**: Secure Docker socket proxy
+- **Nginx Proxy Manager (NPM)**: Reverse proxy with SSL
+
+#### Media & Arr Stack
+- **Prowlarr**: Indexer manager
+- **Radarr**: Movie management
+- **Sonarr**: TV show management
+- **Lidarr**: Music management
+- **Bazarr**: Subtitle management
+- **Jellyseerr**: Media request management
+- **Jellyfin**: Media server with hardware transcoding
+- **qBittorrent**: Download client
+- **Flaresolverr**: Cloudflare bypass
+- **Slskd**: Soulseek client
+- **Soulsync**: Music sync automation
+
+#### Document & AI Suite
+- **Paperless-ngx**: Document management system
+  - MariaDB database
+  - Redis broker
+  - Tika for text extraction
+  - Gotenberg for PDF rendering
+- **Paperless-AI**: AI-powered document analysis
+- **Stirling-PDF**: PDF manipulation tools
+- **Open WebUI**: LLM interface
+- **LiteLLM**: LLM proxy/gateway with PostgreSQL
+
+#### Photo & Data Management
+- **Immich**: Photo management and backup
+  - PostgreSQL with vector extensions
+  - Redis cache
+  - Optimized storage: SSD for thumbnails/profiles, NAS for full resolution
+- **Syncthing**: File synchronization (Obsidian vaults)
+
+#### Development
+- **Gitea**: Self-hosted Git service with MySQL
+
+#### Genealogy
+- **GrampsWeb**: Web interface for genealogy research
+  - Two instances: Jamie and Helen family trees
+  - Shared Redis broker
+
+#### Utilities
+- **Speedtest Tracker**: Network performance monitoring with MariaDB
+- **RustDesk**: Remote desktop (hbbs + hbbr servers)
+- **RedBot**: Discord bot
+- **iperf3**: Network performance testing
+- **Newt**: Custom application
+
+#### Finance
+- **WYGIWYH**: Financial tracking with PostgreSQL
+
+#### Music & Scrobbling
+- **Maloja**: Music scrobbling server
+- **Multi-Scrobbler**: Cross-platform scrobbler
+
+## Storage Structure
+
+### Local Docker Configs (`/docker/`)
+```
+/docker/
+├── Homepage/
+├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
+├── qBittorrent/
+├── slskd/
+├── soulsync/
+├── paperless/
+├── stirling/
+├── immich/
+├── obsidian/
+├── syncthing/
+├── gitea/
+├── gramps-jamie/
+├── gramps-helen/
+├── speedtest-tracker/
+├── rustdesk/
+├── redbot/
+├── maloja/
+├── scrobble/
+└── litellm/
+```
+
+### NAS Storage (`/mnt/Nas-Storage/data/`)
+```
+/mnt/Nas-Storage/data/
+├── media/
+│   ├── movies/
+│   ├── tv/
+│   ├── music/
+│   └── images/
+└── torrents/
+    └── soulsync/
+```
+
+## Prerequisites
+
+### Required Environment Variables (.env file)
+
+```bash
+# System
+PUID=1000
+PGID=1000
+TZ=Pacific/Auckland
+
+# URLs
+JELLYFIN_URL=https://your-jellyfin-domain.com
+PAPERLESS_URL=https://your-paperless-domain.com
+WYGIWYH_URL=https://your-wygiwyh-domain.com
+HOMEPAGE_ALLOWED_HOSTS=your-homepage-domain.com
+
+# Nginx Proxy Manager
+NPM_PASSWORD=your_secure_password
+
+# Paperless
+PAPERLESS_DB_ROOT_PASSWORD=your_secure_password
+PAPERLESS_DB_NAME=paperless
+PAPERLESS_DB_USER=paperless
+PAPERLESS_DB_PASSWORD=your_secure_password
+PAPERLESS_SECRET_KEY=your_secret_key
+
+# Immich
+IMMICH_POSTGRES_PASSWORD=your_secure_password
+
+# LiteLLM
+GROQ_API_KEY=your_groq_api_key
+LITELLM_MASTER_KEY=your_master_key
+
+# Slskd
+SLSKD_USERNAME=your_username
+SLSKD_PASSWORD=your_secure_password
+
+# Speedtest Tracker
+SPEEDTEST_DB_NAME=speedtest
+SPEEDTEST_DB_USER=speedtest
+SPEEDTEST_DB_PASSWORD=your_secure_password
+SPEEDTEST_APP_KEY=base64:your_generated_key
+
+# WYGIWYH
+WYGIWYH_DB_DATABASE=wygiwyh
+WYGIWYH_DB_USER=wygiwyh
+WYGIWYH_DB_PASSWORD=your_secure_password
+WYGIWYH_SECRET_KEY=your_secret_key
+WYGIWYH_ALLOWED_HOSTS=localhost,127.0.0.1,your-domain.com
+
+# RedBot
+REDBOT_TOKEN=your_discord_bot_token
+```
+
+### System Requirements
+
+- Docker Engine 20.10+
+- Docker Compose V2
+- Sufficient storage for media and databases
+- Intel GPU for Jellyfin hardware transcoding (optional)
+
+## Installation
+
+1. **Clone/Download this repository**
+
+2. **Create required directories**
+   ```bash
+   # See directory_structure.sh for automated setup
+   chmod +x directory_structure.sh
+   ./directory_structure.sh
+   ```
+
+3. **Create .env file**
+   ```bash
+   cp .env.example .env
+   # Edit .env with your values
+   nano .env
+   ```
+
+4. **Set correct permissions**
+   ```bash
+   sudo chown -R $PUID:$PGID /docker
+   sudo chown -R $PUID:$PGID /mnt/Nas-Storage/data
+   ```
+
+5. **Start services**
+   ```bash
+   docker compose up -d
+   ```
+
+## Service Access
+
+Default ports (configure reverse proxy for SSL/domains):
+
+- Homepage: 7575
+- Portainer: 9443
+- Jellyfin: 8096
+- Radarr: 7878
+- Sonarr: 8989
+- Prowlarr: 9696
+- Lidarr: 8686
+- Bazarr: 6767
+- Jellyseerr: 5055
+- qBittorrent: 7070
+- Paperless: 8100
+- Stirling-PDF: 8090
+- Open WebUI: 3000
+- Immich: 2283
+- Syncthing: 8384
+- Gitea: 8418
+- GrampsWeb Jamie: 5511
+- GrampsWeb Helen: 5512
+- Speedtest Tracker: 8180
+- NPM Admin: 81
+- Maloja: 42010
+- Multi-Scrobbler: 9078
+- WYGIWYH: 9008
+
+## Maintenance
+
+### Backup Strategy
+
+**Critical Data to Backup:**
+- `/docker/` - All service configurations
+- Database volumes (see docker-compose.yaml)
+- `.env` file (store securely, contains secrets)
+
+**Optional (can be regenerated):**
+- Media files on NAS
+- Immich thumbnails (can be regenerated)
+
+### Updates
+
+Watchtower automatically updates containers daily. To manually update:
+
+```bash
+docker compose pull
+docker compose up -d
+```
+
+### Logs
+
+View logs for any service:
+```bash
+docker compose logs -f [service_name]
+```
+
+### Resource Limits
+
+- Portainer: 512MB RAM limit
+- Log rotation: 10MB max, 3 files
+
+## Hardware Acceleration
+
+Jellyfin is configured for Intel GPU transcoding:
+- Device: `/dev/dri/renderD128`
+- Group: `104` (render group)
+
+Verify GPU access:
+```bash
+ls -l /dev/dri/renderD128
+```
+
+## Security Considerations
+
+1. **Secrets Management**: Store `.env` securely, never commit to version control
+2. **Network Segmentation**: Database network is internal-only
+3. **Container Security**: `no-new-privileges:true` on supported services
+4. **Reverse Proxy**: Use NPM for SSL termination and authentication
+5. **Updates**: Watchtower keeps containers updated
+
+## Troubleshooting
+
+### Service won't start
+```bash
+docker compose logs [service_name]
+docker compose restart [service_name]
+```
+
+### Database connection issues
+```bash
+# Check database is healthy
+docker compose ps
+# Verify network connectivity
+docker compose exec [service] ping [db_service]
+```
+
+### Permission errors
+```bash
+# Verify ownership
+ls -la /docker/[service]/
+# Fix if needed
+sudo chown -R $PUID:$PGID /docker/[service]/
+```
+
+### Storage full
+```bash
+# Check Docker disk usage
+docker system df
+# Clean up unused resources
+docker system prune -a
+```
+
+## Migration/Restoration
+
+1. Install Docker and Docker Compose on new system
+2. Restore `/docker/` directory structure
+3. Restore `.env` file
+4. Restore database volumes (if applicable)
+5. Mount NAS storage at `/mnt/Nas-Storage/data/`
+6. Run `docker compose up -d`
+
+## Contributing Services
+
+To add new services:
+1. Add service definition to appropriate section
+2. Assign to correct network(s)
+3. Add volume mounts under `/docker/[service]/`
+4. Update this README with service description
+5. Add required environment variables to `.env.example`
+
+## License
+
+This is a personal infrastructure setup. Adapt as needed for your use case.
+
+## Support
+
+For issues with specific services, consult their official documentation:
+- Jellyfin: https://jellyfin.org/docs/
+- Paperless-ngx: https://docs.paperless-ngx.com/
+- Immich: https://immich.app/docs/
+- And so on...
+
+---
+
+**Last Updated**: December 2025