Files
Homelab/README.md

391 lines
11 KiB
Markdown

# Docker Compose Infrastructure Documentation
## Overview
This infrastructure manages a comprehensive self-hosted environment including media management (Jellyfin, *arr stack), document management (Paperless-ngx), AI services (Open WebUI, LiteLLM), photo management (Immich), genealogy tools (GrampsWeb), and various utilities.
## Architecture
### Network Segmentation
- **media_net**: Media services and *arr applications
- **db_net**: Database services (internal only)
- **web_net**: Web-accessible services
- **internal_net**: Internal service communication
### Service Categories
#### Management & Infrastructure
- Glances: System monitoring dashboard
- Portainer: Container management UI
- Watchtower: Automatic container updates
- Homepage: Service dashboard
- Docker Proxy: Secure Docker socket proxy
- Nginx Proxy Manager (NPM): Reverse proxy with SSL
#### Media & Arr Stack
- Prowlarr: Indexer manager
- Radarr: Movie management
- Sonarr: TV show management
- Lidarr: Music management
- Bazarr: Subtitle management
- Jellyseerr: Media request management
- Jellyfin: Media server with hardware transcoding
- qBittorrent: Download client
- Flaresolverr: Cloudflare bypass
- Slskd: Soulsearch client
- Soulsync: Music sync automation
#### Document & AI Suite
- Paperless-ngx: Document management system
- MariaDB database
- Redis broker
- Tika for text extraction
- Gotenberg for PDF rendering
- Paperless-AI: AI-powered document analysis
- OnlyOffice: Document collaboration server
- Stirling-PDF: PDF manipulation tools
- Open WebUI: LLM interface
- LiteLLM: LLM proxy/gateway with PostgreSQL
#### Photo & Data Management
- Immich: Photo management and backup
- PostgreSQL with vector extensions
- Redis cache
- SSD/NAS storage for thumbnails/profiles and full resolution
- Syncthing: File synchronization (Obsidian vaults)
#### Development
- Gitea: Self-hosted Git service with MySQL
#### Genealogy
- GrampsWeb: Web interface for genealogy research
- Two instances: Jamie and Helen family trees
- Shared Redis broker
#### Utilities
- Speedtest Tracker: Network performance monitoring with MariaDB
- RustDesk: Remote desktop (hbbs + hbbr servers)
- RedBot: Discord bot
- iperf3: Network performance testing
- Newt: Custom application
#### Finance
- WYGIWYH: Financial tracking with PostgreSQL
#### Music & Scrobbling
- Maloja: Music scrobbling server
- Multi-Scrobbler: Cross-platform scrobbler
## Storage Structure
### Local Docker Configs (`/docker/`)
```
/docker/
├── Homepage/
├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
├── qBittorrent/
├── slskd/
├── soulsync/
├── paperless/
├── stirling/
├── immich/
├── obsidian/
├── syncthing/
├── gitea/
├── gramps-jamie/
├── gramps-helen/
├── speedtest-tracker/
├── rustdesk/
├── redbot/
├── maloja/
├── scrobble/
└── litellm/
```
### NAS Storage (`/mnt/Nas-Storage/data/`)
```
/mnt/Nas-Storage/data/
├── media/
│ ├── movies/
│ ├── tv/
│ ├── music/
│ └── images/
└── torrents/
└── soulsync/
```
## Prerequisites
### Required Environment Variables (.env file)
```
# System
PUID=1000
PGID=1000
TZ=Pacific/Auckland
# URLs
JELLYFIN_URL=https://your-jellyfin-domain.com
PAPERLESS_URL=https://your-paperless-domain.com
WYGIWYH_URL=https://your-wygiwyh-domain.com
HOMEPAGE_ALLOWED_HOSTS=your-homepage-domain.com
# Nginx Proxy Manager
NPM_PASSWORD=your_secure_password
# Paperless
PAPERLESS_DB_ROOT_PASSWORD=your_secure_password
PAPERLESS_DB_NAME=paperless
PAPERLESS_DB_USER=paperless
PAPERLESS_DB_PASSWORD=your_secure_password
PAPERLESS_SECRET_KEY=your_secret_key
# Immich
IMMICH_POSTGRES_PASSWORD=your_secure_password
# LiteLLM
GROQ_API_KEY=your_groq_api_key
LITELLM_MASTER_KEY=your_master_key
# Slskd
SLSKD_USERNAME=your_username
SLSKD_PASSWORD=your_secure_password
# Speedtest Tracker
SPEEDTEST_DB_NAME=speedtest
SPEEDTEST_DB_USER=speedtest
SPEEDTEST_DB_PASSWORD=your_secure_password
SPEEDTEST_APP_KEY=base64:your_generated_key
# WYGIWYH
WYGIWYH_DB_DATABASE=wygiwyh
WYGIWYH_DB_USER=wygiwyh
WYGIWYH_DB_PASSWORD=your_secure_password
WYGIWYH_SECRET_KEY=your_secret_key
WYGIWYH_ALLOWED_HOSTS=localhost,127.0.0.1,your-domain.com
# RedBot
REDBOT_TOKEN=your_discord_bot_token
```
### System Requirements
- Docker Engine 20.10+
- Docker Compose V2
- Sufficient storage for media and databases
- Intel GPU for Jellyfin hardware transcoding (optional)
## Installation
1. Clone/Download this repository
2. Create required directories
```bash
# See directory_structure.sh for automated setup
chmod +x directory_structure.sh
./directory_structure.sh
```
3. Create .env file
```bash
cp .env.example .env
# Edit .env with your values
nano .env
```
4. Set correct permissions
```bash
sudo chown -R $PUID:$PGID /docker
sudo chown -R $PUID:$PGID /mnt/Nas-Storage/data
```
5. Start services
```bash
docker compose up -d
```
## Service Access
Default ports (configure reverse proxy for SSL/domains):
- Homepage: 7575
- Portainer: 9443
- Jellyfin: 8096
- Radarr: 7878
- Sonarr: 8989
- Prowlarr: 9696
- Lidarr: 8686
- Bazarr: 6767
- Jellyseerr: 5055
- qBittorrent: 7070
- Paperless: 8100
- Stirling-PDF: 8090
- OnlyOffice: 8091
- Open WebUI: 3000
- Immich: 2283
- Syncthing: 8384
- Gitea: 8418
- GrampsWeb Jamie: 5511
- GrampsWeb Helen: 5512
- Speedtest Tracker: 8180
- NPM Admin: 81
- Maloja: 42010
- Multi-Scrobbler: 9078
## Maintenance
### Backup Strategy
**Critical Data to Backup:**
- `/docker/` - All service configurations
- Database volumes (see docker-compose.yaml)
- `.env` file (store securely, contains secrets)
**Optional (can be regenerated):**
- Media files on NAS
- Immich thumbnails (can be regenerated)
### Updates
Watchtower automatically updates containers daily. To manually update:
```
docker compose pull
docker compose up -d
```
### Logs
View logs for any service:
```
docker compose logs -f [service_name]
```
### Resource Limits
- Portainer: 512MB RAM limit
- Log rotation: 10MB max, 3 files
## Hardware Acceleration
Jellyfin is configured for Intel GPU transcoding:
- Device: `/dev/dri/renderD128`
- Group: `104` (render group)
Verify GPU access:
```
ls -l /dev/dri/renderD128
```
## Security Considerations
1. Secrets Management: Store `.env` securely, never commit to version control
2. Network Segmentation: Database network is internal-only
3. Container Security: `no-new-privileges:true` on supported services
4. Reverse Proxy: Use NPM for SSL termination and authentication
5. Updates: Watchtower keeps containers updated
## Troubleshooting
### Service won't start
```
docker compose logs [service_name]
docker compose restart [service_name]
```
### Database connection issues
```
# Check database is healthy
docker compose ps
# Verify network connectivity
docker compose exec [service] ping [db_service]
```
### Permission errors
```
# Verify ownership
ls -la /docker/[service]/
# Fix if needed
sudo chown -R $PUID:$PGID /docker/[service]/
```
### Storage full
```
# Check Docker disk usage
docker system df
# Clean up unused resources
docker system prune -a
```
## Internal Docker Services\n\n| Service | Role / Component | Web UI Port | Notes |
|---|---|---:|---|
| Jellyfin | Media server with hardware transcoding | 8096 | - |
| Jellyseerr | Media request management | 5055 | - |
| Radarr | Movie management | 7878 | - |
| Sonarr | TV show management | 8989 | - |
| Lidarr | Music management | 8686 | - |
| Bazarr | Subtitle management | 6767 | - |
| Prowlarr | Indexer manager | 9696 | - |
| qBittorrent | Download client | 7070 | - |
| Paperless | Document management interface | 8100 | - |
| Stirling-PDF | PDF manipulation tools | 8090 | - |
| OnlyOffice | Document collaboration server | 8091 | - |
| Open WebUI | LLM interface | 3000 | - |
| Immich | Photo management | 2283 | - |
| Syncthing | File synchronization | 8384 | - |
| Gitea | Git service | 8418 | - |
| GrampsWeb Jamie | Genealogy UI | 5511 | - |
| GrampsWeb Helen | Genealogy UI | 5512 | - |
| Speedtest Tracker | Network performance | 8180 | - |
| NPM Admin | NPM admin UI | 81 | - |
| Maloja | Music scrobbling | 42010 | - |
| Multi-Scrobbler | Cross-platform scrobbler | 9078 | - |
| WYGIWYH | Financial tracking | 9008 | - |
| Portainer | Container management UI | 9443 | - |
| Homepage | Service dashboard | 7575 | - |
## External Non-Docker Services\n\n| Service | IP | Role / Description | Web UI Port | Notes |\n|---|---|---|---:|---|\n| npm (Nginx Proxy Manager) | 192.168.0.10 | Web UI; admin: 81 | 80/443 | - |\n| adguard | 192.168.0.11 | DNS/ad-blocking | 3000 | - |\n| vaultwarden | 192.168.0.12 | Password manager | 80/8080 | - |\n| crafty (Crafty Controller) | 192.168.0.13 | Minecraft server management | 3000 | - |\n| nextcloud | 192.168.0.14 | File sync and collaboration | 443 | - |\n| homeassistant | 192.168.0.15 | Home automation | 8123 | - |\n| foundryvtt | 192.168.0.16 | Foundry VTT | 30000 | - |\n| openmediavault | 192.168.0.17 | NAS management | 80/443 | - |\n| wordpress-irodori | 192.168.0.18 | WordPress site | 80 | - |\n| wordpress-dustin | 192.168.0.19 | WordPress site | 80 | - |\n\nIf you want, I can tighten these with exact ports from your Proxmox/VM configs and add links to each service's docs.\n\n## Migration/Restoration
Your homelab includes a number of LXCs/VMs outside Docker. This section documents their host IPs, brief descriptions, and typical access ports. Verify in your Proxmox/VM configs.
- npm (Nginx Proxy Manager) — 192.168.0.10 — Web UI and reverse proxy management; typical host ports: 80/443, admin: 81
- adguard (AdGuard Home) — 192.168.0.11 — DNS/ad-blocking; UI: 3000
- vaultwarden (Vaultwarden) — 192.168.0.12 — Password manager; UI: 80/8080
- crafty (Crafty Controller) — 192.168.0.13 — Home automation central
- nextcloud — 192.168.0.14 — File sync and collaboration; UI: 443
- homeassistant — 192.168.0.15 — Home automation; UI: 8123
- foundryvtt — 192.168.0.16 — Foundry VTT; UI: 30000
- openmediavault (OMV) — 192.168.0.17 — NAS management; UI: 80/443
- wordpress-irodori — 192.168.0.18 — WordPress site; UI: 80
- wordpress-dustin — 192.168.0.19 — WordPress site; UI: 80
1. Install Docker and Docker Compose on new system
2. Restore `/docker/` directory structure
3. Restore `.env` file
4. Restore database volumes
5. Mount NAS storage at `/mnt/Nas-Storage/data/`
6. Run `docker compose up -d`
## Contributing Services
To add new services:
1. Add service definition to appropriate section
2. Assign to correct network(s)
3. Add volume mounts under `/docker/[service]/`
4. Update this README with service description
5. Add required environment variables to `.env.example`
## License
This is a personal infrastructure setup. Adapt as needed for your use case.
## Support
For issues with specific services, consult their official documentation:
- Jellyfin: https://jellyfin.org/docs/
- Paperless-ngx: https://docs.paperless-ngx.com/
- Immich: https://immich.app/docs/
- And so on...
---
**Last Updated**: January 2026