355 lines
8.2 KiB
Markdown
355 lines
8.2 KiB
Markdown
# Docker Compose Infrastructure Documentation
|
|
|
|
## Overview
|
|
|
|
This infrastructure manages a comprehensive self-hosted environment including media management (Jellyfin, *arr stack), document management (Paperless-ngx), AI services (Open WebUI, LiteLLM), photo management (Immich), genealogy tools (GrampsWeb), and various utilities.
|
|
|
|
## Architecture
|
|
|
|
### Network Segmentation
|
|
|
|
- **media_net**: Media services and *arr applications
|
|
- **db_net**: Database services (internal only)
|
|
- **web_net**: Web-accessible services
|
|
- **internal_net**: Internal service communication
|
|
|
|
### Service Categories
|
|
|
|
#### Management & Infrastructure
|
|
- **Glances**: System monitoring dashboard
|
|
- **Portainer**: Container management UI
|
|
- **Watchtower**: Automatic container updates
|
|
- **Homepage**: Service dashboard
|
|
- **Docker Proxy**: Secure Docker socket proxy
|
|
- **Nginx Proxy Manager (NPM)**: Reverse proxy with SSL
|
|
|
|
#### Media & Arr Stack
|
|
- **Prowlarr**: Indexer manager
|
|
- **Radarr**: Movie management
|
|
- **Sonarr**: TV show management
|
|
- **Lidarr**: Music management
|
|
- **Bazarr**: Subtitle management
|
|
- **Jellyseerr**: Media request management
|
|
- **Jellyfin**: Media server with hardware transcoding
|
|
- **qBittorrent**: Download client
|
|
- **Flaresolverr**: Cloudflare bypass
|
|
- **Slskd**: Soulseek client
|
|
- **Soulsync**: Music sync automation
|
|
|
|
#### Document & AI Suite
|
|
- **Paperless-ngx**: Document management system
|
|
- MariaDB database
|
|
- Redis broker
|
|
- Tika for text extraction
|
|
- Gotenberg for PDF rendering
|
|
- **Paperless-AI**: AI-powered document analysis
|
|
- **OnlyOffice**: Document collaboration server
|
|
- **Stirling-PDF**: PDF manipulation tools
|
|
- **Open WebUI**: LLM interface
|
|
- **LiteLLM**: LLM proxy/gateway with PostgreSQL
|
|
|
|
#### Photo & Data Management
|
|
- **Immich**: Photo management and backup
|
|
- PostgreSQL with vector extensions
|
|
- Redis cache
|
|
- Optimized storage: SSD for thumbnails/profiles, NAS for full resolution
|
|
- **Syncthing**: File synchronization (Obsidian vaults)
|
|
|
|
#### Development
|
|
- **Gitea**: Self-hosted Git service with MySQL
|
|
|
|
#### Genealogy
|
|
- **GrampsWeb**: Web interface for genealogy research
|
|
- Two instances: Jamie and Helen family trees
|
|
- Shared Redis broker
|
|
|
|
#### Utilities
|
|
- **Speedtest Tracker**: Network performance monitoring with MariaDB
|
|
- **RustDesk**: Remote desktop (hbbs + hbbr servers)
|
|
- **RedBot**: Discord bot
|
|
- **iperf3**: Network performance testing
|
|
- **Newt**: Custom application
|
|
|
|
#### Finance
|
|
- **WYGIWYH**: Financial tracking with PostgreSQL
|
|
|
|
#### Music & Scrobbling
|
|
- **Maloja**: Music scrobbling server
|
|
- **Multi-Scrobbler**: Cross-platform scrobbler
|
|
|
|
## Storage Structure
|
|
|
|
### Local Docker Configs (`/docker/`)
|
|
```
|
|
/docker/
|
|
├── Homepage/
|
|
├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
|
|
├── qBittorrent/
|
|
├── slskd/
|
|
├── soulsync/
|
|
├── paperless/
|
|
├── stirling/
|
|
├── immich/
|
|
├── obsidian/
|
|
├── syncthing/
|
|
├── gitea/
|
|
├── gramps-jamie/
|
|
├── gramps-helen/
|
|
├── speedtest-tracker/
|
|
├── rustdesk/
|
|
├── redbot/
|
|
├── maloja/
|
|
├── scrobble/
|
|
└── litellm/
|
|
```
|
|
|
|
### NAS Storage (`/mnt/Nas-Storage/data/`)
|
|
```
|
|
/mnt/Nas-Storage/data/
|
|
├── media/
|
|
│ ├── movies/
|
|
│ ├── tv/
|
|
│ ├── music/
|
|
│ └── images/
|
|
└── torrents/
|
|
└── soulsync/
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
### Required Environment Variables (.env file)
|
|
|
|
```bash
|
|
# System
|
|
PUID=1000
|
|
PGID=1000
|
|
TZ=Pacific/Auckland
|
|
|
|
# URLs
|
|
JELLYFIN_URL=https://your-jellyfin-domain.com
|
|
PAPERLESS_URL=https://your-paperless-domain.com
|
|
WYGIWYH_URL=https://your-wygiwyh-domain.com
|
|
HOMEPAGE_ALLOWED_HOSTS=your-homepage-domain.com
|
|
|
|
# Nginx Proxy Manager
|
|
NPM_PASSWORD=your_secure_password
|
|
|
|
# Paperless
|
|
PAPERLESS_DB_ROOT_PASSWORD=your_secure_password
|
|
PAPERLESS_DB_NAME=paperless
|
|
PAPERLESS_DB_USER=paperless
|
|
PAPERLESS_DB_PASSWORD=your_secure_password
|
|
PAPERLESS_SECRET_KEY=your_secret_key
|
|
|
|
# Immich
|
|
IMMICH_POSTGRES_PASSWORD=your_secure_password
|
|
|
|
# LiteLLM
|
|
GROQ_API_KEY=your_groq_api_key
|
|
LITELLM_MASTER_KEY=your_master_key
|
|
|
|
# Slskd
|
|
SLSKD_USERNAME=your_username
|
|
SLSKD_PASSWORD=your_secure_password
|
|
|
|
# Speedtest Tracker
|
|
SPEEDTEST_DB_NAME=speedtest
|
|
SPEEDTEST_DB_USER=speedtest
|
|
SPEEDTEST_DB_PASSWORD=your_secure_password
|
|
SPEEDTEST_APP_KEY=base64:your_generated_key
|
|
|
|
# WYGIWYH
|
|
WYGIWYH_DB_DATABASE=wygiwyh
|
|
WYGIWYH_DB_USER=wygiwyh
|
|
WYGIWYH_DB_PASSWORD=your_secure_password
|
|
WYGIWYH_SECRET_KEY=your_secret_key
|
|
WYGIWYH_ALLOWED_HOSTS=localhost,127.0.0.1,your-domain.com
|
|
|
|
# RedBot
|
|
REDBOT_TOKEN=your_discord_bot_token
|
|
```
|
|
|
|
### System Requirements
|
|
|
|
- Docker Engine 20.10+
|
|
- Docker Compose V2
|
|
- Sufficient storage for media and databases
|
|
- Intel GPU for Jellyfin hardware transcoding (optional)
|
|
|
|
## Installation
|
|
|
|
1. **Clone/Download this repository**
|
|
|
|
2. **Create required directories**
|
|
```bash
|
|
# See directory_structure.sh for automated setup
|
|
chmod +x directory_structure.sh
|
|
./directory_structure.sh
|
|
```
|
|
|
|
3. **Create .env file**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your values
|
|
nano .env
|
|
```
|
|
|
|
4. **Set correct permissions**
|
|
```bash
|
|
sudo chown -R $PUID:$PGID /docker
|
|
sudo chown -R $PUID:$PGID /mnt/Nas-Storage/data
|
|
```
|
|
|
|
5. **Start services**
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
## Service Access
|
|
|
|
Default ports (configure reverse proxy for SSL/domains):
|
|
|
|
- Homepage: 7575
|
|
- Portainer: 9443
|
|
- Jellyfin: 8096
|
|
- Radarr: 7878
|
|
- Sonarr: 8989
|
|
- Prowlarr: 9696
|
|
- Lidarr: 8686
|
|
- Bazarr: 6767
|
|
- Jellyseerr: 5055
|
|
- qBittorrent: 7070
|
|
- Paperless: 8100
|
|
- Stirling-PDF: 8090
|
|
- OnlyOffice: 8091
|
|
- Open WebUI: 3000
|
|
- Immich: 2283
|
|
- Syncthing: 8384
|
|
- Gitea: 8418
|
|
- GrampsWeb Jamie: 5511
|
|
- GrampsWeb Helen: 5512
|
|
- Speedtest Tracker: 8180
|
|
- NPM Admin: 81
|
|
- Maloja: 42010
|
|
- Multi-Scrobbler: 9078
|
|
- WYGIWYH: 9008
|
|
|
|
## Maintenance
|
|
|
|
### Backup Strategy
|
|
|
|
**Critical Data to Backup:**
|
|
- `/docker/` - All service configurations
|
|
- Database volumes (see docker-compose.yaml)
|
|
- `.env` file (store securely, contains secrets)
|
|
|
|
**Optional (can be regenerated):**
|
|
- Media files on NAS
|
|
- Immich thumbnails (can be regenerated)
|
|
|
|
### Updates
|
|
|
|
Watchtower automatically updates containers daily. To manually update:
|
|
|
|
```bash
|
|
docker compose pull
|
|
docker compose up -d
|
|
```
|
|
|
|
### Logs
|
|
|
|
View logs for any service:
|
|
```bash
|
|
docker compose logs -f [service_name]
|
|
```
|
|
|
|
### Resource Limits
|
|
|
|
- Portainer: 512MB RAM limit
|
|
- Log rotation: 10MB max, 3 files
|
|
|
|
## Hardware Acceleration
|
|
|
|
Jellyfin is configured for Intel GPU transcoding:
|
|
- Device: `/dev/dri/renderD128`
|
|
- Group: `104` (render group)
|
|
|
|
Verify GPU access:
|
|
```bash
|
|
ls -l /dev/dri/renderD128
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
1. **Secrets Management**: Store `.env` securely, never commit to version control
|
|
2. **Network Segmentation**: Database network is internal-only
|
|
3. **Container Security**: `no-new-privileges:true` on supported services
|
|
4. **Reverse Proxy**: Use NPM for SSL termination and authentication
|
|
5. **Updates**: Watchtower keeps containers updated
|
|
|
|
## Troubleshooting
|
|
|
|
### Service won't start
|
|
```bash
|
|
docker compose logs [service_name]
|
|
docker compose restart [service_name]
|
|
```
|
|
|
|
### Database connection issues
|
|
```bash
|
|
# Check database is healthy
|
|
docker compose ps
|
|
# Verify network connectivity
|
|
docker compose exec [service] ping [db_service]
|
|
```
|
|
|
|
### Permission errors
|
|
```bash
|
|
# Verify ownership
|
|
ls -la /docker/[service]/
|
|
# Fix if needed
|
|
sudo chown -R $PUID:$PGID /docker/[service]/
|
|
```
|
|
|
|
### Storage full
|
|
```bash
|
|
# Check Docker disk usage
|
|
docker system df
|
|
# Clean up unused resources
|
|
docker system prune -a
|
|
```
|
|
|
|
## Migration/Restoration
|
|
|
|
1. Install Docker and Docker Compose on new system
|
|
2. Restore `/docker/` directory structure
|
|
3. Restore `.env` file
|
|
4. Restore database volumes (if applicable)
|
|
5. Mount NAS storage at `/mnt/Nas-Storage/data/`
|
|
6. Run `docker compose up -d`
|
|
|
|
## Contributing Services
|
|
|
|
To add new services:
|
|
1. Add service definition to appropriate section
|
|
2. Assign to correct network(s)
|
|
3. Add volume mounts under `/docker/[service]/`
|
|
4. Update this README with service description
|
|
5. Add required environment variables to `.env.example`
|
|
|
|
## License
|
|
|
|
This is a personal infrastructure setup. Adapt as needed for your use case.
|
|
|
|
## Support
|
|
|
|
For issues with specific services, consult their official documentation:
|
|
- Jellyfin: https://jellyfin.org/docs/
|
|
- Paperless-ngx: https://docs.paperless-ngx.com/
|
|
- Immich: https://immich.app/docs/
|
|
- And so on...
|
|
|
|
---
|
|
|
|
**Last Updated**: January 2026
|