# Docker Compose Infrastructure Documentation

## Overview

This infrastructure manages a comprehensive self-hosted environment including media management (Jellyfin, *arr stack), document management (Paperless-ngx), AI services (Open WebUI), photo management (Immich), and various utilities.

The deployment is split into **5 independent compose stacks** sharing a common `.env` file and Docker networks.

## Compose Stacks

| Stack | File | Services |
|-------|------|----------|
| **Infrastructure** | `docker-compose.infra.yml` | portainer, ntopng, wud, homepage, dockerproxy, newt, npm, npm-db |
| **Media** | `docker-compose.media.yml` | prowlarr, flaresolverr, qbittorrent, radarr, sonarr, lidarr, bazarr, seerr, jellyfin, slskd, metube, maloja, multi-scrobbler |
| **Documents** | `docker-compose.documents.yml` | onlyoffice, paperless-ngx (db, broker, tika, gotenberg, webserver, ai), stirling-pdf, open-webui |
| **Photo & ROM Library** | `docker-compose.photo-roms.yml` | immich (server, postgres, ml, redis), syncthing, retrom (db, adminer, service, jaeger) |
| **Utilities** | `docker-compose.utils.yml` | gitea (server + db), tandoor (db + web), speedtest-tracker (app + db), rustdesk (hbbs + hbbr), redbot, iperf3-server, neolink, linkwarden (app + db + meilisearch) |

## Architecture

### Internal Docker Services

| Stack | Service | Role | Port |
|-------|---------|------|------|
| Infrastructure | Nginx Proxy Manager | SSL termination & reverse proxy | 80, 443, 81 |
| Infrastructure | Portainer | Container management UI | 9443 |
| Infrastructure | Homepage | Service dashboard | 7575 |
| Infrastructure | WUD | Docker image update monitoring | 3666 |
| Infrastructure | ntopng | Network traffic monitoring | 3939 |
| Infrastructure | Docker Socket Proxy | Secure Docker API access | 2375 |
| Infrastructure | Newt | Pangolin tunnel client | - |
| Media | Jellyfin | Media server (HW transcoding) | 8096 |
| Media | Jellyseerr (seerr) | Media request management | 5055 |
| Media | Radarr | Movie management | 7878 |
| Media | Sonarr | TV show management | 8989 |
| Media | Lidarr | Music management | 8686 |
| Media | Bazarr | Subtitle management | 6767 |
| Media | Prowlarr | Indexer manager | 9696 |
| Media | FlareSolverr | Cloudflare bypass | 8191 |
| Media | qBittorrent | Download client | 7070 |
| Media | Slskd | Soulseek file sharing | 5030, 5031 |
| Media | Metube | YouTube downloader | 8081 |
| Media | Maloja | Music scrobbling | 42010 |
| Media | Multi-Scrobbler | Cross-platform scrobbler | 9078 |
| Documents | Paperless-ngx | Document management | 8100 |
| Documents | Paperless-AI | AI document processing | 3040 |
| Documents | Stirling-PDF | PDF manipulation | 8090 |
| Documents | OnlyOffice | Document collaboration | 8091 |
| Documents | Open WebUI | LLM interface | 3000 |
| Photo & ROM | Immich | Photo management | 2283 |
| Photo & ROM | Syncthing | File synchronization | 8384 |
| Photo & ROM | Retrom | ROM library management | 5111 |
| Utilities | Gitea | Git service | 8418 |
| Utilities | Tandoor Recipes | Recipe management | 8450 |
| Utilities | Speedtest Tracker | Network monitoring | 8180 |
| Utilities | RustDesk | Remote desktop (host mode) | - |
| Utilities | RedBot | Discord bot | - |
| Utilities | iperf3-server | Network performance test | 5201 |
| Utilities | Neolink | Reolink camera bridge | 8554 |
| Utilities | Linkwarden | Bookmark manager | 3400 |

### Network Segmentation

- **media_net**: Media services and *arr applications
- **db_net**: Database services (internal only)
- **web_net**: Web-accessible services
- **internal_net**: Internal service communication

Networks are created by `infra.yml` and referenced as external by all other stacks.

## Storage Structure

### Local Docker Configs (`/docker/`)
```
/docker/
├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
├── Homepage/
├── immich/
├── paperless/
├── discord-agent/ (Separate stack)
├── gitea/
├── gramps-jamie/ (Separate stack)
├── gramps-helen/ (Separate stack)
├── kasm/ (Separate stack)
├── foundry-watcher/ (Separate stack)
├── litellm/ (Separate stack)
├── linkwarden/
├── maloja/
├── neolink/
├── npm/
├── ntopng/
├── qBittorrent/
├── redbot/
├── retrom/
├── rustdesk/
├── scrobble/ (Multi-scrobbler)
├── slskd/
├── speedtest-tracker/
├── stirling/
├── surmai/
├── syncthing/
├── tandoor/
├── wud/
└── .env (shared environment variables)
```

### NAS Storage (`/mnt/nas-storage/data/`)
```
/mnt/nas-storage/data/
├── media/
│   ├── movies/
│   ├── tv/
│   ├── music/
│   └── romms/
└── torrents/
    ├── metube/
    └── soulsync/
```

## Deployment

### First-time deploy (or after network cleanup)

```bash
# Deploy infrastructure first (creates shared networks)
docker compose -f docker-compose.infra.yml up -d

# Then deploy remaining stacks in any order
docker compose -f docker-compose.media.yml up -d
docker compose -f docker-compose.documents.yml up -d
docker compose -f docker-compose.photo-roms.yml up -d
docker compose -f docker-compose.utils.yml up -d
```

### Deploy all stacks

```bash
for f in docker-compose.*.yml; do
  if [ "$f" != "docker-compose.full.yaml.bak" ]; then
    docker compose -f "$f" up -d
  fi
done
```

### Stop/start individual stacks

```bash
docker compose -f docker-compose.media.yml down
docker compose -f docker-compose.media.yml up -d
```

### View logs

```bash
docker compose -f docker-compose.media.yml logs -f jellyfin
```

## Prerequisites

### Required Environment Variables (.env file)

```
# System
PUID=1000
PGID=1000
TZ=Pacific/Auckland

# URLs
JELLYFIN_URL=https://your-jellyfin-domain.com
PAPERLESS_URL=https://your-paperless-domain.com
HOMEPAGE_ALLOWED_HOSTS=your-homepage-domain.com

# Nginx Proxy Manager
NPM_PASSWORD=your_secure_password

# Paperless
PAPERLESS_DB_ROOT_PASSWORD=your_secure_password
PAPERLESS_DB_NAME=paperless
PAPERLESS_DB_USER=paperless
PAPERLESS_DB_PASSWORD=your_secure_password
PAPERLESS_SECRET_KEY=your_secret_key

# Immich
IMMICH_POSTGRES_PASSWORD=your_secure_password

# Slskd
SLSKD_USERNAME=your_username
SLSKD_PASSWORD=your_secure_password

# Speedtest Tracker
SPEEDTEST_DB_NAME=speedtest
SPEEDTEST_DB_USER=speedtest
SPEEDTEST_DB_PASSWORD=your_secure_password
SPEEDTEST_APP_KEY=base64:your_generated_key

# RedBot
REDBOT_TOKEN=your_discord_bot_token

# Tandoor
TANDOOR_SECRET_KEY=your_secret_key
TANDOOR_POSTGRES_DB=tandoor
TANDOOR_POSTGRES_USER=tandoor
TANDOOR_POSTGRES_PASSWORD=your_secure_password

# Linkwarden
LINKWARDEN_DB_PASSWORD=your_secure_password
NEXTAUTH_SECRET=your_secret
NEXTAUTH_URL=https://your-linkwarden-domain.com

# WUD
WUD_REGISTRY_HUB_0_LOGIN=your_dockerhub_username
WUD_REGISTRY_HUB_0_TOKEN=your_dockerhub_token

# OnlyOffice
OO_JWT_SECRET=your_secret

# Maloja
MALOJA_FORCE_PASSWORD=your_password
MALOJA_URL=http://maloja:42010
MALOJA_API_KEY=your_key

# Jellyfin
JELLYFIN_APIKEY_MS=your_api_key_for_multiscrobbler
JELLYFIN_USER=your_username
```

### System Requirements

- Docker Engine 20.10+
- Docker Compose V2
- Sufficient storage for media and databases
- Intel GPU for Jellyfin hardware transcoding (optional)

## Installation

1. Clone this repository
2. Create `.env` from `.env.example` and fill in values
3. Run infrastructure:
   ```bash
   docker compose -f docker-compose.infra.yml up -d
   ```
4. Deploy desired stacks:
   ```bash
   docker compose -f docker-compose.media.yml up -d
   ```

## Maintenance

### Backup Strategy

**Critical Data to Backup:**
- `/docker/` — all service configurations
- Named volumes: `portainer_data`, `docker_dbdata`, `open-webui`
- `.env` file (contains secrets)

**Optional (can be regenerated):**
- Media files on NAS

### Updates

```bash
for f in docker-compose.infra.yml docker-compose.media.yml docker-compose.documents.yml docker-compose.photo-roms.yml docker-compose.utils.yml; do
  docker compose -f "$f" pull
  docker compose -f "$f" up -d
done
```

### Resource Limits

- Portainer: 512MB RAM limit
- Log rotation: 10MB max, 3 files (on services with logging config)

## Hardware Acceleration

Jellyfin is configured for Intel GPU transcoding:
- Device: `/dev/dri/renderD128`
- Group: `104` (render group)

Verify GPU access:
```bash
ls -l /dev/dri/renderD128
```

## Security Considerations

1. Secrets Management: Store `.env` securely, never commit to version control
2. Network Segmentation: Database network is internal-only (`db_net`)
3. Container Security: `no-new-privileges:true` on supported services
4. Reverse Proxy: Use NPM for SSL termination and authentication
5. Updates: Regularly update containers as shown above

## Troubleshooting

### Service won't start
```bash
docker compose -f docker-compose.media.yml logs service_name
docker compose -f docker-compose.media.yml restart service_name
```

### Database connection issues
```bash
docker compose -f docker-compose.documents.yml ps
docker compose -f docker-compose.documents.yml exec paperless-webserver ping paperless-db
```

### Permission errors
```bash
ls -la /docker/service_name/
sudo chown -R $PUID:$PGID /docker/service_name/
```

### Storage full
```bash
docker system df
docker system prune -a
```

## Contributing Services

To add new services:
1. Add service definition to the appropriate compose file
2. Assign to correct network(s)
3. Add volume mounts under `/docker/[service]/`
4. Update this README with service description
5. Add required environment variables to `.env.example`

## Independent Stacks

The following services have their own compose files (not part of the main split):
- **discord-agent/** — Discord Agent Bot
- **foundry-watcher/** — Foundry VTT monitoring
- **gramps-jamie/** — Genealogy (Jamie)
- **gramps-helen/** — Genealogy (Helen)
- **kasm/** — Browser workspace
- **litellm/** — AI gateway
- **mbi-poller/** — MBI poller service

## License

This is a personal infrastructure setup. Adapt as needed for your use case.

---

**Last Updated**: May 2026