Jamie/Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b2f4d37f194fb7fe86d9ccf0b36f8b8a94f6d75d
Homelab/README.md
KansaiGaijin b2f4d37f19 Split monolithic compose into 5 independent stacks 
- docker-compose.infra.yml: core infrastructure (portainer, npm, homepage, wud, etc.)
- docker-compose.media.yml: media stack (arrs, jellyfin, qbittorrent, scrobbling)
- docker-compose.documents.yml: paperless-ngx, onlyoffice, stirling, open-webui
- docker-compose.photo-roms.yml: immich, syncthing, retrom
- docker-compose.utils.yml: gitea, tandoor, speedtest, linkwarden, rustdesk, etc.

Each stack has its own project name (docker-infra, docker-media, etc.) to prevent
orphan warnings. Networks defined in infra.yml, referenced as external by others.
Original preserved as docker-compose.full.yaml.bak.
Updated .gitignore, README, AGENTS.md, and RESTORE.md to reflect new structure.
2026-05-14 22:58:11 +12:00

345 lines
9.9 KiB
Markdown
Raw Blame History

# Docker Compose Infrastructure Documentation
## Overview
This infrastructure manages a comprehensive self-hosted environment including media management (Jellyfin, *arr stack), document management (Paperless-ngx), AI services (Open WebUI), photo management (Immich), and various utilities.
The deployment is split into **5 independent compose stacks** sharing a common `.env` file and Docker networks.
## Compose Stacks
| Stack | File | Services |
|-------|------|----------|
| **Infrastructure** | `docker-compose.infra.yml` | portainer, ntopng, wud, homepage, dockerproxy, newt, npm, npm-db |
| **Media** | `docker-compose.media.yml` | prowlarr, flaresolverr, qbittorrent, radarr, sonarr, lidarr, bazarr, seerr, jellyfin, slskd, metube, maloja, multi-scrobbler |
| **Documents** | `docker-compose.documents.yml` | onlyoffice, paperless-ngx (db, broker, tika, gotenberg, webserver, ai), stirling-pdf, open-webui |
| **Photo & ROM Library** | `docker-compose.photo-roms.yml` | immich (server, postgres, ml, redis), syncthing, retrom (db, adminer, service, jaeger) |
| **Utilities** | `docker-compose.utils.yml` | gitea (server + db), tandoor (db + web), speedtest-tracker (app + db), rustdesk (hbbs + hbbr), redbot, iperf3-server, neolink, linkwarden (app + db + meilisearch) |
## Architecture
### Internal Docker Services
| Stack | Service | Role | Port |
|-------|---------|------|------|
| Infrastructure | Nginx Proxy Manager | SSL termination & reverse proxy | 80, 443, 81 |
| Infrastructure | Portainer | Container management UI | 9443 |
| Infrastructure | Homepage | Service dashboard | 7575 |
| Infrastructure | WUD | Docker image update monitoring | 3666 |
| Infrastructure | ntopng | Network traffic monitoring | 3939 |
| Infrastructure | Docker Socket Proxy | Secure Docker API access | 2375 |
| Infrastructure | Newt | Pangolin tunnel client | - |
| Media | Jellyfin | Media server (HW transcoding) | 8096 |
| Media | Jellyseerr (seerr) | Media request management | 5055 |
| Media | Radarr | Movie management | 7878 |
| Media | Sonarr | TV show management | 8989 |
| Media | Lidarr | Music management | 8686 |
| Media | Bazarr | Subtitle management | 6767 |
| Media | Prowlarr | Indexer manager | 9696 |
| Media | FlareSolverr | Cloudflare bypass | 8191 |
| Media | qBittorrent | Download client | 7070 |
| Media | Slskd | Soulseek file sharing | 5030, 5031 |
| Media | Metube | YouTube downloader | 8081 |
| Media | Maloja | Music scrobbling | 42010 |
| Media | Multi-Scrobbler | Cross-platform scrobbler | 9078 |
| Documents | Paperless-ngx | Document management | 8100 |
| Documents | Paperless-AI | AI document processing | 3040 |
| Documents | Stirling-PDF | PDF manipulation | 8090 |
| Documents | OnlyOffice | Document collaboration | 8091 |
| Documents | Open WebUI | LLM interface | 3000 |
| Photo & ROM | Immich | Photo management | 2283 |
| Photo & ROM | Syncthing | File synchronization | 8384 |
| Photo & ROM | Retrom | ROM library management | 5111 |
| Utilities | Gitea | Git service | 8418 |
| Utilities | Tandoor Recipes | Recipe management | 8450 |
| Utilities | Speedtest Tracker | Network monitoring | 8180 |
| Utilities | RustDesk | Remote desktop (host mode) | - |
| Utilities | RedBot | Discord bot | - |
| Utilities | iperf3-server | Network performance test | 5201 |
| Utilities | Neolink | Reolink camera bridge | 8554 |
| Utilities | Linkwarden | Bookmark manager | 3400 |
### Network Segmentation
- **media_net**: Media services and *arr applications
- **db_net**: Database services (internal only)
- **web_net**: Web-accessible services
- **internal_net**: Internal service communication
Networks are created by `infra.yml` and referenced as external by all other stacks.
## Storage Structure
### Local Docker Configs (`/docker/`)
```
/docker/
├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
├── Homepage/
├── immich/
├── paperless/
├── discord-agent/ (Separate stack)
├── gitea/
├── gramps-jamie/ (Separate stack)
├── gramps-helen/ (Separate stack)
├── kasm/ (Separate stack)
├── foundry-watcher/ (Separate stack)
├── litellm/ (Separate stack)
├── linkwarden/
├── maloja/
├── neolink/
├── npm/
├── ntopng/
├── qBittorrent/
├── redbot/
├── retrom/
├── rustdesk/
├── scrobble/ (Multi-scrobbler)
├── slskd/
├── speedtest-tracker/
├── stirling/
├── surmai/
├── syncthing/
├── tandoor/
├── wud/
└── .env (shared environment variables)
```
### NAS Storage (`/mnt/nas-storage/data/`)
```
/mnt/nas-storage/data/
├── media/
│ ├── movies/
│ ├── tv/
│ ├── music/
│ └── romms/
└── torrents/
├── metube/
└── soulsync/
```
## Deployment
### First-time deploy (or after network cleanup)
```bash
# Deploy infrastructure first (creates shared networks)
docker compose -f docker-compose.infra.yml up -d
# Then deploy remaining stacks in any order
docker compose -f docker-compose.media.yml up -d
docker compose -f docker-compose.documents.yml up -d
docker compose -f docker-compose.photo-roms.yml up -d
docker compose -f docker-compose.utils.yml up -d
```
### Deploy all stacks
```bash
for f in docker-compose.*.yml; do
if [ "$f" != "docker-compose.full.yaml.bak" ]; then
docker compose -f "$f" up -d
fi
done
```
### Stop/start individual stacks
```bash
docker compose -f docker-compose.media.yml down
docker compose -f docker-compose.media.yml up -d
```
### View logs
```bash
docker compose -f docker-compose.media.yml logs -f jellyfin
```
## Prerequisites
### Required Environment Variables (.env file)
```
# System
PUID=1000
PGID=1000
TZ=Pacific/Auckland
# URLs
JELLYFIN_URL=https://your-jellyfin-domain.com
PAPERLESS_URL=https://your-paperless-domain.com
HOMEPAGE_ALLOWED_HOSTS=your-homepage-domain.com
# Nginx Proxy Manager
NPM_PASSWORD=your_secure_password
# Paperless
PAPERLESS_DB_ROOT_PASSWORD=your_secure_password
PAPERLESS_DB_NAME=paperless
PAPERLESS_DB_USER=paperless
PAPERLESS_DB_PASSWORD=your_secure_password
PAPERLESS_SECRET_KEY=your_secret_key
# Immich
IMMICH_POSTGRES_PASSWORD=your_secure_password
# Slskd
SLSKD_USERNAME=your_username
SLSKD_PASSWORD=your_secure_password
# Speedtest Tracker
SPEEDTEST_DB_NAME=speedtest
SPEEDTEST_DB_USER=speedtest
SPEEDTEST_DB_PASSWORD=your_secure_password
SPEEDTEST_APP_KEY=base64:your_generated_key
# RedBot
REDBOT_TOKEN=your_discord_bot_token
# Tandoor
TANDOOR_SECRET_KEY=your_secret_key
TANDOOR_POSTGRES_DB=tandoor
TANDOOR_POSTGRES_USER=tandoor
TANDOOR_POSTGRES_PASSWORD=your_secure_password
# Linkwarden
LINKWARDEN_DB_PASSWORD=your_secure_password
NEXTAUTH_SECRET=your_secret
NEXTAUTH_URL=https://your-linkwarden-domain.com
# WUD
WUD_REGISTRY_HUB_0_LOGIN=your_dockerhub_username
WUD_REGISTRY_HUB_0_TOKEN=your_dockerhub_token
# OnlyOffice
OO_JWT_SECRET=your_secret
# Maloja
MALOJA_FORCE_PASSWORD=your_password
MALOJA_URL=http://maloja:42010
MALOJA_API_KEY=your_key
# Jellyfin
JELLYFIN_APIKEY_MS=your_api_key_for_multiscrobbler
JELLYFIN_USER=your_username
```
### System Requirements
- Docker Engine 20.10+
- Docker Compose V2
- Sufficient storage for media and databases
- Intel GPU for Jellyfin hardware transcoding (optional)
## Installation
1. Clone this repository
2. Create `.env` from `.env.example` and fill in values
3. Run infrastructure:
```bash
docker compose -f docker-compose.infra.yml up -d
```
4. Deploy desired stacks:
```bash
docker compose -f docker-compose.media.yml up -d
```
## Maintenance
### Backup Strategy
**Critical Data to Backup:**
- `/docker/` — all service configurations
- Named volumes: `portainer_data`, `docker_dbdata`, `open-webui`
- `.env` file (contains secrets)
**Optional (can be regenerated):**
- Media files on NAS
### Updates
```bash
for f in docker-compose.infra.yml docker-compose.media.yml docker-compose.documents.yml docker-compose.photo-roms.yml docker-compose.utils.yml; do
docker compose -f "$f" pull
docker compose -f "$f" up -d
done
```
### Resource Limits
- Portainer: 512MB RAM limit
- Log rotation: 10MB max, 3 files (on services with logging config)
## Hardware Acceleration
Jellyfin is configured for Intel GPU transcoding:
- Device: `/dev/dri/renderD128`
- Group: `104` (render group)
Verify GPU access:
```bash
ls -l /dev/dri/renderD128
```
## Security Considerations
1. Secrets Management: Store `.env` securely, never commit to version control
2. Network Segmentation: Database network is internal-only (`db_net`)
3. Container Security: `no-new-privileges:true` on supported services
4. Reverse Proxy: Use NPM for SSL termination and authentication
5. Updates: Regularly update containers as shown above
## Troubleshooting
### Service won't start
```bash
docker compose -f docker-compose.media.yml logs service_name
docker compose -f docker-compose.media.yml restart service_name
```
### Database connection issues
```bash
docker compose -f docker-compose.documents.yml ps
docker compose -f docker-compose.documents.yml exec paperless-webserver ping paperless-db
```
### Permission errors
```bash
ls -la /docker/service_name/
sudo chown -R $PUID:$PGID /docker/service_name/
```
### Storage full
```bash
docker system df
docker system prune -a
```
## Contributing Services
To add new services:
1. Add service definition to the appropriate compose file
2. Assign to correct network(s)
3. Add volume mounts under `/docker/[service]/`
4. Update this README with service description
5. Add required environment variables to `.env.example`
## Independent Stacks
The following services have their own compose files (not part of the main split):
- **discord-agent/** — Discord Agent Bot
- **foundry-watcher/** — Foundry VTT monitoring
- **gramps-jamie/** — Genealogy (Jamie)
- **gramps-helen/** — Genealogy (Helen)
- **kasm/** — Browser workspace
- **litellm/** — AI gateway
- **mbi-poller/** — MBI poller service
## License
This is a personal infrastructure setup. Adapt as needed for your use case.
---
**Last Updated**: May 2026
Reference in New Issue View Git Blame Copy Permalink