Homelab/AGENTS.md

AGENTS - Docker Infrastructure Documentation

Overview

This document provides a comprehensive overview of the Docker infrastructure managed in /docker/. This self-hosted ecosystem contains 50+ services organized into 5 compose stacks, all integrated through a reverse proxy and secured with proper network segmentation.

Compose Stack Organization

The infrastructure is split into 5 independent compose files sharing a common .env:

Stack	File	Services
Infrastructure	docker-compose.infra.yml	Core: portainer, npm, dockerproxy, homepage, wud, ntopng, newt
Media	docker-compose.media.yml	Media: arr stack, jellyfin, qbittorrent, slskd, metube, maloja, scrobbler
Documents	docker-compose.documents.yml	Paperless-ngx + AI, onlyoffice, stirling-pdf, open-webui
Photo & ROM Library	docker-compose.photo-roms.yml	Immich, syncthing, retrom
Utilities	docker-compose.utils.yml	Gitea, tandoor, speedtest-tracker, rustdesk, redbot, linkwarden, neolink, iperf3, ntfy, mailrise

Core Infrastructure Agents (infra.yml)

Management & Monitoring

• Portainer - Container management UI (port 9443)
• Homepage - Service dashboard (port 7575)
• WUD (WhatsUpDocker) - Docker image management utility
• ntopng - Network traffic monitoring (port 3939)

Reverse Proxy & Security

• Nginx Proxy Manager - SSL termination and reverse proxy (ports 80, 443, 81)
• Docker Socket Proxy - Secure Docker API access (port 2375)
• Newt - Pangolin tunnel client

Media & Entertainment Stack (media.yml)

Media Server

• Jellyfin - Media server with hardware transcoding (port 8096)
  • Intel GPU acceleration via render device
  • NAS storage integration

Download & Index Management

• qBittorrent - Torrent client (ports 56881, 7070)
• Prowlarr - Indexer manager (port 9696)
• FlareSolverr - Cloudflare bypass service (port 8191)
• Metube - YouTube video downloader (port 8081)

Content Management

• Radarr - Movie management (port 7878)
• Sonarr - TV show management (port 8989)
• Lidarr - Music management (port 8686)
• Bazarr - Subtitle management (port 6767)
• Jellyseerr - Media request system (port 5055)

File Sharing

• Slskd - Soulseek file sharing client (ports 5030, 5031, 50300)

Music & Scrobbling

• Maloja - Music scrobbling service (port 42010)
• Multi-Scrobbler - Cross-platform scrobbling (port 9078)
  • Integrates with Jellyfin and Last.fm

Document & AI Suite (documents.yml)

Document Processing

• Paperless-ngx - Document management (port 8100)
  • MariaDB database, Redis broker, Tika text extraction, Gotenberg PDF processing
• Paperless-AI - AI-powered document processing for Paperless-ngx
• Stirling-PDF - PDF manipulation tools (port 8090)
• OnlyOffice - Document collaboration (port 8091)

AI Interface

• Open WebUI - LLM interface (port 3000)

Photo Management & Library (photo-roms.yml)

Photo Management

• Immich - AI-powered photo management (port 2283)
  • PostgreSQL with vector search, Valkey/Redis cache
  • Local SSD storage for config and thumbs

File Synchronization

• Syncthing - File synchronization (ports 21027, 22000, 8384)
  • Obsidian vault synchronization
  • Multiple shared folders

ROM Management

• Retrom - ROM library management service
  • retrom: Main ROM service container (port 5111)
  • retrom-db: PostgreSQL database for Retrom
  • retrom-adminer: Adminer interface for database management (port 8080)
  • retrom-jaeger: Distributed tracing for performance monitoring

Utilities Stack (utils.yml)

Development Tools

• Gitea - Git service (ports 222, 8418)
  • MySQL database

Personal Tools

• Tandoor Recipes - Self-hosted recipe management (port 8450)
  • PostgreSQL database

Finance & Tracking

• Speedtest Tracker - Network monitoring (port 8180)
  • MariaDB database, automated testing

Bookmark Management

• Linkwarden - Collaborative bookmark manager (port 3400)
  • PostgreSQL database
  • Meilisearch for full-text search

Remote Access

• RustDesk - Remote desktop (host mode)

Home Automation

• Neolink - Reolink camera bridge for Frigate/Home Assistant integration

Discord Bots

• RedBot - Discord bot with custom cogs (internal_net)

Notifications

• Ntfy - Push notification service (port 8086)
  • Token-based authentication
  • Subscribable topics: alerts, backups, paperless
• Mailrise - SMTP-to-ntfy bridge (port 8025)
  • Converts email alerts into push notifications
  • Works with any service that supports SMTP notifications

Network Tools

• iperf3-server - Network performance testing (port 5201)

Independent Stacks (separate compose files)

These services are deployed independently and are not part of the main 5-stack split:

Service	Directory	Role
Discord Agent Bot	discord-agent/	AI-powered Discord bot using Ollama LLM
Foundry Watcher	foundry-watcher/	Foundry VTT player monitoring
GrampsWeb Jamie	gramps-jamie/	Genealogy UI (port 5511)
GrampsWeb Helen	gramps-helen/	Genealogy UI (port 5512)
Kasm	kasm/	Browser isolation workspace
LiteLLM	litellm/	AI gateway (port 4000)
MBI Poller	mbi-poller/	MBI data polling service

Network Architecture

Docker Networks

• media_net - Media services and *arr stack
• db_net - Database services (internal only)
• web_net - Web-accessible services
• internal_net - Internal service communication

Networks are defined in docker-compose.infra.yml and referenced as external: true by all other stacks.

Storage Structure

/docker/
├── Arrs/ (Media stack: Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
├── immich/ (Photo management)
├── paperless/ (Document management)
├── discord-agent/ (Independent — Discord bot with Ollama)
├── gitea/ (Git service)
├── gramps-jamie/ (Independent — Genealogy)
├── gramps-helen/ (Independent — Genealogy)
├── npm/ (Nginx Proxy Manager)
├── qBittorrent/ (Download client)
├── slskd/ (Soulseek client)
├── speedtest-tracker/ (Network monitoring)
├── maloja/ (Music scrobbling)
├── scrobble/ (Multi-scrobbler)
├── syncthing/ (File sync)
├── Homepage/ (Dashboard)
├── retrom/ (ROM library management)
├── wud/ (Docker image management)
├── ntopng/ (Network traffic monitoring)
├── kasm/ (Independent — Browser workspace)
├── litellm/ (Independent — AI gateway)
├── neolink/ (Reolink camera bridge)
├── linkwarden/ (Bookmark manager)
├── surmai/ (Flight tracking)
├── tandoor/ (Recipe management)
├── foundry-watcher/ (Independent — Foundry VTT)
├── rustdesk/ (Remote desktop)
├── redbot/ (Discord bot)
├── stirling/ (PDF tools)
├── ntfy/ (Notification configs)
└── Various other service configs

Deployment

Deploy all main stacks

for f in docker-compose.infra.yml docker-compose.media.yml docker-compose.documents.yml docker-compose.photo-roms.yml docker-compose.utils.yml; do
  docker compose -f "$f" up -d
done

Deploy a specific stack

docker compose -f docker-compose.media.yml up -d

Stop a specific stack

docker compose -f docker-compose.media.yml down

Key Features & Configuration

Security

• Container security with no-new-privileges:true
• Network segmentation (databases internal-only via db_net)
• Reverse proxy with SSL termination

Performance

• Intel GPU hardware acceleration for Jellyfin
• Log rotation (10MB max, 3 files)
• Resource limits (Portainer: 512MB RAM)

Data Management

• Comprehensive backup script included (backup.sh)
• Separate volumes for critical data
• NAS storage integration for media files

Environment

• System configured for Pacific/Auckland timezone
• PUID/PGID for proper file permissions
• Extensive environment variable configuration
• Shared .env file across all stacks

Service Dependencies

Database Services

• MariaDB: npm, paperless, speedtest-tracker
• PostgreSQL: immich, retrom, tandoor, linkwarden
• MySQL: gitea
• Redis/Valkey: paperless-broker, immich-redis
• Meilisearch: linkwarden

Network Dependencies

• infra.yml must be deployed first (creates shared networks)
• All web services route through Nginx Proxy Manager
• Internal services communicate via internal_net
• Media services isolated on media_net
• Database services on internal-only db_net

Maintenance

Automated Tasks

• Backup Script (backup.sh) - Regular data backups
• Speedtest Tracker - Automated network testing
• WUD - Automatic Docker image update monitoring

Manual Tasks

• Service monitoring via Portainer
• Container management via Portainer
• Service configuration via Nginx Proxy Manager

Development & Debugging

Service Access

• Web Services: Access through reverse proxy (80/443)
• Admin Interfaces: Direct ports for management (e.g., 9443, 7575)
• Internal Services: Host-only access for security

Troubleshooting

• Container logs accessible via Portainer
• Network diagnostics via iperf3-server
• Per-stack logs: docker compose -f docker-compose.<stack>.yml logs -f <service>

This Docker infrastructure represents a comprehensive self-hosted ecosystem covering media management, document processing, AI services, development tools, and personal productivity applications, all integrated through a reverse proxy and organized into independent compose stacks with proper network segmentation.