Docs: update Discord agent configuration and fix syntax issues

- Update AGENTS.md with current Discord agent configuration - Add discord-agent service details - Update storage structure to include discord-agent directory - Update README.md Discord agent integration section - Correct LLM configuration from OpenCode to Ollama - Update environment variables and configuration details - Fix storage structure documentation - Update RESTORE.md Discord agent restoration steps - Correct service configuration for Ollama integration - Add proper troubleshooting steps - Update environment variable documentation - Fix docker-compose.yaml Discord agent environment variables - Add DISCORD_BOT_TOKEN, OLLAMA_ENDPOINT, OLLAMA_MODEL - Correct database and network configuration - Add AGENTS.md to .gitignore tracked files - Remove old discord agent versions and test files - Create working discord_agent.py and health_check.py
2026-02-06 04:34:43 +00:00
parent 5029b13f02
commit a3f6855f32
5 changed files with 398 additions and 58 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -7,6 +7,7 @@
 !.gitignore
 !README.md
 !RESTORE.md
+!AGENTS.md

 # Never track actual secrets
 .env
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -0,0 +1,211 @@
+# AGENTS - Docker Infrastructure Documentation
+
+## Overview
+This document provides a comprehensive overview of the Docker infrastructure managed in `/docker/`. This self-hosted ecosystem contains 30+ services organized into functional categories, all integrated through a reverse proxy and secured with proper network segmentation.
+
+## Core Infrastructure Agents
+
+### Management & Monitoring
+- **Glances** - System monitoring dashboard with hardware access
+- **Portainer** - Container management UI (port 9443)
+- **Watchtower** - Automated container updates (daily polling)
+- **Homepage** - Service dashboard (port 7575)
+
+### Reverse Proxy & Security
+- **Nginx Proxy Manager** - SSL termination and reverse proxy (ports 80, 443, 81)
+- **Docker Socket Proxy** - Secure Docker API access (port 2375)
+
+## Media & Entertainment Stack
+
+### Media Server
+- **Jellyfin** - Media server with hardware transcoding (port 8096)
+  - Intel GPU acceleration via render device
+  - NAS storage integration
+
+### Download & Index Management
+- **qBittorrent** - Torrent client (ports 56881, 7070)
+- **Prowlarr** - Indexer manager (port 9696)
+- **FlareSolverr** - Cloudflare bypass service (port 8191)
+
+### Content Management
+- **Radarr** - Movie management (port 7878)
+- **Sonarr** - TV show management (port 8989)
+- **Lidarr** - Music management (port 8686)
+- **Bazarr** - Subtitle management (port 6767)
+- **Jellyseerr** - Media request system (port 5055)
+
+### File Sharing
+- **Slskd** - Soulseek file sharing client (ports 5030, 5031, 50300)
+
+## Document & Data Management
+
+### Document Processing
+- **Paperless-ngx** - Document management (port 8100)
+  - MariaDB database, Redis broker, Tika text extraction, Gotenberg PDF processing
+- **Stirling-PDF** - PDF manipulation tools (port 8090)
+- **OnlyOffice** - Document collaboration (port 8091)
+
+### Photo Management
+- **Immich** - AI-powered photo management (port 2283)
+  - PostgreSQL with vector search, Redis cache
+  - Local SSD storage for config and thumbs
+
+### File Synchronization
+- **Syncthing** - File synchronization (ports 21027, 22000, 8384)
+  - Obsidian vault synchronization
+  - Multiple shared folders
+
+## AI & Development Services
+
+### AI Infrastructure
+- **Open WebUI** - LLM interface (port 3000)
+- **LiteLLM** - AI gateway (port 4000)
+  - PostgreSQL database for configuration
+  - Supports multiple AI providers
+
+### Development Tools
+- **Gitea** - Git service (ports 222, 8418)
+  - MySQL database
+- **Newt** - AI service integration
+
+## Genealogy & Personal Tools
+
+### Family History
+- **GrampsWeb Jamie** - Genealogy UI (port 5511)
+- **GrampsWeb Helen** - Genealogy UI (port 5512)
+  - Shared Redis broker
+  - Separate trees for different family branches
+
+### Finance & Tracking
+- **WYGIWYH** - Financial tracking (port 9008)
+  - PostgreSQL database
+- **Speedtest Tracker** - Network monitoring (port 8180)
+  - MariaDB database, automated testing
+
+### Music & Scrobbling
+- **Maloja** - Music scrobbling service (port 42010)
+- **Multi-Scrobbler** - Cross-platform scrobbling (port 9078)
+  - Integrates with Jellyfin and Last.fm
+
+## Utilities & External Services
+
+### Remote Access
+- **RustDesk** - Remote desktop (host mode)
+
+### Network Tools
+- **iperf3-server** - Network performance testing (port 5201)
+
+### Discord Bot (OpenCode)
+- **Service**: Discord Agent Bot
+- **Role**: AI-powered Discord bot using Ollama LLM
+- **Container**: `discord-agent`
+- **Network**: `internal_net`, `db_net`
+- **Configuration**: `/docker/discord-agent/config/agent-config.yaml`
+- **Data**: `/docker/discord-agent/data/`
+- **LLM**: Ollama (mistral-3:8b) at `http://192.168.0.31:11434`
+- **Features**: Discord commands, service integrations, AI chat capabilities
+- **Database**: MySQL for conversation persistence
+- **Cogs**: Modular architecture with base_cog and integration_cog
+
+### RedBot
+- **Service**: RedBot Discord Bot
+- **Role**: Alternative Discord bot with custom cogs
+- **Container**: `redbot`
+- **Network**: `web_net`
+
+### Other Tools
+- **ByteStash** - File sharing service
+- **KitchenOwl** - Recipe management
+- **PinchFlat** - Cycling tracking
+
+## Network Architecture
+
+### Docker Networks
+- **media_net** - Media services and *arr stack
+- **db_net** - Database services (internal only)
+- **web_net** - Web-accessible services
+- **internal_net** - Internal service communication
+
+### Storage Structure
+```
+/docker/
+├── Arrs/ (Media stack: Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
+├── immich/ (Photo management)
+├── paperless/ (Document management)
+├── litellm/ (AI gateway)
+├── discord-agent/ (Discord bot with Ollama LLM integration)
+├── gitea/ (Git service)
+├── gramps-jamie/ (Genealogy)
+├── gramps-helen/ (Genealogy)
+├── npm/ (Nginx Proxy Manager)
+├── qBittorrent/ (Download client)
+├── slskd/ (Soulseek client)
+├── speedtest-tracker/ (Network monitoring)
+├── maloja/ (Music scrobbling)
+├── scrobble/ (Multi-scrobbler)
+├── syncthing/ (File sync)
+├── Homepage/ (Dashboard)
+└── Various other service configs
+```
+
+## Key Features & Configuration
+
+### Security
+- Container security with `no-new-privileges:true`
+- Network segmentation (databases internal-only)
+- Automated updates via Watchtower
+- Reverse proxy with SSL termination
+
+### Performance
+- Intel GPU hardware acceleration for Jellyfin
+- Log rotation (10MB max, 3 files)
+- Resource limits (Portainer: 512MB RAM)
+
+### Data Management
+- Comprehensive backup script included
+- Separate volumes for critical data
+- NAS storage integration for media files
+
+### Environment
+- System configured for Pacific/Auckland timezone
+- PUID/PGID for proper file permissions
+- Extensive environment variable configuration
+
+## Service Dependencies
+
+### Database Services
+- MariaDB instances for media stack, tracking, and document services
+- PostgreSQL for AI services, photo management, and finance
+- Redis for caching and message brokering
+
+### Network Dependencies
+- All web services route through Nginx Proxy Manager
+- Internal services communicate via internal_net
+- Media services isolated on media_net
+- Database services on internal-only db_net
+
+## Maintenance Agents
+
+### Automated Tasks
+- **Watchtower** - Daily container updates
+- **Backup Script** - Regular data backups
+- **Speedtest Tracker** - Automated network testing
+
+### Manual Tasks
+- Service monitoring via Glances
+- Container management via Portainer
+- Service configuration via Nginx Proxy Manager
+
+## Development & Debugging
+
+### Service Access
+- **Web Services**: Access through reverse proxy (80/443)
+- **Admin Interfaces**: Direct ports for management (e.g., 9443, 7575)
+- **Internal Services**: Host-only access for security
+
+### Troubleshooting
+- Container logs accessible via Portainer
+- System metrics via Glances
+- Network diagnostics via iperf3-server
+
+This Docker infrastructure represents a comprehensive self-hosted ecosystem covering media management, document processing, AI services, development tools, and personal productivity applications, all integrated through a reverse proxy and organized with proper network segmentation.
--- a/README.md
+++ b/README.md
@@ -35,19 +35,58 @@ This infrastructure manages a comprehensive self-hosted environment including me
 | Management & Infrastructure | Portainer | Container management UI | 9443 |
 | Management & Infrastructure | Homepage | Service dashboard | 7575 |

-### External Non-Docker Services
+### Discord Agent Integration

-| Service Categories| Service | Role / Description | Web UI Port |
-|---|---|---|---|
-| Management & Infrastructure | Adguard | DNS/ad-blocking | 3000 |
-| Data Management | Vaultwarden | Password manager | 80/8080 |
-| Entertainment | Crafty (Crafty Controller) | Minecraft server management | 3000 |
-| Data Management | Nextcloud | File sync and collaboration | 443 |
-| Automation | Homeassistant | Home automation | 8123 |
-| Entertainment | Foundryvtt | Foundry VTT | 30000 |
-| Data Management | Openmediavault | NAS management | 80/443 |
-| Website Hosting | Wordpress-irodori | WordPress site | 80 |
-| Website Hosting | Wordpress-dustin | WordPress site | 80 |
+### Discord Bot Service
+- **Service**: Discord Agent Bot
+- **Role**: AI-powered Discord bot using Ollama LLM
+- **Container**: `discord-agent`
+- **Network**: `internal_net`, `db_net`
+- **Web UI**: Discord (no direct web interface)
+- **Configuration**: `/docker/discord-agent/config/agent-config.yaml`
+- **Data**: `/docker/discord-agent/data/`
+
+### Features
+- **AI Integration**: Uses Ollama LLM (mistral-3:8b) at `http://192.168.0.31:11434`
+- **Modular Architecture**: Cogs for service integrations
+- **Database Support**: MySQL for conversation persistence
+- **Caching**: Redis for improved performance (configurable)
+- **Security**: Discord role-based permissions
+- **Logging**: Configurable logging levels (default: INFO)
+- **Rate Limiting**: Built-in rate limiting for API calls
+
+### Available Commands
+- `!agent` - Chat with the AI assistant
+- `!status` - Check bot status and latency
+- `!help` - Show available commands
+- Service integration commands (configurable)
+
+### Configuration
+- **Main Config**: `config/agent-config.yaml` (YAML-based with env var substitution)
+- **Discord Token**: Sourced from `DISCORD_BOT_TOKEN` environment variable
+- **Ollama Config**: Model, endpoint, temperature, top_p, top_k parameters
+- **Integrations**: Configurable service integrations in YAML
+
+### Environment Variables
+```
+DISCORD_BOT_TOKEN=your_discord_bot_token_here
+OLLAMA_ENDPOINT=http://192.168.0.31:11434
+OLLAMA_MODEL=mistral-3:8b
+TZ=Pacific/Auckland
+```
+
+### Bot Architecture
+- **Main Bot**: `discord_agent.py` - Core Discord bot implementation
+- **Base Cog**: `cogs/base_cog.py` - Base functionality and commands
+- **Integration Cog**: `cogs/integration_cog.py` - Service integrations
+- **Config**: YAML-based configuration with environment variable substitution
+- **Database**: MySQL with asyncpg driver for conversation history
+
+### Security
+- Runs on internal networks only
+- Discord role-based access control
+- Configurable rate limiting to prevent abuse
+- No direct database access from external networks


 ### Network Segmentation
@@ -65,12 +104,10 @@ This infrastructure manages a comprehensive self-hosted environment including me
 ├── Homepage/
 ├── Arrs/ (Prowlarr, Radarr, Sonarr, Lidarr, Bazarr, Jellyfin, Jellyseerr)
 ├── qBittorrent/
-├── slskd/
-├── soulsync/
+├── discord-agent/ (Discord bot with Ollama LLM)
 ├── paperless/
 ├── stirling/
 ├── immich/
-├── obsidian/
 ├── syncthing/
 ├── gitea/
 ├── gramps-jamie/
--- a/RESTORE.md
+++ b/RESTORE.md
@@ -17,19 +17,102 @@ If you are restoring, follow the standard restoration steps below, and refer to
 - Access to NAS storage (if applicable)
 - Root or sudo access

-## Step 1: Extract Backup
+## Discord Agent Service

+### Step 1: Create Discord Agent Directory
 ```bash
-# Navigate to your backup location
-cd /path/to/backups
-
-# Extract the backup archive
-tar xzf docker-backup-YYYYMMDD_HHMMSS.tar.gz
-
-# Navigate into the extracted backup
-cd YYYYMMDD_HHMMSS
+mkdir -p /docker/discord-agent/config /docker/discord-agent/data/logs /docker/discord-agent/data/database /docker/discord-agent/data/cache /docker/discord-agent/cogs /docker/discord-agent/scripts
 ```

+### Step 2: Copy Configuration Files
+```bash
+# Copy agent configuration
+cp agent-config.yaml /docker/discord-agent/config/
+cp permissions.json /docker/discord-agent/config/
+
+# Copy scripts
+cp startup.sh /docker/discord-agent/scripts/
+cp health_check.sh /docker/discord-agent/scripts/
+```
+
+### Step 3: Copy Python Files
+```bash
+cp discord_agent.py /docker/discord-agent/
+cp base_cog.py /docker/discord-agent/cogs/
+cp integration_cog.py /docker/discord-agent/cogs/
+```
+
+### Step 4: Copy Requirements
+```bash
+cp requirements.txt /docker/discord-agent/
+cp Dockerfile /docker/discord-agent/
+```
+
+### Step 5: Update Environment Variables
+Add these to your `.env` file:
+```bash
+# Discord Agent
+DISCORD_BOT_TOKEN=your_discord_bot_token_here
+OLLAMA_ENDPOINT=http://192.168.0.31:11434
+OLLAMA_MODEL=mistral-3:8b
+TZ=Pacific/Auckland
+```
+
+### Step 6: Update Configuration
+Edit `/docker/discord-agent/config/agent-config.yaml` with your specific settings:
+```yaml
+discord:
+  token: ${DISCORD_BOT_TOKEN}
+  prefix: "!"
+  status: "AI Assistant | !help"
+
+ollama:
+  endpoint: "${OLLAMA_ENDPOINT:http://192.168.0.31:11434}"
+  model: "${OLLAMA_MODEL:mistral-3:8b}"
+  parameters:
+    temperature: 0.7
+    top_p: 0.9
+    top_k: 40
+  timeout: 60
+```
+
+### Step 7: Build and Start Service
+```bash
+# Build the Discord agent image
+docker compose build discord-agent
+
+# Start the service
+docker compose up -d discord-agent
+
+# Verify the service is running
+docker compose logs discord-agent
+```
+
+### Step 8: Verify Integration
+```bash
+# Check Discord bot connection
+docker exec discord-agent python3 -c "import discord; print('Discord library available')"
+
+# Check Ollama connection
+curl http://192.168.0.31:11434/api/tags
+
+# Check MySQL database
+docker compose exec agent-db mysql -u agent -pagent -e "SHOW DATABASES;"
+```
+
+### Step 9: Test Bot Functionality
+- The bot should appear in your Discord server
+- Test with `!help` command to verify functionality
+- Test with `!agent <message>` to verify Ollama integration
+- Test with `!status` to check bot status
+
+### Troubleshooting
+- If bot doesn't start, check logs: `docker compose logs discord-agent`
+- Verify DISCORD_BOT_TOKEN is set correctly in `.env` file
+- Ensure Ollama is running and accessible at the configured endpoint
+- Verify internal_net and db_net networks are available
+- Check that MySQL database agent-db is running and healthy
+
 ## Step 2: Restore Directory Structure

 ```bash
--- a/docker-compose.yaml
+++ b/docker-compose.yaml
@@ -189,7 +189,7 @@ services:
      - PUID=${PUID}
      - PGID=${PGID}
    volumes:
-      - /mnt/Nas-Storage/data:/data
+      - /mnt/nas-storage/data:/data
      - /docker/qBittorrent/config:/config
    logging: *default-logging

@@ -206,7 +206,7 @@ services:
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
-      - /mnt/Nas-Storage/data/:/data
+      - /mnt/nas-storage/data/:/data
      - /docker/Arrs/Radarr/config:/config
    logging: *default-logging

@@ -223,7 +223,7 @@ services:
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
-      - /mnt/Nas-Storage/data:/data
+      - /mnt/nas-storage/data:/data
      - /docker/Arrs/Sonarr/config:/config
    logging: *default-logging

@@ -240,7 +240,7 @@ services:
      - PUID=${PUID}
      - PGID=${PGID}
    volumes:
-      - /mnt/Nas-Storage/data:/data
+      - /mnt/nas-storage/data:/data
      - /docker/Arrs/Lidarr/config:/config
    logging: *default-logging

@@ -257,7 +257,7 @@ services:
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
-      - /mnt/Nas-Storage/data:/data
+      - /mnt/nas-storage/data:/data
      - /docker/Arrs/Bazarr/config:/config
    logging: *default-logging

@@ -295,13 +295,13 @@ services:
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    volumes:
-      - /mnt/Nas-Storage/data:/data
+      - /mnt/nas-storage/data:/data
      - /docker/Arrs/Jellyfin/cache:/cache
      - /docker/Arrs/Jellyfin/config:/config
    logging: *default-logging

  slskd:
-    image: slskd/slskd
+    image: slskd/slskd:latest
    container_name: slskd
    restart: unless-stopped
    networks:
@@ -317,33 +317,11 @@ services:
      - SLSKD_USERNAME=${SLSKD_USERNAME}
      - SLSKD_PASSWORD=${SLSKD_PASSWORD}
    volumes:
-      - /mnt/Nas-Storage/data/torrents/soulsync/complete:/downloads
-      - /mnt/Nas-Storage/data/torrents/soulsync/incomplete:/incomplete
+      - /mnt/nas-storage/data/torrents/soulsync/complete:/data/torrents/soulsync/complete
+      - /mnt/nas-storage/data/torrents/soulsync/incomplete:/data/torrents/soulsync/incomplete
      - /docker/slskd:/app
    logging: *default-logging

-  soulsync-webui:
-    image: boulderbadgedad/soulsync:latest
-    container_name: soulsync-webui
-    restart: unless-stopped
-    networks:
-      media_net:
-    ports:
-      - "8887:8008"
-    extra_hosts:
-      - "host.docker.internal:host-gateway"
-    environment:
-      - TZ=${TZ}
-      - PUID=${PUID}
-      - PGID=${PGID}
-      - FLASK_ENV=production
-    volumes:
-      - /mnt/Nas-Storage/data/media/music:/music:ro
-      - /mnt/Nas-Storage/data/torrents/soulsync:/app/downloads
-      - /docker/soulsync/logs:/app/logs
-      - docker_soulsync:/app/database
-    logging: *default-logging
-
  # --- DOCUMENT & AI SUITE ---
  onlyoffice:
    image: onlyoffice/documentserver:latest
@@ -473,7 +451,8 @@ services:
      - "8090:8080"
    environment:
      - UI_APPNAME=Stirling-PDF
-      - SHOW_SURVEY=true
+      - SHOW_SURVEY=false
+      - SECURITY_ENABLELOGIN=false
      - SYSTEM_MAXFILESIZE=100
      - PUID=${PUID}
      - PGID=${PGID}
@@ -689,6 +668,7 @@ services:
    ports:
      - "5512:5000"
    environment:
+      - GRAMPSWEB_TREE=Helen Tree
      - GRAMPSWEB_CELERY_CONFIG__broker_url=redis://grampsweb_redis:6379/0
    volumes:
      - /docker/gramps-helen/cache:/app/cache
@@ -773,6 +753,24 @@ services:
    volumes:
      - /docker/redbot:/data
    logging: *default-logging
+    
+  discord-agent:
+    build: ./discord-agent
+    container_name: discord-agent
+    restart: unless-stopped
+    networks:
+      - internal_net
+    volumes:
+      - ./discord-agent/data:/app/data
+    env_file:
+      - .env
+    environment:
+      - DISCORD_BOT_TOKEN=${DISCORD_BOT_TOKEN}
+      - OLLAMA_ENDPOINT=${OLLAMA_ENDPOINT:-http://ollama:11434}
+      - OLLAMA_MODEL=${OLLAMA_MODEL:-llama3.2}
+      - OPENCODE_ENDPOINT=${OPENCODE_ENDPOINT:-http://192.168.0.10:4096}
+      - OPENCODE_API_KEY=${OPENCODE_API_KEY}
+    logging: *default-logging

  iperf3-server:
    image: networkstatic/iperf3
@@ -828,6 +826,8 @@ services:
    image: krateng/maloja:latest
    container_name: maloja
    restart: unless-stopped
+    environment:
+      - MALOJA_FORCE_PASSWORD=${MALOJA_FORCE_PASSWORD}
    networks:
      - internal_net
      - web_net
@@ -847,7 +847,14 @@ services:
    ports:
      - "9078:9078"
    environment:
-      - MALOJA_URL=http://maloja:42010
+      - MALOJA_URL=${MALOJA_URL}
+      - MALOJA_API_KEY=${MALOJA_API_KEY}
+      - JELLYFIN_URL=${JELLYFIN_URL}
+      - JELLYFIN_APIKEY=${JELLYFIN_APIKEY_MS}
+      - JELLYFIN_USER=${JELLYFIN_USER}
+      - JELLYFIN_TRANSFORMS=musicbrainz
+      - MB_CONTACT=${MB_CONTACT}
+      - MB_PRESETS=default,sensible,native
    volumes:
      - /docker/scrobble/config:/config
    logging: *default-logging
@@ -865,7 +872,7 @@ networks:
  internal_net:
    name: internal_net
    driver: bridge
-
+    
 volumes:
  docker_aidata:
  docker_dbdata:
@@ -875,3 +882,4 @@ volumes:
  docker_soulsync:
  open-webui:
  portainer_data:
+  discord_agent_data: