Docker Compose Generation

Overview

This document describes how docker-compose files are managed in relation to the configuration system.

Current Approach

Instead of generating docker-compose files from YAML config, we maintain the compose files directly with references to the config system:

1. Infrastructure (infra/docker/infrastructure.yml) - PostgreSQL, Keycloak, Kong, shared networks/volumes.
2. API Services (infra/docker/services-api.yml) - llm-api, media-api, response-api.
3. MCP Services (infra/docker/services-mcp.yml) - mcp-tools, vector-store, sandbox helpers.
4. Observability (infra/docker/observability.yml) - Prometheus, Grafana, Jaeger, OTEL collector.
5. Inference (infra/docker/inference.yml) - vLLM GPU/CPU profiles.
6. Development overlay (infra/docker/dev-full.yml) - adds host.docker.internal mapping for hybrid workflows.

The root docker-compose.yml stitches the profiles together (infrastructure + services + MCP + observability). Profiles such as full, mcp, monitor, and dev-full map directly to the files above.

Configuration Integration

Each service in docker-compose references the standardized environment variables. Most values are pulled from .env (generated by make quickstart) and fall back to the defaults defined in config/defaults.yaml and pkg/config/types.go:

services:
 llm-api:
 environment:
 # Database - constructed DSN from config defaults
 DB_POSTGRESQL_WRITE_DSN: "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@api-db:5432/${POSTGRES_DB}?sslmode=disable"
 
 # All other variables reference config/defaults.yaml structure
 HTTP_PORT: ${HTTP_PORT:-8080}
 LOG_LEVEL: ${LOG_LEVEL:-info}

Why Direct Maintenance?

1. Simplicity - Docker Compose is already declarative and easy to read
2. Flexibility - Allows docker-specific optimizations (healthchecks, extra hosts, bind mounts)
3. Version Control - Changes are clearly visible in git diffs
4. No Generation Overhead - No build step required
5. Profiles - Different dev/prod/monitoring stacks can be launched with a single make target

Future: Optional Generation

If needed, a generator can be built using pkg/config/compose/generator.go that:

• Reads config/defaults.yaml
• Applies environment overrides
• Generates docker-compose YAML files
• Validates output

Validation

To validate compose files:

# Validate syntax
docker compose -f docker/infrastructure.yml config

# Validate with current environment
docker compose -f docker-compose.yml config

# Dry-run full stack
docker compose --profile full config

# Verify dev overlay
docker compose --profile dev-full config

Typical networks and volumes:

• Networks: jan-server_default (core), jan-server_mcp-network (MCP helpers)
• Volumes: api-db-data, keycloak-db-data, vector-store-data, grafana-data

All of the above are declared in the compose snippets so they can be inspected with docker compose config.

Sprint 4 Status

OK COMPLETE - Docker compose files are consistent with config system OK COMPLETE - All services use standardized environment variables OK COMPLETE - Validation process documented

Rationale: Direct maintenance is simpler and more maintainable than generation for this use case. The generator infrastructure exists in pkg/config/compose/ if needed in the future.