> ## Documentation Index > Fetch the complete documentation index at: https://docs.hoodcloud.io/llms.txt > Use this file to discover all available pages before exploring further. # Environment Variables > Complete reference for all HoodCloud environment variables across services ## Quick Start Copy `.env.example` to `.env` and configure required variables. All configuration can also be provided via the `config/hoodcloud.yaml` file with environment variable overrides. ## Configuration Hierarchy HoodCloud uses a two-tier configuration system: 1. **Primary**: YAML configuration file (`config/hoodcloud.yaml`) 2. **Override**: Environment variables (take precedence over YAML) The `HOODCLOUD_CONFIG` environment variable can specify a custom config file path. ## Variables by Category ### Configuration Management | Variable | Required | Default | Security | Description | | ----------------- | -------- | ------- | -------- | ---------------------------------------------------------------------------------------------------------------- | | HOODCLOUD\_CONFIG | No | - | public | Path to YAML configuration file (if not specified, uses env-only mode) | | ENVIRONMENT | No | dev | public | Deployment environment (dev, staging, production). Used for logging context and production readiness validation. | | LOG\_LEVEL | No | info | public | Logging level (debug, info, warn, error) | ### HTTP Server | Variable | Required | Default | Security | Description | | ---------------------- | -------- | ------- | --------- | --------------------------------------------- | | SERVER\_HOST | No | 0.0.0.0 | public | HTTP server bind address | | SERVER\_PORT | No | 8080 | public | HTTP server port | | SERVER\_PUBLIC\_URL | Yes | - | sensitive | Public URL for API server (used by ops-agent) | | SERVER\_READ\_TIMEOUT | No | 30s | public | HTTP server read timeout | | SERVER\_WRITE\_TIMEOUT | No | 30s | public | HTTP server write timeout | ### Database (PostgreSQL) | Variable | Required | Default | Security | Description | | ------------- | -------- | --------- | --------- | --------------------------------------------------------------------------------------------------------------- | | DB\_HOST | Yes | localhost | sensitive | PostgreSQL host | | DB\_PORT | No | 5432 | public | PostgreSQL port | | DB\_USER | Yes | hoodcloud | sensitive | PostgreSQL username | | DB\_PASSWORD | Yes | - | secret | PostgreSQL password | | DB\_NAME | Yes | hoodcloud | sensitive | PostgreSQL database name | | DB\_SSL\_MODE | No | require | public | PostgreSQL SSL mode (disable, require, verify-ca, verify-full). Default changed to `require` in docker-compose. | ### Database Pool Configuration | Variable | Required | Default | Security | Description | | ------------------------- | -------- | ------------ | -------- | --------------------------------------------------------------------------------------------------------------- | | DB\_MAX\_CONNS | No | 10 | public | Maximum number of connections in the pool | | DB\_MIN\_CONNS | No | 2 | public | Minimum number of connections in the pool | | DB\_MAX\_CONN\_LIFETIME | No | 1h | public | Maximum lifetime of a connection before it is closed and replaced | | DB\_MAX\_CONN\_IDLE\_TIME | No | 30m | public | Maximum time a connection can be idle before it is closed | | DB\_STATEMENT\_TIMEOUT | No | 0 (disabled) | public | PostgreSQL statement\_timeout per connection (e.g., 2s, 5s, 10s). Queries exceeding this duration are canceled. | **Per-Service Connection Pool Sizing:** | Service | MaxConns | MinConns | StatementTimeout | Rationale | | ---------------- | -------- | -------- | ---------------- | ----------------------------------------- | | Health Evaluator | 5 | 2 | 5s | Single instance, periodic batch queries | | Auth Server | 5 | 2 | 2s | Lightweight auth queries | | API Server | 10 | 2 | 2s | User-facing, variable load | | Agent Gateway | 10 | 2 | 2s | Agent heartbeats, moderate load | | Orchestrator | 10 | 2 | 10s | Workflow activities, Terraform operations | **Connection Budget Formula:** `sum(MaxConns × instances) < PostgreSQL max_connections` Example: With 2 instances per service: `(5×2) + (5×2) + (10×2) + (10×2) + (10×2) = 80 < 100 (default max_connections)` ### Redis | Variable | Required | Default | Security | Description | | --------------- | -------- | --------- | --------- | --------------------------------------------- | | REDIS\_HOST | Yes | localhost | sensitive | Redis host | | REDIS\_PORT | No | 6379 | public | Redis port | | REDIS\_PASSWORD | No | - | secret | Redis password (if authentication is enabled) | | REDIS\_DB | No | 0 | public | Redis database number | ### Temporal | Variable | Required | Default | Security | Description | | --------------------- | -------- | --------------- | --------- | ------------------------ | | TEMPORAL\_HOST | Yes | localhost | sensitive | Temporal server host | | TEMPORAL\_PORT | No | 7233 | public | Temporal server port | | TEMPORAL\_NAMESPACE | No | hoodcloud | public | Temporal namespace | | TEMPORAL\_TASK\_QUEUE | No | hoodcloud-tasks | public | Temporal task queue name | ### HashiCorp Vault Vault is the secrets provider for all application secrets. AWS credentials may still be needed for S3 key backups (either via environment or Vault AWS secrets engine). **Local Development:** All Vault variables below are auto-configured by `docker-compose.dev.yml`. The `vault-dev-init.sh` sidecar seeds secrets and writes AppRole credentials to a shared volume. No manual Vault configuration is needed for local dev. | Variable | Required | Default | Security | Description | | ----------------------------- | ----------- | ------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | VAULT\_ADDR | Yes | - | sensitive | Vault server address (e.g., `https://:8200`). TLS is enabled by default in vault.hcl. | | VAULT\_ROLE\_ID | Yes | - | sensitive | AppRole role\_id for authentication | | VAULT\_SECRET\_ID\_PATH | Yes | - | sensitive | Path to file containing AppRole secret\_id (e.g., `/secrets/vault-secret-id`). File must be readable by the container user (chmod 644). | | VAULT\_NAMESPACE | No | - | public | Vault namespace (enterprise feature, leave empty for OSS) | | VAULT\_CACHE\_TTL | No | 5m | public | Cache TTL for Vault secrets | | VAULT\_TLS\_CA\_FILE | Conditional | - | sensitive | Path to CA certificate for Vault TLS verification (e.g., `/secrets/vault-ca.crt`). Required when Vault uses self-signed TLS certificates. | | VAULT\_TLS\_SKIP\_VERIFY | No | false | public | Skip TLS certificate verification (dev only). **Rejected in production.** | | VAULT\_MASTER\_KEY\_PATH | Yes | - | sensitive | Vault KV path for master encryption key | | VAULT\_APP\_CREDENTIALS\_PATH | Yes | - | sensitive | Vault KV path for application credentials | | VAULT\_TRANSIT\_KEY\_NAME | No | - | public | Transit engine key name for DEK encryption (optional) | | VAULT\_PKI\_ROLE | No | - | public | PKI role for gRPC TLS certificates (optional) | | VAULT\_PKI\_CERT\_TTL | No | 720h | public | PKI certificate TTL (30 days default) | | VAULT\_PKI\_RENEW\_BEFORE | No | 24h | public | Renew PKI certificates this long before expiry | > **See also:** [Vault Secret Structure](./vault#secret-structure) for the full Vault KV layout and all credential fields. **Key notes:** * `sealed_box_public_key` and `sealed_box_private_key` are **mandatory** — api-server and orchestrator fail fast at startup without them. See [Vault - Generate X25519 Keypair](./vault#9-generate-x25519-sealed-box-keypair). * Payment mTLS fields (`payment_client_cert`, `payment_client_key`, `payment_ca_cert`) are optional — only needed when payment service integration is enabled. * Incident notification fields are optional. When present, health-evaluator enables the corresponding notification channels. If none are configured, incidents are tracked in DB only. **AppRole Authentication:** 1. Generate role\_id: `vault read -field=role_id auth/approle/role/hoodcloud-control-plane/role-id` 2. Generate secret\_id: `vault write -f -field=secret_id auth/approle/role/hoodcloud-control-plane/secret-id` 3. Deploy secret\_id to `/opt/hoodcloud/secrets/vault-secret-id` on the service host with `chmod 644` 4. The host path is mounted into containers as `/secrets/vault-secret-id` via the `/opt/hoodcloud/secrets:/secrets:ro` volume mount 5. Service reads the secret\_id file during AppRole authentication ### AWS (S3 Key Backups) | Variable | Required | Default | Security | Description | | ------------------------ | ----------- | ------------ | --------- | ------------------------------------------------------------------------- | | AWS\_REGION | No | eu-central-1 | public | AWS region for S3 | | AWS\_ENDPOINT\_URL | No | - | sensitive | Custom AWS endpoint URL (for LocalStack in testing) | | AWS\_ACCESS\_KEY\_ID | Conditional | - | secret | AWS access key ID (required if using AWS S3 without Vault AWS engine) | | AWS\_SECRET\_ACCESS\_KEY | Conditional | - | secret | AWS secret access key (required if using AWS S3 without Vault AWS engine) | | AWS\_S3\_BUCKET | Yes | - | sensitive | S3 bucket for encrypted key backups | | AWS\_S3\_BACKUP\_PREFIX | No | backups/ | public | S3 prefix for backup objects | ### gRPC Server (Ops Agent Communication) | Variable | Required | Default | Security | Description | | -------------------------- | ----------- | ------- | --------- | ------------------------------------------------------------------ | | GRPC\_HOST | No | 0.0.0.0 | public | gRPC server bind address | | GRPC\_PORT | No | 9090 | public | gRPC server port | | GRPC\_PUBLIC\_URL | Yes | - | sensitive | Public gRPC address in host:port format (for ops-agent connection) | | GRPC\_USE\_TLS | No | false | public | Enable TLS with system root CAs for ops-agent gRPC connection | | GRPC\_CERT\_FILE | Conditional | - | sensitive | Path to TLS certificate file (required if using custom TLS) | | GRPC\_KEY\_FILE | Conditional | - | secret | Path to TLS private key file (required if using custom TLS) | | GRPC\_CA\_FILE | Conditional | - | sensitive | Path to TLS CA certificate file (for mTLS) | | GRPC\_CONFIG\_SIGNING\_KEY | No | - | secret | Path to ECDSA private key for signing configuration commands | ### gRPC Rate Limiting | Variable | Required | Default | Security | Description | | ------------------------------------ | -------- | ------- | -------- | --------------------------------------------------- | | GRPC\_RATE\_LIMIT\_RPS | No | 10 | public | Requests per second per node for gRPC rate limiting | | GRPC\_RATE\_LIMIT\_BURST | No | 20 | public | Maximum burst size for gRPC rate limiting | | GRPC\_RATE\_LIMIT\_CLEANUP\_INTERVAL | No | 5m | public | Interval for cleaning up stale rate limiters | ### Cloud Provider Defaults | Variable | Required | Default | Security | Description | | ----------------- | -------- | ------- | -------- | ------------------------------------ | | DEFAULT\_PROVIDER | No | hetzner | public | Default cloud provider (hetzner) | | DEFAULT\_REGION | No | fsn1 | public | Default region for node provisioning | ### Terraform | Variable | Required | Default | Security | Description | | -------------------------- | ----------- | --------------- | --------- | ----------------------------------------------------------- | | TERRAFORM\_BINARY | No | terraform | public | Path to Terraform binary | | TERRAFORM\_MODULE\_PATH | No | - | public | Path to Terraform module (provider-specific) | | TERRAFORM\_STATE\_DIR | No | terraform-state | public | Directory for Terraform state files (local backend) | | TERRAFORM\_PARALLELISM | No | 10 | public | Terraform parallelism setting | | SKIP\_TERRAFORM | No | false | public | Skip Terraform execution (for local dev) | | DEV\_MODE | No | false | public | Enable development mode features | | TERRAFORM\_STATE\_BACKEND | No | local | public | State backend type: `local` or `s3` | | TERRAFORM\_S3\_BUCKET | Conditional | - | sensitive | S3 bucket for state storage (required when backend=s3) | | TERRAFORM\_S3\_REGION | No | eu-central-1 | public | AWS region for S3 state bucket | | TERRAFORM\_DYNAMODB\_TABLE | Conditional | - | sensitive | DynamoDB table for state locking (required when backend=s3) | #### Terraform Operation Timeouts | Variable | Required | Default | Security | Description | | --------------------------- | -------- | ------- | -------- | --------------------------------------- | | TERRAFORM\_INIT\_TIMEOUT | No | 5m | public | Timeout for `terraform init` command | | TERRAFORM\_PLAN\_TIMEOUT | No | 10m | public | Timeout for `terraform plan` command | | TERRAFORM\_APPLY\_TIMEOUT | No | 30m | public | Timeout for `terraform apply` command | | TERRAFORM\_DESTROY\_TIMEOUT | No | 20m | public | Timeout for `terraform destroy` command | | TERRAFORM\_OUTPUT\_TIMEOUT | No | 1m | public | Timeout for `terraform output` command | ### Command Execution | Variable | Required | Default | Security | Description | | ---------------------- | -------- | ------- | -------- | ------------------------------------------------------------------------------------------------------ | | COMMAND\_PROGRESS\_TTL | No | 24h | public | TTL for command progress data in Redis. Progress data is automatically cleaned up after this duration. | ### Chain Configurations #### Dynamic Chain Config Provider The control plane services can load chain configs from either local filesystem or S3 with hot-reload support. | Variable | Required | Default | Security | Description | | ----------------------- | ----------- | ----------------------- | --------- | --------------------------------------------------------------------------- | | CHAIN\_PROFILES\_DIR | No | hoodcloud-chain-configs | public | Directory containing chain profile definitions (local mode) | | CHAIN\_CONFIGS\_SOURCE | No | auto | public | Config source: `local`, `s3`, or `auto` (auto uses S3 if bucket configured) | | CHAINS\_S3\_BUCKET | Conditional | - | sensitive | S3 bucket for chain configs (required for S3 mode) | | CHAINS\_S3\_REGION | No | AWS\_REGION | public | AWS region for chain configs S3 bucket | | CHAINS\_S3\_PREFIX | No | (empty) | public | S3 key prefix for chain config files | | CHAIN\_PROFILE\_VERSION | No | latest | public | Config version: `latest` (auto-update) or pinned like `v1.0.1` | | CHAINS\_CHECK\_INTERVAL | No | 1m | public | Interval to check for version updates (auto-update mode only) | | CHAIN\_CONFIGS\_WATCH | No | false | public | Enable fsnotify file watching for hot reload (local mode) | **Version Modes:** | Mode | CHAIN\_PROFILE\_VERSION | Behavior | | ----------- | ----------------------- | ----------------------------------------------------------- | | Auto-update | `latest` (default) | Downloads from `latest/`, polls `checksums.txt` for changes | | Pinned | `v1.0.1` | Downloads specific version once, no polling | **S3 Bucket Structure:** ``` s3://hoodcloud-chain-configs/ ├── latest/ │ ├── checksums.txt # SHA256 hash for change detection │ └── configs.tar.gz # Current production tarball ├── v1.0.0/ │ ├── checksums.txt │ └── configs-v1.0.0.tar.gz ├── v1.0.1/ │ └── ... └── v1.0.2/ ├── checksums.txt └── configs-v1.0.2.tar.gz ``` **Tarball Contents:** ``` chains/ ├── celestia-mocha/ │ ├── profile.yaml # Chain metadata, provisioning config │ ├── runtime.yaml # Node lifecycle configuration │ └── observation.yaml # Health policies, metrics collectors ├── ethereum-holesky/ └── ... recipes/ ├── celestia/ ├── ethereum/ └── ... ``` #### Config-as-Code (Ops Agent Runtime) These variables control config bundle distribution to ops-agents (separate from control plane loading): | Variable | Required | Default | Security | Description | | --------------------------- | ----------- | ----------- | --------- | --------------------------------------------------------------------------------- | | CHAIN\_CONFIGS\_ENABLED | No | false | public | Enable config-as-code mode for ops-agent | | CONFIG\_BUNDLE\_VERSION | Conditional | - | public | Current config bundle version (e.g., v1.2.3) - required if config-as-code enabled | | CHAIN\_CONFIGS\_S3\_BUCKET | Conditional | - | sensitive | S3 bucket for config bundles - required if config-as-code enabled | | CHAIN\_CONFIGS\_S3\_REGION | No | AWS\_REGION | public | S3 bucket region for config bundles | | CHAIN\_CONFIGS\_URL\_EXPIRY | No | 1h | public | Pre-signed URL expiry duration | | CHAIN\_CONFIGS\_LOCAL\_DIR | No | - | public | Local directory for config bundles (development/testing) | | S3\_PUBLIC\_ENDPOINT\_URL | No | - | sensitive | Public S3 endpoint for pre-signed URLs (E2E testing) | ### Ops Agent | Variable | Required | Default | Security | Description | | ------------------------------- | -------- | ------------------- | --------- | -------------------------------------------------------- | | OPS\_AGENT\_VERSION | No | 0.1.0 | public | Ops agent version to deploy | | OPS\_AGENT\_HEARTBEAT\_INTERVAL | No | 15s | public | Heartbeat interval for keep-alive | | OPS\_AGENT\_DOWNLOAD\_URL | No | SERVER\_PUBLIC\_URL | sensitive | Base URL for ops agent binary downloads | | OPS\_AGENT\_BINARIES\_DIR | No | - | public | Directory containing ops agent binaries for download | | REQUIRE\_CONFIG\_SIGNATURE | No | true | public | Require ECDSA signature verification for config commands | ### Ops Agent Runtime (on VMs) The ops-agent supports two configuration modes: 1. **YAML + Environment Override**: Set `OPS_AGENT_CONFIG` to a YAML config file path. Environment variables override YAML values. 2. **Environment-Only (Legacy)**: If `OPS_AGENT_CONFIG` is not set, configuration loads purely from environment variables. See `config/ops-agent.yaml.example` for a YAML configuration template. | Variable | Required | Default | Security | Description | | ---------------------------- | ----------- | --------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------- | | OPS\_AGENT\_CONFIG | No | - | public | Path to YAML configuration file. If set, loads from file with env overrides. | | NODE\_ID | Yes | - | sensitive | Unique node identifier | | NODE\_TOKEN | No | - | secret | Authentication token for control plane (nt\_xxx format) | | CONTROL\_PLANE\_URL | Yes | - | sensitive | Control plane gRPC URL (host:port) for backward compatibility | | GRPC\_URL | Yes | - | sensitive | Control plane gRPC address (host:port) - preferred over CONTROL\_PLANE\_URL | | GRPC\_ADDRESS | No | - | sensitive | Alias for GRPC\_URL in YAML config | | GRPC\_USE\_TLS | No | false | public | Use TLS with system root CAs for control plane connection | | GRPC\_PORT | No | 9091 | public | gRPC server port on ops-agent | | TLS\_CERT\_FILE | Conditional | - | sensitive | Path to TLS certificate for mTLS | | TLS\_KEY\_FILE | Conditional | - | secret | Path to TLS private key for mTLS | | TLS\_CA\_FILE | Conditional | - | sensitive | Path to CA certificate for gRPC mTLS verification. Not needed for NATS (uses system root CAs with Let's Encrypt). | | CHAIN\_PROFILE\_ID | No | - | public | Chain profile identifier (e.g., celestia-mocha) | | CHAIN\_DATA\_DIR | No | /opt/hoodcloud/data/chain | public | Chain data directory | | CHAIN\_RPC\_PORT | No | 0 | public | Chain-specific RPC port (0 = auto-detect) | | CHAIN\_TYPE | No | - | public | Chain type (cosmos, ethereum, celestia) - loaded from runtime.yaml | | HEARTBEAT\_INTERVAL | No | 5m | public | Keep-alive heartbeat interval | | GRACE\_PERIOD | No | 30s | public | Graceful shutdown grace period | | CONFIG\_SIGNING\_PUBLIC\_KEY | No | /opt/hoodcloud/config/signing.pub | sensitive | Path to ECDSA public key for config signature verification | | REQUIRE\_CONFIG\_SIGNATURE | No | true | public | Require signature verification for config commands | | CONFIG\_DIR | No | - | public | Directory containing chain config bundles | | CONFIG\_VERSION | No | - | public | Version of config bundle | | METRICS\_ENABLED | No | true | public | Enable Prometheus metrics endpoint on ops-agent | | METRICS\_PORT | No | 9101 | public | Port for Prometheus metrics endpoint | | OBSERVATION\_ENABLED | No | true | public | Enable observation collectors and NATS metrics transport (RFC-007) | | NATS\_URL | No | nats\://localhost:4222 | sensitive | NATS server URL for metrics transport | | HOST\_ID | No | - | public | Host identifier for metrics labels | | SYSTEM\_METRICS\_INTERVAL | No | 60s | public | How often to collect system metrics (disk, CPU, memory) | | SYSTEM\_METRICS\_DATA\_PATH | No | / | public | Path to monitor for disk usage (defaults to root filesystem) | ### Health Evaluator | Variable | Required | Default | Security | Description | | ------------------------------- | -------- | ------- | -------- | --------------------------------------------------------------- | | HEALTH\_EVALUATION\_INTERVAL | No | 30s | public | How often to evaluate node health | | HEALTH\_HEARTBEAT\_TIMEOUT | No | 60s | public | Heartbeat timeout before marking node as DOWN | | HEALTH\_MIGRATION\_COOLDOWN | No | 1h | public | Cooldown period between automatic migrations | | SUBSCRIPTION\_CLEANUP\_INTERVAL | No | 1h | public | How often to check for expired subscriptions | | MAINTENANCE\_CLEANUP\_INTERVAL | No | 5m | public | How often to check for nodes stuck in maintenance state | | MAINTENANCE\_CLEANUP\_TIMEOUT | No | 30m | public | How long a node can stay in maintenance before considered stuck | #### Uptime Worker (Hardcoded Defaults) The UptimeWorker runs inside `health-evaluator` and uses hardcoded defaults (no environment variables). Values are defined in `internal/defaults/defaults.go`: | Setting | Default | Description | | ------------- | ------- | ------------------------------------------------------- | | Interval | 5m | How often the worker materializes hourly uptime buckets | | BatchSize | 500 | Number of nodes processed per batch | | GracePeriod | 10m | Provisioning grace period counted as uptime | | RetentionDays | 90 | Hourly buckets older than this are purged automatically | ### Observability | Variable | Required | Default | Security | Description | | ------------------------------------ | -------- | ---------------------------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------- | | LOKI\_URL | No | [http://localhost:3100](http://localhost:3100) | sensitive | Loki URL for log aggregation | | PROMETHEUS\_URL | No | [http://localhost:9090](http://localhost:9090) | sensitive | Prometheus URL for metrics collection | | VICTORIA\_METRICS\_URL | No | - | sensitive | Victoria Metrics URL for observation metrics TSDB (RFC-007). Enables metrics ingester and policy evaluation when set. | | METRICS\_CONSUMER\_NAME | No | metrics-ingester | public | NATS consumer name for metrics ingester (useful for multi-instance deployments) | | EVALUATION\_CONCURRENCY | No | 10 | public | Number of chains to evaluate in parallel for policy evaluation | | CIRCUIT\_BREAKER\_FAILURE\_THRESHOLD | No | 5 | public | Number of consecutive failures before opening the circuit breaker | | CIRCUIT\_BREAKER\_SUCCESS\_THRESHOLD | No | 3 | public | Number of successes in half-open state before closing the circuit | | CIRCUIT\_BREAKER\_TIMEOUT | No | 30s | public | How long to stay open before transitioning to half-open state | #### Telemetry (OpenTelemetry) | Variable | Required | Default | Security | Description | | -------------------- | -------- | ------------------- | --------- | ------------------------------------------------------------- | | OTEL\_ENABLED | No | false | public | Enable OpenTelemetry distributed tracing | | OTEL\_COLLECTOR\_URL | No | otel-collector:4317 | sensitive | OTLP gRPC endpoint for the OTel Collector | | OTEL\_SAMPLE\_RATIO | No | 1.0 | public | Trace sampling ratio (0.0 to 1.0). 1.0 = sample all traces | | OTEL\_INSECURE | No | true | public | Disable TLS for OTLP exporter connection (true for local dev) | ### Authentication (Auth Server) #### JWT RS256 (Asymmetric) RS256 uses asymmetric cryptography: the auth-server signs tokens with a private key, and api-server verifies with the public key. | Variable | Required | Default | Security | Description | | ------------------------ | ----------------- | ------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | JWT\_PRIVATE\_KEY\_FILE | Yes (auth-server) | - | secret | Path to RSA private key PEM file (auth-server only) | | JWT\_PUBLIC\_KEY\_FILE | Yes | - | sensitive | Path to RSA public key PEM file (all services) | | JWT\_ACCESS\_TOKEN\_TTL | No | 15m | public | JWT access token time-to-live | | JWT\_REFRESH\_TOKEN\_TTL | No | 168h | public | JWT refresh token time-to-live (7 days) | | JWT\_ISSUER | No | hoodcloud | public | JWT issuer claim | | JWT\_AUDIENCE | No | hoodcloud-api | public | JWT audience claim. Auth-server embeds this in issued tokens; all verifying services reject tokens with a mismatched audience (returns 401 Unauthorized). Must be identical across auth-server and api-server. | | JWT\_KEY\_ID | No | - | public | Key ID for JWT header (kid), enables key rotation | **Audience claim:** When `JWT_AUDIENCE` is set, the auth-server includes it in every issued token and all verifying services (api-server, agent-gateway) validate that incoming tokens contain this exact audience value. A mismatch causes token validation to fail with 401 Unauthorized. If deploying multiple services, ensure `JWT_AUDIENCE` is consistent across all of them. Omitting `JWT_AUDIENCE` skips audience validation entirely (backward compatibility). **Key Generation:** ```bash theme={null} # Generate 2048-bit RSA key pair openssl genrsa -out jwt_private.pem 2048 openssl rsa -in jwt_private.pem -pubout -out jwt_public.pem # Store in Vault (production) vault kv put secret/hoodcloud/app-credentials \ jwt_private_key=@jwt_private.pem \ jwt_public_key=@jwt_public.pem \ # ... other existing fields ``` **Key Resolution Order:** 1. Vault (`jwt_private_key`, `jwt_public_key` fields) 2. File paths (`JWT_PRIVATE_KEY_FILE`, `JWT_PUBLIC_KEY_FILE`) \| SIWE\_DOMAIN | No | localhost | public | **(Deprecated)** SIWE message domain. Only used when `AUTH_PROVIDER=siwe` or `both`. | \| SIWE\_URI | No | [http://localhost:3000](http://localhost:3000) | public | **(Deprecated)** SIWE message URI. Only used when `AUTH_PROVIDER=siwe` or `both`. | \| AUTH\_NONCE\_EXPIRY | No | 5m | public | **(Deprecated)** Auth nonce expiry time. Only used when `AUTH_PROVIDER=siwe` or `both`. | \| AUTH\_RATE\_LIMIT\_PER\_MINUTE | No | 20 | public | Rate limit for auth endpoints per IP | \| AUTH\_MAX\_REFRESH\_TOKENS\_PER\_USER | No | 10 | public | Maximum concurrent active sessions per user | \| AUTH\_ALLOWED\_ORIGINS | No | - | public | Comma-separated allowed CORS origins for auth server | ### Clerk (Primary Auth Provider) | Variable | Required | Default | Security | Description | | ------------------------------------- | ----------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | AUTH\_PROVIDER | No | clerk | public | Auth provider: `clerk` (default), `siwe` (legacy), or `both` (migration) | | AUTH\_CLERK\_SECRET\_KEY | Conditional | - | secret | Clerk secret key for JWKS JWT verification (required when provider is `clerk` or `both`) | | AUTH\_CLERK\_AUTHORIZED\_PARTY | Conditional | - | public | Comma-separated list of allowed frontend origins for Clerk JWT `azp` claim, e.g. `https://hoodcloud.io,https://app.hoodcloud.io` (required when provider is `clerk` or `both`) | | AUTH\_CLERK\_WEBHOOK\_SIGNING\_SECRET | Conditional | - | secret | Clerk webhook signing secret for svix signature verification (required for auth-server when provider is `clerk` or `both`) | ### API Security | Variable | Required | Default | Security | Description | | -------------------------- | -------- | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | API\_AUTH\_ENABLED | No | false | public | Enable API key authentication | | DANGEROUS\_DISABLE\_AUTH | No | false | public | Must be set to `true` to confirm disabling authentication when `API_AUTH_ENABLED=false`. Panics at startup if auth is disabled without this flag. | | API\_RATE\_LIMIT\_ENABLED | No | false | public | Enable rate limiting | | API\_RATE\_LIMIT\_REQUESTS | No | 100 | public | Requests per minute per API key | | API\_RATE\_LIMIT\_REDIS | No | false | public | Enable Redis-backed distributed rate limiting. When true, uses Redis INCR+EXPIRE sliding window instead of in-memory token bucket. Includes circuit breaker: after 5 consecutive Redis failures, falls back to in-memory limiter automatically. | ### NATS Event Streaming | Variable | Required | Default | Security | Description | | ------------------------ | -------- | ---------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | NATS\_ENABLED | No | false | public | Enable NATS event streaming | | NATS\_URL | No | nats\://localhost:4222 | sensitive | NATS server URL (internal, for control plane) | | NATS\_PUBLIC\_URL | No | - | sensitive | Public NATS URL (for ops-agent on external VMs) | | NATS\_ACCOUNT | No | - | sensitive | NATS account name for multi-account auth | | NATS\_STREAM\_NAME | No | HOODCLOUD\_EVENTS | public | JetStream stream name | | NATS\_CONSUMER\_NAME | No | control-plane | public | Durable consumer name | | NATS\_MAX\_MESSAGE\_SIZE | No | 1048576 | public | Maximum message size in bytes (default: 1MB) | | NATS\_STREAM\_REPLICAS | No | 1 | public | JetStream stream replica count. Set to 3 for production with a 3-node NATS cluster. Applies to HOODCLOUD\_EVENTS and HOODCLOUD\_METRICS streams. | > **Cross-repo dependency:** The PAYMENTS stream is managed by the payment service repository. Its replica count must be configured separately in the payment service. #### NATS Multi-Account Authentication When `NATS_ACCOUNTS_ENABLED=true`, each service component connects with its own account and subject-level permissions. Tokens are delivered to ops-agents via gRPC after registration (no shared token in cloud-init). | Variable | Required | Default | Security | Description | | ------------------------------------ | ----------- | ------- | -------- | ----------------------------------------------------------------------------------------- | | NATS\_ACCOUNTS\_ENABLED | Yes (prod) | false | public | Enable per-service NATS account auth. Required `true` in production when NATS is enabled. | | NATS\_ACCOUNT\_OPS\_AGENT\_TOKEN | Conditional | - | secret | NATS token for ops-agent account (delivered via gRPC) | | NATS\_ACCOUNT\_PAYMENT\_SVC\_TOKEN | Conditional | - | secret | NATS token for payment-service account | | NATS\_ACCOUNT\_CONTROL\_PLANE\_TOKEN | Conditional | - | secret | NATS token for control-plane account | | NATS\_ACCOUNT\_HEALTH\_EVAL\_TOKEN | Conditional | - | secret | NATS token for health-evaluator account | ### Payment Service Integration The payment service runs as a separate microservice with its own database. These variables configure the main app's connection to the payment service. #### Payment Service Client (gRPC) | Variable | Required | Default | Security | Description | | ------------------------------ | ----------- | ------- | --------- | ----------------------------------------------------------------------------- | | PAYMENT\_SERVICE\_ENABLED | No | false | public | Enable payment service integration | | PAYMENT\_SERVICE\_ADDRESS | Conditional | - | sensitive | Payment service gRPC address (e.g., payment-service.hoodcloud.internal:50051) | | PAYMENT\_SERVICE\_CERT\_FILE | Conditional | - | sensitive | Client certificate file for mTLS (see note below) | | PAYMENT\_SERVICE\_KEY\_FILE | Conditional | - | secret | Client key file for mTLS (see note below) | | PAYMENT\_SERVICE\_CA\_FILE | Conditional | - | sensitive | CA certificate for server verification (see note below) | | PAYMENT\_SERVICE\_SERVER\_NAME | No | - | public | Expected server name for TLS verification (SNI) | | PAYMENT\_SERVICE\_TIMEOUT | No | 30s | public | Default timeout for gRPC calls | | PAYMENT\_SERVICE\_INSECURE | No | false | public | Skip TLS (development only) | **mTLS Credentials via Vault:** When using Vault for secrets management, payment service mTLS credentials can be stored in the app credentials secret instead of using file paths. Add these fields to your app credentials JSON: ```json theme={null} { "payment_client_cert": "-----BEGIN CERTIFICATE-----\n...", "payment_client_key": "-----BEGIN RSA PRIVATE KEY-----\n...", "payment_ca_cert": "-----BEGIN CERTIFICATE-----\n..." } ``` When these fields are present in Vault, they take precedence over the file path environment variables. This is the recommended approach for production deployments. #### Payment Event Consumer (NATS) | Variable | Required | Default | Security | Description | | ----------------------------------- | -------- | ---------------- | --------- | ------------------------------------------ | | PAYMENT\_CONSUMER\_ENABLED | No | false | public | Enable payment event consumer | | PAYMENT\_CONSUMER\_URL | No | NATS\_URL | sensitive | NATS server URL for payment events | | PAYMENT\_CONSUMER\_STREAM\_NAME | No | PAYMENTS | public | JetStream stream name for payment events | | PAYMENT\_CONSUMER\_NAME | No | main-app-payment | public | Durable consumer name | | PAYMENT\_CONSUMER\_ACK\_WAIT | No | 30s | public | Time before unacked message is redelivered | | PAYMENT\_CONSUMER\_MAX\_DELIVER | No | 5 | public | Maximum delivery attempts | | PAYMENT\_CONSUMER\_IDEMPOTENCY\_TTL | No | 24h | public | TTL for processed idempotency keys | ### Payment Service (Separate Microservice) The payment service runs as an isolated microservice with its own configuration. See `payment-service/config/config.example.yaml` for full configuration reference. | Variable | Required | Default | Security | Description | | ------------------------- | ----------- | ---------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------- | | PAYMENT\_CONFIG | No | - | public | Path to YAML configuration file | | GRPC\_ADDRESS | No | :50051 | public | gRPC server listen address | | HTTP\_ADDRESS | No | :8080 | public | HTTP server (health checks) | | METRICS\_ADDRESS | No | :9090 | public | Prometheus metrics server listen address | | DB\_HOST | Yes | localhost | sensitive | PostgreSQL host | | DB\_PORT | No | 5432 | public | PostgreSQL port | | DB\_USER | Yes | payment | sensitive | PostgreSQL user | | DB\_PASSWORD | Yes | - | secret | PostgreSQL password | | DB\_NAME | Yes | payment | sensitive | PostgreSQL database | | DB\_SSL\_MODE | No | require | public | PostgreSQL SSL mode | | NATS\_URL | No | nats\://localhost:4222 | sensitive | NATS server URL | | NATS\_STREAM\_NAME | No | PAYMENTS | public | JetStream stream name | | NATS\_CTRL\_SIGNING\_SEED | Conditional | - | secret | CTRL account signing seed for JWT auth (from Vault in production) | | NATS\_CTRL\_ACCOUNT\_PUB | Conditional | - | sensitive | CTRL account public key for JWT auth | | PRICING\_CONFIG\_PATH | No | config/pricing.yaml | public | Path to pricing configuration | | TLS\_CERT\_FILE | Conditional | - | sensitive | Server TLS certificate (mTLS) | | TLS\_KEY\_FILE | Conditional | - | secret | Server TLS private key (mTLS) | | TLS\_CA\_FILE | Conditional | - | sensitive | CA certificate for client verification (mTLS) | | TLS\_INSECURE | No | false | public | Disable TLS (development only) | | TLS\_ALLOWED\_CN | No | - | sensitive | Required Common Name for client certificate authentication. When set, gRPC clients must present a certificate with this CN. | #### Payment Service Vault Integration When Vault is enabled for the payment service, it provides AWS credentials for S3 operations instead of static environment variables. | Variable | Required | Default | Security | Description | | ----------------------------- | ------------------ | ------------------------- | --------- | ---------------------------------------------------------------------------------------------------- | | VAULT\_ADDRESS | Yes (if Vault) | - | sensitive | Vault server address (e.g., `https://:8200`). Note: uses `VAULT_ADDRESS` not `VAULT_ADDR`. | | VAULT\_ROLE\_ID | Yes (if Vault) | - | sensitive | AppRole role\_id for authentication | | VAULT\_SECRET\_ID\_PATH | Yes (if Vault) | - | sensitive | Path to file containing AppRole secret\_id | | VAULT\_NAMESPACE | No | - | public | Vault namespace (enterprise feature) | | VAULT\_TLS\_SKIP\_VERIFY | No | false | public | Skip TLS certificate verification. **Rejected in production.** | | VAULT\_TLS\_CA\_FILE | Yes (if Vault TLS) | - | sensitive | Path to CA certificate for Vault TLS verification | | VAULT\_APP\_CREDENTIALS\_PATH | No | hoodcloud/app-credentials | sensitive | Vault KV path for payment service credentials | | VAULT\_CACHE\_TTL | No | 5m | public | Cache TTL for Vault secrets | **Note:** Payment service uses koanf for configuration (same pattern as main app). Environment variables override YAML configuration values. #### Tempo Crypto Provider The payment service supports Tempo network TIP-20 stablecoins for crypto payments. Tempo uses `TransferWithMemo` events for exact payment matching via memo field (payment ID as bytes32). | Variable | Required | Default | Security | Description | | ------------------------ | -------------- | ---------------------------------------------------------------- | --------- | ----------------------------------------- | | TEMPO\_ENABLED | No | false | public | Enable Tempo crypto payment provider | | TEMPO\_RPC\_URL | No | [https://rpc.moderato.tempo.xyz](https://rpc.moderato.tempo.xyz) | public | Tempo network RPC endpoint | | TEMPO\_RECEIVER\_ADDRESS | Yes (if Tempo) | - | sensitive | Wallet address for receiving payments | | TEMPO\_CHAIN\_ID | No | 42431 | public | Tempo chain ID (42431 = Moderato testnet) | | TEMPO\_POLL\_INTERVAL | No | 10s | public | Interval for polling transfer events | | TEMPO\_TOKEN\_ALPHAUSD | No | 0x20c0...001 | public | AlphaUSD TIP-20 contract address | | TEMPO\_TOKEN\_BETAUSD | No | - | public | BetaUSD TIP-20 contract address | #### Stripe Payment Provider The payment service supports Stripe Checkout for card payments. Stripe secrets are sourced from Vault at `secret/payment-service/credentials`. | Variable | Required | Default | Security | Description | | --------------- | -------- | ------- | -------- | ------------------------------ | | STRIPE\_ENABLED | No | false | public | Enable Stripe payment provider | **Vault-Sourced Secrets:** | Vault Field | Vault Path | Description | | ----------------------- | ------------------------------------ | ------------------------------------------------------ | | `stripe_secret_key` | `secret/payment-service/credentials` | Stripe API secret key (`sk_live_...` or `sk_test_...`) | | `stripe_webhook_secret` | `secret/payment-service/credentials` | Stripe webhook signing secret (`whsec_...`) | These values are loaded by the payment service Vault client at startup and injected into the Stripe adapter configuration. They are not set via environment variables. ### Cloud Provider Credentials #### Hetzner | Variable | Required | Default | Security | Description | | ------------- | ------------- | ------- | -------- | ------------------------------------------------------- | | HCLOUD\_TOKEN | Yes (Hetzner) | - | secret | Hetzner Cloud API token for infrastructure provisioning | #### OVH OVH credentials are required only when using OVH providers (`ovh-public-cloud`, `ovh-vps`, `ovh-dedicated`). Create API credentials at: [https://api.ovh.com/createToken/](https://api.ovh.com/createToken/) | Variable | Required | Default | Security | Description | | ---------------------------- | ------------------- | ------- | --------- | --------------------------------------------------------------------- | | OVH\_ENDPOINT | No | ovh-eu | public | OVH API endpoint (ovh-eu, ovh-ca, ovh-us, kimsufi-eu, soyoustart-eu) | | OVH\_APPLICATION\_KEY | Yes (OVH) | - | secret | OVH Application Key from createToken page | | OVH\_APPLICATION\_SECRET | Yes (OVH) | - | secret | OVH Application Secret from createToken page | | OVH\_CONSUMER\_KEY | Yes (OVH) | - | secret | OVH Consumer Key from createToken page | | OVH\_CLOUD\_PROJECT\_SERVICE | Yes (Public Cloud) | - | sensitive | OVH Cloud Project Service ID (required for ovh-public-cloud provider) | | OVH\_SUBSIDIARY | Yes (VPS/Dedicated) | EU | public | OVH billing subsidiary (EU, FR, GB, DE, US, CA) | **OVH Provider Types:** | Provider | Description | Use Case | | ------------------ | --------------------------------- | -------------------------------------- | | `ovh-public-cloud` | Public Cloud instances (Beta API) | Most Hetzner-like, supports cloud-init | | `ovh-vps` | Traditional VPS (order-based) | Lower cost, no external volumes | | `ovh-dedicated` | Bare-metal dedicated servers | High performance, 2h delivery time | ### E2E Testing | Variable | Required | Default | Security | Description | | ------------------------- | -------- | ---------------------------------------------- | --------- | ----------------------------------- | | E2E\_API\_URL | No | [http://localhost:8081](http://localhost:8081) | sensitive | API base URL for E2E tests | | E2E\_GRPC\_ADDRESS | No | localhost:9091 | sensitive | gRPC address for E2E tests | | E2E\_DB\_HOST | No | localhost | sensitive | Database host for E2E tests | | E2E\_DB\_PORT | No | 5433 | public | Database port for E2E tests | | E2E\_DB\_USER | No | hoodcloud | sensitive | Database user for E2E tests | | E2E\_DB\_PASSWORD | No | e2e\_test\_password | secret | Database password for E2E tests | | E2E\_DB\_NAME | No | hoodcloud\_e2e | sensitive | Database name for E2E tests | | E2E\_TEMPORAL\_HOST | No | localhost | sensitive | Temporal host for E2E tests | | E2E\_TEMPORAL\_PORT | No | 7234 | public | Temporal port for E2E tests | | E2E\_SSH\_KEY\_PATH | No | \~/.ssh/e2e\_test\_key | sensitive | Path to SSH key for E2E VM access | | E2E\_SSH\_KEY\_NAME | No | hoodcloud-e2e-test | public | SSH key name in Hetzner | | E2E\_TEST\_REGION | No | fsn1 | public | Hetzner region for E2E tests | | E2E\_TEST\_INSTANCE\_TYPE | No | cpx22 | public | Hetzner instance type for E2E tests | | E2E\_CLEANUP\_ON\_FAILURE | No | true | public | Clean up resources on test failure | ### Docker Compose Environment | Variable | Required | Default | Security | Description | | ------------------------------ | ------------- | ----------------- | --------- | ---------------------------------------------------------------------------- | | POSTGRES\_USER | No | hoodcloud | sensitive | PostgreSQL user for Docker Compose | | POSTGRES\_PASSWORD | Yes (Compose) | - | secret | PostgreSQL password for Docker Compose | | POSTGRES\_DB | No | hoodcloud | public | PostgreSQL database name for Docker Compose | | GRAFANA\_ADMIN\_USER | No | admin | sensitive | Grafana admin username | | GRAFANA\_ADMIN\_PASSWORD | Yes | - | secret | Grafana admin password (no default — must be set explicitly) | | NGROK\_AUTHTOKEN | Conditional | - | secret | ngrok auth token (required for E2E with external VMs) | | OPS\_AGENT\_VERSION | No | latest | public | Ops agent version for orchestrator deployment | | NATS\_JWT\_ENABLED | No | false | public | Enable NATS JWT operator mode authentication | | NATS\_JWT\_CTRL\_ACCOUNT\_PUB | Conditional | - | sensitive | CTRL account public key (from nats-jwt-setup) | | NATS\_JWT\_AGENT\_ACCOUNT\_PUB | Conditional | - | sensitive | AGENT account public key (from nats-jwt-setup) | | NATS\_JWT\_EXTERNAL\_URL | Conditional | - | sensitive | Public NATS URL for ops-agents (tls\://...) | | DOMAIN\_API | No | api.localhost | public | Domain for API server reverse proxy (Caddy) | | DOMAIN\_GRAFANA | No | grafana.localhost | public | Domain for Grafana dashboard reverse proxy (Caddy) | | DOMAIN\_STATUS | No | status.localhost | public | Domain for Gatus status page reverse proxy (Caddy) | | DOMAIN\_PAY | No | pay.localhost | public | Domain for payment service reverse proxy (Caddy, docker-compose.payment.yml) | #### Payment Service (docker-compose.payment.yml) These variables are used when running the payment service via `docker compose -f docker-compose.yml -f docker-compose.payment.yml up`: | Variable | Required | Default | Security | Description | | --------------------- | -------- | ------- | --------- | ----------------------------------- | | PAYMENT\_DB\_USER | No | payment | sensitive | Payment service PostgreSQL user | | PAYMENT\_DB\_PASSWORD | Yes | - | secret | Payment service PostgreSQL password | | PAYMENT\_DB\_NAME | No | payment | public | Payment service PostgreSQL database | ## Variables by Service ### auth-server **Required:** * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME * REDIS\_HOST (for Redis-backed rate limiting) * JWT\_PRIVATE\_KEY\_FILE + JWT\_PUBLIC\_KEY\_FILE (or via Vault) * AUTH\_CLERK\_SECRET\_KEY, AUTH\_CLERK\_AUTHORIZED\_PARTY, AUTH\_CLERK\_WEBHOOK\_SIGNING\_SECRET (when using Clerk, which is the default) **Optional:** * SERVER\_HOST, SERVER\_PORT * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS, DB\_MAX\_CONN\_LIFETIME, DB\_MAX\_CONN\_IDLE\_TIME, DB\_STATEMENT\_TIMEOUT * REDIS\_PORT, REDIS\_PASSWORD, REDIS\_DB * JWT\_ACCESS\_TOKEN\_TTL, JWT\_REFRESH\_TOKEN\_TTL, JWT\_ISSUER, JWT\_AUDIENCE * AUTH\_PROVIDER (default: `clerk`) * AUTH\_RATE\_LIMIT\_PER\_MINUTE, AUTH\_ALLOWED\_ORIGINS * ENVIRONMENT, LOG\_LEVEL **Deprecated (SIWE legacy):** * SIWE\_DOMAIN, SIWE\_URI, AUTH\_NONCE\_EXPIRY (only needed when `AUTH_PROVIDER=siwe` or `both`) ### api-server **Required:** * SERVER\_PUBLIC\_URL * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME * REDIS\_HOST * TEMPORAL\_HOST * AWS\_S3\_BUCKET **Optional:** * SERVER\_HOST, SERVER\_PORT, SERVER\_READ\_TIMEOUT, SERVER\_WRITE\_TIMEOUT * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS, DB\_MAX\_CONN\_LIFETIME, DB\_MAX\_CONN\_IDLE\_TIME, DB\_STATEMENT\_TIMEOUT * REDIS\_PORT, REDIS\_PASSWORD, REDIS\_DB * TEMPORAL\_PORT, TEMPORAL\_NAMESPACE, TEMPORAL\_TASK\_QUEUE * AWS\_REGION, AWS\_ENDPOINT\_URL, AWS\_S3\_BACKUP\_PREFIX * OPS\_AGENT\_BINARIES\_DIR * CHAIN\_PROFILES\_DIR, CHAIN\_CONFIGS\_SOURCE, CHAINS\_S3\_BUCKET, CHAINS\_S3\_REGION, CHAINS\_S3\_PREFIX, CHAINS\_CHECK\_INTERVAL, CHAIN\_CONFIGS\_WATCH * JWT\_PUBLIC\_KEY\_FILE, JWT\_AUDIENCE for JWT validation (RS256) * AUTH\_PROVIDER, AUTH\_CLERK\_SECRET\_KEY, AUTH\_CLERK\_AUTHORIZED\_PARTY (when Clerk provider enabled; no webhook secret — api-server does not handle webhooks) * API\_AUTH\_ENABLED, API\_RATE\_LIMIT\_ENABLED, API\_RATE\_LIMIT\_REQUESTS * PAYMENT\_CONSUMER\_ENABLED, PAYMENT\_CONSUMER\_URL, PAYMENT\_CONSUMER\_STREAM\_NAME (for processing payment events) * ENVIRONMENT, LOG\_LEVEL ### agent-gateway **Required:** * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME * REDIS\_HOST * GRPC\_PUBLIC\_URL **Optional:** * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS, DB\_MAX\_CONN\_LIFETIME, DB\_MAX\_CONN\_IDLE\_TIME, DB\_STATEMENT\_TIMEOUT * REDIS\_PORT, REDIS\_PASSWORD, REDIS\_DB * AWS\_REGION, AWS\_ENDPOINT\_URL * GRPC\_HOST, GRPC\_PORT, GRPC\_USE\_TLS, GRPC\_CERT\_FILE, GRPC\_KEY\_FILE, GRPC\_CA\_FILE * GRPC\_CONFIG\_SIGNING\_KEY * GRPC\_RATE\_LIMIT\_RPS, GRPC\_RATE\_LIMIT\_BURST, GRPC\_RATE\_LIMIT\_CLEANUP\_INTERVAL * COMMAND\_PROGRESS\_TTL * NATS\_ENABLED, NATS\_URL, NATS\_STREAM\_NAME, NATS\_MAX\_MESSAGE\_SIZE * ENVIRONMENT, LOG\_LEVEL ### orchestrator **Required:** * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME * REDIS\_HOST * TEMPORAL\_HOST * AWS\_S3\_BUCKET * SERVER\_PUBLIC\_URL, GRPC\_PUBLIC\_URL * HCLOUD\_TOKEN (if using Hetzner provider) **Optional:** * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS, DB\_MAX\_CONN\_LIFETIME, DB\_MAX\_CONN\_IDLE\_TIME, DB\_STATEMENT\_TIMEOUT * REDIS\_PORT, REDIS\_PASSWORD, REDIS\_DB * TEMPORAL\_PORT, TEMPORAL\_NAMESPACE, TEMPORAL\_TASK\_QUEUE * AWS\_REGION, AWS\_ENDPOINT\_URL, AWS\_S3\_BACKUP\_PREFIX * GRPC\_USE\_TLS, GRPC\_CONFIG\_SIGNING\_KEY * TERRAFORM\_BINARY, TERRAFORM\_STATE\_DIR, TERRAFORM\_PARALLELISM * TERRAFORM\_STATE\_BACKEND, TERRAFORM\_S3\_BUCKET, TERRAFORM\_S3\_REGION, TERRAFORM\_DYNAMODB\_TABLE * TERRAFORM\_INIT\_TIMEOUT, TERRAFORM\_PLAN\_TIMEOUT, TERRAFORM\_APPLY\_TIMEOUT, TERRAFORM\_DESTROY\_TIMEOUT, TERRAFORM\_OUTPUT\_TIMEOUT * COMMAND\_PROGRESS\_TTL * SKIP\_TERRAFORM, DEV\_MODE * CHAIN\_PROFILES\_DIR, CHAIN\_CONFIGS\_SOURCE, CHAINS\_S3\_BUCKET, CHAINS\_S3\_REGION, CHAINS\_S3\_PREFIX, CHAINS\_CHECK\_INTERVAL, CHAIN\_CONFIGS\_WATCH * CHAIN\_CONFIGS\_ENABLED, CONFIG\_BUNDLE\_VERSION, CHAIN\_CONFIGS\_S3\_BUCKET, CHAIN\_CONFIGS\_S3\_REGION, CHAIN\_CONFIGS\_URL\_EXPIRY, CHAIN\_CONFIGS\_LOCAL\_DIR, S3\_PUBLIC\_ENDPOINT\_URL (for ops-agent config bundles) * OPS\_AGENT\_VERSION * DEFAULT\_PROVIDER, DEFAULT\_REGION * OVH\_ENDPOINT, OVH\_APPLICATION\_KEY, OVH\_APPLICATION\_SECRET, OVH\_CONSUMER\_KEY, OVH\_CLOUD\_PROJECT\_SERVICE (if using OVH providers) * ENVIRONMENT, LOG\_LEVEL ### health-evaluator **Required:** * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME * TEMPORAL\_HOST **Optional:** * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS, DB\_MAX\_CONN\_LIFETIME, DB\_MAX\_CONN\_IDLE\_TIME, DB\_STATEMENT\_TIMEOUT * TEMPORAL\_PORT, TEMPORAL\_NAMESPACE, TEMPORAL\_TASK\_QUEUE * AWS\_REGION, AWS\_S3\_BUCKET, AWS\_S3\_BACKUP\_PREFIX (for backup cleanup) * HEALTH\_EVALUATION\_INTERVAL, HEALTH\_HEARTBEAT\_TIMEOUT, HEALTH\_MIGRATION\_COOLDOWN * SUBSCRIPTION\_CLEANUP\_INTERVAL * MAINTENANCE\_CLEANUP\_INTERVAL, MAINTENANCE\_CLEANUP\_TIMEOUT * NATS\_ENABLED, NATS\_URL, NATS\_STREAM\_NAME, NATS\_CONSUMER\_NAME, NATS\_MAX\_MESSAGE\_SIZE, NATS\_STREAM\_REPLICAS * VICTORIA\_METRICS\_URL, METRICS\_CONSUMER\_NAME (for observation metrics ingestion and policy evaluation) * EVALUATION\_CONCURRENCY (for parallel chain policy evaluation) * CIRCUIT\_BREAKER\_FAILURE\_THRESHOLD, CIRCUIT\_BREAKER\_SUCCESS\_THRESHOLD, CIRCUIT\_BREAKER\_TIMEOUT (for metrics ingester protection) * CHAIN\_PROFILES\_DIR, CHAIN\_CONFIGS\_SOURCE, CHAINS\_S3\_BUCKET, CHAINS\_S3\_REGION, CHAINS\_S3\_PREFIX, CHAINS\_CHECK\_INTERVAL (for loading observation specs and policies) * ENVIRONMENT, LOG\_LEVEL **Built-in workers (no env vars needed):** * UptimeWorker — materializes rolling uptime buckets (5m interval, 500 batch size, 90-day retention). See [Deployment & Operations - Rolling Uptime](./deployment-and-operations#rolling-uptime). **Vault-sourced (incident notifications):** * `incident_slack_webhook_url` — Slack incoming webhook URL (optional, enables Slack notifications) * `incident_telegram_bot_token` — Telegram bot token (optional, enables Telegram notifications) * `incident_telegram_chat_id` — Telegram chat ID (required if bot token is set) * `incident_email_api_url` — Email delivery service API URL (optional, enables Email notifications) * `incident_email_api_key` — Email delivery service API key (required if API URL is set) * `incident_email_from` — Sender email address for incident notifications * `incident_email_to` — Recipient email address for incident notifications * `incident_webhook_url` — Outbound webhook URL (optional, enables Webhook notifications) * `incident_webhook_secret` — Shared secret sent in `X-Webhook-Secret` header for request verification These are read from Vault at `secret/hoodcloud/app-credentials`, not from environment variables. The health-evaluator initializes the incident notification pipeline at startup based on which credentials are present. ### ops-agent (on VMs) **Required:** * NODE\_ID * GRPC\_URL (or CONTROL\_PLANE\_URL for backward compatibility) **Optional:** * OPS\_AGENT\_CONFIG (path to YAML configuration file) * NODE\_TOKEN * GRPC\_USE\_TLS, GRPC\_PORT, GRPC\_ADDRESS * TLS\_CERT\_FILE, TLS\_KEY\_FILE, TLS\_CA\_FILE * CHAIN\_PROFILE\_ID, CHAIN\_DATA\_DIR, CHAIN\_RPC\_PORT, CHAIN\_TYPE * HEARTBEAT\_INTERVAL, GRACE\_PERIOD * CONFIG\_SIGNING\_PUBLIC\_KEY, REQUIRE\_CONFIG\_SIGNATURE * CONFIG\_DIR, CONFIG\_VERSION * METRICS\_ENABLED, METRICS\_PORT * OBSERVATION\_ENABLED, NATS\_URL, HOST\_ID (for observation metrics transport; NATS auth via JWT operator mode) * SYSTEM\_METRICS\_INTERVAL, SYSTEM\_METRICS\_DATA\_PATH (for system metrics collection) * ENVIRONMENT, LOG\_LEVEL ### payment-service (Isolated Microservice) **Required:** * DB\_HOST, DB\_USER, DB\_PASSWORD, DB\_NAME **Conditional (for mTLS):** * TLS\_CERT\_FILE, TLS\_KEY\_FILE, TLS\_CA\_FILE (unless TLS\_INSECURE=true) * TLS\_ALLOWED\_CN (for client certificate CN verification) **Conditional (for Vault):** * VAULT\_ADDRESS, VAULT\_ROLE\_ID, VAULT\_SECRET\_ID\_PATH * VAULT\_TLS\_CA\_FILE, VAULT\_APP\_CREDENTIALS\_PATH **Optional:** * PAYMENT\_CONFIG * GRPC\_ADDRESS, HTTP\_ADDRESS, METRICS\_ADDRESS * DB\_PORT, DB\_SSL\_MODE, DB\_MAX\_CONNS, DB\_MIN\_CONNS * NATS\_URL, NATS\_STREAM\_NAME, NATS\_CTRL\_SIGNING\_SEED, NATS\_CTRL\_ACCOUNT\_PUB * PRICING\_CONFIG\_PATH * TLS\_INSECURE (development only) * VAULT\_NAMESPACE, VAULT\_TLS\_SKIP\_VERIFY (dev only — rejected in production), VAULT\_CACHE\_TTL * OTEL\_ENABLED, OTEL\_COLLECTOR\_URL, OTEL\_SAMPLE\_RATIO * TEMPO\_ENABLED, TEMPO\_RPC\_URL, TEMPO\_RECEIVER\_ADDRESS, TEMPO\_CHAIN\_ID, TEMPO\_POLL\_INTERVAL, TEMPO\_TOKEN\_ALPHAUSD, TEMPO\_TOKEN\_BETAUSD * STRIPE\_ENABLED (Stripe secrets sourced from Vault, not environment variables) ## Security Classifications ### Secret (Must Never Be Exposed) These variables contain credentials, tokens, or cryptographic keys: * JWT\_PRIVATE\_KEY\_FILE (file content is secret) * DB\_PASSWORD * REDIS\_PASSWORD * AWS\_ACCESS\_KEY\_ID * AWS\_SECRET\_ACCESS\_KEY * VAULT\_ROLE\_ID * VAULT\_SECRET\_ID\_PATH (file content is secret) * GRPC\_KEY\_FILE * GRPC\_CONFIG\_SIGNING\_KEY * TLS\_KEY\_FILE * HCLOUD\_TOKEN * OVH\_APPLICATION\_KEY * OVH\_APPLICATION\_SECRET * OVH\_CONSUMER\_KEY * E2E\_DB\_PASSWORD * POSTGRES\_PASSWORD * GRAFANA\_ADMIN\_PASSWORD * NGROK\_AUTHTOKEN * NATS\_JWT\_CTRL\_ACCOUNT\_PUB (sensitive, not secret — public key only) * NATS\_CTRL\_SIGNING\_SEED (secret — loaded from Vault in production) * AUTH\_CLERK\_SECRET\_KEY * AUTH\_CLERK\_WEBHOOK\_SIGNING\_SECRET **Best Practices:** * Never commit to version control * Store in secure secret management systems (HashiCorp Vault) * Use environment-specific .env files (gitignored) * Rotate regularly * Use IAM roles/instance profiles instead of static credentials where possible ### Sensitive (Should Not Be Public) These variables contain internal URLs, usernames, or configuration that should not be publicly disclosed: * SERVER\_PUBLIC\_URL * GRPC\_PUBLIC\_URL * DB\_HOST, DB\_USER, DB\_NAME * REDIS\_HOST * TEMPORAL\_HOST * AWS\_S3\_BUCKET * AWS\_ENDPOINT\_URL * VAULT\_ADDR * VAULT\_MASTER\_KEY\_PATH, VAULT\_APP\_CREDENTIALS\_PATH * VAULT\_TLS\_CA\_FILE * JWT\_PUBLIC\_KEY\_FILE * GRPC\_CERT\_FILE, GRPC\_CA\_FILE * TLS\_CERT\_FILE, TLS\_CA\_FILE * NODE\_ID * CONTROL\_PLANE\_URL, GRPC\_URL * CONFIG\_SIGNING\_PUBLIC\_KEY * LOKI\_URL, PROMETHEUS\_URL, VICTORIA\_METRICS\_URL * NATS\_URL, NATS\_PUBLIC\_URL * OPS\_AGENT\_DOWNLOAD\_URL * S3\_PUBLIC\_ENDPOINT\_URL * E2E\_API\_URL, E2E\_GRPC\_ADDRESS, E2E\_DB\_HOST, E2E\_DB\_USER, E2E\_DB\_NAME, E2E\_TEMPORAL\_HOST, E2E\_SSH\_KEY\_PATH * POSTGRES\_USER * GRAFANA\_ADMIN\_USER * CHAIN\_CONFIGS\_S3\_BUCKET * OVH\_CLOUD\_PROJECT\_SERVICE * TERRAFORM\_S3\_BUCKET * TERRAFORM\_DYNAMODB\_TABLE ### Public (Safe to Commit) These variables contain non-sensitive configuration: * All timeouts, intervals, and thresholds * Port numbers * Feature flags (API\_AUTH\_ENABLED, NATS\_ENABLED, etc.) * Default values and limits * ENVIRONMENT, LOG\_LEVEL * Public configuration like CHAIN\_PROFILES\_DIR ## Production Checklist > **See:** [Deployment Checklist](./deployment-checklist) for the full pre-deployment checklist covering configuration, authentication, security, and infrastructure validation. ## Variable Reference by File ### Source Code References **internal/config/config.go** * Authoritative source for all environment variable mappings * Implements koanf-based configuration with env var overrides * TerraformConfig: timeout fields (init, plan, apply, destroy, output) * CommandsConfig: progress\_ttl field **internal/config/loader.go** * Environment variable to config path mappings * TERRAFORM\_\*\_TIMEOUT and COMMAND\_PROGRESS\_TTL mappings **internal/opsagent/config.go** * Ops agent configuration loading with YAML support * LoadFromFile() for YAML configuration * LoadConfig() with OPS\_AGENT\_CONFIG support * All NODE\_ID, GRPC\_URL, CHAIN\_\* variables **cmd/api-server/main.go** * ENVIRONMENT variable usage for logging * HTTP-only API server (gRPC moved to agent-gateway) **cmd/agent-gateway/main.go** * gRPC server for ops-agent communication * Uses COMMAND\_PROGRESS\_TTL for Redis progress storage * GRPC\_\* variables for server configuration **cmd/orchestrator/main.go** * ENVIRONMENT variable usage * HCLOUD\_TOKEN direct usage (passed to Terraform) * TERRAFORM\_\*\_TIMEOUT variables for provisioning operations * COMMAND\_PROGRESS\_TTL for progress reader **cmd/health-evaluator/main.go** * ENVIRONMENT variable usage * HOODCLOUD\_CONFIG file path loading **internal/logging/logger.go** * LOG\_LEVEL parsing and configuration **internal/terraform/runner.go** * Timeout configuration from Config struct * Falls back to defaults.Terraform.\* when not configured **internal/commandqueue/progress.go** * ProgressStore with configurable TTL * Uses functional options pattern (WithProgressTTL) * Falls back to defaults.Commands.ProgressTTL **tests/e2e/helpers/environment.go** (Lines 58-79) * All E2E\_\* test configuration variables ### Configuration File References **docker-compose.dev.yml** * Development environment defaults * All service environment configurations **tests/e2e/docker-compose.e2e.yml** * E2E testing environment * Isolated ports and test-specific configuration **infrastructure/docker/docker-compose.yml** * Production-like Docker Compose setup **.env.example** * Template with all user-configurable variables **config/hoodcloud.yaml** * YAML configuration structure * Default values for all settings ## Troubleshooting ### Missing Required Variables If services fail to start with missing variable errors: 1. Check `.env` file exists and contains required variables 2. Verify environment variables are exported in your shell 3. For Docker Compose, ensure `.env` is in the same directory as `docker-compose.yml` 4. Check service-specific required variables in "Variables by Service" section ### Configuration Not Applied If environment variables seem to be ignored: 1. YAML file values take precedence unless overridden by environment variables 2. Check `HOODCLOUD_CONFIG` is pointing to the correct file 3. Verify variable names match exactly (case-sensitive) 4. For Docker, check environment variables are passed through in docker-compose.yml ### Connection Issues If services can't connect to each other: 1. Verify hostnames match service names in Docker Compose 2. Check port numbers match between services 3. Ensure URLs include protocol (http\://, nats\://, etc.) 4. For external VMs, ensure \*\_PUBLIC\_URL variables are set to externally-accessible addresses ## Migration Guide ### From Environment-Only to YAML + Environment 1. Create `config/hoodcloud.yaml` with your base configuration 2. Set environment variables only for values that differ from YAML or are secrets 3. Set `HOODCLOUD_CONFIG=config/hoodcloud.yaml` 4. Restart services ### From Development to Production 1. Copy `.env.example` to `.env.production` 2. Replace all placeholder values with production values 3. Move all secrets to HashiCorp Vault 4. Configure Vault AppRole authentication 5. Enable security features (TLS, auth, rate limiting) 6. Set appropriate timeouts and intervals for production scale