Skip to content

Configuration Guide

Doc Version: 1.0.2
Last Updated: 2026-05-21 Git Commit: 61384b4a
Author: Lincoln

JAiRouter provides flexible configuration options to meet various deployment scenarios. This guide covers all configuration aspects from basic setup to advanced features.

Modular Configuration Overview

Starting from v1.0.0, JAiRouter adopts a modular configuration structure, separating different functional configurations into independent configuration files to improve maintainability and readability.

Configuration File Structure

src/main/resources/
├── application.yml                    # Main config: only the spring.config.import list
├── config/
│   ├── common/                        # server / webclient / logging / scheduling / cluster
│   ├── config-service/core.yml        # Config service (dynamic-config backend)
│   ├── router/                        # adapter / loadbalancer / ratelimit / circuitbreaker /
│   │                                  # fallback / services / rules / pools
│   ├── auth/                          # jwt / api-key / audit / sanitization
│   ├── monitor/                       # slow-query-alerts / error-tracking /
│   │                                  # persistence-monitoring-base
│   ├── tracing/tracing-base.yml       # Tracing
│   ├── persistence/state-persistence-base.yml   # State persistence
│   ├── base/                          # server-base / model-services-base / monitoring-base
│   └── security/                      # security-base / persistence-base
├── application-dev.yml                # Development profile
├── application-staging.yml            # Staging profile
├── application-prod.yml               # Production profile
├── application-migrate.yml            # Data migration profile
└── application-legacy.yml             # Backward-compatibility profile

Configuration Import Mechanism

The main configuration file application.yml imports various module configurations through the spring.config.import mechanism:

# application.yml — the authoritative list (28 entries)
spring:
  config:
    import:
      - classpath:config/common/server.yml
      - classpath:config/common/webclient.yml
      - classpath:config/common/logging.yml
      - classpath:config/common/scheduling.yml
      - classpath:config/common/cluster.yml
      - classpath:config/config-service/core.yml
      - classpath:config/router/adapter.yml
      - classpath:config/router/loadbalancer.yml
      - classpath:config/router/ratelimit.yml
      - classpath:config/router/circuitbreaker.yml
      - classpath:config/router/fallback.yml
      - classpath:config/router/services.yml
      - classpath:config/router/rules.yml
      - classpath:config/router/pools.yml
      - classpath:config/auth/jwt.yml
      - classpath:config/auth/api-key.yml
      - classpath:config/auth/audit.yml
      - classpath:config/auth/sanitization.yml
      - classpath:config/base/monitoring-base.yml
      - classpath:config/monitor/slow-query-alerts.yml
      - classpath:config/monitor/error-tracking.yml
      - classpath:config/monitor/persistence-monitoring-base.yml
      - classpath:config/tracing/tracing-base.yml
      - classpath:config/persistence/state-persistence-base.yml
      - classpath:config/base/server-base.yml
      - classpath:config/base/model-services-base.yml
      - classpath:config/security/security-base.yml
      - classpath:config/security/persistence-base.yml

Note: the tree and the list above are checked against the repository rather than being an illustrative sketch. If they drift, src/main/resources/application.yml wins.

This approach makes the configuration clearer and easier to maintain and extend.

Configuration Overview

JAiRouter supports two main configuration approaches:

  1. Static Configuration: Defined in YAML files, loaded at startup
  2. Dynamic Configuration: Updated at runtime via REST APIs

Configuration Hierarchy

Configuration is loaded in the following order (later sources override earlier ones):

  1. Default configuration (embedded in JAR)
  2. application.yml (classpath)
  3. ./application.yml (current directory)
  4. ./config/application.yml (config directory)
  5. Environment variables
  6. Command line arguments
  7. Dynamic configuration (runtime updates)

Basic Structure

server:
  port: 8080

model:
  services:
    <service-type>:
      load-balance:
        type: <strategy>
      rate-limit:
        algorithm: <algorithm>
        # ... rate limit settings
      circuit-breaker:
        enabled: true
        # ... circuit breaker settings
      fallback:
        strategy: <fallback-type>
        # ... fallback settings
      instances:
        - name: <model-name>
          baseUrl: <service-url>
          path: <api-path>
          weight: <load-balance-weight>
          # ... instance-specific settings

store:
  type: <storage-backend>
  # ... storage settings

Service Types

JAiRouter supports the following service types:

Service TypeDescriptionDefault Path
chatChat completions/v1/chat/completions
embeddingText embeddings/v1/embeddings
rerankText reranking/v1/rerank
ttsText-to-speech/v1/audio/speech
sttSpeech-to-text/v1/audio/transcriptions
imgGenImage generation/v1/images/generations
imgEditImage editing/v1/images/edits

Configuration Sections

1. Load Balancing

Configure how requests are distributed across service instances:

model:
  services:
    chat:
      load-balance:
        type: round-robin  # random, round-robin, least-connections, ip-hash

Available Strategies:

  • random: Random selection of instances
  • round-robin: Sequential rotation through instances
  • least-connections: Route to instance with fewest active connections
  • ip-hash: Consistent routing based on client IP hash

2. Rate Limiting

Control request rates to prevent service overload:

model:
  services:
    chat:
      rate-limit:
        algorithm: token-bucket
        capacity: 100          # Maximum tokens in bucket
        rate: 10               # Tokens added per second
        client-ip-enable: true # Enable per-client-IP rate limiting

Available Algorithms:

  • token-bucket: Allow bursts up to bucket capacity
  • leaky-bucket: Smooth, constant rate limiting
  • sliding-window: Rate limit over time windows
  • warm-up: Gradually increase rate limit

3. Circuit Breaking

Prevent cascading failures with circuit breaker pattern:

model:
  services:
    chat:
      circuit-breaker:
        enabled: true
        failure-threshold: 5      # Failures before opening circuit
        recovery-timeout: 30000   # Time before attempting recovery (ms)
        success-threshold: 3      # Successes needed to close circuit

4. Fallback Strategies

Define fallback behavior when services are unavailable:

model:
  services:
    chat:
      fallback:
        strategy: default
        message: "Service temporarily unavailable"
        # OR
        strategy: cache
        cache-ttl: 300000  # Cache TTL in milliseconds

Available Types:

  • default: Return predefined message
  • cache: Return cached responses
  • none: No fallback (return error)

5. Service Instances

Define the actual service endpoints:

model:
  services:
    chat:
      instances:
        - name: "qwen2.5:7b"
          baseUrl: "http://server1:11434"
          path: "/v1/chat/completions"
          weight: 2
          timeout: 30000
          headers:
            Authorization: "Bearer token"
            Custom-Header: "value"
        - name: "qwen2.5:14b"
          baseUrl: "http://server2:11434"
          path: "/v1/chat/completions"
          weight: 1

Instance Properties:

  • name: Model name identifier
  • baseUrl: Service base URL
  • path: API endpoint path
  • weight: Load balancing weight (higher = more traffic)
  • timeout: Request timeout in milliseconds
  • headers: Custom headers to include in requests

6. Storage Configuration

Dynamic configuration is persisted to a database through JPA. store.type accepts exactly two values: jpa (default) and h2 — both are the same JPA implementation, and h2 only means "JPA is pointed at H2", not a different storage engine. Any other value is rejected at startup by ConfigIntegrityValidator:

store:
  type: jpa                  # jpa (production) or h2 (embedded, dev and single node)
spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/jairouter
    username: jairouter
    password: ${DB_PASSWORD}
    hikari:
      maximum-pool-size: 10
      minimum-idle: 2
      connection-timeout: 30000
      max-lifetime: 1800000

Those four pool values are the explicitly written defaults (config/config-service/core.yml). Read the measured guidance in Database deployment before changing them — a bigger pool does not raise throughput.

There is no "automatic backup" switch: keys such as store.path, auto-backup and backup-interval do not exist. Backup and restore are operational actions (pg_dump, or copying the H2 data file while stopped); see Database deployment and Storage configuration.

Environment-Specific Configuration

JAiRouter supports multiple environment configuration files:

  • Development Environment: application-dev.yml
  • Staging Environment: application-staging.yml
  • Production Environment: application-prod.yml
  • Compatibility Mode: application-legacy.yml
  • Security Example: application-security-example.yml

Environment configuration files only contain the differences from the base configuration, following Spring Boot's configuration override mechanism.

Required environment variables per profile

ProfileRequiredBehavior when missing
dev / defaultnoneThe JWT secret and the admin password fall back to random values and are printed once in the startup log (they change on every restart)
stagingJWT_SECRETjairouter.security.jwt.secret resolves to an empty string and binds without error (the length check was moved out of the binding annotation into runtime validation, see #821). staging is not covered by the startup gate, so the process starts; every JWT issue/validation then fails with WeakKeyException because the key is too short
prodJWT_SECRET + INITIAL_ADMIN_PASSWORDThe startup gate (StartupSecurityGate) refuses to boot

Every profile reads only JWT_SECRET and, for the admin password, INITIAL_ADMIN_PASSWORD. The three per-profile aliases DEV_JWT_SECRET / STAGING_JWT_SECRET / PROD_JWT_SECRET and the secondary name ADMIN_PASSWORD were removed (#674): they were read in a different place than the configuration binding, so setting both made the checked credential differ from the effective one. Startup logs a warning when a removed name is still set.

Environment Variables

Override configuration using environment variables, for example:

# Server port (relaxed binding to server.port)
export SERVER_PORT=8080

# Disable JWT authentication (equivalent to jairouter.security.jwt.enabled=false)
export JAiROUTER_SECURITY_JWT_ENABLED=false

# Point at an external Redis (equivalent to spring.data.redis.host / port)
export SPRING_DATA_REDIS_HOST=redis.internal
export SPRING_DATA_REDIS_PORT=6379

For how variable names map to configuration keys, which variables are mandatory, and the per-profile defaults, see Environment Variables Reference.

The MODEL_LOAD_BALANCE_TYPE / MODEL_RATE_LIMIT_TYPE variables shown in older versions of this page do not exist in the configuration tree; setting them has no effect.

List-typed settings cannot be overridden item by item through environment variables

model.services.<type>.instances is a list. Overriding a single item through an environment variable makes Spring Boot replace the whole list with a one-element list containing only the overridden field; every other field is left null:

# Intended: turn off rate limiting for the first chat instance only
# Actual: instances becomes a single stub entry with only rate-limit set, name and baseUrl are null
export MODEL_SERVICES_CHAT_INSTANCES_0_RATELIMIT_ENABLED=false

Since v3.6.1 the application stops at startup when an instance has no name or baseUrl, printing which service and which position is broken, instead of failing later on NULL not allowed for column "base_url".

To change instance-level settings, either edit the configuration file (config/router/services.yml) or supply the complete instance list, with name, base-url and the other fields filled in for every entry.

Command Line Arguments

Override configuration via command line:

java -jar jairouter.jar \
  --server.port=8080 \
  --jairouter.security.audit.log-level=DEBUG \
  --spring.profiles.active=prod

Configuration Validation

JAiRouter validates configuration at startup. Common validation errors:

Missing Required Fields

# ❌ Invalid - missing baseUrl
model:
  services:
    chat:
      instances:
        - name: "model"
          path: "/v1/chat/completions"

# ✅ Valid
model:
  services:
    chat:
      instances:
        - name: "model"
          baseUrl: "http://localhost:11434"
          path: "/v1/chat/completions"

Invalid Configuration Values

# ❌ Invalid - unsupported load balance type
model:
  services:
    chat:
      load-balance:
        type: invalid-type

# ✅ Valid
model:
  services:
    chat:
      load-balance:
        type: round-robin

Configuration Best Practices

1. Use Meaningful Names

# ✅ Good - descriptive names
model:
  services:
    chat:
      instances:
        - name: "qwen2.5-7b-fast"
          baseUrl: "http://fast-gpu-server:11434"
        - name: "qwen2.5-14b-accurate"
          baseUrl: "http://high-memory-server:11434"

2. Set Appropriate Timeouts

# ✅ Good - reasonable timeouts
model:
  services:
    chat:
      instances:
        - name: "model"
          baseUrl: "http://server:11434"
          timeout: 30000  # 30 seconds for chat
    embedding:
      instances:
        - name: "embedding"
          baseUrl: "http://server:11434"
          timeout: 10000  # 10 seconds for embeddings

3. Configure Health Checks

# ✅ Good - declare the probe mode and cadence (defaults live in config/router/health-check.yml)
jairouter:
  health-check:
    mode: http          # the default; tcp = port only, http also checks the status code (<500 is healthy)
    http-path: /v1/models  # probe path for http mode; 404/405 also count as healthy (process is serving)
    interval-ms: 30000  # wait this long after the previous round finished
    timeout-ms: 5000    # per-instance probe timeout
    parallelism: 4      # how many service types are checked concurrently
    failure-threshold: 2  # consecutive failed rounds before an instance is evicted (#855)

The older model.load-balance.health-check.* block (enabled / interval / timeout / failure-threshold / success-threshold) was never bound, so it had no effect; since v3.5.4 health checking is controlled by jairouter.health-check.*.

4. Use Weights for Gradual Rollouts

# ✅ Good - gradual rollout with weights
model:
  services:
    chat:
      instances:
        - name: "stable-model-v1"
          baseUrl: "http://stable-server:11434"
          weight: 9  # 90% traffic
        - name: "new-model-v2"
          baseUrl: "http://new-server:11434"
          weight: 1  # 10% traffic

Next Steps