Configuration Guide¶
Doc Version: 1.0.2
Last Updated: 2026-05-21 Git Commit: 61384b4a
Author: Lincoln
JAiRouter provides flexible configuration options to meet various deployment scenarios. This guide covers all configuration aspects from basic setup to advanced features.
Modular Configuration Overview¶
Starting from v1.0.0, JAiRouter adopts a modular configuration structure, separating different functional configurations into independent configuration files to improve maintainability and readability.
Configuration File Structure¶
src/main/resources/
├── application.yml # Main config: only the spring.config.import list
├── config/
│ ├── common/ # server / webclient / logging / scheduling / cluster
│ ├── config-service/core.yml # Config service (dynamic-config backend)
│ ├── router/ # adapter / loadbalancer / ratelimit / circuitbreaker /
│ │ # fallback / services / rules / pools
│ ├── auth/ # jwt / api-key / audit / sanitization
│ ├── monitor/ # slow-query-alerts / error-tracking /
│ │ # persistence-monitoring-base
│ ├── tracing/tracing-base.yml # Tracing
│ ├── persistence/state-persistence-base.yml # State persistence
│ ├── base/ # server-base / model-services-base / monitoring-base
│ └── security/ # security-base / persistence-base
├── application-dev.yml # Development profile
├── application-staging.yml # Staging profile
├── application-prod.yml # Production profile
├── application-migrate.yml # Data migration profile
└── application-legacy.yml # Backward-compatibility profile
Configuration Import Mechanism¶
The main configuration file application.yml imports various module configurations through the spring.config.import mechanism:
# application.yml — the authoritative list (28 entries)
spring:
config:
import:
- classpath:config/common/server.yml
- classpath:config/common/webclient.yml
- classpath:config/common/logging.yml
- classpath:config/common/scheduling.yml
- classpath:config/common/cluster.yml
- classpath:config/config-service/core.yml
- classpath:config/router/adapter.yml
- classpath:config/router/loadbalancer.yml
- classpath:config/router/ratelimit.yml
- classpath:config/router/circuitbreaker.yml
- classpath:config/router/fallback.yml
- classpath:config/router/services.yml
- classpath:config/router/rules.yml
- classpath:config/router/pools.yml
- classpath:config/auth/jwt.yml
- classpath:config/auth/api-key.yml
- classpath:config/auth/audit.yml
- classpath:config/auth/sanitization.yml
- classpath:config/base/monitoring-base.yml
- classpath:config/monitor/slow-query-alerts.yml
- classpath:config/monitor/error-tracking.yml
- classpath:config/monitor/persistence-monitoring-base.yml
- classpath:config/tracing/tracing-base.yml
- classpath:config/persistence/state-persistence-base.yml
- classpath:config/base/server-base.yml
- classpath:config/base/model-services-base.yml
- classpath:config/security/security-base.yml
- classpath:config/security/persistence-base.yml
Note: the tree and the list above are checked against the repository rather than being an illustrative sketch. If they drift,
src/main/resources/application.ymlwins.
This approach makes the configuration clearer and easier to maintain and extend.
Configuration Overview¶
JAiRouter supports two main configuration approaches:
- Static Configuration: Defined in YAML files, loaded at startup
- Dynamic Configuration: Updated at runtime via REST APIs
Configuration Hierarchy¶
Configuration is loaded in the following order (later sources override earlier ones):
- Default configuration (embedded in JAR)
application.yml(classpath)./application.yml(current directory)./config/application.yml(config directory)- Environment variables
- Command line arguments
- Dynamic configuration (runtime updates)
Basic Structure¶
server:
port: 8080
model:
services:
<service-type>:
load-balance:
type: <strategy>
rate-limit:
algorithm: <algorithm>
# ... rate limit settings
circuit-breaker:
enabled: true
# ... circuit breaker settings
fallback:
strategy: <fallback-type>
# ... fallback settings
instances:
- name: <model-name>
baseUrl: <service-url>
path: <api-path>
weight: <load-balance-weight>
# ... instance-specific settings
store:
type: <storage-backend>
# ... storage settings
Service Types¶
JAiRouter supports the following service types:
| Service Type | Description | Default Path |
|---|---|---|
chat | Chat completions | /v1/chat/completions |
embedding | Text embeddings | /v1/embeddings |
rerank | Text reranking | /v1/rerank |
tts | Text-to-speech | /v1/audio/speech |
stt | Speech-to-text | /v1/audio/transcriptions |
imgGen | Image generation | /v1/images/generations |
imgEdit | Image editing | /v1/images/edits |
Configuration Sections¶
1. Load Balancing¶
Configure how requests are distributed across service instances:
model:
services:
chat:
load-balance:
type: round-robin # random, round-robin, least-connections, ip-hash
Available Strategies:
- random: Random selection of instances
- round-robin: Sequential rotation through instances
- least-connections: Route to instance with fewest active connections
- ip-hash: Consistent routing based on client IP hash
2. Rate Limiting¶
Control request rates to prevent service overload:
model:
services:
chat:
rate-limit:
algorithm: token-bucket
capacity: 100 # Maximum tokens in bucket
rate: 10 # Tokens added per second
client-ip-enable: true # Enable per-client-IP rate limiting
Available Algorithms:
- token-bucket: Allow bursts up to bucket capacity
- leaky-bucket: Smooth, constant rate limiting
- sliding-window: Rate limit over time windows
- warm-up: Gradually increase rate limit
3. Circuit Breaking¶
Prevent cascading failures with circuit breaker pattern:
model:
services:
chat:
circuit-breaker:
enabled: true
failure-threshold: 5 # Failures before opening circuit
recovery-timeout: 30000 # Time before attempting recovery (ms)
success-threshold: 3 # Successes needed to close circuit
4. Fallback Strategies¶
Define fallback behavior when services are unavailable:
model:
services:
chat:
fallback:
strategy: default
message: "Service temporarily unavailable"
# OR
strategy: cache
cache-ttl: 300000 # Cache TTL in milliseconds
Available Types:
- default: Return predefined message
- cache: Return cached responses
- none: No fallback (return error)
5. Service Instances¶
Define the actual service endpoints:
model:
services:
chat:
instances:
- name: "qwen2.5:7b"
baseUrl: "http://server1:11434"
path: "/v1/chat/completions"
weight: 2
timeout: 30000
headers:
Authorization: "Bearer token"
Custom-Header: "value"
- name: "qwen2.5:14b"
baseUrl: "http://server2:11434"
path: "/v1/chat/completions"
weight: 1
Instance Properties:
name: Model name identifierbaseUrl: Service base URLpath: API endpoint pathweight: Load balancing weight (higher = more traffic)timeout: Request timeout in millisecondsheaders: Custom headers to include in requests
6. Storage Configuration¶
Dynamic configuration is persisted to a database through JPA. store.type accepts exactly two values: jpa (default) and h2 — both are the same JPA implementation, and h2 only means "JPA is pointed at H2", not a different storage engine. Any other value is rejected at startup by ConfigIntegrityValidator:
store:
type: jpa # jpa (production) or h2 (embedded, dev and single node)
spring:
datasource:
url: jdbc:postgresql://localhost:5432/jairouter
username: jairouter
password: ${DB_PASSWORD}
hikari:
maximum-pool-size: 10
minimum-idle: 2
connection-timeout: 30000
max-lifetime: 1800000
Those four pool values are the explicitly written defaults (config/config-service/core.yml). Read the measured guidance in Database deployment before changing them — a bigger pool does not raise throughput.
There is no "automatic backup" switch: keys such as store.path, auto-backup and backup-interval do not exist. Backup and restore are operational actions (pg_dump, or copying the H2 data file while stopped); see Database deployment and Storage configuration.
Environment-Specific Configuration¶
JAiRouter supports multiple environment configuration files:
- Development Environment:
application-dev.yml - Staging Environment:
application-staging.yml - Production Environment:
application-prod.yml - Compatibility Mode:
application-legacy.yml - Security Example:
application-security-example.yml
Environment configuration files only contain the differences from the base configuration, following Spring Boot's configuration override mechanism.
Required environment variables per profile¶
| Profile | Required | Behavior when missing |
|---|---|---|
dev / default | none | The JWT secret and the admin password fall back to random values and are printed once in the startup log (they change on every restart) |
staging | JWT_SECRET | jairouter.security.jwt.secret resolves to an empty string and binds without error (the length check was moved out of the binding annotation into runtime validation, see #821). staging is not covered by the startup gate, so the process starts; every JWT issue/validation then fails with WeakKeyException because the key is too short |
prod | JWT_SECRET + INITIAL_ADMIN_PASSWORD | The startup gate (StartupSecurityGate) refuses to boot |
Every profile reads only JWT_SECRET and, for the admin password, INITIAL_ADMIN_PASSWORD. The three per-profile aliases DEV_JWT_SECRET / STAGING_JWT_SECRET / PROD_JWT_SECRET and the secondary name ADMIN_PASSWORD were removed (#674): they were read in a different place than the configuration binding, so setting both made the checked credential differ from the effective one. Startup logs a warning when a removed name is still set.
Environment Variables¶
Override configuration using environment variables, for example:
# Server port (relaxed binding to server.port)
export SERVER_PORT=8080
# Disable JWT authentication (equivalent to jairouter.security.jwt.enabled=false)
export JAiROUTER_SECURITY_JWT_ENABLED=false
# Point at an external Redis (equivalent to spring.data.redis.host / port)
export SPRING_DATA_REDIS_HOST=redis.internal
export SPRING_DATA_REDIS_PORT=6379
For how variable names map to configuration keys, which variables are mandatory, and the per-profile defaults, see Environment Variables Reference.
The
MODEL_LOAD_BALANCE_TYPE/MODEL_RATE_LIMIT_TYPEvariables shown in older versions of this page do not exist in the configuration tree; setting them has no effect.
List-typed settings cannot be overridden item by item through environment variables¶
model.services.<type>.instances is a list. Overriding a single item through an environment variable makes Spring Boot replace the whole list with a one-element list containing only the overridden field; every other field is left null:
# Intended: turn off rate limiting for the first chat instance only
# Actual: instances becomes a single stub entry with only rate-limit set, name and baseUrl are null
export MODEL_SERVICES_CHAT_INSTANCES_0_RATELIMIT_ENABLED=false
Since v3.6.1 the application stops at startup when an instance has no name or baseUrl, printing which service and which position is broken, instead of failing later on NULL not allowed for column "base_url".
To change instance-level settings, either edit the configuration file (config/router/services.yml) or supply the complete instance list, with name, base-url and the other fields filled in for every entry.
Command Line Arguments¶
Override configuration via command line:
java -jar jairouter.jar \
--server.port=8080 \
--jairouter.security.audit.log-level=DEBUG \
--spring.profiles.active=prod
Configuration Validation¶
JAiRouter validates configuration at startup. Common validation errors:
Missing Required Fields¶
# ❌ Invalid - missing baseUrl
model:
services:
chat:
instances:
- name: "model"
path: "/v1/chat/completions"
# ✅ Valid
model:
services:
chat:
instances:
- name: "model"
baseUrl: "http://localhost:11434"
path: "/v1/chat/completions"
Invalid Configuration Values¶
# ❌ Invalid - unsupported load balance type
model:
services:
chat:
load-balance:
type: invalid-type
# ✅ Valid
model:
services:
chat:
load-balance:
type: round-robin
Configuration Best Practices¶
1. Use Meaningful Names¶
# ✅ Good - descriptive names
model:
services:
chat:
instances:
- name: "qwen2.5-7b-fast"
baseUrl: "http://fast-gpu-server:11434"
- name: "qwen2.5-14b-accurate"
baseUrl: "http://high-memory-server:11434"
2. Set Appropriate Timeouts¶
# ✅ Good - reasonable timeouts
model:
services:
chat:
instances:
- name: "model"
baseUrl: "http://server:11434"
timeout: 30000 # 30 seconds for chat
embedding:
instances:
- name: "embedding"
baseUrl: "http://server:11434"
timeout: 10000 # 10 seconds for embeddings
3. Configure Health Checks¶
# ✅ Good - declare the probe mode and cadence (defaults live in config/router/health-check.yml)
jairouter:
health-check:
mode: http # the default; tcp = port only, http also checks the status code (<500 is healthy)
http-path: /v1/models # probe path for http mode; 404/405 also count as healthy (process is serving)
interval-ms: 30000 # wait this long after the previous round finished
timeout-ms: 5000 # per-instance probe timeout
parallelism: 4 # how many service types are checked concurrently
failure-threshold: 2 # consecutive failed rounds before an instance is evicted (#855)
The older
model.load-balance.health-check.*block (enabled / interval / timeout / failure-threshold / success-threshold) was never bound, so it had no effect; since v3.5.4 health checking is controlled byjairouter.health-check.*.
4. Use Weights for Gradual Rollouts¶
# ✅ Good - gradual rollout with weights
model:
services:
chat:
instances:
- name: "stable-model-v1"
baseUrl: "http://stable-server:11434"
weight: 9 # 90% traffic
- name: "new-model-v2"
baseUrl: "http://new-server:11434"
weight: 1 # 10% traffic
Next Steps¶
- Application Configuration - Detailed application settings
- Dynamic Configuration - Runtime configuration management
- Load Balancing - Load balancing strategies
- Rate Limiting - Rate limiting algorithms
- Circuit Breaker - Circuit breaker configuration
- Storage Configuration - Storage and auto-merge configuration
- Version Management - Configuration version management: Apply vs Rollback
- JWT Persistence Configuration - JWT token persistence and security audit configuration
- KV Cache - Prefix-cache-friendly gateway: sticky routing and cache metrics (v2.9.0)
- Response Cache - Full response cache: deterministic requests reuse the cached response and skip the downstream; P1 adds streaming SSE cache + invalidation API + rate-limit short-circuit (v2.9.10)