Skip to content

JAiRouter¶

AI Model Service Unified Gateway
Connect to all models with a single line of code

🎯 Core Value¶

Feature	Description
🔌 Unified API	OpenAI-compatible interface, seamlessly connect to Ollama, vLLM, GPUStack, etc.
⚖️ Smart Load Balancing	5 strategies for automatic request distribution, health checks to remove failed nodes
🛡️ Traffic Control	Rate limiting, circuit breaking, degradation for service stability
🔐 Dual Authentication	JWT + API Key with audit logs and blacklist support
📊 Full-chain Tracing	OpenTelemetry integration, performance bottlenecks at a glance
💾 State Persistence	Redis / file storage, supports high-availability cluster deployment

🏗️ Architecture Overview¶

┌─────────────────┐     ┌─────────────────────┐     ┌─────────────────┐
│   OpenAI SDK    │────▶│     JAiRouter       │────▶│    Ollama       │
│   Langchain     │     │  • Load Balancing   │     │    vLLM         │
│   HTTP Client   │     │  • Rate Limiting    │     │    GPUStack     │
└─────────────────┘     │  • Authentication   │     │    Xinference   │
                        └─────────────────────┘     └─────────────────┘

⚡ 1-Minute Quick Start¶

# Start the service
docker run -d --name jairouter -p 8080:8080 sodlinken/jairouter:latest

# Access the console
open http://localhost:8080

I want to...	Recommended Docs
Get started quickly	Quick Start Guide
Deploy in production	Deployment Guide
Configure services	Configuration Guide
Integrate via API	API Reference
Set up monitoring	Monitoring Guide
Troubleshoot issues	Troubleshooting

📈 Project Status¶

LTS Release: v2.6.11 is the final Long-Term Support version, maintained until 2028-05

🔧 Supported Backends¶

Backend	Description
GPUStack	GPUStack private model service
Ollama	Local LLM runtime
vLLM	High-performance LLM inference engine
Xinference	Distributed model inference framework
LocalAI	OpenAI-compatible local inference
OpenAI	Official OpenAI API

🤝 Community Support¶

GitHub Issues - Bug reports
GitHub Discussions - Discussions
Contributing Guide - Join development

If JAiRouter helps you, please give it a ⭐️ Star!