nexus/routing
Domain agent orchestration
python · azure functions · semantic kernel · azure openai · reciprocal rank fusion · asyncio
[problem]
One retrieval strategy couldn't reliably serve the range of enterprise questions. Compliance, product, facility, network, and pricing queries each needed a different approach — without sacrificing latency, resilience to partial failures, or a safe rollout path.
[architecture]
An LLM router classifies intent into one of five domains through an explicit state machine. Every routing decision lands in an enumerated, testable outcome. Domain agents retrieve in parallel, results merge through Reciprocal Rank Fusion, and timeouts preserve partial results instead of zeroing out the response.
Chose deterministic routing over free-form agent planning — traded flexibility for auditability and testability.
One tradeoff was accepted deliberately: a measured +7% in end-to-end latency bought an 18% reduction in response-generation time. That's the part of the wait users actually feel.
[measured]
- 88% measured routing accuracy [measured]
- Fallback to the legacy path reduced from 32% to 0% after a router fix [measured]
- Parallel router classification saves 500–1,500 ms per query [measured]
- 360 automated tests [verified]
Rolled out behind feature flags and validated in shadow mode — the new path ran alongside the legacy path, with per-prompt deltas, before activation. Now live in production as the platform's retrieval backbone.