nexus/routing
Domain agent orchestration
python · azure functions · semantic kernel · azure openai · reciprocal rank fusion · asyncio
[problem]
One retrieval strategy couldn't reliably serve the range of enterprise questions. Compliance, product, facility, network, and pricing queries each needed a different approach — without sacrificing latency, resilience to partial failures, or a safe rollout path.
[architecture]
An LLM router classifies intent into one of five domains through an explicit state machine — every routing decision lands in an enumerated, testable outcome rather than free-form agent planning. Domain agents retrieve in parallel, results merge through Reciprocal Rank Fusion, and timeouts preserve partial results instead of zeroing out the response.
Chose deterministic routing over free-form agent planning — traded flexibility for auditability and testability.
One tradeoff was accepted deliberately: a measured +7% in end-to-end latency bought an 18% reduction in response-generation time — improving the part of the wait users actually feel.
[measured]
- 88% measured routing accuracy [measured]
- Fallback to the legacy path reduced from 32% to 0% after a router fix [measured]
- Parallel router classification saves 500–1,500 ms per query [measured]
- 360 automated tests [verified]
Rolled out behind feature flags and validated in shadow mode — the new path ran alongside the legacy path, with per-prompt deltas, before activation. Now live in production as the platform's retrieval backbone.