nexus/routing
Domain agent orchestration
python · azure functions · semantic kernel · azure openai · reciprocal rank fusion · asyncio
[problem]
One retrieval strategy couldn't reliably serve the range of enterprise questions. Compliance, product, facility, network, and pricing queries each needed a different approach — without sacrificing latency, resilience to partial failures, or a safe rollout path.
[architecture]
An LLM router classifies intent into one of five domains through an explicit state machine. Every routing decision lands in an enumerated, testable outcome. Domain agents retrieve in parallel, results merge through Reciprocal Rank Fusion, and timeouts preserve partial results instead of zeroing out the response.
Chose deterministic routing over free-form agent planning — traded flexibility for auditability and testability.
One tradeoff was accepted deliberately: a measured +7% in end-to-end latency bought an 18% reduction in response-generation time. That's the part of the wait users actually feel.
[measured]
- 69%→95%routing accuracy, 170-question golden benchmark[verified in prod]
- 50%→86%end-to-end retrieval success; errors and empty results eliminated[measured]
- 5.3s→1.4sretrieval-pipeline latency, median — a 74% cut, same benchmark[measured]
- 32%→0%fallback to the legacy path, after a router fix[measured]
- 360automated tests[verified]
First validated in shadow mode; the 2026 rebuild found the routing ceiling was architectural, made the fast-paths advisory, shipped dark behind feature flags, and passed an acceptance gate against production data before the flip — verified live the same day. Now the platform's retrieval backbone.