Consilium — a Citation-Gated Multi-RAG Switchboard
Routes a question across subject-specialized knowledge modules, binds every claim to a source span, and abstains instead of guessing — measured head-to-head against a naive RAG baseline on the identical corpus.
- 0/5 vs 5/5
- unsafe outputs vs. a naive-RAG baseline measured same cases, corpus, and embedder; zero in-scope recall cost (11/11 both)
- 1.000
- routing + fan-out + abstention + citation accuracy measured eval.py
- 11/11
- integrity-hardening checks (conflict resolution + poison quarantine) measured
Context
A single vector store answers everything with the same confidence, whether the question is in its wheelhouse or nowhere near it. Consilium is the opposite bet: route the question to the module that should own it, make every claim cite a source, and say “I don’t know” when nothing qualifies.
What I built
- A routing switchboard that dispatches a query to subject-specialized modules (and fans out when more than one applies).
- Citation-gating — a claim ships only if it is bound to a retrieved span.
- Abstention — out-of-scope questions are refused, not hallucinated.
- Integrity hardening — conflict resolution across modules and quarantine of poisoned or contradictory sources.
Why it matters
Run head-to-head on the identical corpus, a naive RAG baseline produced five unsafe outputs out of five (out-of-scope answers plus a fabricated claim); Consilium produced zero — and gave up no in-scope recall to do it. Routing, fan-out, abstention, and citation accuracy all measured 1.000 on this demo corpus and threshold set — the repo documents their provenance and does not claim they generalize beyond it. It is the spine the other retrieval projects here plug into.