MetaNaviT: Agentic RAG over Research Files
Find the current file, trace the evidence, and review proposed changes before an agent applies them.
Result & current limits
SciFact, 300 queries: hybrid retrieval plus reranking improves paired nDCG@10 by 0.075 (95% bootstrap CI 0.053–0.101) over BM25. The fp16/cap-512 GPU configuration cuts in-memory benchmark query p50 from 923 to 191 ms with no statistically detected ranking change.
Problem
Research folders contain active configs, archived drafts, logs, and checkpoints. Finding a plausible answer is easier than identifying the current file and returning traceable evidence.
Solution
Combined BM25 and dense retrieval with reciprocal rank fusion, query routing, staleness filtering, and source tracing. Exposed search and proposed filesystem changes through an MCP interface.
What I built
Contributed retrieval and data APIs on the original six-person team. Independent 2026 extensions add experiment-file provenance, external retrieval evaluations, service-load measurements, and signed approval grants for file operations.
Technologies
More demos, engineering decisions, and evaluation notes
Follow the answer back to a file
The walkthrough starts with conflicting copies of a configuration. Retrieval surfaces the current source and its location; the user can inspect that evidence instead of relying on an unsupported answer.
Make proposed changes reviewable
Mutation tools return a plan before application. Current file operations require a signed, single-use approval grant after plan review. SciFact experiments use actual embedding and reranking backends, evaluated separately from the illustrated retrieval traces.
Test across different corpora
October 7: on 323 NFCorpus queries, hybrid nDCG@10 is 0.3637 versus BM25 at 0.3207; adding the reranker reduces it to 0.3555. On 648 FiQA queries, dense retrieval leads hybrid, 0.3848 versus 0.3645. The best retrieval recipe depends on the corpus.
Retrieve the right experiment file
Adding path context raises Recall@10 from 0.058 to 0.671 on 240 generated robotics questions with disjoint experiment families. This measures source-file retrieval, not answer accuracy. A separate H100 HTTP check records p95 of 252/422/1319 ms at concurrency 1/4/16, with only 16 measured requests per level.
Approval grants under attack
Each approval grant is signed, short-lived, single-use, and bound to one exact reviewed change. In deterministic enforcement tests, all 220 malicious fixtures across 22 attack families were denied, with the corpus and files outside it unchanged, and all 100 benign controls succeeded. These tests check the enforcement code, not a model’s resistance to prompt injection. Results.