REPRODUCIBLE EVALUATION SUITE (2026-10-15)

Live E2E Benchmarks: Ristria v1.5 vs Tavily & Baseline

Every figure below was measured on live network traffic on 2026-10-14/15 and stored as raw JSON. Tavily figures come from its MCP server on the same queries; the baseline is Ristria's previous local cascade. Methodology and raw results are available on request.

WARM SERPCACHE HIT

2 ms

Normalized SHA-256 L1/L2 cache

COLD SEARCH + CHUNKS

1.08s – 2.34s

Includes live 3-page BM25 extraction

MULTILINGUAL P@5 (IT)

5/5 vs 0/5

Ristria vs Tavily on query s03 (Italian technical docs)

FALSE POSITIVES (s04)

0 / 5

Clean abstention on invented products

HEAD-TO-HEAD LATENCY & PRECISION@5

Complete 8-Workload Evaluation Ledger

Re-run Queries in Playground

ID	WORKLOAD	TAVILY MCP	TAVILY P@5	RISTRIA v1.5	RISTRIA P@5 & CHUNKS	BASELINE	RESULT
s01	Technical Search EN (Python asyncio TaskGroup exception handling)	1,940 ms	5/5 (docs.python.org at #4)	1,926 ms	5/5 (anyio #1, docs.python.org #2 + 2 chunks)	5,142 ms	-62.5% vs baseline
s02	Academic Paper (Cormack, Clarke, Buettcher SIGIR 2009 RRF)	1,196 ms	5/5	1,929 ms	5/5 (research.google #2, dl.acm.org #3, 0 errors)	7,744 ms	-75.1% vs baseline
s03	Italian Technical Docs (Google Search Central sitemap robots.txt)	1,709 ms	0/5 (off-topic results)	2,340 ms	5/5 (developers.google.com IT #1 + 3 chunks)	4,056 ms	5/5 vs 0/5 Tavily
s04	Negative Control (Invented product "Velrion QZ-9274")	1,462 ms	5 False Positives (unrelated manuals)	777 ms	0 False Positives (status: empty)	5,668 ms	-86.3% vs baseline
s05	News Search + Freshness Filter (category: news, time_range: week)	Not compared	Not compared	747 ms	5/5 dated news within the weekly freshness window	Unsupported	Sub-800ms News
s06	Token Budget Packing (max_tokens: 600)	Not compared	Not compared	1,078 ms	docs.python.org #1 packed inside 600 tokens	Unsupported	Exact context cap
e01	Query-Aware Passage Extraction (docs.python.org asyncio-task)	324 ms	1,732 chars extracted	247 ms	3 BM25 chunks (ExceptionGroup, CancelledError)	479 ms	23.7% faster than Tavily
HTTP	Shared SerpCache Repeat Query (POST /v1/search normalized SHA-256)	Not compared	Not compared	2 ms	cache_hit: true (0 upstream calls)	4,000+ ms	0 upstream calls

HYBRID RANKING PIPELINE

Why Ristria Ranks Official Docs Above SEO Slop

After retrieval, Ristria applies three deterministic ranking stages:

1. URL-Aware Okapi BM25 + Docs Domain Boost

Scores title, URL and snippet tokens with Okapi BM25 and boosts official documentation domains such as docs.*, developers.* and *.readthedocs.io.

2. Multiplicative Recency × Proof-Count Boost

Fresher pages and URLs confirmed by several independent providers are promoted multiplicatively, so one noisy source cannot dominate the ranking.

3. Entity-Overlap Graph Score & Token Budget Packing

Pages sharing technical entities with the query rank higher; results are then packed best-first into max_tokens, skipping oversized chunks to fit shorter ones behind them.

CASE STUDY // MULTILINGUAL FAILURE ANALYSIS (TASK s03)

Query s03: Italian Technical Documentation

When queried with "Google Search Central sitemap robots.txt documentazione italiano":

TAVILY MCP OUTPUT (0/5 RELEVANT — 1,709 ms)

None of the 5 results were Italian robots.txt or sitemap documentation; the term txt was matched to unrelated entertainment and government pages.

RISTRIA v1.5 OUTPUT (5/5 RELEVANT — 2,340 ms)

Passed explicit Italian locale parameters to the search providers, preferred the official documentation domain via include_domains_mode: "prefer", and extracted the exact Sitemap: https://example.com/my_sitemap.xml directive into highlights at Rank #1.