API (FastAPI, Docker) → Hybrid search (RRF fusion)
Hybrid search (RRF fusion) → Claude (search_result blocks)
What it finds
Measured on 40 reference questions in French and English, 32 of which have their answer somewhere in the documentation's 421 passages; for each, the right file and heading are known.
Search
Right passage 1st
in the first 3
in the first 5
MRR
Keywords (BM25)
0.41
0.56
0.66
0.51
Meaning (vectors)
0.31
0.63
0.75
0.49
Hybrid (both, fused)
0.53
0.69
0.81
0.62
Share of the questions with a right passage among the first results; MRR: mean of 1 / rank of the first right passage. Embeddings: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2, run locally. Recomputed by the CI whenever the documentation changes.
What it answers
Prototype: the evaluation of Claude's answers (right citations, right refusals) is written and tested with a fake model, but has not been run, for want of an API key; the CI's "answers" job will produce it with a key.
Tech stack
Python
FastAPI
Qdrant
Claude
pytest
Docker
Who does what
Python
Ingestion (split by headings, lines kept), hybrid search and evaluation.
FastAPI
The search and question routes, request validation and the OpenAPI documentation.
Qdrant
The vector database: the passages' multilingual embeddings, searched by similarity.
Claude
Writes the answer from the retrieved passages only, each sentence cited (search_result blocks).
pytest
Tests of chunking, search and citations, with a fake model, on Linux, Windows and macOS.
Docker
The API and Qdrant, started by docker compose.
Definition of Done
Code reviewed, diff by diff
Unit and integration tests green
Green CI on Linux, Windows and macOS when compiled