Technical Deep Dive: RAG & Hybrid Search
Embedding-model selection, document indexing, BM25 retrieval, vector search, reranking, and relevance evaluation for ESS Explorer.
← Back to the client-facing case study
This supplement describes the hybrid retrieval stack that was built. It does not invent benchmark scores.
1. Scope and retrieval requirements
The ESS Explorer corpus comprised approximately 10,000 documents, many tens of pages long. The need was to locate relevant passages and preserve their source context for users searching across lengthy technical and reference material.
A retrieval system needed to address two search patterns:
- Semantic questions: when the query and relevant text describe the same concept using different words.
- Exact-match questions: identifiers, names, phrases, numbers and specialized vocabulary where semantic similarity alone can be insufficient.
The quality requirement is passage relevance and traceability, not simply whether a language model can produce a convincing response.
2. Document ingestion and source-aware chunking
The documented retrieval flow extracted and prepared document text, organized it into searchable chunks and associated metadata such as document ID, title, source, page/section and chunk location.
Chunking is a quality decision: oversized chunks can dilute relevance and consume context; overly small chunks can omit the language needed to interpret a passage. Structure-aware boundaries and provenance are particularly valuable for PDFs containing sections, tables and repeated terminology.
Source provenance matters: search results should retain sufficient metadata for a user to return to the originating document and verify a claim.
3. Embedding benchmark: three real candidates
The GenAI Lab benchmark compared three open-source sentence-transformer models. The objective was to select a usable retrieval model for this particular corpus, rather than assuming that the smallest or largest model was automatically best.
| Model | Dimensions | Relative finding | Outcome |
|---|---|---|---|
all-MiniLM-L6-v2 |
384 | Fastest/lowest-cost baseline; lower retrieval relevance | Not selected |
all-MiniLM-L12-v2 |
384 | Intermediate quality; still below MPNet | Not selected |
all-mpnet-base-v2 |
768 | Best relative relevance in internal validation, at acceptable footprint | Selected |
Evaluations considered Top-K retrieval quality, MRR, nDCG-style relevance checks, query-time latency and cost/footprint. Numeric Recall@K, MRR and nDCG scores are not in the shareable source report, so this document does not fabricate them.
Because index generation is largely an offline process, embedding throughput was less important than the relevance/latency/cost balance of live retrieval. MPNet’s larger vectors increased storage and search footprint compared with 384-dimensional candidates, but the tradeoff was acceptable for the corpus.
4. Vector retrieval as the evidenced foundation
The documented core architecture followed a standard but testable retrieval pattern:
- Parse and chunk source documents, attaching traceable metadata.
- Generate passage embeddings with the selected model.
- Index vectors in a searchable database.
- Embed each user query with the same model.
- Retrieve and rank relevant passages by similarity, retaining source references.
This provided the evaluated semantic-search layer underpinning the product direction. Generated answers may use those passages as context, but the supplied model card supports retrieval capability; it does not establish a separately measured, fully deployed end-to-end answer-generation system.
5. Hybrid retrieval
The search stack combined:
- BM25 keyword retrieval: match exact terms and keyword-rich passages.
- Vector similarity: retrieve semantically related passages when terms do not align.
- Reciprocal rank fusion (RRF): merge the two candidate rankings without assuming each method produces directly comparable raw scores.
- Reranking: rescore the combined candidate set so the most relevant documents rise to the top.
A query with an exact identifier and a conceptual description can use both lexical precision and semantic coverage. Metadata filters can narrow candidates by document attributes before ranking.
No numeric result in the shareable materials shows how much the hybrid stack improved on vector-only search. The architecture is what was built; a measured lift is not claimed here.
6. Evaluation design and failure modes
The useful retrieval metrics were Recall@K or Top-K hit rates (is relevant evidence found?), MRR (how highly does the first relevant item appear?), nDCG@K (how well ordered are results with graded relevance?), latency and indexing/query cost.
A solid evaluation process compares retrieval configurations on the same query–relevance judgments, including exact identifiers, conceptual queries and difficult near-matches. Any claim that hybrid search outperformed vector-only retrieval should be backed by comparable measured results—not assumed from architecture alone.
Additional production considerations include ingestion quality, PDF tables, metadata completeness, document updates, permissions-aware filtering and the risk of unsupported generated answers downstream.
7. Delivered evidence, unresolved details, next evaluation
| Area | Supported by project materials | To confirm / measure |
|---|---|---|
| Corpus | ~10,000 documents | Exact current indexed volume |
| Embedding selection | Three candidates; MPNet selected | Numeric Top-K/MRR/nDCG outputs if publication is allowed |
| Semantic search | Embedded passages + metadata-backed retrieval foundation | Product deployment/user adoption details |
| Hybrid architecture | BM25 keyword retrieval combined with vector search, then RRF | Measured lift versus vector-only search |
| Reranking | Reranker applied to the combined candidate set | Marginal gain versus fusion alone |
| Business value | Improved search capabilities as intended use | Measured task time savings or user efficiency evidence |
The important lesson is to treat retrieval as a measurable product component. Index design, embedding choice, rank fusion and citations each affect whether a search or RAG workflow earns users’ trust.
Technical themes: Python · document parsing · sentence-transformers · vector indexing · metadata provenance · BM25 · RRF · reranking · Recall@K · MRR · nDCG · retrieval latency/cost.