Triple Search: Why Legal RAG Needs Keyword, Semantic and Metadata Retrieval

One question, three kinds of search

Watch a lawyer use a research tool and you will see three kinds of question, often in the same sentence:

• Exact: a statute article, a case reference or a defined legal term that must match precisely.
• Conceptual: "When is an employer liable for an employee's misconduct?", phrased in words that may never appear in the judgment.
• Structural: "Only from this court of appeal, only since 2020, and show me what happened on appeal."

No single search method handles all three. When I architected Datagreffe Justice, an AI platform over more than 50,000 French-language court decisions, the starting point was clear: keyword search was failing, and lawyers needed answers that understood legal meaning, cited the exact decision and passage, and followed the chain of appeals.

The answer was triple search: keyword, semantic and metadata retrieval, fused into a single ranking. The fused ranking retrieved 15% better than any single method on its own. Here is why each leg exists and how they fit together.

Leg one: keyword search with BM25

BM25 is the classic ranking function behind full-text search engines. It scores a passage by how often the query terms appear in it, weighted by how rare those terms are across the whole corpus.

In legal research, exact matching is a feature, not a limitation. Statute numbers, case references, Latin maxims and defined terms must match precisely. A lawyer searching for a specific article does not want passages about a neighboring article that merely looks similar.

What makes keyword search work on legal text:

• Language-aware analysis. French text needs handling for elisions such as "l'article" and "d'appel", for accents and for plural forms, or obvious matches are missed.
• Exact fields for references. Article numbers and case references should be indexed in a form that is never stemmed or split apart.
• Phrase matching. Many legal concepts are multi-word terms whose meaning changes when the words are separated.

Its weakness is paraphrase. If the judgment says "dismissal" and the lawyer types "termination", BM25 sees two unrelated words.

Leg two: semantic vector search

Semantic search turns passages and questions into embeddings: vectors that place similar meanings close together. It finds the passage that answers a question even when the two share almost no words.

On Datagreffe Justice, semantic search runs on Pinecone. Documents are split into 512-token chunks with a 50-token overlap, so context that crosses a boundary is not lost. I benchmarked 256, 512 and 1,024-token chunks rather than assuming a default, and 512 with overlap worked best for dense, structured legal paragraphs. Across the corpus, that produces around 200,000 vectors.

Semantic search has the opposite weakness to BM25. Embeddings are good at meaning and weak at exact identifiers. "Article 12" and "Article 21" can look almost identical to an embedding model, and a confident near-miss is the worst kind of result in legal work, because it looks right.

Leg three: metadata and decision-chain search

The third leg is structured search over facts about each decision: the court, the date, the jurisdiction and the decision's place in an appeal chain. On Datagreffe Justice it runs on Elasticsearch, which is built for exactly this kind of filtering.

Metadata does two jobs:

• Filtering. "Only this court, only since 2020" narrows the candidates before ranking, which improves both precision and speed.
• Context. Results can be grouped or boosted by court level or recency when the question calls for it.

The decision chain deserves its own mention. A first-instance judgment may have been confirmed, modified or overturned on appeal, and then reviewed again by a higher court. Depending on the subject matter, the final word may come from a national supreme court or, for disputes under the OHADA uniform acts, from the Common Court of Justice and Arbitration. A research tool that returns first-instance reasoning without showing what happened next can lead a lawyer to rely on a ruling that no longer stands. Linking decisions into chains at ingestion time lets the system follow a case through its appeals.

Good metadata depends on good extraction. Court names, dates and case references have to be pulled reliably from each document during ingestion, and checked, because a wrong date silently removes a decision from every filtered search.

Fusing three result lists into one ranking

Each leg returns its own ranked list, and their scores are not comparable. A BM25 score and a cosine similarity live on different scales, so simply adding them up rewards whichever leg happens to produce bigger numbers.

The common fusion options are:

• Rank-based fusion, such as reciprocal rank fusion, which combines positions rather than raw scores and is robust to differences in scale.
• Normalized, weighted scores, where each leg's scores are rescaled and weighted, sometimes differently for different types of question.
• A reranking stage, where a reranker model rescores the top candidates from all legs together.

In practice, the metadata leg often does double duty: it filters the candidates the other two legs consider, and it contributes matches of its own, such as decisions linked through an appeal chain. Query-aware weighting helps too. If a question contains an article number or a case reference, a cheap pattern check can detect it and give the keyword leg more weight.

Which combination wins is an empirical question, and the answer depends on your corpus and on how your users ask questions. That is why the evaluation set comes before the tuning.

Evaluate retrieval separately from generation

When a RAG answer is wrong, the first question to ask is whether the right passage was retrieved at all. If it wasn't, no prompt will fix the answer.

For Datagreffe Justice, 200 question-and-answer pairs validated with a lawyer gated every architecture change. These are the retrieval measures I track:

• Whether the expected decision appears in the top results, and how high it ranks
• Results broken down by question type: exact references, conceptual questions and filtered questions
• Whether filters ever exclude a correct result
• Latency for each leg and for the fused result

The breakdown by question type matters. A change that helps conceptual questions can quietly hurt exact-reference questions, and an overall average hides it.

Three searches, still under two seconds

Running three searches sounds slow, but it doesn't have to be. The legs can run in parallel, metadata filters cut the candidate sets early, and only a small number of top candidates go forward to fusion and generation. Datagreffe Justice returns cited answers in under 2 seconds end to end, with three-tier role-based access control enforced at query time.

Access control belongs inside retrieval, not after it. If permissions are only applied in the interface, a document the user is not allowed to see can still shape the answer the model writes.

When you don't need triple search

Not every system needs all three legs. A support assistant over a few hundred well-written help articles may do fine with semantic search alone, or with a keyword and semantic hybrid.

You need the full triple when:

• Users search for exact references, codes or identifiers
• Questions are phrased differently from the source text
• Users filter by structured facts such as court, date, region or document type
• Documents relate to one another, as decisions do through appeals or contracts do through amendments
• A confident wrong answer has real professional or legal consequences

Legal research ticks every box. So do many compliance, regulatory and financial document collections.

Building retrieval that lawyers trust

If your users search case law, contracts, regulations or any large professional corpus, retrieval design is where answer quality is won or lost. Our RAG development services page explains how we design and evaluate hybrid retrieval, and the Datagreffe Justice case study shows triple search in production, alongside anonymization and AI drafting. For the build story in more depth, read how I built Datagreffe Justice.