Multi-Jurisdiction PII Anonymization for Legal Documents with LLMs

Why legal documents are a hard anonymization problem

Most PII detection tools are built for forms and customer records, where an email address sits neatly in an email field. Court decisions are different. Personal data is woven through long, dense prose: the parties and their relatives, witnesses, addresses, identification numbers, employers, bank details, health information and the circumstances of the dispute itself.

On Datagreffe Justice, an enterprise legal AI platform I architected, more than 50,000 French-language court decisions had to be anonymized before any of them could be indexed or published. The decisions included information about minors and vulnerable people, and the requirements came from data-protection rules and from the OHADA legal standards that apply across several jurisdictions.

The anonymization engine reached 98% PII detection accuracy, and the platform met 100% of its data-protection and OHADA compliance requirements. This article explains how I approach the problem and the decisions that matter most if you are planning something similar.

What "multi-jurisdiction" really means

OHADA harmonizes business law across 17 African member states, but harmonized business law does not mean harmonized rules for publishing court decisions. Requirements can differ by country, by court and by type of case.

Before writing any code, I work with the client's legal team to answer the same set of questions for each jurisdiction:

• Which categories of personal data must be removed?
• Should a name be deleted, reduced to initials or replaced with a consistent pseudonym?
• Do the names of judges, lawyers and court officials stay visible?
• How are companies and other legal entities treated?
• Which extra protections apply to minors and vulnerable parties?
• Which metadata, such as the court, date and case number, may remain?

The answers become a versioned rule set for each jurisdiction. I don't decide what the law requires; the client's legal team does. My job is to turn their decisions into rules that a pipeline can apply and a test suite can check.

It is also worth being precise about terms. Under the GDPR, pseudonymized data is still personal data, because it can be linked back to a person with additional information. Many legal publishing workflows are really pseudonymization plus the removal of identifying details, and the compliance target should state which one is required.

Why rules alone and LLMs alone both fail

There are three families of detection methods, and each has a blind spot:

• Pattern rules reliably catch structured identifiers: national ID and passport numbers, phone numbers, email addresses and bank account numbers. They miss names written in context and anything that doesn't follow a fixed format.
• Named-entity recognition (NER) finds people, places and organizations, but struggles with OCR noise, unusual names, names that are also common words, and the formal register of legal French.
• LLMs understand context. They can tell that "the appellant's employer" or "the only pharmacy in the village" points to a specific person, even with no name in sight. But they can be inconsistent when asked to rewrite text freely.

That is why I layer all three, so that one method's blind spot is covered by another. On Datagreffe Justice, the engine was built on Claude and guided by jurisdiction-specific rules.

The pipeline I recommend

1. Extract and normalize the text

Decisions arrive as PDFs and Word documents, and sometimes as scans. Text extraction, OCR where needed, and normalization of whitespace, accents and hyphenated line breaks come first. A name split across two lines by a hyphen is a name a detector may miss.

2. Detect with layered methods

Pattern rules run first on structured identifiers. NER and the LLM then work through the text in overlapping sections, so an entity that falls on a section boundary is never lost.

I ask the model for structured output: each span, its category and the reason it qualifies, rather than a rewritten document. Structured detections can be checked, measured and applied deterministically. A rewritten document can quietly change words you needed to keep.

3. Resolve each person across the document

The same person may appear as "Mr. Koné", "Paul Koné", "the defendant" and "his client". If each mention gets a different placeholder, the decision becomes unreadable, and the legal reasoning depends on knowing who did what.

Linking those mentions into one entity, with one consistent pseudonym per person in each decision, keeps the text useful to lawyers while removing the identity.

4. Apply the jurisdiction's rules

Only now does the jurisdiction's rule set decide what happens to each entity: remove it, replace it with a pseudonym, generalize it (a full address becomes a city) or keep it. Separating detection from the rules means a change in the rules doesn't require re-engineering detection.

5. Apply stricter rules for minors and vulnerable people

Minors and vulnerable parties get stricter rules and dedicated checks, as they did on Datagreffe Justice. Indirect identifiers matter most here: a school, a small village or a family relationship can identify a child as surely as a name. When in doubt, the rules for these groups lean toward removal.

6. Verify, and route uncertainty to people

A verification pass looks for anything the first pass missed, such as a name that survived in a heading or a quoted letter. Low-confidence detections go to a human review queue with the context needed to decide quickly, so the automation never silently guesses.

7. Index only anonymized text

Anonymization happens before indexing, so personal data never reaches a search index, an embedding or a generation prompt. Compliance doesn't depend on a downstream filter remembering to do its job.

Using an LLM safely inside an anonymization pipeline

If an LLM does part of the detection, the anonymization stage is the one place where raw text is processed. Treat it as the most sensitive component in the system:

• Choose a deployment that matches the data. Options include enterprise API terms, models served through Amazon Bedrock or Azure OpenAI under your existing cloud agreements, or open-weight models hosted in your own environment when documents cannot leave it.
• Keep generation settings deterministic, so the same document produces the same result every time.
• Log decisions without logging the personal data itself.
• Version prompts and rules together, so every anonymized document can be traced to the exact configuration that produced it.

Measuring accuracy: what the numbers should mean

An accuracy figure is only useful if you know what it measures. I measure detection on labelled samples of the client's own documents, broken down by category, because missing a passport number and missing a city name are not equally serious.

Two kinds of error matter:

• Missed personal data is a compliance failure, and minimizing it is the priority.
• Over-redaction is a usefulness failure. If the pipeline removes court names, statute references or dates that the legal analysis depends on, lawyers stop trusting the output.

Track both, and rerun the evaluation whenever rules, prompts or models change. New decisions keep arriving, so anonymization is a continuous process, not a one-off clean-up.

Common mistakes I see

• Anonymizing at display time. If personal data is already in the index, it can leak through search results, embeddings or logs.
• Inconsistent placeholders. Different labels for the same person make a decision impossible to follow.
• Forgetting the edges. File names, document metadata, headers, footers and signatures often contain names.
• Ignoring OCR errors. A misread name doesn't match a dictionary or a pattern, so it slips through.
• No human review path. Without one, uncertain cases are either guessed or skipped, and both are worse than a quick review.

Planning an anonymization project

If you hold court decisions, contracts, medical records or any document set that must be shared or made searchable without exposing people, start with two things: a clear compliance target for each jurisdiction and a labelled sample of your own documents. Everything else follows from those.

Our document AI and anonymization services page describes how we design these pipelines, and the Datagreffe Justice case study shows the full platform: 50,000+ court decisions, 98% PII detection and 100% data-protection and OHADA compliance. For the retrieval side of the same platform, read how I built Datagreffe Justice, and if you work in the sector, see how we support legal teams.