A lawyer asks the assistant “what does article 12 of the labour law say about notice periods?” in Arabic. Three things make this harder than a typical RAG demo. The same Arabic word can be written several ways. A law is a tree of books, chapters and articles, and “article 12” means nothing without the law it belongs to. And a confident wrong answer about the law is worse than no answer at all.
This is how retrieval works in Nibras, the multi-tenant legal platform I lead (the architecture overview covers the rest of the system). It runs inside the same ASP.NET Core API on .NET 10 and the same PostgreSQL database as everything else, with pgvector for embeddings. There is no separate vector database. Snippets are simplified rewrites, not the production source.
In short:
- Fold Arabic spelling variants the same way on both sides of the search, but keep the original text for display.
- Chunk legislation by article, and only accept an “article” heading when its number is plausible.
- If the question names an article and a law, look it up exactly before any vector search.
- Run vector and keyword search together, with a similarity floor calibrated on real questions.
- Put the firm and case filter in the SQL, on every read path.
- Rebuild the index in the background, debounced, and never leave it half-built.
- If no source is found, the turn stops instead of answering law from memory.
Where the law comes from
Platform staff maintain a national library of Jordanian law inside the product: the official gazette issues, the legislation published in each issue, the articles of each law as a tree, court decisions, and the relationships between texts. It is global reference data. No firm owns it, every firm reads it, and only staff write it.
The tree has no hard-coded vocabulary. Node types (book, part, chapter, article) are rows of data, each declaring which parent types it accepts and whether it can contain legal text, so the engine never branches on the Arabic word for “article”. Moving a node is refused if it would make a node its own ancestor, which is a simple prefix test on the stored path.
Relationships between texts (repeals, amends, replaces, suspends, interprets, relies on, refers to) are proposed by a citation analyzer and confirmed by a person. The analyzer is a deliberately deterministic set of regular expressions: it finds references such as “law number N of year Y” or “article N”, picks the nearest relating verb in the 60 characters before the citation, defaults to “refers to”, and returns the evidence snippet. It never writes anything. The design note in the code says it plainly: a regex that misses a citation is visibly wrong, while a model that invents a repeal poisons the chain of authority. So a human confirms each proposal, and the database refuses duplicates and self-links.
When retrieval returns a piece of legislation, it carries the law’s number, year and status label with it, so the answer can say which law it is quoting and whether that law is still in force.
Normalizing Arabic, twice
Arabic text from scanned gazettes and PDFs arrives messy: presentation forms, invisible direction marks, decorative stretching, inconsistent hamza, diacritics on some words and not others. A lawyer typing a question will spell things one way; the gazette may spell them another.
There are two levels of cleaning, and keeping them separate matters.
The display clean is applied to everything stored and to the question before it is embedded. It applies Unicode NFKC (which folds the Arabic presentation forms that PDF extraction produces), removes bidirectional and zero-width marks and tatweel, and tidies whitespace. It keeps the diacritics, because a lawyer reading a quoted article should see the text as it was published.
The search fold goes further, and is applied to the question for keyword matching:
// Simplified: the search fold. Stored text keeps its diacritics for display;
// the same fold is applied to the indexed column at query time.
static string FoldForSearch(string text)
{
var s = DisplayClean(text); // NFKC, bidi and zero-width marks, tatweel
s = Regex.Replace(s, "[ً-ٰٟ]", ""); // harakat and the superscript alef
s = s.Replace('أ', 'ا').Replace('إ', 'ا').Replace('آ', 'ا').Replace('ٱ', 'ا')
.Replace('ى', 'ي') // alef maqsura to yaa
.Replace('ة', 'ه'); // taa marbuta to haa
s = ArabicIndicDigitsToAscii(s); // ٠-٩ and ۰-۹ to 0-9
return Regex.Replace(s, @"\s+", " ").Trim();
}The stored text is not folded. Instead, the keyword query applies the same fold to the column inside SQL, with a regex replace for the marks and a translate() for the letters. That keeps one copy of the text, readable, while both sides of a keyword match meet in the middle. Hamza on waw and yaa is not folded.
The exact rules are a product decision as much as a technical one. Fold too little and searches miss; fold too much and different words collide.
Chunking a statute by article
Every chunk is at most 1,800 characters, and the chunker picks a strategy by source type. Court decisions split at their section headings (the facts, the grounds of appeal, the decision). General documents are chunked as prose with a small sentence overlap. Legislation is chunked by article, and that is where most of the care went.
The chunker reads a law line by line. Heading lines for book, part, chapter and section set a heading path such as “Part 2 > Chapter 3”, and a short line right after a heading is merged in as its title. A new heading resets the levels below it.
Article headings are the trap. Laws refer to other articles constantly, and sometimes a line starts with exactly the same words as a real article heading: “Article 7 of this law applies…”. Splitting there would cut an article in half and attach the second half to the wrong number. So a candidate heading is only accepted when its number is plausible:
// Simplified: "المادة (12)" starts a new article only if 12 is a plausible next number.
// A cross-reference at the start of a line usually is not.
static bool IsNextArticle(int number, bool repeated, int? previous, bool partialDocument)
{
if (repeated) return previous == number; // "مكرر": the same number again
if (previous is null) return partialDocument || number <= 3;
return number > previous && number <= previous + 5;
}If no article heading is accepted at all, the document falls back to prose chunking rather than producing one giant chunk. Text before the first article becomes a preamble chunk.
Long articles are split at their clause markers (a letter or number followed by a dash, dot or bracket). If the article has a short lead-in sentence before its clauses (“the employer shall:”), that lead-in is repeated at the top of every part, so a chunk containing clause (c) still says what clause (c) is a clause of. With no clause markers, the split falls back to sentences, then to word boundaries. Parts are labelled “part i of n”.
What gets embedded is not just the chunk. Each chunk is prefixed with a header built from the law’s title, number and year, the heading path, and the article number. The embedding of “the employer shall give one month’s notice” then sits close to questions about that specific law, not just about notice periods in general. Page numbers from text extraction are mapped back onto each chunk so an answer can cite a page when one is known.
Indexing without a half-built index
Staff edit laws in bursts: fix a title, adjust three articles, add a relationship. Re-embedding the whole library after every keystroke would be wasteful, and running two rebuilds at once would be worse.
Every write path that touches legislation calls one debounce helper. It schedules a single background job three minutes out and ignores further requests until that job starts. If nine minutes pass without the job starting, the next request treats it as lost and schedules a new one, so a dropped job heals itself. Edits made while a rebuild is running schedule the next one.
The rebuild is all or nothing. It embeds every chunk first. If any embedding call fails, it abandons the run and the existing index stays in place. Only when every vector is ready does it delete the old chunks and insert the new ones in one transaction. Jobs retry three times and never run concurrently. A separate bootstrap job runs every 15 minutes and rebuilds only if the index is empty or holds vectors of mixed dimensions. Single items and a firm’s own case material are reindexed individually when they change.
On the database side, pgvector’s HNSW index has a dimension limit: 2,000 for vector, 4,000 for halfvec. The embeddings here are 2,048-dimensional, so the column stores full-precision vectors and the index is built on an expression that casts them to half precision:
-- Simplified: full precision stored, half precision indexed.
CREATE INDEX ON chunks USING hnsw ((embedding::halfvec(2048)) halfvec_cosine_ops);A startup task pins the column to a fixed dimension only when every stored row agrees on one, and builds or rebuilds the index when the operator class or dimension is wrong. A query vector of a different size is rejected rather than compared.
The query path
Retrieval runs in a fixed order, and the first step that can answer precisely wins.
Parsing. The question is search-folded, then a parser looks for an article number (with an optional “repeated” marker) and a law name introduced by words like “of”, “under” or “according to”, followed by law, regulation, constitution, instructions or decision. Without an article reference it still tries to pick out a standalone law name, trimmed at question words and capped at eight words. Keyword tokens drop Arabic stop words.
Resolving the law. A law name typed by a user rarely matches the official title character for character. It is turned into a pattern that tolerates hamza forms, alef maqsura and taa marbuta variants, and optional diacritics or tatweel after any letter, and matched against chunk titles. It tries the full name, then its first three words. If nothing matches, the law name is ignored rather than guessed.
Exact lookup. An article number on its own matches dozens of laws, so the exact path only runs when both the article and the law are known. It matches the law on the chunk’s title only (article bodies rarely repeat the law’s name, and matching on content picks up cross-references) and the article number in either digit system, so article 5 does not match 15, 25 or 50. Results are ranked by title match, then by where the number appears, and the top three go first. No similarity score is claimed for them, because none was computed.
Vector and keyword together. Nearest-neighbour search always returns something, even for a question about cooking. So vector hits need a cosine score of at least 0.35. That floor was calibrated on real questions: on-topic questions scored roughly 0.46 to 0.62 against their best chunk, off-topic ones roughly 0.18 to 0.31. It has to be recalibrated whenever the embedding model changes. When the user named a law, vector search is restricted to that law and the floor is dropped, because the user already told us where to look.
Keyword search runs alongside, using PostgreSQL full-text search with the simple configuration over the folded text. Query tokens of three or more letters become prefix terms joined with OR, and a word starting with the definite article “ال” is also tried without it. The two result lists are interleaved and deduplicated until the limit is reached. If the embedding call fails, keyword search alone answers and the user sees a warning.
The final context is a numbered list of sources with title, type, page and text, plus instructions to cite them as [1], [2], to say what needs verification when the sources are insufficient, and never to guess a page number.
Firm-scoped retrieval, in the SQL
The library is global, but the same index also holds material a firm adds from its own cases. A firm must never retrieve another firm’s chunks, and inside a firm, a lawyer who can only see their assigned cases must not retrieve passages from the others.
That rule lives in one SQL predicate that every read path shares: the index-state count, the law-name check, the exact lookup, vector search and keyword search.
-- Simplified: one access predicate, applied to every retrieval query.
WHERE c.tenant_id IS NULL -- the national library
OR (c.tenant_id = @tenant AND (
@sees_all_cases
OR (c.kind = 'case' AND c.case_id = ANY(@case_ids))
OR (c.kind = 'document' AND c.matter_id = ANY(@case_ids))))The tenant and case list come from the server-side session and are passed as typed parameters, never assembled from strings and never taken from the model’s tool arguments. The model cannot ask for a wider scope, because the scope is not something it controls. A firm’s case material only enters the index when the firm adds it to a knowledge space, and deleting a case purges its chunks.
No source, no answer
Retrieval is one tool the assistant can call. What happens when it finds nothing is the most important behaviour in the whole design.
If the library returns nothing and web search is configured, the tool falls back to web results, with an explicit warning that they need jurisdiction and effective-date verification. Web results go into the conversation as delimited data on the user side, never as system instructions, and the library tool stays available, because a web page is not a verified source of law.
If after that there is still no source in the whole turn, the turn stops. The user gets a fixed message in Arabic or English saying that no ungrounded legal answer was generated, and the model never writes one. A later empty search does not throw away sources found earlier in the same turn.
Citations are kept honest by numbering them before the model sees them. Each tool result is wrapped with the exact number each source will have in the list the user sees, so [3] in the answer points at the third source on screen. The interface drops any citation number with no matching source, and labels each section of the answer as coming from the platform’s records when it cites one, or as analysis when it does not.
What I would do the same, and differently
- Exact before similar. A lawyer who names an article wants that article. A lookup that runs before vector search and refuses to guess the law is worth more than any embedding model upgrade.
- Calibrate the floor on real questions. A similarity threshold picked by feel is a guess. Measure on-topic and off-topic questions, write the numbers next to the constant, and redo it when the model changes.
- Keep the scope out of the model’s hands. Firm and case filters belong in the SQL, built from the session, on every query path.
- Make rebuilds atomic. An index that is either the old one or the complete new one is far easier to reason about than one that can be half-built.
- Prefer deterministic tools for the chain of authority. Proposing relationships with a regex and confirming them by hand is slower, but nothing in the chain is invented.
- Keep two versions of the text. Folded text is for matching, original text is for reading. Storing one and folding the other at query time kept both honest.
If you are building retrieval over Arabic or legal text, I’d like to hear what worked for you. You can reach me through my site.