Build note: why I put boring search under the AI, not a vector database

The default architecture for "AI over your documents" in 2026 is still: embed everything, store the vectors, retrieve by similarity, hand the chunks to a model. It is a good default. For this system it was the wrong one, and the reasoning generalises to a lot of businesses.

What the users actually needed

Two different questions were being asked of the same knowledge base, and they wanted different machinery.

The first was lookup: find the thing I know exists. A specific process type, a named concept, a client matter. The user knows roughly what they want and needs it in under a second, often on a poor connection, often between other tasks.

The second was reasoning: has this situation come up before, and what worked? That one genuinely needs a model, because it spans documents and requires judgement.

Vector search handles the second question well and the first one badly. Embedding similarity is fuzzy by design — excellent when you can't name what you want, frustrating when you can. Asking for a specific identifier and getting five thematically-adjacent results is a worse experience than a keyword search from 1998.

The boring layer

So the base layer is SQLite FTS5 with BM25 ranking. Unfashionable, and very good at this job.

A few things made it work properly rather than just work:

Field weighting. Titles and aliases are boosted heavily over body text — roughly six to eight times. Someone searching a concept name wants the concept page, not the forty matters that mention it in passing.

Porter stemming, so plural and inflected forms match without the user thinking about it.

Domain synonym expansion. Every industry has jargon where two or three terms mean the same thing, and staff use them interchangeably depending on who trained them. The index expands these at query time, so a search finds the right material regardless of which vocabulary the searcher grew up with. This mattered more than any other single tuning decision, and it is not something a general-purpose embedding model gets right for a niche domain.

Heading-level chunking, so a hit lands on the relevant section rather than dumping a whole document.

The result is deterministic, explainable, fast, and works with no network. If someone gets an unexpected result, I can tell them exactly why — which is not a small thing in a business that has to justify its process.

The AI layer on top

The reasoning question is served by an MCP server exposing about a dozen typed tools over the same vault: list and read knowledge pages, search with citations, read a matter, pull a case story, trace deadlines, look up prior relationships.

The important design choice is that the assistant gets tools, not a context dump. It queries the same index a human would, follows links, and cites what it used. Three consequences:

• Answers are attributable. Every claim traces to a source the user can open.
• The model can be swapped without rebuilding retrieval. The tools are the stable interface.
• It degrades honestly. If the vault doesn't contain the answer, the assistant says so, rather than producing a fluent average of adjacent documents.

The general lesson

Deterministic search and AI search aren't competitors. They're layers, and most businesses need the boring one first.

In this system, the index answers the large majority of questions instantly and offline. The assistant handles the harder minority that need reasoning across matters. Build them the other way round — model first, retrieval as an afterthought — and you get something that demos beautifully and frustrates people by week three.

I'd add one more thing, which is a business point rather than a technical one. A keyword index is legible to the people who have to trust it. A partner can understand why a result appeared. That legibility is often what gets a system adopted in a conservative practice, and no amount of benchmark performance substitutes for it.

Context: the ingestion pipeline that filled this index, and the two-layer architecture that made it safe to query.

---

Written with the firm's identifying details removed.