Lightening Tech Evidence, not opinion

AI & Machine Learning

An assistant that answers from your own documents

Your manuals, policies, contracts and past job records made answerable in plain language, with a citation on every line and a written rule that says it does not know.

From
$2,000–$18,000
Typical time
3–8 weeks
Usually bought by
Ops-heavy teams

What this is actually for

Three situations bring companies here, and they are all versions of the same problem. Someone who has been there fifteen years is retiring, and everyone knows the answers live in their head rather than in a file. New starters take months to become useful because they have to ask, which stops two people working instead of one. Or nobody is leaving and nothing is wrong, except that finding an answer takes fifteen minutes and it happens several hundred times a week across the company.

That third one is worth measuring before anything is built, because it is the number that justifies the project. The information is not missing — it is in a two-hundred-page operations manual, in supplier contracts, in job reports going back years, in a folder on a shared drive. It is all written down. What is missing is anything that can read it and answer a specific question with a reference. That is the entire gap this fills, and it is worth being clear that filling it is a document problem before it is a technology problem.

How the build runs

  1. Inventory the documents and settle what is current. Every document gets an owner and a version, and superseded files are explicitly excluded. This sounds administrative and it prevents the single worst failure this system has. Three to five days.
  2. Extract clean text from everything. PDFs are print files, not reading files — columns interleave, tables collapse into a row of stray numbers, headers repeat on every page. Scans need character recognition before they contain any text at all. This is the least interesting step and often the longest. Three to seven days.
  3. Split and index. Passages are cut at headings first, then to around four hundred words with a sixty-word overlap so nothing important dies on a boundary. Tables are never split, whatever the size limit says. Every passage carries its document name and section heading at the top — which fixes the problem of a clause that reads it shall notify within five days fifteen pages after the sentence that said who it is. Three to five days.
  4. Build retrieval that searches two ways at once. Meaning-based search finds the old job report that described the same situation in different words. Keyword search finds the exact part number or clause reference, which meaning-based search loses completely. Both run, the results are merged, and a reranking pass cuts a dozen candidates down to the best few. Three to five days.
  5. Write the answer rules, citations and permissions. Answer only from the retrieved passages, cite document, section and page on every claim, use one exact sentence when the answer is not there, and add a verification warning on anything touching safety. Access control is applied here. Three to five days.
  6. Build a test set from your own questions and score it. Fifty real questions your people actually ask, with the correct answers written by your people, not by us. That spreadsheet is the most valuable thing in the project — it produces a baseline score, and every change afterwards is re-scored against it, so improvement is a number rather than an impression. Four to six days.
  7. Put it where your staff already are, and schedule re-indexing. Usually a bot in Slack or Teams, because an assistant on a separate website does not get used. Three to five days.

The scale is worth knowing before you picture a large infrastructure project. A two-hundred-page manual plus the surrounding material comes to a few hundred searchable passages and an index measured in single-digit megabytes. Indexing the whole thing costs a few cents of compute. The expense in this work is the judgement, the testing and the document archaeology — never the storage.

What changes the quote

The state of the documents. Clean, current, machine-readable files sit at the bottom of the range. Scanned contracts, four versions of the same policy and a shared drive nobody has audited sit at the top, and the difference is weeks rather than days.

Where the documents live. Reading from a folder is one job. Pulling continuously from SharePoint, Confluence or Google Drive, each with its own permissions model to respect, is another, and each connector is its own piece of work.

Whether it can run on hosted infrastructure. If your answer to can this data sit with a cloud provider is no, the build changes shape entirely — self-hosted models, your own infrastructure — and both the price and the timeline roughly double. Ask that question in week one, not week five.

Where these go wrong

Two versions of the same document are indexed together. The folder contained Policy v2 and Policy v3 FINAL and both went in. Now the assistant sometimes answers from the retired rule, and that answer looks completely correct and carries a citation to a real document. This is the most dangerous failure available here, because nothing about it looks wrong. The only defence is deciding on day one what is current and refusing the rest at ingestion.

A table gets cut in half by the passage boundary. Twelve rows of equipment specifications, seven in one passage and five in the next, with the heading row stranded in the first. The second passage is now a column of numbers with nothing saying what they measure, and it will be retrieved and quoted as if it meant something. Tables are kept whole even when that breaks the size rule.

Searching by meaning only. A specific part number or clause reference has no neighbourhood in a meaning-based index — ask for the torque specification of a named component and nothing comes back, because the code is just a string. The mirror failure is just as costly — a past job report describing exactly the right precedent is missed because it used a different word for the same thing. One method alone fails in both directions, which is why both always run.

See where the site stands before you brief anyone.

If the site is already live, start from what it scores rather than from a guess. The audit reports what it checked, what it found, and — the part most tools leave out — what it could not check, and why.

See the website audit Talk about this work

Questions people ask before they commit.

How is this different from a chatbot on our website?
The reader. A website bot talks to customers about a small, stable set of policies. This one talks to your own staff about the whole body of what your company knows, so the questions are harder, more specific, and the cost of a confident wrong answer is much higher. That difference shows up in the retrieval quality, in access control, and in how much testing the work carries.
How do we know it is not inventing answers?
Two things, and they work together. Every answer carries a reference back to the document, section and page it came from, so anybody can check it in seconds. And the system is explicitly permitted to say the answer is not in the documents — that sentence is written into its instructions and specifically tested. Without that permission a language model will always produce something.
Our documents are a mess and some are scanned. Is that a problem?
It is the job, not an obstacle. Scanned PDFs are images rather than text, so they need optical character recognition first, and that typically adds around two weeks. The bigger issue is usually version confusion rather than scanning, which is why the first deliverable is a list of what is current and who owns it.
We only have about forty pages of policies. Do we still need this?
Probably not, and it would be dishonest to sell you one. Forty pages fits inside a modern model's context window with room to spare, so the whole document set can simply be sent with every question. That is simpler, has no retrieval to go wrong, and takes a week less to build. We will tell you when you are in that situation.
Can we control who sees which documents?
Yes, and for HR, finance or client-confidential material it is not optional. Permissions are applied at retrieval, so a question from someone without access to a document never sees a passage from it, and never sees a citation revealing it exists. Bolting this on afterwards is far harder than building it in.
What happens when our documents change?
Changed documents are re-indexed and the old version is removed rather than left alongside. For routine updates that is part of the running arrangement. Where a large share of the knowledge base is rewritten at once, the re-indexing and re-testing is quoted separately, and that threshold is written into the contract so neither of us is guessing.

Often taken together.

Start with the brief, not the invoice.

Send what you are trying to achieve. You get a written scope, a price and a date — or an honest “this is not the right service for that”.

Talk about RAG knowledge assistant All AI & Machine Learning services