Small manufacturers answer the same handful of questions over and over. Will it fit my boat? Can I use it in salt water? How long does installation take? What’s the warranty? The answers are almost always written down somewhere already, in an FAQ, a product page, or an instructions PDF. Customers just don’t find them, so they email or call, and someone stops what they’re doing to retype an answer that’s already on the website.
That’s a good fit for AI, with one big condition: it can’t make things up. A support assistant that confidently invents an installation step or a safety rule is worse than no assistant at all. So I built a prototype for a small company that makes a boat docking product, using nothing but their public documents, and designed it around one rule: answer from the documents or say you don’t know.
It’s a prototype, not something their customers use. I built it on spec to show what this would look like for a business their size.
Start with the documents, not the model
Before any AI, the first job was collecting what the company has actually published: their FAQ, product page, About page, return policy, contact page, a page written for boat owners, and the official Instructions for Use PDF. That came to 28 passages from seven sources.
The instructions PDF was the one that mattered most, and it was the hardest to find. It isn’t in the site’s menu or the FAQ. It’s linked from one spot on the product page. That PDF holds the safety-critical material, like how to approach the dock and how to tie the boat down in wind and current. My guess going in was that this kind of content wouldn’t be public at all. It was public, just buried. That’s a finding in itself: if I had trouble finding it, customers do too.
Finding the right passage
When a customer asks a question, the assistant first looks for the passages most likely to answer it. It uses a classic keyword-weighting method (TF-IDF): words that are rare across the documents, like “salt” or “tritoon,” count for more than common ones. Each question gets compared against all 28 passages, and the top four go forward, but only if they clear a minimum match score. If nothing clears the bar, the assistant is told that nothing relevant was found.
The word weights are computed ahead of time by a small batch job written in Apache Spark, which builds the vocabulary (449 terms) and figures out how rare each word is across the whole collection. Spark is overkill for 28 passages, and I’ll admit I partly used it for the practice. But the split is the right one: the heavy, whole-collection math runs offline, and answering a question only does a quick comparison. The same job would handle a few thousand pages of manuals without changing shape.
Why not the newer approach, where an AI model converts every passage into an “embedding” for meaning-based search? At this size, keyword matching works well, and it’s easy to inspect: you can see exactly why a passage was picked. If the document set grew, or customers started phrasing questions very differently from the docs, embeddings would be the next step.
The rule that matters: answer only from the material
The passages and the question go to Claude with strict instructions. Answer using only the reference material. Never invent installation steps, specifications, safety guidance, warranty terms, or policies that aren’t in it. If the material doesn’t answer the question, say so plainly and give the customer the support email and phone number, and never guess about anything safety-related. Keep answers to two to four sentences, in plain language rather than long quotes.
Every answer also shows its sources underneath, so a customer (or the business owner reviewing it) can see whether it came from the FAQ, the return policy, or the instructions PDF.
- 01QuestionTyped by the customer
- 02Find passagesTop four of 28, above a minimum match score
- 03Grounded answerClaude answers only from those passages
- 04Sources shownEvery answer lists where it came from
- 05Or hand offNo match means support’s email and phone
What it looks like
Here are two example exchanges. The first answer is written from the company’s actual FAQ wording; the second shows what happens when the documents don’t cover something. Company details are removed.
The second answer is the one I care about. A lot of chatbots would have produced a plausible-sounding number there. This one doesn’t, because nothing in the documents supports one.
What it would take to go live
Right now the prototype runs inside Claude’s own environment, which handles the AI connection for it. Putting it on a real website takes a small server-side piece to hold the AI key (you can’t safely put that in a web page) and pass questions through. Beyond that, a production version would want two more things:
- A way to refresh the documents. When the FAQ or a policy changes, the batch job reruns so the assistant never quotes last year’s return window.
- A log of the questions it couldn’t answer.That list is the most useful output of the whole thing: it’s a to-do list of what’s missing from the company’s documentation. Fix the docs, and the assistant gets better without touching the AI at all.
The running cost is small. Each answer is a short request to the AI with four passages attached, which works out to about a cent or less per answer, depending on the model.
Where this fits for other businesses
The pattern isn’t specific to boats. It fits any business where the same questions keep coming in and the answers already exist in writing: manufacturers with install guides and warranties, property managers fielding tenant questions, clinics sending pre-visit instructions, software companies with help docs.
It doesn’t fit when the answers live in someone’s head instead of on paper, or when every question is genuinely different. In that case the first project isn’t AI. It’s writing things down. That’s the same test from where small businesses should (and shouldn’t) use AI: repetitive questions with written answers are one of the few places it reliably earns its keep.
The most useful thing a support assistant can say is “I don’t know, but here’s who does.”