I asked an AI about medicines and made it prove every sentence

Ask a chatbot about the maximum dose of a medicine and you’ll get a confident, well-written answer. Whether it’s right is a different question, and in medicine that question is the whole point. A wrong dose is worse than no answer at all.

Doctors and pharmacists in Romania already have the right answers. They’re sitting in the European Medicines Agency’s product information: long PDFs, written for regulators, a pain to search on a busy day. So I wanted to find out something simple: could I put a chat on top of those documents that never, not once, says anything the documents don’t?

This isn’t only a medical problem. It’s the thing I hear most from companies right now. They want answers from their own documents, answers they can trust, without their data disappearing into someone else’s system. Medicine just happens to be where a wrong answer is easiest to spot, and hardest to forgive.

#How it works

I started by feeding it 450 medicines approved across the EU, from Abasaglar to Fabrazyme: tens of thousands of pages of Romanian PDFs, cut into passages. Each passage stays inside one section of its document, so “Doses” never bleeds into “Contraindications”.

Then, every time someone asks a question, four things happen:

1. It looks for the right passages, two ways at once: by meaning, with a vector search using the BAAI/bge-m3 model, and by exact words, with Postgres full-text search. Romanian diacritics are folded, so someone typing “contraindicatii” on a phone still finds “contraindicații”. If you name a medicine, it only searches that medicine’s documents.
2. It picks the best five. A small ranking model, BAAI/bge-reranker-base, reads the top candidates and keeps the five that answer the question best. If even the best passage is barely relevant, it stops there and refuses, and nothing is sent to Claude.
3. Claude writes the answer. Claude Haiku 4.5 gets your question and those five passages, and nothing else. It has to attach the exact text it’s quoting to every sentence it writes.
4. Code checks every single quote. This is the part I care about. Code, not the model, searches for each quote inside its passage, word for word. If a quote isn’t there, that sentence is deleted before you see it. If nothing survives, you get one fixed sentence: “Nu am găsit această informație în documente.”

The search runs on our own server. The only thing that goes out is your question and those five passages, on their way to Claude, and each request is logged before it’s sent.

Try it: ask “Ce contraindicații are Abilify?” Then ask “Cât costă Abilify în farmaciile din România?” and watch it refuse, because the leaflet simply doesn’t say. Open “Detalii de verificare” under any answer to see exactly what was checked.

A screenshot from the app: a stored example answer, generated by the real pipeline on 13 October 2026. The highlighted text is what the code found in the passage, word for word. Source text: EMA product information for Abasaglar.

#How I tested it

I wrote 60 test questions in two kinds. 46 of them have a real answer somewhere in the documents: doses, side effects, how to store an insulin pen once it’s open. The other 14 are traps: medicines that aren’t in there, like Nurofen, questions about prices and reimbursement, a human medicine asked about for cats, and a couple about football and sarmale.

For each question I checked three things. Did the search find the right medicine’s document? Did the answer contain the fact it should? And for the traps, did it refuse instead of making something up?

Then I ran the whole set twice, on our server on 12 October 2026 with Claude Haiku 4.5 (claude-haiku-4-5-20251001): once looking at what Claude wrote on its own, and once at what the reader actually sees after the check. That comparison is the interesting part.

8 of the 14 traps never reached Claude: Nurofen, Augmentin, sarmale and the football question scored so low in the search that the server refused them on its own. Claude saw the other 6 and answered 2 of them.

Even a good model, doing its best, occasionally quoted a sentence that isn’t in the document or answered a trap it had no basis for. Rare, but in medicine rare isn’t good enough. With the check in place, every one of those slips was caught, and none of the correct answers were lost.

I drafted the questions with AI help from the documents themselves, and no clinician has reviewed them yet. So it’s a tough test, not a medical exam.

#What went wrong along the way

My first ranking model was far too slow. On my laptop, BAAI/bge-reranker-v2-m3 was instant. On a small 2-core server it took over ten seconds per question (13.4 s median over the 60 test questions). I switched to a smaller ranking model, bge-reranker-base, that looks at fewer candidates, and the search dropped to about 3 seconds (2.96 s median, same server, same questions), and still found the right document for all 46 answerable questions.

The same model gave different answers on different processors. This one surprised me. To fit a small server I compressed both models, the search model and the ranking model, to 8 bits, tested them on my laptop, and they matched the full versions. On the server’s processor, the exact same file scored some passages noticeably lower, enough to push the right section out of the top five, and “Ce contraindicații are Abilify?” got a polite refusal it didn’t deserve. The fix for now is a rule: the best exact-word match always stays in the top five. The real fix is running the full-size model, which needs a bigger server to stay quick.

Vague questions were a coin toss. “Ce știi despre Cialis?” sometimes got a great answer and sometimes a refusal, because the search grabbed the warnings section instead of the one explaining what the medicine is for. Now, when you name a medicine and don’t ask anything specific, it looks for “what is it and what is it used for”.

People ask about what isn’t there. The first thing everyone typed was Nurofen. But the EMA only covers medicines approved across the whole EU. Nurofen and paracetamol are approved country by country, so they aren’t in these documents, and the honest answer is a refusal.

What it takes to run: one small server and about 0.4 cents per question for Claude (in the 12 October run, 52 of the 60 questions reached Claude and used 174,403 input and 3,537 output tokens, at Haiku 4.5’s list price; the other 8 were refused on our server before anything was sent; that’s the Claude API only, not the server). There’s no sign-in, so the chat caps how many questions each visitor, and everyone together, can ask per day, which keeps the bill predictable.

#What it means for you

If your product answers questions from your own documents, the idea fits in a sentence: let the model quote, check the quotes in code, and make “I didn’t find it” an answer you’re proud to show.

#My take

The interesting part of this experiment isn’t the AI. It’s about 100 lines of plain code that refuse to show you a sentence they can’t find in the document. With them, a cheap, fast model gives answers I’d be comfortable putting in front of a pharmacist. Without them, I wouldn’t.

And that changes what comes next. If the check is what makes the answers trustworthy, the model doing the writing doesn’t have to be the biggest one out there. So the next step is to take Claude out completely and run an open model on our own server. The search already runs there. The writing is the last thing that leaves, and once it doesn’t, your question, the documents and the answer all stay in one place. The check stays exactly as it is, so a smaller local model gets no free pass either. What I don’t know yet is how often a local model will quote correctly, and so how often it’ll have to say “I don’t know”. That’s what the next round of testing will show.

After that: the rest of the EMA catalogue, the Romanian national leaflets so everyday medicines like Nurofen are covered, and a test set written and reviewed by real clinicians.

Should you try something like this? If you answer questions from documents where a wrong sentence costs something (medicine, law, insurance, your own internal policies), yes. If your documents don’t contain the answers in the first place, don’t bother. No amount of checking will make up what isn’t there. It’ll just say “I don’t know” more often, which, to be fair, is still better than making something up.