Your Retriever Found the Paragraph and Lost the Heading Above It

The assistant answered questions about the pricing policy. Somebody asked how long they had to ask for a refund. Thirty days, it said, and quoted the policy. The customer was on the basic plan. The basic plan allows fourteen. The quote was real. It came from the section for business accounts. The policy document had a heading for each plan, and under each heading a paragraph that began the same way. Refunds are available within. Before anything was indexed, a script cut the document into pieces of a few hundred tokens, counted from the top, with no idea where a section began. The heading landed at the end of one piece. The paragraph landed at the start of the next. So the index held two paragraphs about refunds, nearly identical, neither of which said which plan it was about. Asked about refunds, the retriever found both. It ranks by resemblance, and they resembled the question equally. The model received two confident answers with no labels, and picked one. Nothing was wrong with the model. It answered from exactly what it was given. Text means less the further you cut it from what surrounds it. A row without its column names. A step without its procedure. A limit without the product it limits. So stop cutting by count. Split along the structure the author already built, headings, sections, list items, rows that keep their header. Then give every piece its address. Before you embed a chunk, put the document title and the path of headings above it at the front. Pricing policy, basic plan, refunds. Store the plan as a field you can filter on, and filter when you know who is asking. Show the model the heading along with the text, and ask it to cite both. And write the test that catches this. Questions whose answer differs by section. Same words, different plan, different number. A paragraph is not a fact. It is a fact plus where it was written. Keep the where. – Serguey Asael Shinder