The smallest RAG loop: retrieve, cite, answer

RAG connects external knowledge to generation by retrieving relevant material before asking a model to answer. It does not automatically eliminate hallucinations or enforce document permissions.

Separate retrieval and generation so you can locate a failure: was the evidence missing, or did the model misuse evidence that was already available?

Two pipelines, one loop

The offline pipeline imports and parses documents, creates chunks, records metadata, and updates an index. The online pipeline receives a question, restricts accessible material, retrieves and ranks candidates, assembles evidence, and generates an answer.

Retain the original source, document version, and chunk location. A citation should lead to specific supporting material rather than an enormous file with no useful position.

For multiple users or projects, enforce authorization during retrieval. Do not send all documents to a model and ask it to ignore material the user should never have received.

A first version does not require a vector database

Keyword matching is enough to learn the boundaries of a pipeline. It is not semantic retrieval and does not provide sophisticated tokenization, but it makes the inputs and results visible.

This offline example uses constructed English sentences and whitespace tokenization. It filters accessible material before selecting evidence with matching words. It does not call a model.

documents = [
    {"id": "public-cache", "scope": "public", "text": "cache reduces repeated computation"},
    {"id": "public-retry", "scope": "public", "text": "retry needs an idempotent operation"},
    {"id": "private-note", "scope": "private", "text": "cache configuration is restricted"},
]

def retrieve(question, allowed_scopes):
    query = set(question.lower().split())
    candidates = []
    for doc in documents:
        if doc["scope"] not in allowed_scopes:
            continue
        score = len(query & set(doc["text"].lower().split()))
        if score:
            candidates.append((score, doc["id"]))
    return [doc_id for _, doc_id in sorted(candidates, reverse=True)]

assert retrieve("cache", {"public"}) == ["public-cache"]
assert retrieve("payment", {"public"}) == []
print(retrieve("retry operation", {"public"}))

The allowed scopes come from trusted application code, not model-generated arguments. A real system must connect this boundary to the current user and its resource authorization system.

Allow an answer to be unavailable

A retrieved passage does not necessarily answer the question. A high similarity score is not evidence that a statement is true. Check relevance, freshness, and whether the passage supports the particular claim.

When evidence is insufficient, return a clear unavailable state or ask for needed information. Do not fill a source gap from model memory and then attach unrelated citations.

Place citations close to the claims they support so the reader can follow the relationship.

Keep intermediate results traceable

Record retrieved document identifiers, ranking results, and the sources used in the final context. Avoid logging complete questions and document bodies by default. Handle necessary debugging samples in a separate controlled process.

For an incorrect answer, first check whether the right document reached the candidate set, then whether it entered the context, and finally whether generation used it faithfully. Changing a prompt before these checks can hide the real cause.

Try it next

Build a small collection from three public descriptions. Prepare a question with a clear answer, one with no answer, and one whose required document is inaccessible.

Verify retrieval and provenance before adding generation. The next guide examines how chunking changes evidence quality and citation completeness.