Build knowledge applications whose failures can be located

A knowledge assistant can fail because ingestion missed a document, chunking separated context, retrieval selected the wrong passage, or generation ignored evidence. Wrapping everything as one RAG call hides the repair. Preserve inspectable stages without putting private text in general logs.

The case has fictional spaces T-1 and T-2. The user may access only T-1. A candidate from T-2 must not reach the user or an unauthorized model context.

Separate ingestion from querying

Ingestion reads, parses, chunks, annotates, embeds, and indexes sources. Queries select conditions, retrieve and rank candidates, assemble evidence, generate, and validate. A successful query does not establish complete ingestion, and a successful index does not establish correct answers.

The original RAG paper combines parameterized models with external memory for knowledge-intensive tasks. Modern applications vary; a vector search is not necessarily the paper's implementation. The common problem is converting external material into supported answers.

Version each stage. Ingestion retains source, content, parsing, and index versions. Queries retain authorization scope, candidate IDs, ranking information, and answer state. Sensitive source and question text need separate access rules.

Preserve context when chunking

Small chunks can separate conditions from conclusions; large chunks add irrelevant material and consume context. Inspect representative headings, tables, code, and page boundaries before choosing sizes. A character threshold does not guarantee semantic completeness.

Store document and chunk IDs, versions, locations, and heading paths so citations reach actual evidence. Define invalidation and replacement during updates and avoid mixing versions in one query.

Embeddings map content to numerical representations. Similarity indicates proximity in that space, not factual support. Keyword retrieval, filters, and reranking offer different signals; compare combinations using task examples rather than a universal ranking of techniques.

Enforce permission before exposure

Tenant and document scope must come from trusted identity and resource rules. Retrieval filtering reduces unauthorized candidates, but verify everything entering model context, user output, and logs. Removing a private citation after generation cannot retract information already supplied to the model.

This function filters candidate IDs and excludes unknown documents. It assumes string IDs and a tenant established by trusted authentication.

documents = {"D-1": {"tenant": "T-1", "text": "policy"},
             "D-2": {"tenant": "T-2", "text": "private"}}
def permitted(retrieved, tenant):
    return [doc_id for doc_id in retrieved
            if doc_id in documents and documents[doc_id]["tenant"] == tenant]
assert permitted(["D-2", "D-1"], "T-1") == ["D-1"]
assert permitted(["D-2"], "T-1") == []
assert permitted(["D-missing"], "T-1") == []
print(permitted(["D-2", "D-1"], "T-1"))

The actual output is ['D-1']. D-2 belongs to T-2 and is excluded. Only D-2 or an unknown ID produces no result. The function performs neither vector search nor authentication and is only a tenant-field check.

Real sharing may include document permissions, groups, revocation, and inheritance. Reflect changes in indexes and caches and recheck final reads. Answer caching needs permission and source-version variation rather than unconditional reuse between users.

Define absent and conflicting evidence

No evidence should produce an explicit inability to answer from current materials or a clarification request. Scores need validation: low similarity may still contain evidence and high similarity may not answer the question. Do not present unsupported default knowledge as a library-derived answer.

For conflicts, retain dates, versions, and scope. Apply a defined precedence rule or request review. Old material can score highly yet be inapplicable. Cite passages supporting specific claims rather than adding an unrelated relevant link.

Separate evidence, inference, and unknown information in generation. Citation-ID validity is a structural check; support requires textual comparison. Model judging can help but should be checked against human labels rather than treated as final truth.

Choose orchestration for required control

LangGraph organizes state, nodes, and edges with persistence, streaming, and interruption capabilities. It does not prescribe all prompts or agent architecture, and durable execution does not automatically make external effects happen once.

LlamaIndex provides ingestion and retrieval abstractions whose integrations need version checks. Dify's Workflow and Chatflow visually connect models, retrieval, and branches for different interaction uses. Frameworks reduce wiring without removing permission, evaluation, versioning, and operating responsibilities.

Compare required reranking, recovery, review, testing, maintenance, and deployment control using a small task. A working demonstration does not establish privacy or recoverability.

Evaluate and observe each stage

Check parsing, missing passages, and update delays in ingestion; evidence presence and rank in retrieval; support, omissions, and abstention in answers; and cross-user access and revocation in authorization. Separate records locate failures.

Keep sample and configuration versions for replay. Candidate IDs, necessary scores, model versions, and durations can support investigation without routinely saving private text. Restrict telemetry access and consider derived records during deletion.

Recovery may rebuild part of an index, replace parsing output, disable a source, or restore a workflow. Define affected queries and verify evidence versions afterward. Restarting cannot fix incorrect indexed content.

Exercise: locate the failing stage

Prepare synthetic answered, unanswerable, stale, contradictory, badly parsed, and cross-tenant materials. Specify expected evidence and refusal rules, then inspect ingestion, candidates, answers, and permissions separately.

Accept the design when unauthorized content never enters context, missing answers do not invent sources, citations reach the right versions, and parsing failures are distinct from retrieval failures. Simulate updates and revocation to test index and cache invalidation. The local function proves candidate filtering only; retrieval quality and framework execution need separate tests.