A RAG System Can Retrieve Two Correct Policies and Still Give One Wrong Answer

RAG conflicting sources become an enterprise control problem when several retrieved documents are factually valid but only one governs the question being asked. Similarity can tell a system that a policy is relevant. It cannot, on its own, decide which policy applies to this employee, jurisdiction, product or transaction.

An employee in Germany asks an internal assistant how much hotel spend can be approved without additional authorization. The retrieval system finds two highly relevant documents. The global travel policy says €250, while the Germany travel addendum says €180.

Neither document is obsolete. Neither has been retrieved by mistake. Both accurately describe company policy.

The assistant answers €250.

Retrieval worked. The wrong policy governed the answer.

RAG conflicting sources are not always bad data

Two enterprise sources can disagree without either being incorrect.

A global policy can coexist with a country-specific addendum. Product rules can differ by customer tier. A corporate standard can have an approved exception for one business unit. A regulation can impose a local requirement beyond the organization’s general policy.

Google Research has examined conflicts inside retrieval-augmented generation and found that models still have substantial room to improve in recognizing and responding appropriately when retrieved sources disagree.

In an enterprise setting, the question is often narrower than deciding which statement is universally true. It is which statement governs this case.

For the German employee, both approval limits can be valid. The missing piece is the relationship between the employee and the rules.

Relevance ranking does not establish policy precedence

Vector retrieval can reasonably return both documents because they discuss the same subject.

The global policy may even rank first. Its wording may resemble the question more closely, its relevant passage may be longer, or its chunk may have the stronger similarity score. None of those properties gives it authority over the country-specific rule.

The information that determines applicability may live in metadata: jurisdiction, legal entity, business unit, employee class, customer tier, product, contract type or exception status.

Amazon Bedrock Knowledge Bases supports metadata filters on retrieved content, including equality, inclusion and numeric conditions. AWS also documents metadata-based retrieval constraints such as filtering by modification time.

For enterprise policy retrieval, those attributes can express much more than search preference. A filter that includes country = DE alongside an applicable global scope can retain both the German addendum and the global baseline while excluding policies for unrelated jurisdictions. The application still needs a precedence rule to determine which approval limit governs.

The governing source depends on facts outside the document

The user’s question may not contain enough information to select the applicable rule.

“How much can I spend on a hotel without extra approval?” says nothing about the employee’s legal entity, jurisdiction or policy regime. The application may already know those facts from authenticated user context.

An internal assistant can use the employee’s location, entity or role when forming the retrieval request. A support application may use customer plan and region. A compliance assistant may need the regulated entity and transaction type.

Microsoft Azure AI Search uses a related pattern for security trimming: caller attributes can be supplied at query time so documents outside an authorized set are excluded before results are returned.

Policy applicability uses the same architectural idea for a different purpose. Context held by the application narrows the set of rules that should be allowed to compete for the answer.

Policy hierarchy cannot always be reduced to one filter

Metadata filtering does not solve every policy conflict.

A corporate rule may apply unless a regional addendum overrides one provision. A contract may replace the standard commercial term for one customer. An approved exception may alter a limit for a specific project without replacing the underlying policy.

Several documents can therefore remain applicable after filtering. The system needs to understand their relationship.

A German travel addendum might override the hotel threshold while leaving the rest of the global travel policy intact. Selecting one entire document as authoritative would lose useful rules from the other.

The more precise question becomes:

Which source governs this proposition?

That may require metadata describing scope, effective period, policy level, parent document, override relationships and the particular rule affected by an exception.

The model can synthesize across those sources once the application has supplied the governing structure. It should not have to infer an organization’s policy hierarchy from semantic similarity alone.

Conflict should sometimes stop generation

Some conflicts remain unresolved even after context and precedence rules are applied. Two current policies may appear to govern the same transaction. An exception may have unclear scope, or metadata may be incomplete.

A fluent answer in that situation can conceal a defect in the policy estate.

The system can instead treat unresolved conflict as a workflow state. It can identify the competing sources, isolate the disputed rule and route the question to the policy owner rather than letting the model choose whichever passage happens to rank first.

Google’s research on conflicting RAG sources supports the broader idea that different conflict types should influence how a system responds rather than being treated as ordinary retrieval output.

This also creates useful operational evidence. Repeated conflict around the same rule can reveal policies that need clarification, incomplete metadata or an override relationship that was never encoded.

Source authority should travel with the answer

Citations show where an answer came from. For governed enterprise questions, a reviewer may also need to know why that source was allowed to govern.

The German travel answer could therefore preserve the policy reference together with the applicability context used to select it: German entity, current policy period and local-addendum precedence.

If the answer is later challenged, that context helps distinguish very different failures. The German policy may have been missed. Its metadata may have been wrong. The user may have been assigned to the wrong entity. The precedence rule itself may have been misconfigured.

A source citation alone cannot make those distinctions.

Authority provenance can. It records not only where the answer came from, but the basis on which that source was treated as applicable.

The answer needs a governing rule, not just supporting evidence

Enterprise RAG can retrieve relevant documents, quote them accurately and generate a coherent response while still applying the wrong policy. The failure may sit in scope, hierarchy or case context rather than retrieval quality.

That is why enterprise knowledge needs applicability and precedence to be represented explicitly enough for the application to use them before generation.

Otherwise, the system can do everything that appears technically correct: retrieve relevant documents, quote them accurately and produce a fluent answer. And still apply the wrong rule.

A RAG system needs to know more than which source resembles the question. It needs to know which source gets to decide the answer.