The company has changed. The answer has not.
The company changed its policy last month. Today, its AI still answers from last year’s rules. The answer cites a source: an internal document, approved by the person responsible.
The document was not necessarily wrong. The system failed to establish whether it still applied.
Uploading a policy does not finish the change
Imagine a company changing the scope of its complimentary maintenance service. The new terms take effect for new contracts next month. Existing customers retain their agreed terms for the remainder of their contracts. This is a hypothetical example.
The company needs to retain both policies. Would “use the most recent document” be enough? Quoting the old terms to a new customer would be wrong. Applying the new terms to every existing customer would also be wrong.
The task requires more than finding the newest file. It requires knowing which rule applies, from what date, and to whom.
Companies faced this problem before AI. They now also need to check whether their AI’s answers and actions respect those distinctions.
Finding evidence is not the same as applying it
Presented at LREC in May 2026, “Ragability Benchmark” examined how language models respond to conflicting source material. The researchers tested seven models and found that recognizing a contradiction did not mean a model could reliably answer questions about the conflicting content. [1]
This was not a field study of changes to corporate contracts. Its relevance is narrower: detecting a conflict does not guarantee a sound answer.
A more recent study, TREK, was released on October 18. It separates retrieval from reasoning when assessing questions about rapidly changing knowledge. In that evaluation, reasoning after retrieval emerged as a major bottleneck: conflicting evidence could still mislead models that had retrieved the information needed to answer. [2]
The study is an arXiv preprint; peer-reviewed acceptance has not been verified. It does not establish that every enterprise AI system behaves this way. It does give us a reason to look beyond whether a system can retrieve new material.
“Have we uploaded the new policy?” is a useful first question. The next is harder: “When the old policy is also present, can the system choose the one that applies to this customer?”
Deleting the past creates another problem
Deleting every old document would leave other needs unmet. Staff still have to explain existing contracts, review earlier decisions, and establish what the company promised at the time.
A record can deserve preservation without being authorized to guide every current decision. Keep the history, but distinguish it from the material that governs today’s default answer. Where an earlier policy remains valid for particular customers, preserve those conditions too.
I propose starting with five questions.
| Record when a rule changes | Question to resolve |
|---|---|
| Scope | Which products, regions, customers, or contracts does it cover? |
| Effective date | Is the announcement date also the date the rule takes effect? |
| Continuing validity of earlier terms | For whom do the previous terms remain valid, and until when? |
| Superseded provisions | Which parts of which earlier rule have changed? |
| Decision owner | Who resolves a conflict or an unclear case? |
These are proposed checks for managing change, not a standard whose effectiveness has been established by the cited research.
A correction should survive the conversation
An expert corrects an AI answer. Today’s problem is resolved. What happens when another employee asks the same question tomorrow?
In a technical account published in September 2026, Meta describes a process that turns expert feedback into updates to structured knowledge and procedures. Proposed changes are tested against the original problem and checked for regressions before expert review. The approach updates the knowledge and procedures around the model without retraining the model itself. [3]
This is a company’s account of work in a particular specialist domain, not independent validation or evidence of brand performance. What matters here is the mechanism for carrying a correction into future work.
Our hypothetical maintenance provider could test at least three situations after registering its revised policy:
- A new contract signed after the new policy takes effect.
- An existing contract whose original terms remain in force.
- An inquiry missing the contract date or relevant product details.
The first two require answers grounded in different terms. The third requires a request for more information. Always using the new policy would fail this test. Always using the old one would fail it too.
A changed promise needs a way to take effect
A company can announce a strategic change. Customers encounter that change through their contracts and services.
When a brand revises a promise, it needs to establish both what has changed and whose existing promise it must continue to honor. Employee guidance, AI answers, and operational decisions need to carry that distinction through.
I see this as an operating concern for Brand Design Management: designing how a revised promise takes effect in the customer’s circumstances. That connection is my interpretation of the evidence, not a direct finding of the studies.
A company may have an extensive archive. That alone does not mean it can accurately explain the company as it operates today. Before approving the next policy, ask:
Where will the new rule live? And for whom does the old one still apply?
Key questions and evidence boundaries
Why is the latest document not enough? Effective dates, eligible customers, and exceptions for existing agreements may differ. Should all old documents be deleted? My proposal is to retain the history while distinguishing the conditions under which earlier rules may still govern current work.
[1] Gross, Petrak, and Krenn, LREC 2026, pp. 2323–2333: a benchmark of conflicting contexts, not a measurement of errors in real corporate contracts. [2] Hu et al., arXiv preprint released October 18, 2026: an evaluation of retrieval and reasoning failures; peer-reviewed acceptance has not been verified. [3] Meta Engineering, September 2, 2026: a company technical account, not an independent assessment of outcomes.
The maintenance-contract scenario is hypothetical. The table, test cases, and connection to Brand Design Management are the author’s proposals and interpretation. Earlier research notes addressed organizational memory and consistency across customer touchpoints. This note focuses on the continuing validity of earlier rules when a company changes its policy.