Tracking Brand Hallucinations Over Time

Track brand hallucinations by writing a fact sheet of verified truths about your brand, checking every tracked AI answer against it, logging each error with its severity and any cited source, and reporting the count and share of answers with errors each month on the same prompt set. Our view: fix the sources engines rely on, then confirm the error disappears in later runs.

Points marked "Our view" are interpretation; examples are illustrative.

On this page

• What is a brand hallucination?
• Step 1: How do you build a fact sheet?
• Step 2: Which prompts should you use?
• Step 3: How do you log errors?
• Step 4: How do you rate severity?
• Step 5: How do you trace the cause?
• Step 6: How do you fix it?
• Step 7: How do you report the trend?
• Which pitfalls should you watch for?
• How do you check answers efficiently?
• Who should own it?
• How do you handle errors that won't go away?
• How do you prepare for launches and price changes?
• Exercise
• Checklist
• From MentionX data: where answers draw their facts
• Worked example: Ledgerline tracks a pricing error
• Which facts belong on the fact sheet?
• How do you write fact-checking prompts?
• How do you rate severity?
• What if the wrong fact comes from your own site?
• How long do fixes take to show?
• How should you report hallucinations?
• How do you keep the fact sheet current?
• When should you involve legal or PR?
• Common questions
• Related
• Sources

Checked on 31 October 2026. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.

What is a brand hallucination?

NIST describes "confabulation" as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived" (NIST). For brands, that includes wrong prices, invented features, outdated leadership, incorrect locations or claims that a product is discontinued.

Our view: also track outdated facts that were once true. To a buyer, the effect is the same.

Step 1: How do you build a fact sheet?

List verifiable facts with a source and a "last checked" date:

Our view: include things you don't do; engines often invent features.

Step 2: Which prompts should you use?

Mix factual prompts ("How much does Acme cost?") with category prompts where facts appear incidentally. Run each at least three times per engine. See building a prompt set.

Step 3: How do you log errors?

Step 4: How do you rate severity?

Step 5: How do you trace the cause?

Check cited sources for the false claim. If none are cited, search for the claim in quotes. Our view: common causes are outdated third-party pages, old versions of your own pages, confusion with a similarly named company and missing information that the model fills in.

Step 6: How do you fix it?

• Update your own pages and keep facts in text; Google says important content should be in textual form (Google).
• Ask third-party sites to correct outdated pages.
• Add Organization structured data that matches visible content (Google); see Organization schema.
• Check Wikidata for errors and follow its editing rules; see Wikidata for brands.
• Use engines' feedback tools where offered.

Step 7: How do you report the trend?

• Answers with at least one error ÷ all answers mentioning you.
• High-severity errors count.
• New, recurring and resolved errors.
• Engine breakdown.

Our view: an error is "resolved" only after it's absent from several consecutive runs.

Which pitfalls should you watch for?

• No fact sheet, so labellers guess.
• Counting the same error several times per answer.
• Declaring victory after one clean run.
• Fixing your site but not the third-party source.

How do you check answers efficiently?

Our view: use an AI model as a first-pass checker. Give it the fact sheet and the answer, and ask it to list any claims that contradict or aren't covered by the sheet, quoting the exact sentence. A person then reviews every flagged claim and a random sample of unflagged answers. This catches most errors quickly while keeping a human decision on what counts as wrong. Keep the checker prompt and model fixed between months so changes in the error rate reflect the engines, not your checking method. Store the quoted sentences with dates, because they make the monthly report concrete and help the teams fixing sources find the exact wording to correct.

Who should own it?

Our view: product marketing usually owns the fact sheet, with legal reviewing anything high-risk. Content and PR teams fix sources. Review the fact sheet whenever pricing or features change.

How do you handle errors that won't go away?

Our view: if an error persists for several months after fixes, check whether a widely cited source still repeats it, whether a similarly named company is being confused with you, or whether your correct information is hard to find, for example only in images or PDFs. Add a clear, dated statement on the relevant page, such as "Acme has offered a free plan since 2024". Explicit, specific text gives engines something to retrieve.

How do you prepare for launches and price changes?

Our view: update the fact sheet before the change goes live, update your pages on launch day and ask key third-party sites to update. Then run the factual prompts weekly for a month to see how fast engines pick up the new facts. That tells you how long old information tends to linger.

Exercise

Write a ten-line fact sheet. Run five factual prompts three times each in two engines, log every error with severity and source, and pick one high-severity error to fix.

Checklist

• Fact sheet with sources and dates
• Factual and category prompts
• Error log with severity
• Sources traced
• Fixes on owned and third-party pages
• Monthly trend with resolved errors

From MentionX data: where answers draw their facts

Method. Read-only aggregate queries over MentionX scans from 22 August to 29 October 2026 (UTC): 2,217 engine runs (2,154 completed) across 18 anonymised projects and six engines.

Errors often trace back to the sources an engine cites. Our sample shows which page types are cited most:

And how often engines cite at all:

Our view: when an engine cites, check those pages first; an outdated third-party page is a common source of wrong facts. When an engine gives no citations, the error may come from training data, which changes slowly, so fixing your own pages and the main third-party profiles is still the best route.

Limits. A convenience sample of mostly B2B software projects, run through engine APIs, which can behave differently from the consumer apps. Perplexity, Copilot and DeepSeek figures come from a single day (26 October 2026), so treat them as a snapshot, not a trend.

Worked example: Ledgerline tracks a pricing error

Ledgerline is a fictional accounts-payable tool. Figures are illustrative.

Ledgerline's monthly fact check finds two engines saying it has a free plan. It retired that plan a year ago. The team rates it high severity, because it affects buying decisions.

One engine cites an old review-site profile and a two-year-old blog post comparing tools. The team updates the review-site profile, emails the blog's editor with the correct facts and makes sure its own pricing page states clearly that there is no free plan. Over the next six weeks, the error disappears from the citing engine. The other engine still repeats it occasionally, so the team keeps it on the watch list.

Which facts belong on the fact sheet?

Start with facts that affect buying decisions or trust:

• Pricing, plans and whether a free tier or trial exists.
• Core features and notable features you don't have.
• Integrations and supported platforms.
• Company facts: founding year, headquarters, ownership, leadership.
• Security and compliance certifications.
• Markets and languages served.

Keep one owner per fact and a date it was last confirmed. When a fact changes, update the sheet first, then your site.

How do you write fact-checking prompts?

Ask the way buyers ask. "Does Ledgerline have a free plan?" is better than "Tell me about Ledgerline." Include direct questions, comparison questions and questions that invite assumptions, such as "Is Ledgerline good for large enterprises?" Run each several times, since a wrong fact may appear only in some answers.

How do you rate severity?

Use three levels. High: wrong facts that change buying decisions or create legal or safety risk, such as wrong prices or false compliance claims. Medium: wrong facts that mislead but are less likely to change a decision, such as an outdated feature list. Low: minor errors, such as an old founding date. Fix high first.

What if the wrong fact comes from your own site?

It often does: an old blog post, a forgotten landing page or a cached PDF. Search your site for the wrong fact. Update or remove old pages, add redirects where pages are removed, and check that no sitemap still lists them. Our view: an annual clean-up of old product pages prevents many errors.

How long do fixes take to show?

Engines that search the web at answer time can reflect fixes within days or weeks once the source pages change and are recrawled. Errors that come from training data can persist much longer. Track both, and don't declare a fix done until it's stayed gone for several checks.

How should you report hallucinations?

Report counts by severity, the number fixed and the number still open, with a short note on the worst open issue. A trend line of open high-severity errors is the single most useful chart. Leaders understand "we had five serious errors and now have one" without needing method detail.

How do you keep the fact sheet current?

Review it monthly with product and sales, and whenever pricing, packaging or leadership changes. Link each fact to the page where it's stated publicly, so a reader can check it. When a fact changes, run the related prompts within a week to see how quickly engines catch up, and add the change to your hallucination log as a watch item.

When should you involve legal or PR?

Bring in legal for false claims about safety, compliance, lawsuits or individuals. Bring in PR when an error is spreading or touches reputation. They may have routes to the publisher of the source page that marketing lacks. Keep a record of who was contacted and when.

Common questions

What counts as a brand hallucination? Any false statement about your brand, product or company in an AI answer.

Can I ask an AI company to fix an error? Some offer feedback options. Fixing the source pages is usually more reliable.

How often should I check? Monthly for most brands, and before and after launches or price changes.

Related

• Sentiment in AI answers
• Organization schema for brand entities

Sources

• NIST, AI 600-1: Generative AI Profile
• Google, AI features and your website; Organization structured data

All accessed 30 Oct 2026.