Personalization and Memory: Do AI Answers Vary by User?

Yes, AI answers can vary by user. Some variation is random, from one run to the next. Some comes from context: signed-in history, memory features, custom instructions, location and language. Google has said AI Mode can use past searches and, if users opt in, apps such as Gmail. Measure with a clean baseline first, then test personas.

On this page

• Where does user-level variation come from?
• From MentionX data: how much answers vary before personalisation
• What have engines said about personalisation?
• Why does this matter for brands?
• How do you test personalisation fairly?
• Worked example: Ledgerline's persona test
• What does a persona prompt look like?
• How many runs do you need?
• How should you report it?
• Can you influence personalised answers?
• How much context is too much?
• What about privacy and ethics?
• How many personas do you need?
• What about long chats?
• How do location and language fit in?
• What can you do with persona gaps?
• Common questions
• What mistakes should you avoid?
• Checklist
• Related
• Sources

Updated 31 October 2026. Anything marked "Our view" is MentionX's opinion, not documented engine behaviour. The worked example uses a fictional company.

Where does user-level variation come from?

From MentionX data: how much answers vary before personalisation

Before blaming personalisation, measure the randomness. It's often bigger than people expect.

Method. We ran read-only aggregate queries over MentionX runs. All were API calls with no user account, memory or history, so none of the variation below comes from personalisation. We took 28 prompt-and-engine pairs that were repeated 643 times between 4 September and 19 October 2026 (UTC), across several engines and 18 anonymised projects.

In other words, for more than a third of these pairs, the same question to the same engine sometimes named the brand and sometimes didn't, with no user context at all.

Engines also differed in how often they named the brand. Across the wider set of 2,154 runs from 22 August to 29 October 2026, the share of runs mentioning the tracked brand ranged from 24.0% on DeepSeek (100 runs) to 47.3% on Gemini (803 runs).

Limits. A small, convenience sample of MentionX projects, through APIs rather than consumer apps. We don't yet have persona-versus-baseline comparisons to publish, so we haven't included any.

Our view: if a colleague sees you recommended and the CMO doesn't, randomness is at least as likely an explanation as personalisation. Repeated runs settle it.

What have engines said about personalisation?

• Google AI Mode: In May 2025, Google said AI Mode would soon offer personalised suggestions based on past searches, and that users could opt in to connect other Google apps, starting with Gmail, for more personal context (Google).
• Google Search location: Google's help pages explain that search results can use your device's location, IP address and past activity (Google).
• Assistants: Chat assistants commonly offer memory and custom instructions that users can view and turn off in settings. Check each product's current help pages, as features change often.

Why does this matter for brands?

A buyer in the fintech industry might get a different "best analytics tool" answer than a buyer in retail, especially if the assistant knows their industry. This means:

• Signed-out scans show a baseline, not what every buyer sees.
• Persona tests show how answers shift for your ideal customer profiles.
• Single screenshots from colleagues' accounts are not representative.

How do you test personalisation fairly?

1. Baseline: signed out, new session, fixed location and language, repeated runs.
2. Personas: write 2–4 short persona contexts, such as "I'm the head of finance at a 50-person SaaS company in Berlin."
3. Inject context in the prompt rather than using real accounts; it's repeatable.
4. Run the same prompts for each persona with repeated runs.
5. Compare rates with the baseline.

Worked example: Ledgerline's persona test

Ledgerline is a fictional accounts-payable tool for mid-sized companies. A sales rep shares a ChatGPT screenshot recommending Ledgerline first. The CFO tries the same question and gets a list without it.

The marketing team sets up a proper test:

1. Baseline. 20 recommendation prompts, run through APIs with no account, five times each, in English with a fixed location. Ledgerline is named in a modest share of runs.
2. Three personas, based on the CRM's top segments: a finance manager at a 150-person manufacturer using NetSuite; an AP lead at a 300-person logistics firm using SAP; a controller at a 60-person agency using Xero.
3. Same prompts, same runs, with each persona's context added as the first sentence of each prompt.

The NetSuite persona names Ledgerline much more often than the baseline, because the engine cites Ledgerline's detailed NetSuite integration page. The SAP persona names it less, because Ledgerline has no SAP page and the cited sources favour rivals. The Xero persona is close to the baseline.

The sales rep, it turns out, uses NetSuite and had asked about it earlier in the same chat.

The team doesn't report one blended number. It reports the baseline plus each persona, and puts an SAP integration page at the top of the content plan. Illustrative: the figures and outcomes are made up to show the method.

What does a persona prompt look like?

Illustrative.

Compare this to the plain prompt "What are the best tools to track brand visibility in AI search?" Track which brands appear in each, and whether the reasons given for recommending them change too. A persona that mentions HubSpot may bring integration features to the front of the answer.

How many runs do you need?

Because answers vary on their own, a persona result only means something if the difference from the baseline is bigger than the normal run-to-run swing.

A practical approach:

• Run each prompt at least three times for the baseline and for each persona. Five is better for the prompts that matter most.
• Compare rates, not single answers. "Named in 9 of 15 persona runs vs 3 of 15 baseline runs" is a finding. "Named once for the persona, not for the baseline" is not.
• Use enough prompts. With 20 prompts and three runs each, you have 60 answers per persona. That's enough to spot a big shift, not a small one.
• Keep everything else fixed: same engines, same dates, same language and location settings. Run the baseline and the personas in the same week so model updates don't get mixed in.

Our view: if a persona moves your mention rate by only a few points, treat it as noise until it repeats over two or three months. Big gaps, like being named twice as often, are worth acting on sooner.

How should you report it?

Can you influence personalised answers?

Indirectly. Content that clearly speaks to specific industries, roles and use cases gives engines material to match persona context. Examples include industry pages, case studies by segment and integration pages.

How much context is too much?

Persona prompts can overfit. If you write "I need a tool exactly like [your brand]", you'll be recommended, but it proves nothing. Keep personas realistic:

Base personas on real customer interviews or CRM data, not on what you hope the AI will say.

What about privacy and ethics?

Don't use real customers' accounts or data to test AI answers. Use synthetic personas you write yourself and accounts created for testing, where the product's terms allow it. Record what context was used in each test so results can be reproduced and reviewed.

How many personas do you need?

Start with the two or three segments that bring the most revenue. Each persona multiplies your scan volume, so add more only where you expect answers to differ meaningfully, for example by industry or region rather than small differences in job title.

What about long chats?

In a real conversation, buyers ask follow-up questions. Earlier turns shape later answers. For important journeys, test a short scripted sequence too: a broad question, a narrowing follow-up and a final "which should I choose?" Track whether your brand survives each step.

How do location and language fit in?

Location and language are personalisation you can control in a test. Google explains that results can depend on device location and IP address (Google). Fix both for your baseline, then vary one at a time to see which of your markets get noticeably different answers. See location and language effects for a full method.

What can you do with persona gaps?

A persona gap is a content brief. If a segment sees you less often than the baseline, look at the sources the engine cited for that persona. Usually one of three things is missing: a page about that industry or integration, third-party coverage that mentions you for that segment, or evidence such as a case study from a similar customer. Fix the gap that the cited sources point to, then re-run the same persona prompts.

Common questions

Should we test with signed-in accounts at all? Yes, occasionally, to see what the consumer experience looks like, but use test accounts with memory off and no connected apps. Don't use them for the numbers you report.

Do API answers match what users see? Not exactly. Consumer apps can add memory, history, location and product features that API calls don't have. Treat API results as a consistent baseline, and spot-check the apps.

Can we target personalised answers directly? Not in any controlled way. You can't see or change another user's memory or history. What you can do is make sure the pages engines find for each segment exist and say clearly who you're for. Personal context then has something to match.

What about the earlier messages in a chat? They shape later answers, so a buyer who has already mentioned their tools or budget may see a narrower list. Scripted multi-turn tests, with a broad question followed by one or two narrowing follow-ups, show whether you survive that narrowing.

Does the language of the prompt matter? Often, yes. Engines may draw on sources in that language, which can change which brands appear. Test each market in its own language rather than translating results.

What mistakes should you avoid?

• Using personal, signed-in accounts for official scans
• Treating one colleague's screenshot as proof
• Mixing persona and baseline results in one number
• Forgetting to start new chats between runs
• Testing personas without a baseline to compare against

Checklist

• Signed-out baseline defined
• 2–4 personas written from your ICPs
• Persona context injected in prompts
• Results reported separately
• Segment-specific content planned where gaps appear

Related

• Why AI answers differ every time
• Location and language effects on AI answers

Sources

• Google, AI Mode update, Google I/O 2025, 20 May 2025
• Google Search Help, Understand & manage your location when you search on Google
• Google Search Central, AI features and your website