Measuring Content Impact

To measure content impact, set a baseline before you change a page, choose one primary metric tied to the page's goal, wait a fixed period, then compare against the baseline and against similar pages you didn't change. AI answers vary between runs, so careful measurement prevents you from celebrating noise or abandoning work that's helping.

In this lesson

• Matching metrics to page goals
• Setting baselines and control pages
• Timing and repeat runs
• Reading results honestly
• Exercise and quiz

Which metrics should you use?

Pick one primary metric per page, and one or two secondary ones. Too many metrics make it easy to find something that moved by chance.

How do you set a baseline?

1. Choose prompts that the page should help with, ideally 10 or more.
2. Run them several times on each engine before the change, because answers vary.
3. Record the rates and the date.
4. Choose control pages: similar pages you won't change during the test.

How long should you wait?

There's no fixed answer. Pages need to be recrawled, and AI systems update at different speeds. Many teams check after about four to eight weeks. The key is to set the date before you start and stick to it, rather than checking daily and stopping when the result looks good.

How do you read noisy results?

• Use ranges, not single numbers. For a rate, a confidence interval such as the Wilson interval shows how uncertain it is (NIST).
• Compare with controls. If changed and unchanged pages both rose, something else, such as a model update, may be the cause.
• Look for consistency across engines and prompts.
• Check for confounders: PR coverage, launches, seasonality.
• Record null results. "No measurable change" is useful knowledge.

Which reports should you build?

Keep a simple monthly content-impact table:

Illustrative row. One table like this, kept up to date, is more useful to leadership than a dashboard full of unrelated charts. It also builds an evidence base of which content changes work for your brand, which you can't get from anywhere else.

What other signals are worth watching?

Beyond your primary metric, look at how AI answers describe the page's topic. Are they using your facts, your terminology or your statistics, even without citing you? Are the facts in answers more accurate after a refresh? These qualitative signals don't replace metrics, but they help explain them.

How long should you wait before judging?

Engines that search the web, such as ChatGPT search and Perplexity, can pick up a changed page once it has been recrawled. Answers that rely on training data change only when the model is updated. Give priority changes 2–4 weeks before the first read, and judge them on a trend rather than one run.

A quick example

Illustrative. A team refreshes three guides. Before the change, the guides' prompts produce citations in 8 of 60 runs (13%). Six weeks later, it's 17 of 60 (28%). Control pages moved from 10 of 60 to 11 of 60. The 95% intervals for the changed pages still overlap a little, so the team reports "a promising increase, likely related to the refresh", extends the test to more prompts, and plans a second round.

Measurement checklist

• [ ] One primary metric per page, tied to its goal
• [ ] Baseline with repeat runs before the change
• [ ] Control pages chosen
• [ ] Check date fixed in advance
• [ ] Results reported with ranges
• [ ] Confounders noted
• [ ] Results logged, including no change

Exercise

Pick 3 pages and set the metric you will check in 30 days.

1. Choose three pages you've improved during this course.
2. For each, write the page goal and the primary metric.
3. Record today's baseline.
4. Pick one control page for each.
5. Put the check date in your calendar.

Quiz

1. When should you set your baseline?

a) After the change b) Before the change c) Never Answer: b

1. Why use control pages?

a) To see if something other than your change moved the metric b) To make reports longer c) To hide results Answer: a

1. Why fix the check date in advance?

a) To avoid stopping when results look good by chance b) Calendars require it c) It makes results bigger Answer: a

1. What's a good way to report a noisy rate?

a) A single number only b) A range such as a confidence interval c) A screenshot Answer: b

1. What should you do with a result showing no change?

a) Delete it b) Log it as useful knowledge c) Change the numbers Answer: b

Next: You've finished the AEO Content Masterclass. Continue with Technical GEO, Lesson 1: AI crawlers and user agents

Sources

• Google, Custom channel groups in GA4
• Google, Key events in GA4
• Google, Search Console performance report
• NIST/SEMATECH, Confidence intervals for proportions