Build Your Own AI Visibility Tracker With APIs

You can build a basic AI visibility tracker in an afternoon. Send a fixed set of prompts to the OpenAI, Gemini, Claude and Perplexity APIs with web search switched on, save each answer and its cited URLs, check whether your brand and competitors are mentioned, and report mention and citation rates over time. Each of these APIs documents a web search or grounding option that returns sources ([OpenAI](https://developers.openai.com/api/docs/guides/tools-web-search); [Google](https://ai.google.dev/gemini-api/docs/google-search); [Anthropic](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool); [Perplexity](https://docs.perplexity.ai/docs/agent-api/migrate-from-sonar/overview)).

API features are from each provider's documentation, accessed 31 Oct 2026. API usage is billed by each provider; check their pricing pages. Code is illustrative and simplified. "Our view" marks opinion.

On this page

• What will you build?
• Before you start: is API tracking the same as the apps?
• Step 1: Write the prompt set
• Step 2: Call each API with web search on
• Step 3: Run each prompt several times
• Step 4: Detect mentions and citations
• Step 5: Score and chart
• Troubleshooting
• When should you buy a tool instead?
• From MentionX data: what a tracker should capture
• Worked example: Ledgerline builds a small tracker
• How do you design the data model?
• How do you detect brand mentions reliably?
• How do you handle costs and limits?
• How do you keep it running?
• How do you report results?
• When should you buy instead?
• What should you check before trusting the data?
• How do you write the prompt set?
• How do you handle citations in the answers?
• How do you share the data with your team?
• What privacy and terms issues apply?
• Common questions
• Sources

Checked on 31 October 2026. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.

What will you build?

A script that, once per run:

1. Reads prompts from a CSV file.
2. Sends each prompt to one or more AI APIs with web search on.
3. Saves the full answer text, cited URLs and a timestamp.
4. Flags whether your brand and each competitor appear in the answer and in the citations.
5. Appends the results to a CSV or Google Sheet for charting.

Expected result: a table with one row per prompt, engine and run, which you can turn into mention rate, citation rate and share of voice. See share of voice.

Before you start: is API tracking the same as the apps?

No. The APIs give you control over the model and tools, but not a user's memory, location, account history or app interface. Our view: API tracking is a consistent, repeatable proxy. Use it to see trends and compare yourself with competitors, and spot-check important prompts in the consumer apps by hand. See manual vs automated tracking.

Step 1: Write the prompt set

Create prompts.csv with columns id, prompt, intent. Start with 20 to 50 prompts across category, comparison, problem and brand questions. See building a prompt set and prompt intent classification.

Fictional rows:

Also create brands.csv listing your brand and competitors with name variants, for example "Acme", "Acme App" and "acme.com".

Step 2: Call each API with web search on

Each provider documents a different switch:

Fictional Python for OpenAI (check the current docs for exact parameters):

from openai import OpenAI
client = OpenAI()
resp = client.responses.create(
    model="MODEL_NAME",
    tools=[{"type": "web_search"}],
    input=prompt,
)
text = resp.output_text
urls = [a.url for item in resp.output if item.type == "message"
        for c in item.content for a in (c.annotations or ())
        if a.type == "url_citation"]

Write similar small functions for the other providers, each returning (text, urls). Keep the model name in your saved data, because results change when models change.

OpenAI's documentation says that when you show web results to end users, inline citations must be clearly visible and clickable (OpenAI). That matters if you put answers into client reports.

Step 3: Run each prompt several times

AI answers vary from run to run. Run each prompt 3 to 10 times per period on each engine and store every run. Our view: weekly runs suit most brands; daily runs help on volatile, high-value prompts. See statistical confidence.

Step 4: Detect mentions and citations

For each run:

• Mentioned: any brand name variant appears in the answer text, matched case-insensitively on word boundaries.
• Cited: any cited URL's domain matches the brand's domain.
• Position: the order in which brands are first mentioned.

Save one row per run: date, engine, model, prompt_id, brand, mentioned, cited, position, urls.

Step 5: Score and chart

Compute per engine and period:

• Mention rate = runs with a mention ÷ total runs
• Citation rate = runs citing your domain ÷ total runs
• Share of voice = your mentions ÷ all tracked brands' mentions

Load the CSV into Google Sheets and summarise with QUERY or a pivot table (Google). See Looker Studio dashboard.

Troubleshooting

When should you buy a tool instead?

When you need many engines, consumer-app capture, dashboards for clients, or someone else to maintain the integrations. See AI visibility tools pricing.

From MentionX data: what a tracker should capture

Method. Read-only aggregate queries over MentionX scans from 22 August to 29 October 2026 (UTC): 2,217 engine runs (2,154 completed) across 18 anonymised projects and six engines.

A home-built tracker should capture the same things a commercial one does. In our sample, mention rates and citation habits differed sharply by engine:

And by prompt intent:

Our view: design your tracker to report by engine and by intent from the start. A tracker that only stores a single blended score can't explain the variation shown here.

Limits. A convenience sample of mostly B2B software projects, run through engine APIs, which can behave differently from the consumer apps. Perplexity, Copilot and DeepSeek figures come from a single day (26 October 2026), so treat them as a snapshot, not a trend.

Worked example: Ledgerline builds a small tracker

Ledgerline is a fictional accounts-payable tool. Figures are illustrative.

Ledgerline's analyst writes 60 prompts across four intents and stores them in a sheet with an ID and intent label. A script sends each prompt to two engine APIs with web search enabled, three times each, once a week.

The script saves the full answer text, the cited URLs, the model name and the date. A second step checks each answer for Ledgerline and four competitors, labelled Competitor A to Competitor D, and records mentions, position and citations. Results go into a database table and a simple chart by engine and intent. After a month, the analyst compares a sample of API answers with the consumer apps and notes where they differ in the report.

How do you design the data model?

Use separate tables for prompts, runs and results. The prompts table holds the ID, text, intent and product area. The runs table holds the prompt ID, engine, model, settings, date, run number and raw answer. The results table holds one row per brand per run, with mentioned, position and cited URLs. This structure makes it easy to add engines, brands or prompts without rebuilding.

How do you detect brand mentions reliably?

Start with exact matches on brand names and common variants. Add rules for misspellings and product names. Watch for false positives, such as brand names that are also common words. Review a sample of matches by hand every month. For harder cases, a language model can classify mentions, but test it against human labels first.

How do you handle costs and limits?

API calls cost money and have rate limits. Estimate cost per run before scaling: prompts times engines times runs times price per call. Use batching where available, cache nothing that should be fresh, and add retries with backoff for errors. Set a monthly budget alert.

How do you keep it running?

Engines change models, endpoints and response formats. Schedule a check each month to confirm calls still work and outputs still parse. Keep the code in version control with a short readme. Name an owner, because unowned trackers break quietly.

How do you report results?

Show mention rate and citation share by engine and intent, with ranges across runs. Add competitor comparisons on the same prompts. Include a short note on the method and its limits, especially the difference between API and consumer app answers. Our view: the method note matters as much as the chart, because it stops readers from treating proxy data as exact.

When should you buy instead?

Buy a tool when you need many engines, consumer-app coverage, large prompt sets, team dashboards or support you can't provide yourself. Compare the tool's method with yours before switching, and run both in parallel for a few weeks to understand the differences.

What should you check before trusting the data?

Confirm that answers are stored in full, that mention detection matches a hand-checked sample, that runs happened on schedule and that no engine silently failed. Add simple checks to the script that flag empty answers or missing runs. Clean data matters more than clever charts.

How do you write the prompt set?

Write prompts the way buyers ask questions, not the way marketers describe products. Cover four intents: best-of, comparison, recommendation and problem. Avoid naming your own brand except in a small set of branded prompts kept separate. Give each prompt an ID, an intent label and a product area. Keep the set stable for at least a quarter, and version it when you make changes, so you can tell whether a shift came from the engines or from your edits.

How do you handle citations in the answers?

Different APIs return citations in different formats. Some return a list of sources with the answer; some include links inline in the text; some return none. Write a small parser for each engine that extracts the URLs into the same format. Normalise them by removing tracking parameters and trailing slashes, then record the domain separately. Check a sample by hand after any change to an API.

How do you share the data with your team?

Export a weekly summary to a shared sheet or dashboard with mention rate, citation share and competitor gaps by engine and intent. Link to the raw answers for anyone who wants to read them. Most people will only look at the chart, but the raw answers are what convince sceptics and explain surprising changes.

What privacy and terms issues apply?

Read each provider's terms of use before you start, especially on automated access and how outputs may be stored. Don't put personal or customer data into prompts. Store answers securely and agree how long to keep them. Write the retention period into the readme so the next owner knows the rule.

Common questions

Do APIs give the same answers as the apps? Not always. Settings, models and features can differ.

How many runs per prompt? Three or more, so you can see variation.

Can I track Google AI Overviews by API? Not directly through an official API; consider other methods and note the gap.

Sources

• OpenAI, Web search tool
• Google AI for Developers, Grounding with Google Search
• Anthropic, Web search tool
• Perplexity, Migrate from Sonar to the Agent API; API overview
• Google Docs Editors Help, Google Sheets function list