Course lesson · Senior-Leader
Managing Agencies and Vendors
To manage GEO agencies and tools well, define what you need before you buy, score vendors on methodology and transparency rather than promises, and hold them to measurable deliverables. Be wary of anyone who guarantees placements in AI answers. No vendor controls those answers, so the best partners explain their method, report ranges honestly and transfer knowledge to your team.
In this lesson
- Build, buy or partner
- What to ask agencies and tool vendors
- A scoring framework
- Red flags
- Exercise and quiz
Should you build, buy or partner?
| Option | Best for | Watch out for |
|---|---|---|
| In-house team | Long-term ownership, deep product knowledge | Hiring time, narrow skill set |
| Agency | Capacity, specialist skills such as PR and content | Knowledge leaving with the contract |
| Tools | Repeatable measurement at scale | Methodology you can't inspect |
| Hybrid | Most mid-size and large companies | Unclear ownership between parties |
A common model: your team owns strategy, prompt sets and reporting; tools handle measurement; agencies add capacity.
What should you ask an agency?
- How do you measure AI visibility, and how often do you repeat each prompt?
- Which engines do you cover?
- Which tactics do you use, and do any break search engine spam policies (Google)?
- Show a case study with the baseline, change and measurement method.
- Who will work on our account, and what experience do they have?
- What will we own at the end of the contract (prompt sets, data, playbooks)?
What should you ask a tool vendor?
- Which engines and models are tracked, and how are answers collected?
- How many runs per prompt, and do you show variation?
- Can we export raw answers and citations?
- How are competitors and sentiment classified, and can we correct mistakes?
- What are the limits on prompts, brands and regions, and the cost of scaling?
Use the same prompt set in every trial so you're comparing like with like.
How do you score vendors?
| Criterion | Weight | Score (1–5) |
|---|---|---|
| Methodology transparency | 25% | |
| Relevant experience or coverage | 20% | |
| Data ownership and export | 15% | |
| Reporting quality | 15% | |
| Team or support quality | 10% | |
| Cost and scalability | 15% |
Set the weights before you see proposals so your favourite vendor doesn't shape the criteria.
What are the red flags?
- Guaranteed placements in ChatGPT, Gemini or any other AI answer
- Secret methods they can't explain
- Mass-produced content at a scale that could be seen as scaled content abuse (Google)
- Paid links or fake reviews
- Single-run screenshots presented as results
- No baseline in case studies
How do you manage the relationship?
- Agree deliverables and KPIs in the contract, linked to your KPI tree
- Monthly reviews with data you can check
- Quarterly business reviews against targets
- Knowledge transfer: documented playbooks and shared prompt sets
- Exit plan: what happens to your data and assets if you part ways
How do you run a fair tool trial?
- Pick 20–30 prompts from your prompt set, covering different intents.
- Run them in every tool over the same two weeks.
- Spot-check answers manually in each engine to see whether each tool's data matches reality.
- Compare coverage: engines, regions, languages and how often each tool runs the prompts.
- Test the exports: can you get raw answers and citations into your own sheet?
- Score with the same scorecard you set before the trial.
What should an agency report each month?
- Rates with ranges for priority prompts, compared with the baseline
- Work completed and the work planned for next month
- Notable changes in AI answers, with labelled examples
- Risks, such as inaccurate answers or competitor moves
- Hours or budget used against the plan
If an agency can't produce this, the relationship will be hard to judge.
A quick example
Illustrative. A company issues an RFP to four agencies. One promises "guaranteed ChatGPT top recommendations" and is eliminated. The other three answer the methodology questions; the winner scores highest on transparency and data ownership, despite not being the cheapest. The contract specifies a monthly report with ranges and full access to raw scan data.
Vendor checklist
- [ ] Needs defined before the RFP
- [ ] Scorecard and weights set in advance
- [ ] Same prompt set used in tool trials
- [ ] Red flags checked
- [ ] Deliverables and KPIs in the contract
- [ ] Data ownership and exit plan agreed
Exercise
Write an RFP scorecard for agencies or tools.
- List your top five needs.
- Write eight questions from this lesson that fit your needs.
- Set weights for the six criteria.
- Add your own red flags.
- Score one vendor you know, even informally.
Quiz
- What is the biggest red flag?
a) Guaranteed AI answer placements b) Monthly reports c) Raw data exports Answer: a
- When should scorecard weights be set?
a) Before you see proposals b) After choosing c) Never Answer: a
- Why use the same prompt set in tool trials?
a) To compare like with like b) To save money c) Vendors require it Answer: a
- What should you own at the end of an agency contract?
a) Prompt sets, data and playbooks b) Nothing c) Only invoices Answer: a
- What's a common hybrid model?
a) In-house strategy, tool measurement, agency capacity b) Agency owns everything c) No tools at all Answer: a
Next: Lesson 8: Your 12-month GEO roadmap
Sources
- Google Search Central, Spam policies for Google web search
Questions
Should we hire a GEO agency or build in-house?
Many teams combine both: in-house ownership of strategy and data, with agencies for capacity or specialist skills such as digital PR.
What's the biggest red flag in a GEO vendor?
Guaranteed placements or rankings in AI answers. No one controls what AI systems say.
How do you compare GEO tools?
Run the same prompt set through each tool in a trial and compare engines covered, methodology, repeat runs and export options.