# Competitor monitoring in n8n with dedupe and data tables

> Watch competitor blogs, pages and announcements with n8n: RSS and HTTP checks, Remove Duplicates, data tables, AI tagging and Slack alerts.

Source: https://mentionx.ai/learn/n8n/n8n-competitor-monitoring  
Author: Dipuck Jones, founder of MentionX  
Updated: 4 November 2026  
Level: Practitioner

**A competitor monitor in n8n checks feeds and pages on a schedule, removes anything it has already seen, tags new items with AI, stores them in a data table and posts only new changes to Slack. The trick is deduplication: without it, you get the same alert every day.**

> **Key takeaways**
>
> - Keep the watch list in a data table, not in nodes.
> - Use Remove Duplicates across executions so alerts fire once.
> - Prefer RSS and official APIs to page scraping.
> - Classify changes so the right people see them.

*Checked 4 November 2026 (CET) against n8n documentation and the n8n pricing page. n8n ships updates often and node options, plan limits and prices change, so confirm in your own instance and on the pricing page. "Our view" marks MentionX opinion. Ledgerline is a fictional company used for illustration.*

## What's worth monitoring?

| Signal | Source | Who cares |
|---|---|---|
| New blog posts and guides | RSS | Content, SEO |
| Product launches | Blog feed, changelog, press page | Product marketing |
| Pricing page changes | Page check | Product marketing, sales |
| Webinars and events | Events page or feed | Field marketing |
| Hiring patterns | Careers feed or page | Leadership |
| Mentions in AI answers | Lesson 12 monitor | Everyone |

Start with feeds and changelogs. Pricing pages are valuable but fragile to monitor.

## How dedupe makes alerts useful

Without dedupe, a daily workflow reading an RSS feed returns the same 10 posts every day. Remove Duplicates, set to compare against previous executions, keeps a history of values it has seen (such as the post URL) and lets through only new ones.

For page monitoring, the "value" is the content itself. Extract the relevant part of the page, create a hash with the Crypto node or a Code node, and dedupe on the hash. A new hash means the content changed.

## Tagging and routing

A Text Classifier with categories like product launch, pricing change, hiring and content lets a Switch node send alerts to different channels. Write clear category descriptions and turn on the Other branch so unclear items still reach someone.

Add a short AI summary step if items are long, using only the fetched text. Link to the source so people can read it themselves.

## Weekly roll-up

Daily alerts suit urgent changes. For the rest, a second workflow can read the competitor_changes table each Friday, summarise by competitor with a Basic LLM Chain, and post a digest. The summary should quote and link, not speculate.

*Our view:* pricing and launch alerts justify real-time-ish monitoring; content alerts are better as a weekly digest. Otherwise the Slack channel becomes noise people mute.

> **Pro move:** store the extracted page text, not just the hash. When a change alert fires, a Compare Datasets or a simple diff step can show exactly what changed.

## Remove Duplicates modes

According to the n8n docs, Remove Duplicates (overhauled in 1.64.0) has three operations:

| Operation | Use |
|---|---|
| Remove Items Repeated Within Current Input | Clean a list in one run |
| Remove Items Processed in Previous Executions | Only alert on new items across runs |
| Clear Deduplication History | Reset the stored history |

For previous executions, you choose what to keep: **Value Is New**, **Value Is Higher than any previous value**, or **Value Is a Date Later than any previous date**. Scope can be the node or the whole workflow.

## Data tables

Data tables store data inside n8n, so you don't need a sheet for state. The docs give a default limit of 200 MiB per instance (configurable when self-hosting), and note the Code node can't access data tables directly; use the Data table node. They're good for watch lists, seen-item lists and small result logs.

## Respect the sites you monitor

Prefer RSS feeds and official APIs. If you fetch pages, check the site's terms and robots rules, fetch infrequently, and identify yourself where appropriate. Don't log in to competitor accounts or collect personal data.

## Building the watch list

Keep it short and useful. For each competitor, consider:

- Blog or news feed.
- Changelog or release notes page.
- Pricing page (one selector, the plan table).
- Events or webinars page.
- Careers page, if hiring signals matter to you.

Review the list quarterly. Remove sources that never produced a useful alert.

## Turning alerts into action

| Alert type | Owner | Typical action |
|---|---|---|
| Pricing change | Product marketing | Update battlecards and sales notes |
| Product launch | Product marketing | Brief sales; consider response content |
| New comparison content | Content and SEO | Check claims about you; respond if needed |
| Big hiring push | Leadership | Note in quarterly review |

Store what action was taken in the competitor_changes table. It shows the monitor's value and helps you prune noisy sources.

## Node configuration for page-change detection

| Node | Key settings |
|---|---|
| Schedule Trigger | Daily, 08:00; workflow timezone set |
| Data table | Get rows from "watch_list" where `source_type` is `page` |
| HTTP Request | GET `{{ $json.url }}`; Response Format: String; a polite User-Agent header if your policy requires |
| HTML | Extract HTML Content; key `section`; CSS selector from the watch list, such as `{{ $('Data table').item.json.selector }}` |
| Crypto | Action: Hash; Type: SHA256; Value `{{ $json.section }}`; property name `hash` |
| Remove Duplicates | Remove Items Processed in Previous Executions; Value Is New; dedupe on `{{ $json.hash }}`; Scope: Node |
| Text Classifier | Categories: pricing change, product launch, hiring, content; Other branch on |
| Data table | Insert into "competitor_changes" with competitor, url, category, section text, run date |
| Slack | One message per change, or aggregate first for a single post |

Store the selector per source in the watch list. Different pages need different selectors, and keeping them in data means you fix a broken selector without editing the workflow.

## First-run behaviour

On the very first run, every item is new, so you get an alert for everything. Two ways to handle it:

1. Run the workflow manually once with the Slack node disabled to seed the dedupe history.
2. Or add an IF node that skips alerts when a "seeded" flag in a data table is false, then flip the flag.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| Alert every day for the same page | Dynamic content (dates, testimonials, prices by location) in the extract | Narrow the selector; strip dates with an Edit Fields expression |
| No alerts ever | Selector returns empty, so the hash never changes | Alert when the section is empty |
| 403 errors | Site blocks automated requests | Use the feed or API, or drop the source |
| Too many Slack messages | One message per item | Add an Aggregate node and send one digest |
| Duplicates after clearing history | Clear Deduplication History was run | Reseed as above |

## Cost math

A daily monitor is about 30 or 31 executions a month, whatever the number of sources, because one run loops over all of them. Text Classifier calls happen only for new items, so a quiet week costs almost nothing in model usage. On the Starter plan's 2,500 monthly executions, this workflow uses about 1%.

## Ledgerline walkthrough: seeding and tuning

*Ledgerline is a fictional accounts-payable software company. Details are illustrative.*

Marketing ops built the watch list with eight sources. The first manual run seeded the dedupe history with Slack disabled. The second run, a day later, produced one alert: a genuine new post from Competitor A. Over the first month, the pricing page selector was narrowed once and one careers page was dropped as noisy.

## Worked example: Ledgerline's pricing and launch alerts

*Ledgerline is a fictional accounts-payable software company. Details are illustrative.*

Ledgerline watched Competitors A-D: four blog feeds, two changelogs and two pricing pages.

The first week, one pricing page alert fired daily because a rotating testimonial was inside the extracted area. Narrowing the CSS selector to the plan table fixed it. Content alerts moved to a Friday digest after the team complained about volume.

When Competitor C changed its plan names, the alert reached sales enablement the same morning with the page link, and the battlecard was updated that week.

### Walkthrough

1. **Data table** "watch_list": competitor (A-D), source_type (rss, page, api), url, notes.
2. **Schedule Trigger** daily at 08:00.
3. Get watch list rows. **Switch** by source_type.
4. RSS branch: **RSS Read** node for each feed. Page branch: **HTTP Request** plus **HTML** extract of the price or headline area.
5. **Remove Duplicates**: previous executions, Value Is New, on link (RSS) or on a hash of the extracted text (pages).
6. **Text Classifier**: product launch, pricing change, hiring, content, other.
7. **Data table** insert into "competitor_changes".
8. If anything is new, **Slack** message to #competitive with the change, category and link. Nothing new: no message.

## Try it yourself

Set up the watch list for two competitors with one feed and one page each. Run it manually twice. The second run should send no alerts. Then change the page selector to see how fragile page monitoring is, and decide which sources are worth watching.

## Quick quiz

**1. Which Remove Duplicates option alerts only on new items across runs?**

*Answer:* Remove Items Processed in Previous Executions, with Value Is New.

**2. Can the Code node read a data table directly?**

*Answer:* No. Use the Data table node.

**3. What source is most reliable to watch?**

*Answer:* RSS feeds and official APIs.

## What should you check before the next lesson?

- The watch list lives in a data table.
- The second run sends no duplicate alerts.
- Sources are feeds or APIs where possible.

Next: [Reporting automations for GA4 and Google Ads](https://mentionx.ai/learn/n8n/n8n-reporting-ga4-google-ads). Or return to the [n8n for Marketers track](https://mentionx.ai/learn/n8n/n8n-for-marketers).

## Common questions

**Is monitoring competitor websites legal?** Reading public pages generally is, but check terms of service and local law, and avoid personal data.

**How do I detect a pricing change?** Extract the price area with the HTML node and compare a hash with the last run.

**Will page monitoring break?** Yes, when sites change layout. Prefer feeds and APIs.

## Sources

- n8n Docs, [Remove Duplicates node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.removeduplicates)
- n8n Docs, [Data tables](https://docs.n8n.io/build/work-with-data/data-tables)
- n8n Docs, [RSS Feed Trigger node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.rssfeedreadtrigger)
- n8n Docs, [HTTP Request node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.httprequest)
- n8n Docs, [Text Classifier](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.text-classifier)
- All accessed 4 Nov 2026.

## Frequently asked questions

### Is monitoring competitor websites legal?

Reading public pages generally is, but check terms of service and local law, and avoid personal data.

### How do I detect a pricing change?

Extract the price area with the HTML node and compare a hash with the last run.

### Will page monitoring break?

Yes, when sites change layout. Prefer feeds and APIs.

---

Written and reviewed by Dipuck Jones for MentionX Learn. The HTML version of this page: https://mentionx.ai/learn/n8n/n8n-competitor-monitoring