Competitor monitoring in n8n with dedupe and data tables

#01What's worth monitoring?

Start with feeds and changelogs. Pricing pages are valuable but fragile to monitor.

#02How dedupe makes alerts useful

Without dedupe, a daily workflow reading an RSS feed returns the same 10 posts every day. Remove Duplicates, set to compare against previous executions, keeps a history of values it has seen (such as the post URL) and lets through only new ones.

For page monitoring, the "value" is the content itself. Extract the relevant part of the page, create a hash with the Crypto node or a Code node, and dedupe on the hash. A new hash means the content changed.

#03Tagging and routing

A Text Classifier with categories like product launch, pricing change, hiring and content lets a Switch node send alerts to different channels. Write clear category descriptions and turn on the Other branch so unclear items still reach someone.

Add a short AI summary step if items are long, using only the fetched text. Link to the source so people can read it themselves.

#04Weekly roll-up

Daily alerts suit urgent changes. For the rest, a second workflow can read the competitor_changes table each Friday, summarise by competitor with a Basic LLM Chain, and post a digest. The summary should quote and link, not speculate.

Our view: pricing and launch alerts justify real-time-ish monitoring; content alerts are better as a weekly digest. Otherwise the Slack channel becomes noise people mute.

#05Remove Duplicates modes

According to the n8n docs, Remove Duplicates (overhauled in 1.64.0) has three operations:

For previous executions, you choose what to keep: Value Is New, Value Is Higher than any previous value, or Value Is a Date Later than any previous date. Scope can be the node or the whole workflow.

#06Data tables

Data tables store data inside n8n, so you don't need a sheet for state. The docs give a default limit of 200 MiB per instance (configurable when self-hosting), and note the Code node can't access data tables directly; use the Data table node. They're good for watch lists, seen-item lists and small result logs.

#07Respect the sites you monitor

Prefer RSS feeds and official APIs. If you fetch pages, check the site's terms and robots rules, fetch infrequently, and identify yourself where appropriate. Don't log in to competitor accounts or collect personal data.

#08Building the watch list

Keep it short and useful. For each competitor, consider:

• Blog or news feed.
• Changelog or release notes page.
• Pricing page (one selector, the plan table).
• Events or webinars page.
• Careers page, if hiring signals matter to you.

Review the list quarterly. Remove sources that never produced a useful alert.

#09Turning alerts into action

Store what action was taken in the competitor_changes table. It shows the monitor's value and helps you prune noisy sources.

#10Node configuration for page-change detection

Store the selector per source in the watch list. Different pages need different selectors, and keeping them in data means you fix a broken selector without editing the workflow.

#11First-run behaviour

On the very first run, every item is new, so you get an alert for everything. Two ways to handle it:

1. Run the workflow manually once with the Slack node disabled to seed the dedupe history.
2. Or add an IF node that skips alerts when a "seeded" flag in a data table is false, then flip the flag.

#12Troubleshooting

#13Cost math

A daily monitor is about 30 or 31 executions a month, whatever the number of sources, because one run loops over all of them. Text Classifier calls happen only for new items, so a quiet week costs almost nothing in model usage. On the Starter plan's 2,500 monthly executions, this workflow uses about 1%.

#14Ledgerline walkthrough: seeding and tuning

Ledgerline is a fictional accounts-payable software company. Details are illustrative.

Marketing ops built the watch list with eight sources. The first manual run seeded the dedupe history with Slack disabled. The second run, a day later, produced one alert: a genuine new post from Competitor A. Over the first month, the pricing page selector was narrowed once and one careers page was dropped as noisy.

#15Worked example: Ledgerline's pricing and launch alerts

Ledgerline is a fictional accounts-payable software company. Details are illustrative.

Ledgerline watched Competitors A-D: four blog feeds, two changelogs and two pricing pages.

The first week, one pricing page alert fired daily because a rotating testimonial was inside the extracted area. Narrowing the CSS selector to the plan table fixed it. Content alerts moved to a Friday digest after the team complained about volume.

When Competitor C changed its plan names, the alert reached sales enablement the same morning with the page link, and the battlecard was updated that week.

Walkthrough

1. Data table "watch_list": competitor (A-D), source_type (rss, page, api), url, notes.
2. Schedule Trigger daily at 08:00.
3. Get watch list rows. Switch by source_type.
4. RSS branch: RSS Read node for each feed. Page branch: HTTP Request plus HTML extract of the price or headline area.
5. Remove Duplicates: previous executions, Value Is New, on link (RSS) or on a hash of the extracted text (pages).
6. Text Classifier: product launch, pricing change, hiring, content, other.
7. Data table insert into "competitor_changes".
8. If anything is new, Slack message to #competitive with the change, category and link. Nothing new: no message.

#16Try it yourself

Set up the watch list for two competitors with one feed and one page each. Run it manually twice. The second run should send no alerts. Then change the page selector to see how fragile page monitoring is, and decide which sources are worth watching.

#17Quick quiz

1. Which Remove Duplicates option alerts only on new items across runs?

Answer: Remove Items Processed in Previous Executions, with Value Is New.

2. Can the Code node read a data table directly?

Answer: No. Use the Data table node.

3. What source is most reliable to watch?

Answer: RSS feeds and official APIs.

#18What should you check before the next lesson?

• The watch list lives in a data table.
• The second run sends no duplicate alerts.
• Sources are feeds or APIs where possible.

Next: Reporting automations for GA4 and Google Ads. Or return to the n8n for Marketers track.

#19Common questions

Is monitoring competitor websites legal? Reading public pages generally is, but check terms of service and local law, and avoid personal data.

How do I detect a pricing change? Extract the price area with the HTML node and compare a hash with the last run.

Will page monitoring break? Yes, when sites change layout. Prefer feeds and APIs.