Competitor Re-Analysis Scheduler

The Competitor Re-Analysis Scheduler is a nightly batch workflow that ensures every active competitor in your project is re-analyzed on a consistent 7-day rolling cadence — automatically, with no human intervention required.

Overview

Once a competitor is marked active, the platform takes responsibility for keeping its intelligence fresh. Every day at 06:00 UTC, the scheduler sweeps the competitors table and identifies any record that:

1. Has status = 'active', AND
2. Has either never been analyzed (lastAnalyzedAt IS NULL) or was last analyzed more than 7 days ago (lastAnalyzedAt < NOW() - 7 days)

For each matching competitor, a pipeline/competitor-monitor event is fired. This triggers the downstream competitor monitoring pipeline to re-scrape the competitor and diff the results against the last known state.

Schedule

Property	Value
Cron Expression	0 6 * * *
Frequency	Daily
Run Time	06:00 UTC
Workflow Type	nightly_batch

How It Works

[Scheduler: 06:00 UTC daily]
        │
        ▼
[Query: competitors WHERE status='active'
  AND (lastAnalyzedAt IS NULL
       OR lastAnalyzedAt < NOW() - 7 days)]
        │
        ▼
  For each eligible competitor:
        │
        ▼
[Fire event: pipeline/competitor-monitor]
        │
        ▼
[Re-scrape + Diff vs. last known state]

Event: pipeline/competitor-monitor

Fired once per eligible competitor per cycle. The downstream pipeline triggered by this event is responsible for:

• Re-scraping the competitor's public presence (website, pricing, features, messaging)
• Diffing the new snapshot against the previously stored state
• Surfacing changes to the relevant project for review or automated action

Affected Entities

• competitors — Queried for eligibility; lastAnalyzedAt is updated after each successful analysis cycle.
• projects — Competitor records are scoped to projects; diff results are surfaced at the project level.

Eligibility Rules

Condition	Included in Sweep?
status = 'active', never analyzed	✅ Yes
status = 'active', analyzed > 7 days ago	✅ Yes
status = 'active', analyzed within 7 days	❌ No
status != 'active' (e.g. archived, paused)	❌ No

Notes

• The 7-day window is a rolling threshold computed at query time — it is not a fixed calendar week.
• Competitors added with status = 'active' and no prior analysis (lastAnalyzedAt IS NULL) are picked up in the very next scheduled run.
• The scheduler does not perform the scraping itself; it delegates entirely to the pipeline/competitor-monitor event pipeline, keeping concerns cleanly separated.