Paste-URL Crawl Trigger UI — v0.1.3

Paste-URL Crawl Trigger: Launching a Crawl from the Dashboard

Released in v0.1.3

Starting a new proptech supplier crawl no longer requires engineering intervention. The dashboard now includes a self-service form that lets any authorised team member (allowlist: glyn@agentos.com, dylan@agentos.com) paste a directory URL and fire off a crawl in seconds.

How It Works

1. Open the Dashboard

2. Paste a Directory URL

Copy the URL of a Kerfuffle supplier directory listing and paste it into the URL input field.

https://kerfuffle.com/categories/some-directory

3. Click "Start Crawl"

Press the Start Crawl button. The system will:

Validate the URL.
Dispatch the Kerfuffle crawl Inngest event to queue the job.
Immediately display the current crawl status.

Live Status Updates

Once a crawl is triggered, the UI reflects real-time progress through three states:

Status	Meaning
`Queued`	The crawl job has been accepted and is waiting to be picked up by a worker.
`Running`	The crawler is actively skimming the directory and collecting listings.
`Completed`	The crawl has finished. The number of products discovered is shown alongside the status.

No page refresh is required — status transitions appear automatically.

Duplicate Crawl Prevention

To avoid redundant work and race conditions, the system enforces a one active crawl per URL rule:

If you paste a URL that already has a crawl in the Queued or Running state, the form will block submission and display a warning.
You must wait for the existing crawl to reach Completed before triggering a new one for the same URL.

This applies across all users — if one team member has already started a crawl for a given URL, another member will also be prevented from starting a duplicate.

Notes

Only Kerfuffle directory URLs are supported in this release.
Product analysis and scoring begin automatically once the crawl completes. Results will appear in the ranked dashboard view.