Never Miss a New Listing — Introducing Scheduled Re-Crawls

v0.1.27 | Scheduled Re-Crawl for Monitored Directories

Proptech directories update constantly. New SaaS products appear every week, and manually re-running a crawl to catch them is friction we didn't want to leave in the workflow. In v0.1.27 we've removed it entirely.

The Problem

Up to now, every crawl was a deliberate manual action — paste a URL, kick it off, review results. That's fine for an initial discovery sweep, but it means you only see the latest market picture when you remember to look. A promising new supplier could be listed on Kerfuffle for weeks before anyone notices.

What We Shipped

Monitored Directories

You can now mark any directory URL as monitored with a single toggle. From that point on, the platform re-crawls it automatically every week using an Inngest background cron job — no manual action needed.

"New Since Last Crawl" Badges

After each scheduled run, any product that didn't exist in the previous crawl gets a "New since last crawl" badge in the dashboard. The ranked table makes these instantly visible so you can jump straight to what's changed rather than scanning the full list.

In-App Notifications

When new products are found, the team gets an in-app notification straight away — with a direct link to the filtered view showing only the newly discovered listings. No polling, no manual checks.

How to Get Started

Run a crawl on any directory URL as usual.
Enable the Monitor toggle on that directory's results page.
Come back to fresh, auto-updated results every week.

That's it. The platform handles the rest.

What Comes Next

Scheduled re-crawls are the foundation for more proactive intelligence features — think automatic opportunity score alerts when a product crosses a threshold, or trend tracking across crawl cycles. We'll be building on this in upcoming releases.

v0.1.27 is live for internal users now.