Top-N Auto-Deep-Analysis Selector
Top-N Auto-Deep-Analysis Selector
Available from v0.1.72
Overview
After a skim crawl of a supplier directory (e.g. Kerfuffle.com), the platform automatically identifies the most promising products and queues them for deep analysis. This removes the need to manually sift through every listing — the system surfaces the top candidates based on initial signals, and you confirm before the expensive deep-scrape pipeline runs.
How It Works
1. Paste a Directory URL
Enter a proptech supplier directory URL into the crawler input. The skim crawl runs across all listings, collecting lightweight surface signals for each product.
2. Automatic Scoring & Flagging
Once the skim completes, every discovered product is scored using two surface signals:
| Signal | Description |
|---|---|
| Review count | Higher volume of reviews indicates stronger market presence and data richness |
| Pricing tier | Signals revenue potential and market positioning |
The top N products by this surface score are automatically flagged as candidates for deep analysis. N defaults to 20 and is configurable.
3. Review the Flagged List
Before any deep-scrape work begins, the platform presents the auto-flagged list for your review. At this step you can:
- Confirm the list as-is to launch the deep-scrape pipeline
- Add products that didn't make the cut but interest you
- Remove products you want to exclude
- Reorder candidates by priority
This human checkpoint ensures the more resource-intensive deep-analysis stage is only triggered intentionally.
4. Deep-Analysis Pipeline
Once confirmed, the deep-scrape pipeline runs against the finalised list. Each product receives a full scored dossier covering:
- Replicability — how easy to build with our stack
- Market demand — review volume and sentiment
- Revenue potential — pricing tier and market size
- Competitive gaps — what negative reviews reveal
Results appear on the ranked dashboard, and each product's full dossier is available for drill-down.
Configuration
| Setting | Default | Description |
|---|---|---|
TOP_N_CANDIDATES | 20 | Number of products automatically flagged after a skim crawl |
Notes
- Surface scoring (review count + pricing tier) is a lightweight pre-filter. It is not the same as the full composite opportunity score, which is only computed during deep analysis.
- The confirm step is mandatory — the deep-scrape pipeline cannot be triggered without an explicit user confirmation of the candidate list.