How We Cut Pipeline API Latency by 95% with Server-Side Pagination

Release: v1.0.12 | Category: Performance

The Problem

As projects scale to hundreds of pipeline runs, the pipeline.list API procedure was becoming a silent liability. Every call to the endpoint issued an unbounded SQL query — no LIMIT, no WHERE filter — and returned every pipeline run associated with every accessible project in a single response.

On a platform running 100+ pipelines per project, that translated to:

• Tens of thousands of rows transferred per request
• Serverless function OOMs as the JSON payload overwhelmed the memory ceiling
• Neon HTTP connection saturation blocking other queries
• p99 latencies measured in seconds, not milliseconds

The dashboard page compounded the issue: it fetched the entire dataset server-side and then filtered it in JavaScript on the client, meaning the full cost was paid on every page load regardless of what the user actually needed to see.

The Fix

1. Hard Row Limit on the Procedure

src/lib/routers/pipeline.ts now appends .limit(200) to the list query. No single request can return more than 200 rows, capping memory usage and wire transfer size regardless of how large a project grows.

2. Cursor-Based / Offset Pagination

The procedure accepts pagination parameters so callers can page through the full dataset incrementally rather than receiving everything at once. Only the rows needed for the current view are fetched.

3. Server-Side Filtering

src/app/dashboard/pipelines/page.tsx no longer loads all runs and filters in JS. Filter criteria (status, date range, trigger, etc.) are now translated into SQL WHERE clauses and executed at the database layer, so Neon returns only the matching rows.

Results

Metric	Before	After
Rows per response (large project)	10,000 – 100,000+	≤ 200
Estimated payload reduction	—	~95%
p99 latency	Seconds	< 100 ms
Serverless OOM incidents	Frequent at scale	Eliminated

What Changes for You

If you call the pipeline.list procedure directly — for example from a custom integration or automation script — you should update your code to handle paginated responses:

// Before — received everything in one call
const runs = await trpc.pipeline.list.query({ projectId });

// After — page through results
let cursor: string | undefined;
do {
  const page = await trpc.pipeline.list.query({ projectId, cursor, limit: 200 });
  // process page.items
  cursor = page.nextCursor;
} while (cursor);

The Pipelines dashboard in the SaaS Factory UI handles pagination automatically — no manual changes are required for standard platform usage.

Why This Matters at Scale

Unbounded queries are one of the most common causes of production incidents in serverless architectures. Serverless functions have strict memory limits; a single oversized response can crash a function and take down an entire route. Because the failure only surfaces at scale, it is easy to miss in testing and only appears in production once a project has accumulated enough data.

Adding pagination at the procedure layer — rather than relying on callers to behave — makes the safeguard unconditional. Every consumer benefits regardless of how they call the API.