feat(utils): add keyset option to iterateFind and chunkFind - #78

Summary

New keyset option for iterateFind and chunkFind. Both page with $skip. When the loop changes items so that they leave the query, or others write to the service meanwhile, the pages shift: items are skipped or seen twice, and nothing reports an error. A real loop on main:

// 50 jobs with status 'pending'
for await (const job of iterateFind(app, 'jobs', { params: { query: { status: 'pending' } } })) {
  await app.service('jobs').patch(job.id, { status: 'done' })
}
// processes 30, and 20 stay 'pending'

After the first page, those jobs no longer match, so $skip: 10 jumps over 10 unprocessed ones. $skip is also quadratic on large tables, because the database reads and discards every skipped row on every page, and each page runs a COUNT for total.

With keyset: true, every page asks for the items after the last id of the page before:

{ ...query, id: { $gt: lastId }, $sort: { id: 1 }, $limit }

The find calls run with paginate: false, so there is no COUNT. Every item comes exactly once: the loop above processes all 50.

• The id field is the service's id, falling back to 'id'.
• The cursor is added with addToQuery, so a condition on the id in the query is kept.
• $skip applies to the first page only. $limit is the page size, capped by paginate.max.
• A $select that leaves out the id gets it added. The caller's array is not changed.
• A query.$sort throws, because the items come sorted by id. A combined cursor over a custom sort would be a follow-up.
• If a service hands out the same page again (for example, because it ignores the query), keyset throws instead of looping forever.

Refactor. iterateFind and chunkFind were identical except for three lines. Both now run on one internal pager, findPages in src/common, which also holds the keyset logic. The existing tests pass unchanged.

Docs. Both pages get a generated options table (params, keyset) through exported IterateFindOptions/ChunkFindOptions. A new "Changing Items While Iterating" section explains the $skip pitfall and shows keyset.

Behavior changes

None without keyset. Both option types are now exported.

Test plan

• pnpm test (lint, typecheck, coverage), pnpm docs:build
• iterateFind with keyset:
  • every item is yielded, sorted by id
  • all 50 jobs are processed while the loop moves them out of the query
  • pages are $limit capped by max, with paginate: false
  • a condition on the id is kept, and $skip applies to the first page
  • the id is added to $select
  • a $sort throws, and a page that does not advance throws
  • params are not mutated, including the $select it extends
• chunkFind with keyset: pages of $limit sorted by id, and every job is processed while the loop moves them out of the query
• all existing iterateFind/chunkFind tests pass on the shared pager

🤖 Generated with Claude Code