Visitar URL original
perf(audit-logs): index organization-level rows and read them as two keyset parts by waleedlatif1 · Pull Request #9437 · simstudioai/sim · GitHub
Skip to content

perf(audit-logs): index organization-level rows and read them as two keyset parts - #9437

Merged
waleedlatif1 merged 1 commit into
stagingfrom
perf/audit-log-org-level-index
Oct 29, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
perf/audit-log-org-level-index

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Fixes a regression from #9419 in the organization audit-log listing.

  • perf(db): cut hot-path database work and close copilot cleanup gaps #9419 reads a page's ids as one keyset scan per org workspace plus one organization-level branch (workspace_id IS NULL and the org named in metadata or as the resource). No index served that branch in order, so unless the organization had a page of org-level rows near the top, every listing walked every workspace-less audit row in the table, scattered across the heap. Listings got slower and long reads on the replica began hitting recovery-conflict cancellations
  • Migration 0444 adds audit_log_org_level_created_at_id_idx, a partial index on (metadata->>'organizationId', date_trunc('milliseconds', created_at), id) WHERE workspace_id IS NULL AND metadata->>'organizationId' IS NOT NULL. It's built concurrently with the replay-safe invalid-index handling from 0442, and it only holds workspace-less rows that name an organization
  • The org-level branch is split into two disjoint keyset parts: rows naming the org in metadata (read backward through the new index, stopping after a page) and organization-resource rows that don't (read through audit_log_resource_idx, a handful per org). The index alone isn't enough: under the single OR the planner bitmap-ORs both indexes and sorts every org-level row of the organization
  • buildOrgScopeCondition (single-record read, data drain) is unchanged and shares the same two definitions of an org-level row

Type of Change

  • Bug fix
  • Other: performance

Testing

  • New list-plans.integration.ts: many workspace-less rows from other organizations, then EXPLAINs the SQL queryAuditLogs sends (current members and departed, with and without a cursor). It asserts the metadata part reads the new index and nothing seeks workspace_id IS NULL, and that the index serves it as a backward scan with the cursor in the index condition. Fails on the previous query
  • list-scope.integration.ts adds metadata-only, resource-only and both-matching rows (appear once) and a resource row naming another org, and pages across all parts at several page sizes against the single-predicate reference
  • Autopsy: perf(db): cut hot-path database work and close copilot cleanup gaps #9419's tests compared results only, on a fixture of a few dozen rows where walking every workspace-less row costs nothing; nothing asserted the org-level branch's plan
  • Migration replayed with the index missing, valid and invalid; integration suites pass in migrate and push modes
  • bun run lint, type-check, check:audits, check-test-types, check:migrations, drizzle-kit generate (no changes)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…keyset parts

The organization listing reads its organization-level rows as one keyset
branch: workspace_id IS NULL and (metadata organizationId = org, or the
organization resource itself). No index served that branch, so the planner
seeked workspace_id IS NULL and filtered every workspace-less row in the
table, spread across the heap. An organization without a page of such rows
near the top paid for all of them on every request.

A partial expression index on (metadata->>'organizationId', the millisecond
created_at, id) over workspace-less rows that carry the key now serves the
metadata match, and the branch is split into two disjoint parts so that
part can be read in keyset order and stop after a page: the metadata match,
and the organization-resource match whose metadata does not already name the
organization (a few rows, on audit_log_resource_idx). Disjointness keeps the
union exact; buildOrgScopeCondition keeps its predicate.

The index alone is not enough: under the single OR the planner can only
BitmapOr the two indexes and sort, which reads every organization-level row
of the organization.

The existing list-scope suite compared results only, which the slow plan
returns correctly; a plan suite now asserts the metadata part is read
through the new index, never by a workspace_id IS NULL seek, and that the
index can stream it backward in keyset order. The scope suite gains rows
matching both parts, only the resource part, and another organization.
@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 29, 2026 6:53am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no actionable issues found.

Summary

Adds an index for organization-level audit rows and splits their listing into two non-overlapping parts.

  • Organization audit-log pages find workspace-less rows without scanning other organizations' rows.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Organization audit-log request] --> B[One ordered part per workspace]
  A --> C[Matching organization metadata]
  A --> D[Organization resource without matching metadata]
  C --> E[New partial index]
  D --> F[Existing resource index]
  B --> G[Combine candidates]
  E --> G
  F --> G
  G --> H[Sort and return one page]
Loading

Reviews (1) · Last reviewed commit: "perf(audit-logs): index organization-lev..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit 59bbfd5 into staging Oct 29, 2026
68 checks passed
@waleedlatif1
waleedlatif1 deleted the perf/audit-log-org-level-index branch October 29, 2026 08:04

This branch was successfully deployed

1 active deployment
Preview — 61fd152d Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant