Visitar URL original
cli/cloud: fix `pulumi insights account scan log` pagination by aacotroneo · Pull Request #23256 · pulumi/pulumi · GitHub
Skip to content

cli/cloud: fix pulumi insights account scan log pagination - #23256

Merged
pgavlin merged 1 commit into
masterfrom
aa/insights-scan-log-pagination-fix
May 19, 2026
Merged

pgavlin merged 1 commit into
masterfrom
aa/insights-scan-log-pagination-fix

Conversation

@aacotroneo

@aacotroneo aacotroneo commented May 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

scan log --all doesn't paginate to the end; and step mode comes back empty."

The Insights scan-logs endpoint and the Pulumi Deployments logs endpoint are served by the same server handler. The insights apitype had drifted away from that shape, which is the whole reason the bugs existed.

What this PR copies from apitype.DeploymentLogs / deployment_log.go:

  • The response field is NextToken with json:"nextToken,omitempty" — matches what the server emits.
  • Step mode returns a Lines array (not a raw Output string).
  • The --all loop terminates only when the server emits an empty cursor.

Bug 1 — continuation mode (--all)

The server emits nextToken on the wire but apitype.InsightsScanLogs had the field tagged continuationToken. The struct field decoded to "" on every response, so the page loop terminated after the first page. Renamed ContinuationToken → NextToken with the matching JSON tag — same shape as apitype.DeploymentLogs, which is the same server handler.

Bug 2 — step mode (--job N --step M)

The server returns {lines: [{timestamp, line}, …]} in step mode too — same envelope as continuation mode, just scoped to one step, with nextOffset (an int line index) instead of nextToken. The previous step renderer wrote r.Output (a never-populated string field), so the command printed nothing. Dropped the dead Output field; both modes now share one line-oriented renderer.

--all in step mode follows the nextOffset chain to accumulate every page of lines.

What this PR deliberately does NOT copy from Deployments:

  • The --count / --all semantics on insights account scan log are kept as-is (matching the rest of the Cloud-Ready Insights CLI surface — account list, account scan list).
  • The --org + positional <account> scoping stays — Insights scopes by account, deployments scope by stack.

Follow-up worth considering (not in this PR)

apitype.DeploymentLogs and apitype.InsightsScanLogs are now literally the same shape (Lines []…, NextToken string, plus step-mode NextOffset int64). They could be unified into a single apitype.WorkflowLogs struct shared between the two endpoints. Similarly, the --all loops in deployment_log.go and insights_account_scan_log.go are nearly identical and could share a helper. Both would be refactors that should touch deployments and insights together, not be smuggled into a bug fix.

Sample output (before / after)

--all continuation mode

# Before: stops after one page (~100 lines)
$ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --all | wc -l
100

# After: follows the cursor to the end
$ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --all | wc -l
8099

--job N --step M --all step mode

# Before: empty output
$ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 0 --all
(empty)

# After: prints the step's log lines and follows nextOffset
$ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 0 --all
2026-01-21T15:59:15Z Preparing environment
2026-01-21T15:59:17Z Using runner default image with Pulumi version 3.216.0 …
2026-01-21T15:59:17Z Starting container
2026-01-21T15:59:26Z Container created …
2026-01-21T15:59:26Z Container started …
…

For a step whose log spans multiple nextOffset pages (step 3 in the example above), --all now fetches the full content:

$ pulumi insights account scan log "Michael-Test-AWS/us-east-2" 7bb80f16-… --org pulumi --job 0 --step 3 --all | wc -l
8081

Test plan

  • Updated unit tests for both modes to use the real server response shape (lines array in step mode, nextToken in continuation mode).
  • Updated client_insights_test.go to round-trip the same shape.
  • mise exec -- make format
  • mise exec -- make lint (0 issues across 7 packages)
  • Live smoke against the pulumi org: continuation --all returns 8099 lines (was ~100), step-mode --job 0 --step 0 --all returns the step's log content (was empty), step 3 --all fetches 8081 lines across multiple nextOffset pages.

🤖 Generated with Claude Code

@aacotroneo
aacotroneo requested a review from a team as a code owner May 19, 2026 00:51
Two distinct bugs both surfaced as "the command stops early":

* Continuation mode (`--all`): the server emits `nextToken` on the
  wire but the response apitype was tagged `continuationToken`, so the
  Go field decoded to "" every time and the page loop terminated after
  the first page. Renamed to `NextToken` with the matching JSON tag,
  matching `apitype.DeploymentLogs` (same server-side handler).

* Step mode (`--job N --step M`): the server returns `{lines: [...]}`,
  but the code rendered `r.Output` (a never-populated string field),
  so the command printed nothing. Dropped the dead `Output` field;
  both modes now share a single line-oriented renderer. `--all` in
  step mode follows `nextOffset` to accumulate every page of lines.

Closes pulumi-service#43411 (Issue #6).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@aacotroneo
aacotroneo force-pushed the aa/insights-scan-log-pagination-fix branch from f5cf716 to 4f4bc29 Compare May 19, 2026 01:06
Type string `json:"__type,omitempty"`
Lines []InsightsScanLogLine `json:"lines,omitempty"`
NextToken string `json:"nextToken,omitempty"`
NextOffset int64 `json:"nextOffset,omitempty"`

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type DeploymentLogs struct {
Lines []DeploymentLogLine `json:"lines,omitempty"`
NextToken string `json:"nextToken,omitempty"`
}

@pgavlin
pgavlin enabled auto-merge May 19, 2026 01:27
@pgavlin
pgavlin added this pull request to the merge queue May 19, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 19, 2026
@pgavlin
pgavlin added this pull request to the merge queue May 19, 2026
Merged via the queue into master with commit 90caf84 May 19, 2026
229 of 232 checks passed
@pgavlin
pgavlin deleted the aa/insights-scan-log-pagination-fix branch May 19, 2026 05:31
@pulumi-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped in release v3.242.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants