List pagination silently truncates results across multiple services #4064

What happened?

Several list operations stop after the first page even when the OpenStack API supports pagination and more resources are available.

Both EachPage and AllPages can return incomplete results without reporting an error. This is easy to miss with small collections or default page sizes, but becomes apparent when requesting small pages or exceeding the API's default page size.

Example: server groups

Given at least two server groups visible to the compute client:

page, err := servergroups.List(
    computeClient,
    servergroups.ListOpts{Limit: 1},
).AllPages(ctx)
if err != nil {
    return err
}

groups, err := servergroups.ExtractServerGroups(page)
if err != nil {
    return err
}

fmt.Printf("Retrieved %d server groups\n", len(groups))

Actual: Only one server group is returned, with no error. Using EachPage also stops after the first page.

Expected: AllPages returns the complete collection, and EachPage visits all non-empty pages.

Direct requests to Nova with limit=1 and increasing offset values retrieve the remaining groups.

Affected operations

The investigation identified pagination problems in these resource packages:

For compute security groups, the affected operation is the collection list, not the server-specific association list. Link-based pagination for instance actions, keypairs, and migrations depends on the selected microversion.

Cause

The affected implementations either:

• Embed SinglePageBase despite supporting paginated API responses.
• Inherit the default LinkedPageBase behavior, which expects links.next, while the API returns a collection-specific link array.
• Do not generate the marker or offset needed to request subsequent pages when the API omits usable continuation links.

The fixes can be made in the affected resource packages without changing the shared pagination implementation.

Regression coverage

Acceptance tests have been added for server groups, keypairs, stacks, and security services using limit=1 in #4063 highlighting the issue. They check that EachPage traverses multiple pages and that both traversal methods return every created fixture exactly once.

The server-group test also compares SDK results with explicit-offset API requests to distinguish SDK truncation from server-side behavior.

Failing CI run: https://github.com/gophercloud/gophercloud/actions/runs/40339919437/job/121631515486#step:8:1760

Sources

Examples of the upstream pagination behavior:

• Nova: server-group collection pagination
• Heat: stack continuation-link generation
• Neutron: collection-specific continuation links

Version

main (development)