Repository navigation
Conversation
…onsumers Checkpoint consumers do not use entry buckets: a checkpoint consumer group gives each segment to a single member, and extra members stay idle (the client specification, key-shared.md section 9). Only stream consumers and ungrouped checkpoint consumers preserve per-key ordering across splits and merges; a checkpoint consumer group has no way to wait for a predecessor segment that another member is still reading.
Since apache/pulsar#26838, only stream subscriptions count toward an automatic rebucket rollover. A checkpoint consumer group reads whole segments, one member each, so more entry buckets would not serve it: its member count drives only a split, and on a lower-traffic topic or at the segment limit its extra members stay idle. Also correct the claim that automatic merges preserve the parallelism of every coordinated consumer. The merge guard protects the entry-bucket parallelism of stream consumers; a checkpoint consumer group uses at most one member per segment, so a merge can leave one of its members idle. The versioned 5.0.x pages are unchanged: they still describe the 5.0.x broker, which has neither fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents three V5 checkpoint consumer fixes:
The first two are merged in apache/pulsar#26838, which also carried the commit of apache/pulsar#26836. The ordering fix is apache/pulsar#26837, still open.
Problem. The scalable-topic pages describe checkpoint consumers as if they worked like stream consumers:
Example.
concepts-scalable-topics.md, "Entry buckets and consumer parallelism":Change.
docs/concepts-scalable-topics.md:docs/admin-api-scalable-topics.md:scalableTopicSplitVsRebucketMinMsgRateInThreshold, or at the segment ceiling, bucket capacity grows for stream consumers only, and grouped checkpoint consumers drive only splits.ServiceConfigurationdescription from [fix][broker] Stop checkpoint consumer groups from triggering rebucket rollovers pulsar#26838 on the next docs sync.client-libraries/java-v5.md: a group distributes segments, not entry buckets. Each segment is read by one member, extra members stay idle, and a group does not preserve per-key order across splits and merges.versioned_docs/version-5.0.x/concepts-scalable-topics.md: the entry-bucket and ordering corrections. The old text was wrong for every 5.0.x release, and the new text matches 5.0.x once the group and ordering fixes are backported tobranch-5.0. The auto-scaling corrections are not mirrored there yet: a 5.0.x broker still rebuckets for a group's member count until #26838 is backported.Testing. Docs-only prose edits inside existing paragraphs; no links, anchors or markup added. The statements follow the code in the PRs above and the client specification. The site was not built locally.
✅ Contribution Checklist