Visitar URL original
feat: add a query paginator by mutianf · Pull Request #1530 · googleapis/java-bigtable · GitHub
Skip to content
This repository was archived by the owner on Aug 30, 2026. It is now read-only.

feat: add a query paginator - #1530

Merged
mutianf merged 9 commits into
googleapis:mainfrom
mutianf:paginator
Nov 29, 2022
Merged

feat: add a query paginator#1530
mutianf merged 9 commits into
googleapis:mainfrom
mutianf:paginator

Conversation

@mutianf

@mutianf mutianf commented Nov 22, 2022

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

@mutianf
mutianf requested a review from a team November 22, 2022 18:36
@product-auto-label product-auto-label Bot added size: m Pull request size is medium. api: bigtable Issues related to the googleapis/java-bigtable API. labels Nov 22, 2022
@product-auto-label product-auto-label Bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Nov 23, 2022
igorbernstein2
igorbernstein2 reviewed Nov 29, 2022
View reviewed changes
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
this.query = query;
this.chunkSize = chunkSize;
this.prevSplitPoint = ByteString.EMPTY;
this.firstRun = true;

@igorbernstein2 igorbernstein2 Nov 29, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you need to set the limit on the query here? otherwise gtNextQuery will return the wrong thing

@mutianf mutianf Nov 29, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm this should be handled by advnace and getNextQuery needs to be called after advance:

Query query = Query.create(...).range("a", "z");
Query.QueryPaginator paginator = query.createQueryPaginator(100);
ByteString lastSeenRowKey = ByteString.EMPTY;
while (paginator.advance(lastSeenRowKey)) {
    List<Row> rows = client.readRowsCallable().all().call(paginator.getNextQuery());
      for (Row row : rows) {
      // do some processing
     lastSeenRow = row;
    }
 }

Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
igorbernstein2
igorbernstein2 reviewed Nov 29, 2022
View reviewed changes
Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java Outdated
private ByteString prevSplitPoint;

QueryPaginator(@Nonnull Query query, int pageSize) {
this.hasOverallLimit = query.builder.getRowsLimit() == 0 ? false : true;

@igorbernstein2 igorbernstein2 Nov 29, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.hasOverallLimit = query.builder.getRowsLimit() > 0;

@mutianf mutianf Nov 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦‍♀️

igorbernstein2
igorbernstein2 approved these changes Nov 29, 2022
View reviewed changes

@igorbernstein2 igorbernstein2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm after nit

@mutianf mutianf added the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 29, 2022
@gcf-owl-bot gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 29, 2022
@gcf-owl-bot
gcf-owl-bot Bot requested a review from a team November 29, 2022 21:20
@mutianf
mutianf merged commit 5c8e1f6 into googleapis:main Nov 29, 2022
@mutianf
mutianf deleted the paginator branch November 29, 2022 22:21
@mutianf
mutianf restored the paginator branch November 29, 2022 22:21
@mutianf
mutianf deleted the paginator branch November 29, 2022 22:21
@mutianf
mutianf restored the paginator branch November 29, 2022 22:21
@release-please release-please Bot mentioned this pull request Nov 29, 2022
Merged
gcf-merge-on-green Bot pushed a commit that referenced this pull request Dec 7, 2022
@release-please
chore(main): release 2.17.0 (#1516)
2f0825e 
🤖 I have created a release *beep* *boop*
---


## [2.17.0](https://togithub.com/googleapis/java-bigtable/compare/v2.16.0...v2.17.0) (2022-12-07)


### Features

* Add a query paginator ([#1530](https://togithub.com/googleapis/java-bigtable/issues/1530)) ([5c8e1f6](https://togithub.com/googleapis/java-bigtable/commit/5c8e1f656b15177ecef4709b9e502cef58cca479))
* Add test proxy to java-bigtable client ([#1498](https://togithub.com/googleapis/java-bigtable/issues/1498)) ([132b4e4](https://togithub.com/googleapis/java-bigtable/commit/132b4e4fe340ca316de8eef2e2133b6dabd9dec3))
* Implement cross-project table restore ([#1536](https://togithub.com/googleapis/java-bigtable/issues/1536)) ([335977c](https://togithub.com/googleapis/java-bigtable/commit/335977c499b1776c8fa861d13195ffc49e468e0a))
* Next release from main branch is 2.17.0 ([#1514](https://togithub.com/googleapis/java-bigtable/issues/1514)) ([4fc6a93](https://togithub.com/googleapis/java-bigtable/commit/4fc6a93a25362df7bc107d48f94e1c00b6bd608d))


### Dependencies

* Update dependency com.google.cloud:google-cloud-monitoring-bom … ([#1531](https://togithub.com/googleapis/java-bigtable/issues/1531)) ([ee98338](https://togithub.com/googleapis/java-bigtable/commit/ee9833835a84cee202b142950b28704db682ac0c))
* Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.1.0 ([#1539](https://togithub.com/googleapis/java-bigtable/issues/1539)) ([3244cef](https://togithub.com/googleapis/java-bigtable/commit/3244cefd4d77a44bab6ccaa8f5c03e3d31575625))
* Update dependency kr.motd.maven:os-maven-plugin to v1.7.1 ([#1518](https://togithub.com/googleapis/java-bigtable/issues/1518)) ([8309681](https://togithub.com/googleapis/java-bigtable/commit/830968109a3754a12bd0bc92674fe42ae529b924))
* Update dependency org.graalvm.buildtools:junit-platform-native to v0.9.19 ([#1537](https://togithub.com/googleapis/java-bigtable/issues/1537)) ([7f7c478](https://togithub.com/googleapis/java-bigtable/commit/7f7c478a2163c46d10ed39ea3c1b046f971d4569))
* Update dependency org.graalvm.buildtools:native-maven-plugin to v0.9.19 ([#1538](https://togithub.com/googleapis/java-bigtable/issues/1538)) ([9d3df57](https://togithub.com/googleapis/java-bigtable/commit/9d3df57d7bfeafd9cbcf56ecd58b52cc1b14ba7b))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer
@igorbernstein2 igorbernstein2 igorbernstein2 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigtable Issues related to the googleapis/java-bigtable API. size: l Pull request size is large.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mutianf @igorbernstein2