Visitar URL original
Tuple: support custom update seeds, and check seed hashes by leerho · Pull Request #785 · apache/datasketches-java · GitHub
Skip to content

Tuple: support custom update seeds, and check seed hashes - #785

Open
leerho wants to merge 1 commit into
mainfrom
tuple-custom-seed
Open

leerho wants to merge 1 commit into
mainfrom
tuple-custom-seed

Conversation

@leerho

@leerho leerho commented Oct 29, 2026

Copy link
Copy Markdown
Member

Summary

Java tuple sketches always hashed with the default seed. They wrote a fixed seed hash and never checked the seed hash, either when reading an image or when combining sketches. As a result, Java accepted images created with any seed, and set operations could silently mix sketches built with different seeds. The C++, Go and Rust tuple sketches support custom seeds and reject mismatches. This PR brings Java in line with them, and with Java theta.

API: all changes are additive overloads

No existing signature changes. The new methods and constructors are:

Behavior

  • Writing: compact images record the sketch's seed hash. Empty images record zero, as in theta. Images written with the default seed are byte-for-byte unchanged.
  • Reading: reading a non-empty compact image checks its seed hash against the expected seed. That is the default seed unless the new overload is used. Legacy images (serial versions 1 and 2) were always written with the default seed.
  • Set operations: union, intersection, A-not-B and Jaccard check the seed hash of every non-empty tuple or theta input. Their results carry the operation's seed hash.
  • Updatable form: the deprecated updatable serialized form has no seed hash, so it is always read as the default seed. Serializing an updatable sketch built with a custom seed throws SketchesStateException. Call compact() first. C++ doesn't serialize updatable tuple sketches at all.

For default-seed users nothing changes. The only new exceptions are seed mismatches, which used to produce wrong results silently.

Verification

  • mvn verify passes, including javadoc and style checks.
  • The cross-language checks (check_java_files, check_cpp_files and check_go_files) pass for TupleCrossLanguageTest. Java still reads the C++ and Go tuple_int_* images, now with their seed hash checked.
  • The new test file is TupleCustomSeedTest. It covers:
    • seed hashes through copy(), compact() and serialization;
    • that the default-seed image still carries 37836, as C++ does;
    • the empty-image handling;
    • reading compact images with a matching and with a mismatched seed;
    • the updatable-form behavior;
    • the stateful and stateless union, intersection and A-not-B, with both tuple and theta inputs;
    • Jaccard and Filter;
    • the derived sketches, including FdtSketch.

🤖 Generated with Claude Code

Java tuple sketches always hashed with the default seed, wrote a fixed seed
hash, and never checked the seed hash when reading an image or combining
sketches. C++, Go and Rust support custom seeds and reject mismatches.

All API changes are additive overloads:
- UpdatableTupleSketchBuilder.setSeed(), and a seed constructor on
  UpdatableTupleSketch, DoubleTupleSketch, IntegerTupleSketch,
  ArrayOfStringsTupleSketch and FdtSketch.
- Seed constructors on TupleUnion, TupleIntersection and TupleAnotB, and
  seed overloads of the static TupleAnotB.aNotB methods and of the tuple
  JaccardSimilarity methods.
- TupleSketch.heapifySketch(seg, deserializer, seed) and
  TupleSketch.getSeedHash().

Behavior:
- Compact images record the sketch's seed hash (zero when empty, as in
  theta). Default-seed images are byte-for-byte unchanged.
- Reading a non-empty compact image checks its seed hash.
- Set operations check the seed hash of non-empty tuple and theta inputs,
  and their results carry the operation's seed hash.
- The deprecated updatable form has no seed hash, so it is read as the
  default seed, and serializing an updatable sketch with a custom seed
  throws; compact() it first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho
leerho requested review from proost and tisonkun October 29, 2026 21:11
Comment thread src/main/java/org/apache/datasketches/tuple/TupleSketch.java Dismissed
Comment thread src/test/java/org/apache/datasketches/tuple/TupleCustomSeedTest.java Dismissed
Comment thread src/test/java/org/apache/datasketches/tuple/TupleCustomSeedTest.java Dismissed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants