Visitar URL original
Fix duplicate and biased thread picks in the wall-clock reservoir sampler by kaahos · Pull Request #844 · DataDog/java-profiler · GitHub
Skip to content

Fix duplicate and biased thread picks in the wall-clock reservoir sampler - #844

Merged
kaahos merged 2 commits into
mainfrom
paul.fournillon/fix-reservoir-skip
Oct 9, 2026
Merged

kaahos merged 2 commits into
mainfrom
paul.fournillon/fix-reservoir-skip

Conversation

@kaahos

@kaahos kaahos commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?:

Fixes the skip step in ReservoirSampler (reservoirSampler.h). The wall-clock engines use it to choose which threads to signal on each tick.

  • The sampler implements Algorithm L (Li, 1994). After an item is placed, the next candidate is one past that item plus a geometric skip. The code did target += skip and left out the +1. When the skip is 0, which happens often while the reservoir is still filling, the same input item gets placed again.
  • The step now goes through a small advance(from, n, weight) helper that adds the +1.
  • The helper also compares the skip as a double before converting it to int. A skip larger than INT_MAX, or the -inf you get when 1 - weight rounds to 1.0, ends the scan instead of hitting an undefined float-to-int conversion. That replaces the assert(target >= 0), so <cassert> is no longer included.
  • The initial jump (_size + skip) was already correct and keeps the same behaviour.

Motivation:

The bug has been present since the sampler was added in #126. #794 changed the RNG but kept the same step. It has two effects on wall-clock sampling:

  1. Duplicates. A tick can signal the same thread twice and use up a slot that should go to another thread.
  2. Bias. The item just after the initial fill is chosen more often than the others. That item sits at a fixed position in ThreadFilter::collect() order, so the extra samples always go to the same thread.

A simulation of the exact loop with the default wall_threads_per_tick=16 gives:

threads (n) ticks with a duplicate wasted picks per tick over-sampling of index 16
32 ~99% ~4.2 of 16 ~1.5×
100 ~89% ~2.0 of 16 ~2.8×
1000 ~21% — ~5×

A standalone probe built against the production header shows the same behaviour:

  • n=4, k=1: per-item counts of 250494 / 313019 / 231992 / 204495 over 1M draws, where ~250000 each is expected.
  • n=10, k=3: 35914 of 100000 samples contained a duplicate.

Sampling is only involved when there are more eligible threads than wall_threads_per_tick. With n <= k every thread is taken, and that path is unchanged.

Additional Notes:

How to test the change?:

New gtests in xorshift_ut.cpp, run on the production ReservoirSampler:

  • ReservoirSamplerTest.SampleOfDistinctInputHasNoDuplicates: for (n, k) in {(4,1), (10,3), (17,16), (100,16), (2048,16), (16,16)}, every sample of distinct inputs has exactly k distinct values. After the fix this is deterministic, because target only ever increases.
  • ReservoirSamplerTest.InclusionFrequencyIsUniform: per-item inclusion frequency stays within ±5% of k/n for (4,1), (10,3) and (40,16). The tolerance is far from the buggy values, so the address-derived seed doesn't make it flaky.

Results:

  • Without the fix: duplicate selection for n=10 k=3, plus frequency failures on items 1–3 (n=4) and 0–2 (n=10).
  • With the fix: :ddprof-lib:gtestDebug_xorshift_ut and :ddprof-lib:gtestRelease_xorshift_ut pass, 29/29 on macOS arm64.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: [JIRA-XXXX]

🤖 Generated with Claude Code

kaahos and others added 2 commits October 8, 2026 15:21
ReservoirSampler advanced to the next candidate with `target += skip`,
dropping Algorithm L's +1 for the item just placed. A zero skip placed
the same input item again, so a sample of distinct threads could name
one thread twice and the item right after the initial fill was
over-selected. The advance now goes through advance(), which adds the
+1 and compares the skip as a double before converting it to int, so a
skip larger than INT_MAX or the -inf from `1 - weight` rounding to 1.0
ends the scan instead of being an undefined conversion.

Adds tests for uniqueness on distinct input (k=1, k=n, small n, n=2048)
and for uniform inclusion frequency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaahos

kaahos commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

@codex review

@kaahos
kaahos marked this pull request as ready for review October 8, 2026 13:43
@kaahos
kaahos requested a review from a team as a code owner October 8, 2026 13:43
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T13:46:33.649209Z 2f349bd Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The corrected skip step prevents duplicate selections and safely handles oversized or non-finite skips. Header compilation and an independent distribution simulation support the change.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 2f349bd · @DataDog review to ask questions

@dd-octo-sts

dd-octo-sts Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #37785667838 | Commit: c525ccf | Duration: 16m 48s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-10-08 13:57:55 UTC

@dd-octo-sts

dd-octo-sts Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 2f349bd9

@rkennke rkennke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@kaahos
kaahos merged commit cb3a583 into main Oct 9, 2026
116 checks passed
@kaahos
kaahos deleted the paul.fournillon/fix-reservoir-skip branch October 9, 2026 06:56
@github-actions github-actions Bot added this to the 1.52.0 milestone Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants