Visitar URL original
NativeVectorUtilSupport: lazy MemorySegment promotion adds synchronized overhead on per-node-visit vectors · Issue #782 · datastax/jvector · GitHub
Skip to content

NativeVectorUtilSupport: lazy MemorySegment promotion adds synchronized overhead on per-node-visit vectors #782

Open

Description

@r-devulap

PR #761 changed MemorySegmentVectorFloat(float[]) to store the raw float[] and defer MemorySegment creation until the first native call. The intent was to benefit vectors that live entirely in Java scalar code (e.g. the KMeans distance array in KMeansPlusPlusClusterer).

However, OnDiskGraphIndex.getVector(node) creates a fresh MemorySegmentVectorFloat per node visited during graph traversal and immediately passes it to NativeVectorUtilSupport.cosine/dotProduct which calls .get() to obtain the MemorySegment. Because these vectors are ephemeral (one native call, then discarded), every node visit pays a synchronized block + MemorySegment.ofArray with no amortization benefit. This shows up on the nytimes-256-angular benchmark, causing ~15% native query latency regression compared to the pre-PR761 baseline.

To test this hypothesis, I have a one-line patch (4384f8f):

 MemorySegmentVectorFloat(float[] data) {
-    this.segment = java.util.Objects.requireNonNull(data);
     this.segment = MemorySegment.ofArray(java.util.Objects.requireNonNull(data));
 }

which restores eager MemorySegment wrapping in the float[] constructor, exactly as it was before PR #761. The segment field now starts life as a MemorySegment instead of a raw float[], eliminating the synchronized promotion cost on every short-lived per-node-visit vector. This fixes the query latency regression. See data below:

Image

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions