Pluggable construction build on NVIDIA GPUs #739

Activity

1. mikepapadim

   changed the title [-]Pluggable graph construction build JVector graphs on NVIDIA GPUs[/-] [+]Pluggable construction build JVector graphs on NVIDIA GPUs[/+] on Oct 1, 2026
2. mikepapadim

   changed the title [-]Pluggable construction build JVector graphs on NVIDIA GPUs[/-] [+]Pluggable construction build on NVIDIA GPUs[/+] on Oct 1, 2026
3. akash-shankaran commented on Oct 1, 2026

   

   akash-shankaran

   on Oct 1, 2026

   Contributor

   More actions

   Thanks @mikepapadim. This sounds like a promising direction adding GPU support to Jvector. I would be interested in the outcome from this as well. A couple questions:

   • What would be the additional dependencies pulled in when jvector-gpu is introduced?
   • AFAIK, cuvs uses CAGRA for graph construction. What is the search performance impact (qps, latency) in this case?

   Just an idea, If search were to be run on CPUs, with index construction done on GPUs (which is a pattern in several vector systems), there needs to be a graph translation layer to support both.

   Tagging more people from jvector side @reta @r-devulap @ashkrisk @tlwillke @MarkWolters
4. orionpapadakis commented on Oct 2, 2026

   

   orionpapadakis

   on Oct 2, 2026

   More actions

   Thanks @akash-shankaran, and thanks for looping people in.

   1. Dependencies. None for existing users:

   • jvector-base only gets the SPI (GraphBuildAccelerator) plus a few @Experimental GraphIndexBuilder hooks, about 190 lines. No new dependencies, and it still targets Java 11.
   • jvector-gpu is optional and built only with -Pgpu; default builds and CI are unchanged. It depends on jvector-base and on tornado-api and tornado-cuvs (provided scope), and targets Java 22.
   • At runtime, only when selected with -Djvector.graph.accelerator=tornadovm, it needs the TornadoVM SDK (CUDA backend), cuVS 26.08 with its CUDA libraries (cuvs-env.sh installs them), and an Ampere or newer NVIDIA GPU. If any of that is missing, supports() declines and JVector builds on the CPU.

   2. CAGRA and search performance. We don't build a CAGRA graph. cuVS only provides the candidate k-NN graph (NN-Descent). The rest follows JVector's own rules: Vamana alpha pruning (a TornadoVM tensor-core kernel), reverse edges, and the hierarchy from sampleGraphLevels. The output is a native JVector graph, searched by JVector's unchanged CPU code.

   • ada002-1M: higher recall@100 and fewer visited nodes at every beam (beam 100: 0.9452 with 1,818 visited, against 0.9425 with 1,857).
   • JVector Bench 1M: recall is equal or higher at every overquery. At equal recall, QPS is at parity (−6% to +8%) with the full-precision CPU graph, and +15% to +44% against the PQ-built graph.
   • 10M × 1024: recall@10 is 0.8970 against 0.8752, with about 6–7% fewer visits at equal recall.

   Latency (ada002-1M, single-threaded, 10k queries, top 10, in-memory full-precision vectors; both graphs have the same layer sizes). At the same beam the GPU-built graph is 2–4% slower at p50 but has higher recall. At equal recall it is faster at both p50 and p99:

   recall@10	p50 GPU vs CPU	p99 GPU vs CPU
   0.863	279 vs 290 µs (−4%)	485 vs 512 µs (−5%)
   0.929	408 vs 439 µs (−7%)	680 vs 735 µs (−7%)
   0.967	642 vs 742 µs (−14%)	996 vs 1144 µs (−13%)
   0.986	1089 vs 1308 µs (−17%)	1577 vs 1948 µs (−19%)

   (CPU latency interpolated at the GPU graph's recall; 3 timed passes per graph, alternating order.)

   3. GPU build, CPU search. That's exactly the target, which is why there's no translation layer: the GPU produces JVector's own graph, so cleanup(), the writers (NVQ, fused PQ) and search are untouched. Incremental inserts and deletes stay on the CPU. One open point: cleanup() skips refineFinalGraph for external builds. On the CPU it costs about 14 s at 1M and changed recall by less than 0.001.

   Build time (M=32, efC=100, alpha 1.2, hierarchy; RTX 4090 vs i9-13900K with 32 threads):

   dataset	CPU	GPU	speed-up	recall@10 CPU / GPU
   Cohere 10M × 1024	2270.7 s	207.1 s	11×	0.8752 / 0.8970
   ada002 1M × 1536	268.4 s	14.1 s	19×	0.9692 / 0.9727
   glove-100 1.18M	52.5 s	8.0 s	6.5×	0.7430 / 0.7755
   sift-128 985k	36.3 s	7.9 s	4.6×	0.9564 / 0.9645
   lastfm-64 292k	4.3 s	2.6 s	1.6×	0.9575 / 0.9597

   Thanks @mikepapadim. This sounds like a promising direction adding GPU support to Jvector. I would be interested in the outcome from this as well. A couple questions:

   • What would be the additional dependencies pulled in when jvector-gpu is introduced?
   • AFAIK, cuvs uses CAGRA for graph construction. What is the search performance impact (qps, latency) in this case?

   Just an idea, If search were to be run on CPUs, with index construction done on GPUs (which is a pattern in several vector systems), there needs to be a graph translation layer to support both.

   Tagging more people from jvector side @reta @r-devulap @ashkrisk @tlwillke @MarkWolters
5. mikepapadim

   changed the title [-]Pluggable construction build on NVIDIA GPUs[/-] [+]Withdrawn[/+] on Oct 8, 2026
6. mikepapadim

   closed this as not plannedon Oct 8, 2026
7. mikepapadim

   changed the title [-]Withdrawn[/-] [+]Pluggable construction build on NVIDIA GPUs[/+] on Oct 8, 2026
8. mikepapadim

   reopened this on Oct 8, 2026
9. MarkWolters

   self-assigned this

   on Oct 20, 2026