Bridge: float vectors cross as one binary buffer, not as JSON text or one JPype call per element - #496

Float vectors now cross the Python/Java bridge as one binary buffer per batch. Before, a vector property loaded with insert_many was written by Python as decimal text inside the JSON rows and parsed back by the engine, and in graph_batch.create_vertices, Document.set and the allowed_rids filter JPype converted every list element (or every RID) in its own call. For a property declared ARRAY_OF_FLOATS the engine converts each number to float anyway, so the vectors of a chunk now cross as one float32 matrix (new bridge class FloatVectors.Matrix, one buffer copy) beside the JSON, and Java builds each document's own float[] row. Properties that are not declared ARRAY_OF_FLOATS, and vectors that are ragged, missing or null, keep the exact JSON path. Also: insert_columns accepts a 2-D float32 array as a column of vectors, a numpy array is accepted as a property value everywhere (it raised TypeError before), and the allowed_rids filter builds its RID set in one crossing (GraphCalls.ridSet) instead of two per RID. This is the DSL-first rule applied to transport: the features stay SQL and Cypher, and the glue class carries the data where a crossing costs.

Measured

Two arms built from one base wheel (same engine jars and JRE, only the bindings' Python files and the bridge jar differ): A = origin/main bindings, B = this branch. 128-d float32 vectors on an ARRAY_OF_FLOATS property, JVM heap 4g, one fresh process per run, laptop pinned to 2 cores (x86-64, 16 cores), interleaved ABBA rounds with a quiet gate and a temperature gate before every run, round 0 dropped, tools/ab.py --noise per figure, 10 clean pairs each (2 pairs dropped by the noise rule). Ratios below 1 mean the branch is faster. The table is generated from the study output (ab.txt and the per-run files), and the same figures are in the docs through render_ab_tables.py (docs/benchmarks/ab-results/float-vector-crossing.json).

insert_columns is the one that differs by design: on main the best a column load offers is one float[] per row, so its row compares that with the new 2-D column, and its interval is wide enough that the sign test does not reach significance (8 of 10 pairs better). The other four figures are 10 of 10 pairs better.

The A/B was measured on the pre-rebase commit ad4e8a394e. This branch is that commit rebased onto origin/main at 43d723c9e1 (29 commits); the code is unchanged by the rebase except one conflict in tests/test_python_callbacks.py, where main added an xfail marker (free-threaded heap growth, #454) above test_registering_again_and_again_does_not_grow_the_python_heap and this change added test_a_numpy_array_is_a_return_value in the same place; both are kept. The rebased commit adds the docs table, the figure id in render_ab_tables.py and the result file. The study's base wheel was built before the rebase, so the speed table is not a measurement of the rebased tree.

Tests run

Locally, one batched job through quiet_build (this branch's src and a javac-recompiled bridge jar overlaid on an unpacked linux/amd64 cp312 wheel): test_float_vector_bulk.py, test_vector.py, test_python_callbacks.py, test_graph_batch.py, test_bulk_insert.py, test_insert_columns.py, test_dataframe_ingest.py, test_bulk_json_paths.py, test_bridge_fixes.py: 291 passed, 1 skipped (a polars import in test_dataframe_ingest.py, polars was not installed in the job). Lint: black, isort --profile black, and the repo's pre-commit hooks (including the A/B tables check) pass on the commit; bandit finds nothing at medium or above. The stock wheel build and the CI matrix are the remaining checks. Not run locally: the free-threaded and the other-OS wheels.

For a port to ArcadeData/arcadedb-python: its DocumentBatcher.java, VertexBatcher.java and GraphCalls.java are byte-identical to ours before this change, so the diff of those three files plus the new FloatVectors.java applies as is; it does not have PythonFunctions or PythonRecordEvents (not touched here).

🤖 Generated with Claude Code