Visitar URL original
Bridge: float vectors cross as one binary buffer, not as JSON text or… · humemai/arcadedb-embedded-python@8bbe458 · GitHub
Skip to content

Commit 8bbe458

Browse files
tae898claude
andauthored
Bridge: float vectors cross as one binary buffer, not as JSON text or one JPype call per element (#496)
* Bridge: float vectors cross as one binary buffer, not as JSON text or one JPype call per element A vector load spent most of its time on text: Python wrote every float as a decimal number and the engine parsed it back (insert_many), or JPype converted every list element on its own (create_vertices, Document.set). For a property declared ARRAY_OF_FLOATS the engine converts each number of a list to float anyway, so the vectors of a chunk now cross as one float32 matrix (FloatVectors.Matrix, one buffer copy) beside the JSON, and each document gets its own float[] row Java-side. Properties that are not declared ARRAY_OF_FLOATS, and vectors that are ragged, missing or null, keep the exact JSON path. Also: insert_columns accepts a 2-D float32 array as a column of vectors; a numpy array is accepted as a property value everywhere (Document.set, insert_many, create_vertices; it raised TypeError before); the allowed_rids filter of a vector search builds its RID set in one crossing (GraphCalls.ridSet) instead of two per RID. The speed table in docs/guide/vectors.md is generated by render_ab_tables.py from the laptop A/B study (tools/ab.py, ABBA, 10 clean pairs per figure). Tests: tests/test_float_vector_bulk.py (39 tests: exact float32 equality with the engine's own list conversion on doubles a float32 cannot hold, which values keep the JSON path, every entry point, parallel mode); tests/test_vector.py rid-conversion test now passes a Java RID object so the list takes the per-RID path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests: mark the three test-owned SQL strings of test_float_vector_bulk.py for bandit B608 CI runs bandit at low severity and confidence; the query strings use the test's own type names and RIDs. Same pattern as test_graph_batch.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 43d723c commit 8bbe458

17 files changed

Lines changed: 917 additions & 49 deletions

File tree

‎bindings/python/docs/api/database.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -506,6 +506,8 @@ db.insert_many(type_name: str, rows, commit_every: int = 10_000,
506506
parallel: bool = False) -> int
507507
```
508508

509+
Float vectors take a binary path: for a property declared `ARRAY_OF_FLOATS` whose value in every row of a chunk is a list or a numpy array of numbers of one length, the vectors of the chunk cross as one `float32` matrix beside the JSON, instead of as text. The stored values are the list's (the engine converts a list for such a property to float). A property that is not declared `ARRAY_OF_FLOATS`, or a vector that is ragged, missing or null in a row, stays in the JSON.
510+
509511
Bulk-insert documents with **one FFI crossing per chunk of rows** instead of
510512
several JNI calls per row. The iterable is read 10,000 rows at a time and each
511513
chunk is serialized to one JSON string and looped Java-side, so memory stays
@@ -595,6 +597,8 @@ row-wise bridges gained only 2-12% (tuples plus `dict(zip())`, `DataFrame.to_jso
595597
Same failure contract as `insert_many`: the transaction the call opened is
596598
rolled back on any failure, a transaction the caller opened is left to the caller.
597599

600+
A column can also be a 2-D numpy `float32` array: a column of vectors. The whole matrix crosses as one `float[]` (one buffer copy) and each document gets one row of it, which is what an `ARRAY_OF_FLOATS` property stores; no list, no JSON text. Another dtype (cast with `astype(np.float32)`), or a matrix with no columns, raises `ValueError`.
601+
598602
**Parameters:**
599603

600604
- `type_name` (str): Target document type (must exist)

‎bindings/python/docs/api/vector.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -332,7 +332,8 @@ also the default. If you explicitly disable eager preparation, the first call to
332332
ArcadeDB's default/adaptive search behavior. Anything but a positive integer raises
333333
`ArcadeDBError`.
334334
- `allowed_rids` (List[str]): Optional list of RID strings (e.g. `["#1:0", "#2:5"]`) to
335-
restrict search (default: `None`)
335+
restrict search (default: `None`). RIDs given as text become the filter set in one bridge
336+
call; a list that holds RID objects is added one by one.
336337

337338
**Returns:**
338339

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
{
2+
"format": 1,
3+
"figure": "float-vector-crossing",
4+
"environment": {
5+
"engine_version": "26.11.1",
6+
"wheel_version": "26.11.1.dev0",
7+
"cores": "2 (pinned)",
8+
"machine": "x86-64 laptop, 16 cores"
9+
},
10+
"comparisons": [
11+
{
12+
"case": "insert_many, 100,000 dict rows, 128-d float32, chunks of 20,000",
13+
"metric": "time",
14+
"unit": "ms",
15+
"better": "lower",
16+
"a": "before",
17+
"b": "one buffer",
18+
"a_median": 14214.9033,
19+
"b_median": 2620.5005,
20+
"ratio": 0.185,
21+
"ci_low": 0.18,
22+
"ci_high": 0.191,
23+
"clean_pairs": 10,
24+
"dropped_pairs": 2,
25+
"sign_b_better": 10,
26+
"sign_b_worse": 0,
27+
"sign_p": 0.00195
28+
},
29+
{
30+
"case": "insert_columns, 100,000 rows, 128-d float32",
31+
"metric": "time",
32+
"unit": "ms",
33+
"better": "lower",
34+
"a": "before",
35+
"b": "one buffer",
36+
"a_median": 609.405986,
37+
"b_median": 520.802979,
38+
"ratio": 0.862,
39+
"ci_low": 0.833,
40+
"ci_high": 0.947,
41+
"clean_pairs": 10,
42+
"dropped_pairs": 2,
43+
"sign_b_better": 8,
44+
"sign_b_worse": 2,
45+
"sign_p": 0.109
46+
},
47+
{
48+
"case": "graph_batch create_vertices, 50,000 rows, 128-d float32",
49+
"metric": "time",
50+
"unit": "ms",
51+
"better": "lower",
52+
"a": "before",
53+
"b": "one buffer",
54+
"a_median": 3596.351745,
55+
"b_median": 1082.5297249999999,
56+
"ratio": 0.3,
57+
"ci_low": 0.289,
58+
"ci_high": 0.315,
59+
"clean_pairs": 10,
60+
"dropped_pairs": 2,
61+
"sign_b_better": 10,
62+
"sign_b_worse": 0,
63+
"sign_p": 0.00195
64+
},
65+
{
66+
"case": "find_nearest(k=10) with 5,000 allowed RIDs, 200 queries, 20,000 vectors",
67+
"metric": "time",
68+
"unit": "ms",
69+
"better": "lower",
70+
"a": "before",
71+
"b": "one buffer",
72+
"a_median": 2295.743685,
73+
"b_median": 568.9830085000001,
74+
"ratio": 0.246,
75+
"ci_low": 0.235,
76+
"ci_high": 0.257,
77+
"clean_pairs": 10,
78+
"dropped_pairs": 2,
79+
"sign_b_better": 10,
80+
"sign_b_worse": 0,
81+
"sign_p": 0.00195
82+
},
83+
{
84+
"case": "all four steps of the probe",
85+
"metric": "total time",
86+
"unit": "ms",
87+
"better": "lower",
88+
"a": "before",
89+
"b": "one buffer",
90+
"a_median": 20605.23805,
91+
"b_median": 4784.8637,
92+
"ratio": 0.232,
93+
"ci_low": 0.23,
94+
"ci_high": 0.236,
95+
"clean_pairs": 10,
96+
"dropped_pairs": 2,
97+
"sign_b_better": 10,
98+
"sign_b_worse": 0,
99+
"sign_p": 0.00195
100+
}
101+
]
102+
}

‎bindings/python/docs/development/bridge.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,14 @@ sources live in `bindings/python/src/java/com/arcadedb/python/`:
66
|---|---|
77
| `RowBatcher` | Serializes up to N result rows into one JSON-array string per call (batched row transport) |
88
| `ColumnBatcher` | Encodes up to N rows into one `byte[]` of typed little-endian column buffers plus null bitmaps (binary columnar transport) |
9-
| `DocumentBatcher` | Inserts a whole batch of documents from one JSON-rows string (transactional or async parallel writers), or from whole typed columns (`insertColumns`, one `long[]`, `double[]`, `boolean[]`, or `Object[]` per property); also boxes numpy numeric arrays for `append_samples` |
9+
| `DocumentBatcher` | Inserts a whole batch of documents from one JSON-rows string (transactional or async parallel writers), or from whole typed columns (`insertColumns`, one `long[]`, `double[]`, `boolean[]`, or `Object[]` per property); also boxes numpy numeric arrays for `append_samples`; a float vector of an `ARRAY_OF_FLOATS` property crosses as a binary `FloatVectors.Matrix` beside the JSON (`insertManyJson`) or as a whole 2-D `float32` column |
1010
| `EdgeBatcher` | Buffers a whole batch of edges into `GraphBatch` from one call (RID strings, or JSON rows for edges with properties) |
1111
| `VertexBatcher` | Creates a whole batch of vertices from one JSON-rows string, returning all RIDs as one joined string |
1212
| `TimeSeriesBatcher` | Fills the engine's primitive `TimeSeriesBatch` one column per call, so numeric samples are never boxed |
1313
| `RowAccess` | Hands a row's names and values to Python in one call (`namesAndValues`), and up to N such rows per call (`nextRows`, the fallback for `to_list()` when the bridge jar lacks `TypedRows`); the values are the engine's own objects, so Python converts them with full type fidelity. Also the first row of a result set with the set closed (`firstAndClose`, behind `ResultSet.first()`) and a property or null (`propertyOrNull`, behind `Result.get()`), each in one call instead of two or three |
1414
| `TypedRows` | Up to N rows per call as one JSON string, type-faithful (`nextRows`, which also reports whether the result set is drained): DECIMAL, BigInteger, DATE, DATETIME, Instant and zoned values, and sets are tagged and restored exactly by Python, and every other value (RIDs, embedded documents, `float[]` and `double[]`, non-finite floats, maps with non-String keys, years Python cannot hold) is returned in a side array as the engine's own object and converted by Python as before. Behind `ResultSet.to_list()` (up to 1,000 rows or 4 MiB of JSON per call, whichever comes first) and `ResultSet.iter_dicts()` (one row per call, so a lazy caller reads rows when it asks) |
15-
| `GraphCalls` | Every edge of `Vertex.getEdges(direction, labels)` in one call (`outEdges`, `inEdges`, `bothEdges`, behind `Vertex.get_out_edges()` and its siblings), and the record and score of every vector search hit in one call (`hits`, behind `LSMVectorIndex` search results) |
15+
| `FloatVectors` | Float vectors as one binary buffer instead of JSON text or a JPype call per element: `Matrix` (n vectors in one `float[]`, a row copied into each document), and `declared` (which properties of a type are `ARRAY_OF_FLOATS`); behind `insert_many()`, `insert_columns()` and `GraphBatch.create_vertices()` |
16+
| `GraphCalls` | Every edge of `Vertex.getEdges(direction, labels)` in one call (`outEdges`, `inEdges`, `bothEdges`, behind `Vertex.get_out_edges()` and its siblings), and the record and score of every vector search hit in one call (`hits`, behind `LSMVectorIndex` search results), and the `allowed_rids` set of a vector search from RID strings in one call (`ridSet`) |
1617
| `DbCalls` | One-row statements and lookups in one call: `commandNamed`, `queryNamed`, `commandPositional`, and `queryPositional` take the parameters as the varargs of a single non-overloaded method and build the map Java-side (behind `Database.command()` and `Database.query()`), and `lookupFirst` is the whole of `Database.lookup_by_key()` |
1718
| `PythonFunctions` | Registers a Python callable as an engine `SQLFunction` in the process-wide function factory (so SQL and openCypher resolve it): the arity check, one `CommandExecutionException` naming the function for any failure of the callable, and a revoked flag so an unregistered function fails clearly. Behind `Database.register_function()` and `unregister_function()`. Each call is one JNI round trip |
1819
| `PythonRecordEvents` | One listener class for the six write events (before and after create, update, delete) that forwards each to a Python callback; a `false` from a before event is the veto, and a failure is wrapped in a `DatabaseOperationException` naming the event. Behind `Database.on()` |
@@ -41,10 +42,10 @@ crossing per batch**, receiving a bulk payload it can decode at C speed: the
4142
| `ResultSet.to_json_list()` / `iter_json_batches()` | `RowBatcher` |
4243
| `ResultSet.to_list()` (rows in batches), `ResultSet.iter_dicts()` (one row per call) | `TypedRows` |
4344
| `Result.to_dict()` (one row) | `RowAccess` |
44-
| `Vertex.get_out_edges()` / `get_in_edges()` / `get_both_edges()`, vector search results | `GraphCalls` |
45+
| `Vertex.get_out_edges()` / `get_in_edges()` / `get_both_edges()`, vector search results and their `allowed_rids` set | `GraphCalls` |
4546
| `ResultSet.to_columns()` / fast `to_dataframe()` / `to_arrow()` | `ColumnBatcher` |
46-
| `Database.insert_many()` | `DocumentBatcher` |
47-
| `Database.insert_columns()` | `DocumentBatcher` |
47+
| `Database.insert_many()` | `DocumentBatcher`, `FloatVectors` (vector properties) |
48+
| `Database.insert_columns()` | `DocumentBatcher`, `FloatVectors` (a 2-D `float32` column) |
4849
| `Database.insert_dataframe()` / `insert_arrow()` | `DocumentBatcher` (document types, through `insert_columns()`), `VertexBatcher` (vertex types, through `GraphBatch.create_vertices()`); the Arrow-to-column conversion is Python (`dataframe.py`) |
4950
| `ResultSet.to_polars()` | `ColumnBatcher` (through `to_arrow()`) |
5051
| `AsyncExecutor.append_samples()` (numpy numeric-column boxing) | `DocumentBatcher` |

‎bindings/python/docs/guide/vectors.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,43 @@ Notes:
273273
- For euclidean, skip normalization if magnitude should matter. For dot_product,
274274
normalize: the engine expects unit-length vectors.
275275

276+
### Loading many vectors
277+
278+
Declare the property `ARRAY_OF_FLOATS`, then load the vectors in bulk. The vectors of a chunk cross the bridge as one `float32` buffer, not as JSON text and not one JPype call per element:
279+
280+
```python
281+
db.command("sql", "CREATE DOCUMENT TYPE Doc")
282+
db.command("sql", "CREATE PROPERTY Doc.embedding ARRAY_OF_FLOATS")
283+
284+
# rows with a list or a numpy array per vector (the property must be declared ARRAY_OF_FLOATS)
285+
db.insert_many("Doc", [{"id": i, "embedding": x[i]} for i in range(len(x))])
286+
287+
# or whole columns: a 2-D float32 matrix is one column of vectors
288+
db.insert_columns("Doc", {"id": ids, "embedding": x}) # x: float32, shape (n, dims)
289+
290+
# a graph load takes the same rows
291+
with db.graph_batch() as gb:
292+
gb.create_vertices("Vx", [{"id": i, "embedding": x[i]} for i in range(len(x))])
293+
```
294+
295+
A `float64` list or array stores the same values it always did (the engine converts it to float for this property). Declaring the property is what makes the path apply: an undeclared property keeps the exact doubles in a list. A numpy array is accepted wherever a vector is a property value (`Document.set`, `insert_many`, `create_vertices`).
296+
297+
The figures below compare the bindings before this change (A, JSON text or one JPype call per element) with the bindings after it (B, one binary buffer), built from one base wheel so the engine jars and the JVM are identical. The workload is 128-d `float32` vectors on an `ARRAY_OF_FLOATS` property, a JVM heap of 4 GB, one fresh process per run. `insert_columns` takes a different path in A: a wheel without 2-D columns offers one `float[]` per row, so its row compares that best case with the 2-D column.
298+
299+
<!-- ab-table: float-vector-crossing -->
300+
| Case | Metric | A | B | A median | B median | B vs A (95% interval) | Sign test (B) | Verdict | Clean pairs |
301+
|---|---|---|---|---|---|---|---|---|---|
302+
| insert_many, 100,000 dict rows, 128-d float32, chunks of 20,000 | time (ms, lower is better) | before | one buffer | 14,215 | 2,621 | 0.185 (0.180 to 0.191) | 10 better, 0 worse, p = 0.00195 | B better by 81 to 82 percent | 10 (2 dropped) |
303+
| insert_columns, 100,000 rows, 128-d float32 | time (ms, lower is better) | before | one buffer | 609 | 521 | 0.862 (0.833 to 0.947) | 8 better, 2 worse, p = 0.109 | B better by 5 to 17 percent | 10 (2 dropped) |
304+
| graph_batch create_vertices, 50,000 rows, 128-d float32 | time (ms, lower is better) | before | one buffer | 3,596 | 1,083 | 0.300 (0.289 to 0.315) | 10 better, 0 worse, p = 0.00195 | B better by 68 to 71 percent | 10 (2 dropped) |
305+
| find_nearest(k=10) with 5,000 allowed RIDs, 200 queries, 20,000 vectors | time (ms, lower is better) | before | one buffer | 2,296 | 569 | 0.246 (0.235 to 0.257) | 10 better, 0 worse, p = 0.00195 | B better by 74 to 76 percent | 10 (2 dropped) |
306+
| all four steps of the probe | total time (ms, lower is better) | before | one buffer | 20,605 | 4,785 | 0.232 (0.230 to 0.236) | 10 better, 0 worse, p = 0.00195 | B better by 76 to 77 percent | 10 (2 dropped) |
307+
308+
Medians are over the clean pairs of interleaved runs, one fresh process per run, round 0 dropped. B vs A is the median of the per-round ratios with its bootstrap 95% interval; above 1.000 means B is worse (B / A for a time, A / B for a rate). A verdict needs the interval to exclude 1.000 and at least 7 clean pairs.
309+
310+
Engine 26.11.1, wheel 26.11.1.dev0, cores 2 (pinned), machine x86-64 laptop, 16 cores.
311+
<!-- /ab-table -->
312+
276313
## Preferred Search Surface: SQL / Cypher
277314

278315
For new code, prefer query APIs for search. Bind the query vector as a parameter

‎bindings/python/scripts/render_ab_tables.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@
7373
"udf-cost": "Python function in SQL against the equivalent SQL expression",
7474
"callback-cost": "Per-record callback cost of Database.on(...)",
7575
"heap-pinned": "JVM heap pinned (-Xms equal to -Xmx) against unpinned",
76+
"float-vector-crossing": "Float vectors across the bridge as one binary buffer against JSON text or one JPype call per element",
7677
}
7778

7879
OPEN = re.compile(r"<!-- ab-table: (?P<id>[A-Za-z0-9_-]+) -->")

0 commit comments

Comments
 (0)