Repository navigation
Description
A follow-up to #8983 for a case its fix does not cover. On a NOTUNIQUE_HASH index, a record whose key changes twice in one transaction, away and back (1 -> 4 -> 1), stays listed under that key after a later transaction deletes the record or moves it to another key. The query answer is then wrong: SELECT k FROM T WHERE k = 1 returns a record whose k is 2.
Reproduction (HashMin.java, below): document type T, CREATE PROPERTY T.k INTEGER, CREATE INDEX ON T (k) <type>, one committed record X with k = 1, then the transactions in the table. After the last commit, every key's index.get, SELECT count(*) FROM T WHERE k = ? and the records are compared:
| transactions | NOTUNIQUE_HASH |
NOTUNIQUE (LSM) |
UNIQUE_HASH |
|---|---|---|---|
tx1: update X k 1 -> 4, then 4 -> 1 |
right | right | right |
| tx1 as above; tx2: delete X | index get(1) 1 entry for X, count(*) WHERE k = 1 = 0, SELECT k ... WHERE k = 1 -> [] |
right | right |
tx1 as above; tx2: update X k 1 -> 2 |
index get(1) 1 entry for X, count(*) WHERE k = 1 = 1, SELECT k ... WHERE k = 1 -> [2] |
right | right |
| control: update X 1 -> 2 (one change) | right | right | right |
| control: delete X | right | right | right |
After tx1 alone the index answers right. After tx2, X's entry under k = 1 is still there. Identical on 26.10.2 (e5040d0339) and on main cdaade99c1, eclipse-temurin 25 (-XX:+UseCompactObjectHeaders) and 21. A random single-thread workload (8 keys; inserts, key updates, deletes; every key checked after every commit) breaks a NOTUNIQUE_HASH index in 6 of 20 seeds, while NOTUNIQUE on the same workload breaks in none. In the first one found (seed 1), record #1:1 makes that round trip (k 1 -> 4 ... 4 -> 1) in commit 6 and is deleted in commit 8, numbering from 0.
HashMin.java
// Small fixed scenarios on an index of the given type on T.k: committed records first, then ONE transaction, then the index
// (get per key) and SQL (count per key) held against the records after commit. java HashMin.java <dir> <NOTUNIQUE_HASH|NOTUNIQUE|UNIQUE_HASH>
import com.arcadedb.database.*;
import com.arcadedb.index.*;
import java.util.*;
import java.util.function.*;
public class HashMin {
static Database db; static String TYPE;
public static void main(String[] a) throws Exception {
TYPE = a[1]; DatabaseFactory f = new DatabaseFactory(a[0]); if (f.exists()) f.open().drop(); db = f.create();
run("S1 X(k=1); delete X", new int[] { 1 }, r -> r.get(0).asDocument().delete());
run("S2 X(k=1); update X 1 -> 2", new int[] { 1 }, r -> set(r.get(0), 2));
run("S3 X(k=1), Y(k=1); delete X", new int[] { 1, 1 }, r -> r.get(0).asDocument().delete());
run("S4 X(k=1); delete X, insert Z(k=1)", new int[] { 1 }, r -> { r.get(0).asDocument().delete(); ins(1); });
run("S5 X(k=0); insert A(k=4), insert B(k=4), update X 0 -> 4", new int[] { 0 }, r -> { ins(4); ins(4); set(r.get(0), 4); });
run("S6 X(k=0); update X 0 -> 4", new int[] { 0 }, r -> set(r.get(0), 4));
run("S7 X(k=0), Y(k=4); update X 0 -> 4", new int[] { 0, 4 }, r -> set(r.get(0), 4));
run("S8 X(k=1), W(k=1); delete X, insert A(k=6), update W 1 -> 3, insert B(k=1), delete A", new int[] { 1, 1 },
r -> { r.get(0).asDocument().delete(); RID x = ins(6); set(r.get(1), 3); ins(1); x.asDocument().delete(); });
run("S9 X(k=1), W(k=1); delete X, update W 1 -> 3, insert B(k=1)", new int[] { 1, 1 }, r -> { r.get(0).asDocument().delete(); set(r.get(1), 3); ins(1); });
run("S10 X(k=1), W(k=1); delete X, update W 1 -> 3", new int[] { 1, 1 }, r -> { r.get(0).asDocument().delete(); set(r.get(1), 3); });
run2("S11 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, null);
run2("S12 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1; tx2: delete X", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, r -> r.get(0).asDocument().delete());
run2("S13 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1; tx2: update X 1 -> 2", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, r -> set(r.get(0), 2));
db.close();
}
static void run2(String label, int[] committed, Consumer<List<RID>> tx1, Consumer<List<RID>> tx2) {
run(label, committed, r -> { tx1.accept(r); if (tx2 != null) { db.commit(); db.begin(); tx2.accept(r); } });
}
static RID ins(int k) { MutableDocument d = db.newDocument("T"); d.set("k", k); d.save(); return d.getIdentity(); }
static void set(RID r, int k) { r.asDocument().modify().set("k", k).save(); }
static void run(String label, int[] committed, Consumer<List<RID>> tx) {
if (db.getSchema().existsType("T")) db.getSchema().dropType("T");
db.command("sql", "CREATE DOCUMENT TYPE T").close(); db.command("sql", "CREATE PROPERTY T.k INTEGER").close(); db.command("sql", "CREATE INDEX ON T (k) " + TYPE).close();
List<RID> rs = new ArrayList<>();
try { db.transaction(() -> { for (int k : committed) rs.add(ins(k)); }); }
catch (Exception e) { System.out.println("RESULT " + TYPE + " " + label + " | setup refused " + e.getClass().getSimpleName() + " (the committed records break this index type) | -"); return; }
String res; try { db.begin(); tx.accept(rs); db.commit(); res = "committed"; } catch (Exception e) { if (db.isTransactionActive()) db.rollback(); res = "refused " + e.getClass().getSimpleName(); }
Map<Integer, Set<RID>> want = new TreeMap<>();
for (var it = db.iterateType("T", true); it.hasNext(); ) { Document d = it.next().asDocument(); want.computeIfAbsent(((Number) d.get("k")).intValue(), x -> new TreeSet<>()).add(d.getIdentity()); }
List<String> wrong = new ArrayList<>(); Index idx = db.getSchema().getIndexByName("T[k]");
for (int k = 0; k <= 6; k++) {
Set<RID> w = want.getOrDefault(k, Set.of()), got = new TreeSet<>(); int n = 0; IndexCursor c = idx.get(new Object[] { k }); while (c.hasNext()) { got.add(c.next().getIdentity()); n++; }
long q; try (var r = db.query("sql", "SELECT count(*) AS c FROM T WHERE k = ?", k)) { q = ((Number) r.next().getProperty("c")).longValue(); }
if (!got.equals(w) || n != w.size() || q != w.size()) {
List<Object> vals = new ArrayList<>(); try (var r = db.query("sql", "SELECT k FROM T WHERE k = ?", k)) { while (r.hasNext()) vals.add(r.next().getProperty("k")); }
wrong.add("k=" + k + ": records " + w + ", index get " + got + " (" + n + " entries), SQL count " + q + ", SELECT k FROM T WHERE k = " + k + " -> " + vals);
}
}
System.out.println("RESULT " + TYPE + " " + label + " | " + res + " | " + (wrong.isEmpty() ? "ok" : "WRONG " + String.join("; ", wrong)));
}
}
Activity
tae898 commentedon Oct 18, 2026
The same with a composite key,
CREATE INDEX ON T (a, b) NOTUNIQUE_HASH. X hasa = 1, b = 7, and tx1 setsato 4 and back to 1. If tx2 then deletes X,get(1, 7)still returns 1 entry. If tx2 setsa = 2,get(1, 7)returns 1 entry andSELECT a FROM T WHERE a = 1 AND b = 7returns[2]. The same round trip followed by a delete or an update answers right on aNOTUNIQUELSM index withNULL_STRATEGY INDEX(k 1 -> null -> 1), aFULL_TEXTindex (text away and back), and anLSM_VECTORindex (vector away and back, then the kNN). So far it is the hash indexes only. (Maincdaade99c1, JDK 25.)review(#10194,#10269): failure class only in the rollback-only reason…
review(#10194,#10269): composite LSM key, eager INSERT FROM SELECT fa…
review(#10194,#10264,#10269): guard analyze().execute() too, snapshot…
fix(#10194,#10264,#10269): UPDATE SET reads the row as it was, explic…
lvca commentedon Oct 19, 2026
Fixed in #10347 (milestone 26.11.1).
When a transaction removed and re-added the same key for the same record on a non-unique hash index (for example an update that put a value back), the index could end up holding the same record twice for that key, so a lookup returned a duplicate row. The pair is now replaced instead of added twice. Covered for hash and LSM indexes, unique and non-unique, single and composite keys, including after reopening the database.
❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.
fix(#10194,#10264,#10269): UPDATE SET reads the row as it was, explic…