Visitar URL original
NOTUNIQUE_HASH: a record whose key changes away and back in one transaction stays under that key after a later delete or update; SELECT ... WHERE k = 1 returns a record with k = 2 · Issue #10269 · ArcadeData/arcadedb · GitHub
Skip to content

NOTUNIQUE_HASH: a record whose key changes away and back in one transaction stays under that key after a later delete or update; SELECT ... WHERE k = 1 returns a record with k = 2 #10269

Closed

Description

@tae898

A follow-up to #8983 for a case its fix does not cover. On a NOTUNIQUE_HASH index, a record whose key changes twice in one transaction, away and back (1 -> 4 -> 1), stays listed under that key after a later transaction deletes the record or moves it to another key. The query answer is then wrong: SELECT k FROM T WHERE k = 1 returns a record whose k is 2.

Reproduction (HashMin.java, below): document type T, CREATE PROPERTY T.k INTEGER, CREATE INDEX ON T (k) <type>, one committed record X with k = 1, then the transactions in the table. After the last commit, every key's index.get, SELECT count(*) FROM T WHERE k = ? and the records are compared:

transactions NOTUNIQUE_HASH NOTUNIQUE (LSM) UNIQUE_HASH
tx1: update X k 1 -> 4, then 4 -> 1 right right right
tx1 as above; tx2: delete X index get(1) 1 entry for X, count(*) WHERE k = 1 = 0, SELECT k ... WHERE k = 1 -> [] right right
tx1 as above; tx2: update X k 1 -> 2 index get(1) 1 entry for X, count(*) WHERE k = 1 = 1, SELECT k ... WHERE k = 1 -> [2] right right
control: update X 1 -> 2 (one change) right right right
control: delete X right right right

After tx1 alone the index answers right. After tx2, X's entry under k = 1 is still there. Identical on 26.10.2 (e5040d0339) and on main cdaade99c1, eclipse-temurin 25 (-XX:+UseCompactObjectHeaders) and 21. A random single-thread workload (8 keys; inserts, key updates, deletes; every key checked after every commit) breaks a NOTUNIQUE_HASH index in 6 of 20 seeds, while NOTUNIQUE on the same workload breaks in none. In the first one found (seed 1), record #1:1 makes that round trip (k 1 -> 4 ... 4 -> 1) in commit 6 and is deleted in commit 8, numbering from 0.

HashMin.java
// Small fixed scenarios on an index of the given type on T.k: committed records first, then ONE transaction, then the index
// (get per key) and SQL (count per key) held against the records after commit.   java HashMin.java <dir> <NOTUNIQUE_HASH|NOTUNIQUE|UNIQUE_HASH>
import com.arcadedb.database.*;
import com.arcadedb.index.*;
import java.util.*;
import java.util.function.*;

public class HashMin {
  static Database db; static String TYPE;
  public static void main(String[] a) throws Exception {
    TYPE = a[1]; DatabaseFactory f = new DatabaseFactory(a[0]); if (f.exists()) f.open().drop(); db = f.create();
    run("S1 X(k=1); delete X", new int[] { 1 }, r -> r.get(0).asDocument().delete());
    run("S2 X(k=1); update X 1 -> 2", new int[] { 1 }, r -> set(r.get(0), 2));
    run("S3 X(k=1), Y(k=1); delete X", new int[] { 1, 1 }, r -> r.get(0).asDocument().delete());
    run("S4 X(k=1); delete X, insert Z(k=1)", new int[] { 1 }, r -> { r.get(0).asDocument().delete(); ins(1); });
    run("S5 X(k=0); insert A(k=4), insert B(k=4), update X 0 -> 4", new int[] { 0 }, r -> { ins(4); ins(4); set(r.get(0), 4); });
    run("S6 X(k=0); update X 0 -> 4", new int[] { 0 }, r -> set(r.get(0), 4));
    run("S7 X(k=0), Y(k=4); update X 0 -> 4", new int[] { 0, 4 }, r -> set(r.get(0), 4));
    run("S8 X(k=1), W(k=1); delete X, insert A(k=6), update W 1 -> 3, insert B(k=1), delete A", new int[] { 1, 1 },
        r -> { r.get(0).asDocument().delete(); RID x = ins(6); set(r.get(1), 3); ins(1); x.asDocument().delete(); });
    run("S9 X(k=1), W(k=1); delete X, update W 1 -> 3, insert B(k=1)", new int[] { 1, 1 }, r -> { r.get(0).asDocument().delete(); set(r.get(1), 3); ins(1); });
    run("S10 X(k=1), W(k=1); delete X, update W 1 -> 3", new int[] { 1, 1 }, r -> { r.get(0).asDocument().delete(); set(r.get(1), 3); });
    run2("S11 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, null);
    run2("S12 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1; tx2: delete X", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, r -> r.get(0).asDocument().delete());
    run2("S13 X(k=1); tx1: update X 1 -> 4, update X 4 -> 1; tx2: update X 1 -> 2", new int[] { 1 }, r -> { set(r.get(0), 4); set(r.get(0), 1); }, r -> set(r.get(0), 2));
    db.close();
  }
  static void run2(String label, int[] committed, Consumer<List<RID>> tx1, Consumer<List<RID>> tx2) {
    run(label, committed, r -> { tx1.accept(r); if (tx2 != null) { db.commit(); db.begin(); tx2.accept(r); } });
  }
  static RID ins(int k) { MutableDocument d = db.newDocument("T"); d.set("k", k); d.save(); return d.getIdentity(); }
  static void set(RID r, int k) { r.asDocument().modify().set("k", k).save(); }
  static void run(String label, int[] committed, Consumer<List<RID>> tx) {
    if (db.getSchema().existsType("T")) db.getSchema().dropType("T");
    db.command("sql", "CREATE DOCUMENT TYPE T").close(); db.command("sql", "CREATE PROPERTY T.k INTEGER").close(); db.command("sql", "CREATE INDEX ON T (k) " + TYPE).close();
    List<RID> rs = new ArrayList<>();
    try { db.transaction(() -> { for (int k : committed) rs.add(ins(k)); }); }
    catch (Exception e) { System.out.println("RESULT " + TYPE + " " + label + " | setup refused " + e.getClass().getSimpleName() + " (the committed records break this index type) | -"); return; }
    String res; try { db.begin(); tx.accept(rs); db.commit(); res = "committed"; } catch (Exception e) { if (db.isTransactionActive()) db.rollback(); res = "refused " + e.getClass().getSimpleName(); }
    Map<Integer, Set<RID>> want = new TreeMap<>();
    for (var it = db.iterateType("T", true); it.hasNext(); ) { Document d = it.next().asDocument(); want.computeIfAbsent(((Number) d.get("k")).intValue(), x -> new TreeSet<>()).add(d.getIdentity()); }
    List<String> wrong = new ArrayList<>(); Index idx = db.getSchema().getIndexByName("T[k]");
    for (int k = 0; k <= 6; k++) {
      Set<RID> w = want.getOrDefault(k, Set.of()), got = new TreeSet<>(); int n = 0; IndexCursor c = idx.get(new Object[] { k }); while (c.hasNext()) { got.add(c.next().getIdentity()); n++; }
      long q; try (var r = db.query("sql", "SELECT count(*) AS c FROM T WHERE k = ?", k)) { q = ((Number) r.next().getProperty("c")).longValue(); }
      if (!got.equals(w) || n != w.size() || q != w.size()) {
        List<Object> vals = new ArrayList<>(); try (var r = db.query("sql", "SELECT k FROM T WHERE k = ?", k)) { while (r.hasNext()) vals.add(r.next().getProperty("k")); }
        wrong.add("k=" + k + ": records " + w + ", index get " + got + " (" + n + " entries), SQL count " + q + ", SELECT k FROM T WHERE k = " + k + " -> " + vals);
      }
    }
    System.out.println("RESULT " + TYPE + " " + label + " | " + res + " | " + (wrong.isEmpty() ? "ok" : "WRONG " + String.join("; ", wrong)));
  }
}

Activity

  1. added
    engineCore engine: document/schema API
    severity:majorA feature is broken or returns wrong results, with no practical workaround
    on Oct 18, 2026
  2. tae898 commented on Oct 18, 2026

    @tae898
    ContributorAuthor

    The same with a composite key, CREATE INDEX ON T (a, b) NOTUNIQUE_HASH. X has a = 1, b = 7, and tx1 sets a to 4 and back to 1. If tx2 then deletes X, get(1, 7) still returns 1 entry. If tx2 sets a = 2, get(1, 7) returns 1 entry and SELECT a FROM T WHERE a = 1 AND b = 7 returns [2]. The same round trip followed by a delete or an update answers right on a NOTUNIQUE LSM index with NULL_STRATEGY INDEX (k 1 -> null -> 1), a FULL_TEXT index (text away and back), and an LSM_VECTOR index (vector away and back, then the kNN). So far it is the hash indexes only. (Main cdaade99c1, JDK 25.)

  3. added this to the 26.11.1 milestone on Oct 18, 2026
  4. added
    in progressThe team is actively working on this issue
    on Oct 19, 2026
  5. self-assigned this
    on Oct 19, 2026
  6. added 4 commits that reference this issue on Oct 19, 2026
  7. lvca commented on Oct 19, 2026

    @lvca
    Member

    Fixed in #10347 (milestone 26.11.1).

    When a transaction removed and re-added the same key for the same record on a non-unique hash index (for example an update that put a value back), the index could end up holding the same record twice for that key, so a lookup returned a duplicate row. The pair is now replaced instead of added twice. Covered for hash and LSM indexes, unique and non-unique, single and composite keys, including after reopening the database.


    ❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

  8. removed
    in progressThe team is actively working on this issue
    on Oct 19, 2026
  9. added a commit that references this issue on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugengineCore engine: document/schema APIseverity:majorA feature is broken or returns wrong results, with no practical workaroundsql

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions