Visitar URL original
HA: the late-sample record of a continuous aggregate is shipped right after the append commits, so a leader lost in between drops it · Issue #10673 · ArcadeData/arcadedb · GitHub
Skip to content

HA: the late-sample record of a continuous aggregate is shipped right after the append commits, so a leader lost in between drops it #10673

Closed

Description

@robfrank

Follow-up of #10634 (derived from reading the code, not reproduced on a cluster).

Since #10634 a sample appended behind the watermark of a continuous aggregate ships its late-sample record (#10502) to the followers in a schema recording session of its own (TimeSeriesShard.invalidateContinuousAggregatesBehindWatermark -> LocalSchema.recordContinuousAggregateChange). That session runs right after the append's own transaction committed, not inside it: the append commits through a nested, self-owned transaction under the shard's append lock (#7410/#7657), and the record must not be written before the commit (a refresh reading the source before it would clear it).

Remaining gap: a leader lost between the append's TX_ENTRY and the schema entry carrying the record leaves the followers with the samples and without the record. A follower promoted then never aggregates those late samples into their buckets (until a delete or a TRUNCATE invalidates them again).

Other related, non-gaps (for the record):

Options: carry the record inside the append's replication unit (e.g. an optional section of the TX_ENTRY, which needs a capability per ha-raft/CLAUDE.md), or have a newly elected leader treat the buckets of the samples appended in the last N entries as invalidated.

Activity

  1. added this to the 26.11.1 milestone on Oct 23, 2026
  2. added
    haHigh availability / replication (Raft)
    severity:minorCosmetic, documentation, edge case, or a workaround exists
    on Oct 23, 2026
  3. self-assigned this
    on Oct 23, 2026
  4. added a commit that references this issue on Oct 28, 2026
  5. robfrank commented on Oct 28, 2026

    @robfrank
    CollaboratorAuthor

    Fixed in #10858, with one narrow race left. Under HA, the late-sample record of a continuous aggregate used to reach the followers after the samples committed, so a leader lost in between left followers with the samples and no record. The record is now shipped in a schema session before the samples commit; Raft applies entries in order, so a follower that holds the samples also holds their record. There is no wire-format change. If the pre-commit session fails or the record changed in between, a second session ships after the commit as before. A new two-node HA test fails on the old ordering and passes now.

    Still open: #10857, a refresh that starts after the pre-commit check and finishes before the commit can make the samples late only at commit time. Their record then ships after the commit, as it did before this PR.


    ❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

  6. removed
    in progressThe team is actively working on this issue
    on Oct 28, 2026
  7. added a commit that references this issue on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bughaHigh availability / replication (Raft)severity:minorCosmetic, documentation, edge case, or a workaround existstimeseriesTime series module

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions