Fix query coverage and handoffs across storage-tier retention - #24097

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/web/api/queries/query-execute.c, line 357:

<comment>The coarse-overlap preference can discard valid fine-tier data when the coarse record is a stored gap. Exclude `storage_point_is_gap(sp2)` from `coarser_overlap` so only numeric coarse records can replace an equal-start fine record.</comment>

<file context>
@@ -292,28 +323,56 @@ NOT_INLINE_HOT void rrd2rrdr_query_execute(RRDR *r, size_t dim_id_in_rrdr, QUERY
                     bool finer_total_overlap =
                         ops->point_mode == QUERY_POINT_MODE_TOTAL &&
                         sp2_tier < sp_tier && sp2.start_time_s < sp.end_time_s;
+                    bool coarser_overlap = sp2_tier > sp_tier && !storage_point_is_unset(sp2) &&
+                                           sp2.start_time_s < sp.end_time_s;
 
</file context>

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/web/api/queries/query-plan.c, line 440:

<comment>This singleton branch admits a retained sample exactly at `after_wanted`, violating the query's exclusive lower boundary and allowing a tier>0 backward-expanded scan to add pre-window data to the first bucket. Keep singleton retention points only when they are strictly after `after_wanted` (while retaining the inclusive `before_wanted` endpoint).</comment>

<file context>
@@ -347,10 +362,119 @@ bool query_planer_next_plan(QUERY_ENGINE_OPS *ops, time_t now, time_t last_point
+
+        time_t after = MAX(after_wanted, qm->tiers[tier].db_first_time_s);
+        time_t before = MIN(before_wanted, qm->tiers[tier].db_last_time_s);
+        if(after != before)
+            continue;
+
</file context>

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/web/api/queries/query-execute.c, line 326:

<comment>The fine→coarse handoff protection (skip coarse records already covered by fine reads, clip surviving coarse records at `consumed_end_time`) is wired only into the read-ahead arm that runs when `query_plan_should_switch_plan(ops, sp.end_time_s)` fires mid-point. When the switch instead happens at the top-of-row check (`query_result_plan_should_switch_plan` → `query_planer_next_plan`), the coarse plan's handle begins at its `expanded_after` (several coarse records before `plan[1].after`), and those early coarse records are read raw into the new row with no `consumed` skip and no start-time clipping. Rows right after the handoff can include full out-of-row coarse values (a record spanning the interval before the row) on top of the fine rows already consumed, inflating SUM and distorting min/max/percentile rows; only the AVERAGE path is partly shielded by the seam's per-row duration clip.</comment>

<file context>
@@ -292,28 +323,56 @@ NOT_INLINE_HOT void rrd2rrdr_query_execute(RRDR *r, size_t dim_id_in_rrdr, QUERY
                     ops->db_points_read_per_tier[ops->tier]++;
                     ops->db_total_points_read++;
 
+                    if(sp2_tier > sp_tier) {
+                        // Expanded coarse queries may begin with records already read from the fine tier.
+                        time_t covered_end = consumed_end_time;
</file context>

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/web/api/queries/query-plan.c, line 546:

<comment>The head-candidate search still assumes plan tiers are numerically ordered, so it fails on the non-monotonic retention layout this PR adds. Search for the selected tier’s matching retention-start entry directly; otherwise every plan, including the unnecessary prefix and future plans, is initialized.</comment>

<file context>
@@ -524,12 +536,29 @@ static bool query_plan_build_entries(QUERY_ENGINE_OPS *ops, time_t after_wanted,
+        query_metric_best_tier_for_timeframe(qm, qt->window.after, qt->window.before, qt->window.points) :
+        qm->plan.array[0].tier;
+    size_t selected_plan = 0;
+    while(selected_plan < qm->plan.used && qm->plan.array[selected_plan].tier < selected_tier)
+        selected_plan++;
+    if(selected_plan > 0 && selected_plan < qm->plan.used &&
</file context>

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/web/api/queries/query-execute.c, line 333:

<comment>This fallback is unreachable for production storage points: empty intervals have `count = 1`, while `storage_point_is_unset()` only matches `count = 0`. Remove the dead branch; returned empty intervals already retain their bounds and are not unset.</comment>

<file context>
@@ -318,47 +316,44 @@ NOT_INLINE_HOT void rrd2rrdr_query_execute(RRDR *r, size_t dim_id_in_rrdr, QUERY
                         }
                         if(sp2.end_time_s <= covered_end)
                             storage_point_unset(sp2);
+                        else if(storage_point_is_unset(sp2) && sp2.start_time_s < sp2.end_time_s)
+                            // Preserve a stored empty interval in the count-based read-ahead slot.
+                            storage_point_empty(sp2, sp2.start_time_s, sp2.end_time_s);
</file context>