Visitar URL original
Fix ORC filter pushdown dropping rows for `TIMESTAMP` columns when the writer or reader time zone is not UTC by Algunenano · Pull Request #131274 · ClickHouse/ClickHouse · GitHub
Skip to content

Fix ORC filter pushdown dropping rows for TIMESTAMP columns when the writer or reader time zone is not UTC - #131274

Open
Algunenano wants to merge 1 commit into
ClickHouse:masterfrom
Algunenano:orc-timestamp-pushdown-reader-timezone
Open

Fix ORC filter pushdown dropping rows for TIMESTAMP columns when the writer or reader time zone is not UTC#131274
Algunenano wants to merge 1 commit into
ClickHouse:masterfrom
Algunenano:orc-timestamp-pushdown-reader-timezone

Conversation

@Algunenano

@Algunenano Algunenano commented Nov 13, 2026 •
edited by clickhouse-gh Bot
Loading

Copy link
Copy Markdown
Member

Closes: #129944

The native ORC reader keeps the wall clock of TIMESTAMP values in the reader time zone, but the stripe, row group and bloom filter statistics, and the filter literals, are UTC instants. With input_format_orc_filter_push_down enabled, a predicate on such a column could prune rows that match. The reader now skips pruning on TIMESTAMP leaves unless both the writer time zone of the stripe and the reader time zone (input_format_orc_reader_time_zone_name) are UTC or GMT. IS NULL predicates still prune. The file-level statistics never prune TIMESTAMP leaves, because the writer time zone is stored per stripe.

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix the native ORC reader returning fewer rows than expected for filters on TIMESTAMP columns when input_format_orc_filter_push_down is enabled and the reader or writer time zone is not UTC.

Workflow [PR]
Sync PR [sync-upstream/pr/131274]

@Algunenano @claude
Do not prune ORC TIMESTAMP by statistics when a time zone is not UTC
20ec922 
The reader keeps the wall clock of `TIMESTAMP` values in the reader time zone, while the statistics, bloom filters and filter literals are UTC instants, so the pushdown could drop matching rows. Skip the pruning unless the writer and reader time zones are both `UTC` or `GMT`.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@clickhouse-gh

clickhouse-gh Bot commented Nov 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [20ec922]

Summary: ⏳

job_name test_name status info comment
Performance Comparison (arm_release, master_head, 1/6) ERROR
Performance Comparison (arm_release, master_head, 2/6) ERROR
Performance Comparison (arm_release, master_head, 3/6) ERROR
Performance Comparison (arm_release, master_head, 4/6) ERROR
Performance Comparison (arm_release, master_head, 5/6) ERROR
Performance Comparison (arm_release, master_head, 6/6) ERROR

AI Review

Summary

PR #131274, head 20ec922f06b60af8da20713f80d07a4df071cec4, prevents the native ORC reader from pruning matching TIMESTAMP rows when writer or reader time zones make statistics and filter literals incomparable. It covers file, stripe, row-group, and bloom-filter pruning while retaining null-based pruning, and adds a regression test comparing results with pushdown enabled and disabled. No blockers or majors found.

Missing context / blind spots
  • ⚠️ Runtime validation was unavailable: this checkout has no ClickHouse binary, and the head's functional-test and bugfix-validation jobs were pending during review. Successful CI execution of 05629_orc_timestamp_filter_push_down_time_zones would close this gap.
Final Verdict

Status: ✅ Approve. No review changes requested; functional CI remains pending.

@clickhouse-gh clickhouse-gh Bot added pr-bugfix Pull request with bugfix, not backported by default comp-formats Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.). labels Nov 13, 2026
@clickhouse-gh

clickhouse-gh Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Build profile diff (arm_release)

Comparing 20ec922f0 with master c65bdd819 (stripped binary size, per-symbol sizes and ThinLTO time; compile times per translation unit against the most recent warmup build that recompiled it).

✅ No significant changes.

Binary sizes

programs/clickhouse-stripped: smaller than the master baseline by the known offset between the two builds, so the difference is not shown. A delta that differs from the offset by more than 50% of it is shown, in either direction.

The official master build is compiled with -g and a pull request build is not, and XRay counts debug instructions towards its instrumentation threshold, so master instruments thousands of functions more and its binary is ~0.4% larger no matter what the pull request does.

Slowest function optimization changes (ThinLTO)

Median per-function time ratio to the master baseline: programs/clickhouse ×0.65, programs/clickhouse-keeper ×0.65 (each binary is linked separately, on a different machine and with different build flags); deltas below are relative to it.

Binary Function Master PR Δ vs median
programs/clickhouse DB::registerFunctionConversion(DB::FunctionFactory&) 5.7 s 14.9 s +11.2 s (+299%)
programs/clickhouse _ZN6detail10static_forITk7is_enumN2DB23AttributeUnderlyingTypeEZNS1_29callOnDictionaryAttributeType… 3.4 s 7.6 s +5.4 s (+242%)
Compile time of recompiled translation units

15 translation units recompiled, 68 s compile time in total, 15 of them have a recent master baseline.

Median compile-time ratio to the baselines is ×1.08 (machine-speed difference or a change affecting every TU); per-TU deltas below are relative to that ratio.
The matched translation units cost +4.5 s (+7%) in total before that adjustment.

Job report

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp-formats Input/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.). pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

ORC TIMESTAMP pushdown ignores reader time zone, drops matching rows

1 participant

@Algunenano