Visitar URL original
[feature](timestamp_ns) Add end-to-end TIMESTAMP_NS support (#66761) by jacktengg · Pull Request #68754 · apache/doris · GitHub
Skip to content

[feature](timestamp_ns) Add end-to-end TIMESTAMP_NS support (#66761) - #68754

Open
jacktengg wants to merge 4 commits into
apache:branch-4.2from
jacktengg:pick-4.2-tsns
Open

jacktengg wants to merge 4 commits into
apache:branch-4.2from
jacktengg:pick-4.2-tsns

Conversation

@jacktengg

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@jacktengg
jacktengg requested a review from yiguolei as a code owner October 7, 2026 15:23
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

…6761)

Docs: apache/doris-website#4091

Problem Summary:
Doris DATETIME only support at most six fractional digits, it's not
sufficient for event times, traces, etc which require higher precision
of sub-seconds.

Introduce `TIMESTAMP_NS` as a separate fixed-scale SQL type backed by
signed Int64 Unix-epoch nanoseconds. It always has scale 9 and supports
values from `1677-09-21 00:12:43.145224192` through `2262-04-11
23:47:16.854775807`. Wire the type through Thrift and protobuf metadata,
the FE catalog and Nereids, BE columns and SerDe, storage and indexes,
result protocols, JNI and Java UDF transport, complex and VARIANT types,
and schema metadata.

Add exact casts, comparisons, type inference, and constant folding
across date-like types while preserving nanoseconds and rejecting
partial or precision-losing implicit conversions. Add timestamp-aware
scalar and calendar functions, aggregation, joins and ASOF joins, scan
predicates, runtime filters, partition pruning, query-cache
normalization, automatic partitions, and MTMV roll-up. Keep FE folding
and BE runtime behavior consistent at signed endpoints, leap years, DST
gaps, calendar-clamped boundaries, and fractional formatting and
rounding boundaries.

Keep existing DATETIME and DATETIMEV2 scale semantics unchanged.
FROM_UNIXTIME retains a DECIMAL64 microsecond path for scale 0 through 6
and uses the DECIMAL128 nanosecond path only for scale 7 through 9. The
%f formatter rounds half-up to microseconds with second carry, while %n
preserves the nanosecond fraction.

- Test: Unit Test and Regression Test
- Added targeted BE unit coverage for TIMESTAMP_NS values, casts,
functions, SerDe, predicates, aggregates, joins, and compatibility paths
- Added targeted FE unit coverage for literals, casts, constant folding,
type inference, partitions, query-cache and MTMV boundaries, and
statement-time semantics
- Added regression coverage under datatype_p0/timestamp_ns and focused
MTMV and Java UDF suites
    - Built FE and BE with the standard build scripts
- Behavior changed: Yes. Adds TIMESTAMP_NS and nanosecond function
semantics without changing the scale limits of DATETIME or DATETIMEV2.
- Does this need documentation: Yes. Document TIMESTAMP_NS syntax,
range, casts and coercion rules, function behavior, and integration
support.

Add `TIMESTAMP_NS`, a fixed-scale nanosecond timestamp type stored as
signed Unix-epoch nanoseconds. It supports SQL literals, casts,
date/time functions, storage and indexes, partitions, joins, aggregates,
complex types, Java UDF/JNI transport, and FE constant folding and type
inference. Existing DATETIME and DATETIMEV2 types remain limited to
scale 0 through 6.

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants