Repository navigation
[SPARK-60073][SQL] Return NULL from try_make_timestamp for a year outside the timestamp range - #59292
Open
huangpengtao00-dotcom wants to merge 2 commits into
Conversation
…side the timestamp range MakeTimestamp with failOnError = false only caught DateTimeException. A year outside the microsecond timestamp range is valid for java.time and fails later in instantToMicros with an ArithmeticException, which try_make_timestamp, try_make_timestamp_ntz, try_make_timestamp_ltz and the non-ANSI make_timestamp let through. Return NULL for it as well, in both the interpreted and the codegen path. Generated-by: Claude Code (claude-opus-5-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
MakeTimestampwithfailOnError = falsenow returns NULL when converting the result to microseconds overflows, in both the interpreted and the codegen path. Before, it only caughtDateTimeException, while a year outside the microsecond timestamp range (about |year| > 294247) is valid forjava.timeand only fails later ininstantToMicroswith anArithmeticExceptionfromMath.multiplyExact.try_make_timestamp,try_make_timestamp_ntzandtry_make_timestamp_ltzall buildMakeTimestamp(..., failOnError = false), so the fix covers all three, as well asmake_timestampwith ANSI off.Why are the changes needed?
These functions are documented to return NULL on invalid input, but a large year throws instead:
Does this PR introduce any user-facing change?
Yes.
try_make_timestamp,try_make_timestamp_ntz,try_make_timestamp_ltz, andmake_timestampwithspark.sql.ansi.enabled=false, return NULL instead of throwingjava.lang.ArithmeticException: long overflowwhen the year is outside the timestamp range.make_timestampwith ANSI on is unchanged.How was this patch tested?
Added a test to
DateExpressionsSuitecoveringTIMESTAMPandTIMESTAMP_NTZwith years 999999 and -999999.checkEvaluationruns the interpreted, codegen and constant-folded paths. I checked that the test fails when either the interpreted or the codegen catch is removed.Not covered
The
(date[, time[, timezone]])overloads oftry_make_timestampandtry_make_timestamp_ltzgo throughMakeTimestampFromDateTime, which has no non-failing mode, so a date outside the timestamp range still throws there:try_make_timestamp_ntz(date, time)is wrapped inTryEvaland returns NULL for the same date. Wrapping the other two inTryEvalwould also turn an invalid time zone into NULL, and would change the default column name oftry_make_timestamp_ltz(date, time)frommake_timestamp(...)totryeval(make_timestamp(...)).While checking that, I noticed that
MakeTimestampwithfailOnError = falsehandles an invalid time zone differently in the two paths: the interpreted path throwsINVALID_TIMEZONE, because it resolves the zone before thetry, while the codegen path resolves it inside thetryand returns NULL.I left both out to keep this change to the overflow. Happy to handle either here or in a separate PR, whichever you prefer.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-5-5)
🤖 Generated with Claude Code