Visitar URL original
BUG: Fix signed integer overflow in datetime.c by AnkitAhlawat7742 · Pull Request #31688 · numpy/numpy · GitHub
Skip to content

BUG: Fix signed integer overflow in datetime.c - #31688

Merged
ngoldbaum merged 2 commits into
numpy:mainfrom
AnkitAhlawat7742:fix_Signed_integer_overflow
Jul 3, 2026
Merged

ngoldbaum merged 2 commits into
numpy:mainfrom
AnkitAhlawat7742:fix_Signed_integer_overflow

Conversation

@AnkitAhlawat7742

Copy link
Copy Markdown
Contributor

Fix #31686

PR summary

PR fixes a signed integer overflow in numpy/_core/src/multiarray/datetime.c that occurs when converting datetime64 values near INT64_MIN to Python objects via .item().the days_to_yearsdays() function performs (*days_) - (365*30 + 7) which causes undefined behavior when *days_ is near INT64_MIN . this solution will Reordered arithmetic operations to avoid overflow

First time committer introduction

NA

@MaanasArora MaanasArora left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this would solve the issue I think! Just a refactor suggestion and should improve the test. That said, I actually couldn't reproduce this bug on main, which could be because C behavior with iinfo depends on the environment. Could you check if you can reproduce and if the test fails on main for you? (Fine if not I guess, but then we should maybe try to test this where it does...)

Comment thread numpy/_core/src/multiarray/datetime.c Outdated

if (*days_ < NPY_MIN_INT64 + days_offset) {
/* Extract 400-year cycles first to reduce magnitude */
days = *days_;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could just add days_per_400years here, then we lose the days < 0 check (just always year -= 400). Since days_ is so negative this should be safe.

To make this read even simpler, you could add days_per_400years - days_offset, then you don't need to subtract from days below.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review.
I agree, and I've updated the code accordingly.

Comment thread numpy/_core/src/multiarray/datetime.c
Comment thread numpy/_core/tests/test_datetime.py Outdated
info = np.iinfo(np.int64)

# Test INT64_MIN + 1
dt64 = np.datetime64(info.min + 1, "D")

@MaanasArora MaanasArora Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be folded into the below for loop, already is there actually. (Then you could parametrize if you want, the test is not really that big though.)

Comment thread numpy/_core/tests/test_datetime.py Outdated
result = dt64.item()
# For dates outside Python's datetime range (year 1-9999),
# NumPy returns the raw integer value
assert isinstance(result, (int, datetime.date))

@MaanasArora MaanasArora Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is helpful, given those are the only two plausible dtypes and the segfault tested would happen above. For correctness it would be much better to check the actual values equal if int at least. Not sure, is there any reason we are including datetime.date anyway? These all seem to be small enough values that we should expect int?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree , Test case has been updated .

@AnkitAhlawat7742

Copy link
Copy Markdown
Contributor Author

Thanks, this would solve the issue I think! Just a refactor suggestion and should improve the test. That said, I actually couldn't reproduce this bug on main, which could be because iinfo depends on the environment. Could you check if you can reproduce and if the test fails on main for you? (Fine if not I guess, but then we should maybe try to test this where it does...)

Thanks for the review ...!!
For testing, I've informed the bug reporter(seems like it is happening in panda's side ) to verify it from their side as well, so we can confirm that the issue has been fully resolved.

@MaanasArora MaanasArora left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, sorry, my previous comment was a bit misleading - this should be tested with UBSan, in which case this does fail on main for me, so I think CI will catch it and the test looks good. Would be nice to check with pandas but aside from fixing the comment this looks good to me!

Edit: sorry, UBSan actually doesn't run on CI it seems. Not sure here, I'll wait for maintainer review for other opinions on testing.

npy_int64 year;

if (*days_ < NPY_MIN_INT64 + days_offset) {
/* Extract 400-year cycles first to reduce magnitude */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/* Extract 400-year cycles first to reduce magnitude */
/* Add 400-year cycle to ensure days does not overflow */

Seems clearer!

@ngoldbaum ngoldbaum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I agree that this fixes the bug.

However, doesn't the exact same bug happen elsewhere?

For example:

ret = dts->year - 1970;

ret = 12 * (dts->year - 1970) + (dts->month - 1);

ret = (days - 6) / 7;

ret = days * 24 +
dts->hour;
break;
case NPY_FR_m:
ret = (days * 24 +
dts->hour) * 60 +
dts->min;
break;
case NPY_FR_s:
ret = ((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec;
break;
case NPY_FR_ms:
ret = (((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000 +
dts->us / 1000;
break;
case NPY_FR_us:
ret = (((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000000 +
dts->us;
break;
case NPY_FR_ns:
ret = ((((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000000 +
dts->us) * 1000 +
dts->ps / 1000;
break;
case NPY_FR_ps:
ret = ((((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000000 +
dts->us) * 1000000 +
dts->ps;
break;
case NPY_FR_fs:
/* only 2.6 hours */
ret = (((((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000000 +
dts->us) * 1000000 +
dts->ps) * 1000 +
dts->as / 1000;
break;
case NPY_FR_as:
/* only 9.2 secs */
ret = (((((days * 24 +
dts->hour) * 60 +
dts->min) * 60 +
dts->sec) * 1000000 +
dts->us) * 1000000 +
dts->ps) * 1000000 +
dts->as;

Can't all of these mathematical operations on signed integers possibly overflow?

I would much rather we solve this generically rather than a one-off fix for this one particular issue.

Another way to fix this is to do intermediate calculations in a higher precision integer. We have a 128 bit integer implementation in NumPy for this purpose. That said, only the year is int64, everything else is int32 so you could cast to int64 cheaply for those cases.

Edit: sorry, UBSan actually doesn't run on CI it seems. Not sure here, I'll wait for maintainer review for other opinions on testing.

Yes it does: https://github.com/numpy/numpy/actions/runs/27960070587/job/82738842115?pr=31688

{
const npy_int64 days_per_400years = (400*365 + 100 - 4 + 1);
const npy_int64 days_offset = (365*30 + 7);
/* Adjust so it's relative to the year 2000 (divisible by 400) */

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is orphaned now, move it above the if statement?

@MaanasArora

Copy link
Copy Markdown
Contributor

Yes it does:

Ah sorry again, I just noticed that the test in this PR passes on main even on the UBSAN run (I accidentally opened #31713 and was then surprised). So we may need a stronger test. But I suppose we should be taking the generic approach.

@Alvaro-Kothe

Copy link
Copy Markdown

Yes it does: https://github.com/numpy/numpy/actions/runs/27960070587/job/82738842115?pr=31688

It seems that NumPy suppresses signed integer overflow in CI.

UBSAN_OPTIONS=halt_on_error=1:suppressions=${GITHUB_WORKSPACE}/tools/ci/ubsan_suppressions_arm64.txt \

signed-integer-overflow:*

@ngoldbaum

Copy link
Copy Markdown
Member

Yeah, we want to eventually delete all those suppressions by fixing these bugs!

@AnkitAhlawat7742

Copy link
Copy Markdown
Contributor Author

Thank you for the feedback, @ngoldbaum and @MaanasArora
can we take this fix for the issue which is raised by @Alvaro-Kothe ? and I can create a follow-up PR for a comprehensive overflow
what you guys say ?

@ngoldbaum

Copy link
Copy Markdown
Member

Please go ahead. Also for future notice, you don't need to ask to take up PRs like that. It just causes delays.

@ngoldbaum
ngoldbaum merged commit cd1cfb8 into numpy:main Jul 3, 2026
84 of 87 checks passed
@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label Jul 3, 2026
@charris charris changed the title fix:Signed integer overflow in datetime.c BUG: Fix signed integer overflow in datetime.c Jul 4, 2026
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Jul 4, 2026
charris added a commit that referenced this pull request Jul 4, 2026
BUG: Fix signed integer overflow in datetime.c (#31688)
MaanasArora pushed a commit to MaanasArora/numpy that referenced this pull request Jul 7, 2026
736-c41-2c1-e464fc974 pushed a commit to Swiss-Armed-Forces/Loom that referenced this pull request Aug 16, 2026
This MR contains the following updates:

| Package | Type | Update | Change | OpenSSF |
|---|---|---|---|---|
| [numpy](https://github.com/numpy/numpy) ([changelog](https://numpy.org/doc/stable/release)) | dependencies | patch | `2.5.0` → `2.5.2` | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/numpy/numpy/badge)](https://securityscorecards.dev/viewer/?uri=github.com/numpy/numpy) |
| [uvicorn](https://github.com/Kludex/uvicorn) ([changelog](https://uvicorn.dev/release-notes)) | dependencies | patch | `0.52.1` → `0.52.3` | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/Kludex/uvicorn/badge)](https://securityscorecards.dev/viewer/?uri=github.com/Kludex/uvicorn) |

---

### Release Notes

<details>
<summary>numpy/numpy (numpy)</summary>

### [`v2.5.2`](https://github.com/numpy/numpy/releases/tag/v2.5.2): (Aug 9, 2026)

[Compare Source](numpy/numpy@v2.5.1...v2.5.2)

### NumPy 2.5.2 Release Notes

The NumPy 2.5.2 is a patch release that fixes bugs discovered after the 2.5.1
release. The big news is that it includes wheels for the newly released
Python 3.15.0rc1.

This release supports Python versions 3.12-3.15

#### C API changes

##### `PyArray_StringDTypeObject` is opaque under the abi3t stable ABI

The `PyArray_StringDTypeObject` was accidentally exposed in NumPy
2.5 when targeting the free-threading-compatible stable ABI
(`Py_TARGET_ABI3T`). `PyArray_StringDTypeObject` is now an opaque
struct: extensions compiled that way cannot access its fields, since
the struct layout depends on the size of the object header. Any code
that accessed `PyArray_StringDTypeObject` fields in an abi3t build
would have crashed, so we are making this API change in a bugfix
release.

The `NpyString` allocator API remains usable by passing the
descriptor object pointer, e.g.
`NpyString_acquire_allocator((PyArray_StringDTypeObject *)descr)`.

([gh-31771](numpy/numpy#31771))

#### Contributors

A total of 16 people contributed to this release. People with a "+" by their
names contributed a patch for the first time.

- Abhijeetsingh Meena +
- Charalampos Stratakis
- Charles Harris
- Chris Ninham +
- David Woods
- Geonho +
- Gopu Yeshwanth Reddy +
- Iason Krommydas
- Ijtihed Kilani
- Jelle Zijlstra +
- Joren Hammudoglu
- Kumar Aditya
- Mike Boyle
- Nathan Goldbaum
- Raghuveer Devulapalli
- Sebastian Berg

#### Pull requests merged

A total of 28 pull requests were merged for this release.

- [#&#8203;31864](numpy/numpy#31864): MAINT: Prepare 2.5.x for further development
- [#&#8203;31889](numpy/numpy#31889): TYP: Backport multiple static typing fixes 1.
- [#&#8203;31900](numpy/numpy#31900): TST: add tests for stable ABI numpy extensions ([#&#8203;31822](numpy/numpy#31822))
- [#&#8203;31901](numpy/numpy#31901): BUG: fix `StringDType` coerce flag in binary ufunc promotion...
- [#&#8203;31902](numpy/numpy#31902): BLD: fix meson deprecation warnings ([#&#8203;31892](numpy/numpy#31892))
- [#&#8203;31921](numpy/numpy#31921): TYP: Backport multiple typing fixes 2.
- [#&#8203;31947](numpy/numpy#31947): MAINT: Update x86-simd-sort subproject ([`5adb334`](numpy/numpy@5adb334) → [`fa944ef`](numpy/numpy@fa944ef)) ([#&#8203;31908](numpy/numpy#31908))
- [#&#8203;31949](numpy/numpy#31949): BUG: fix crash on 32 bit systems using abi3t ([#&#8203;31771](numpy/numpy#31771))
- [#&#8203;31950](numpy/numpy#31950): MNT: remove some obsolete string to bool workarounds ([#&#8203;31859](numpy/numpy#31859))
- [#&#8203;31952](numpy/numpy#31952): BUG: centralized helper for output coerce and na\_object in stringdtype...
- [#&#8203;31953](numpy/numpy#31953): BUG: fix CPU feature env diagnostic buffer overruns ([#&#8203;31905](numpy/numpy#31905))
- [#&#8203;31954](numpy/numpy#31954): BUG: restore ndarray.conjugate() for legacy user-defined dtypes...
- [#&#8203;31955](numpy/numpy#31955): TYP: Avoid shadowed dtype annotations
- [#&#8203;32077](numpy/numpy#32077): MAINT: Update verdored-meson/meson to match main.
- [#&#8203;32114](numpy/numpy#32114): BUG: fix refcount leak on overlapping copyto with where=False
- [#&#8203;32115](numpy/numpy#32115): BUG: fix swallowed cast error in fancy indexing assignment ([#&#8203;31975](numpy/numpy#31975))
- [#&#8203;32116](numpy/numpy#32116): BUG: Fix buffered iterator stride after removing multi-index
- [#&#8203;32117](numpy/numpy#32117): BUG: fix `np.fromiter` corruption when reusing a `StringDType`...
- [#&#8203;32119](numpy/numpy#32119): BUG: add a special case for StringDType in np.isdtype ([#&#8203;32030](numpy/numpy#32030))
- [#&#8203;32121](numpy/numpy#32121): BUG: reference leak in `simd_sequence_from_iterable` ([#&#8203;32038](numpy/numpy#32038))
- [#&#8203;32122](numpy/numpy#32122): BUG: ensure lock is held when accessing or writing to RNG state...
- [#&#8203;32123](numpy/numpy#32123): BUG: fully reset cached RNG state for non-MT19937 RNGs ([#&#8203;32062](numpy/numpy#32062))
- [#&#8203;32135](numpy/numpy#32135): TYP: type capabilities max dimensions
- [#&#8203;32158](numpy/numpy#32158): BUG: avoid possible stack overflow in arraydescr\_dealloc ([#&#8203;32133](numpy/numpy#32133))
- [#&#8203;32206](numpy/numpy#32206): MAINT: Update cibuildwheel to v4.2.0
- [#&#8203;32214](numpy/numpy#32214): MAINT: Skip limited\_api tests on some platforms.
- [#&#8203;32220](numpy/numpy#32220): TYP: `isclose` shape-typing fix for 2d array-likes ([#&#8203;32205](numpy/numpy#32205))
- [#&#8203;32221](numpy/numpy#32221): BUG: avoid segfaults when legacy copyswap slot is not defined...

### [`v2.5.1`](https://github.com/numpy/numpy/releases/tag/v2.5.1): (July 4, 2026)

[Compare Source](numpy/numpy@v2.5.0...v2.5.1)

### NumPy 2.5.1 Release Notes

The NumPy 2.5.1 is a patch release that fixes bugs discovered after the 2.5.0
release. The most noticeable is the fix is to the numpy datetime cython API
which should allow downstream to support NumPy versions older than 2.5.
Preparation for Python 3.15 continues along with typing improvements.

This release supports Python versions 3.12-3.14

#### Changes

- The minimum supported GCC version has been updated from 9.3.0 to 10.3.0

  ([gh-31843](numpy/numpy#31843))

#### Contributors

A total of 10 people contributed to this release. People with a "+" by their
names contributed a patch for the first time.

- Adhyan Gupta +
- Ankit Ahlawat
- Charles Harris
- Iason Krommydas
- Joren Hammudoglu
- Kumar Aditya
- Nathan Goldbaum
- Sebastian Berg
- Ties Jan Hefting +
- Vineet Kumar

#### Pull requests merged

A total of 20 pull requests were merged for this release.

- [#&#8203;31707](numpy/numpy#31707): MAINT: Prepare 2.5.x for further development
- [#&#8203;31721](numpy/numpy#31721): CI: fix new `cython-lint` errors ([#&#8203;31711](numpy/numpy#31711))
- [#&#8203;31723](numpy/numpy#31723): MAINT: Update meson to match main
- [#&#8203;31729](numpy/numpy#31729): TST: use setup-sde instead of curl to get SDE binaries ([#&#8203;31727](numpy/numpy#31727))
- [#&#8203;31829](numpy/numpy#31829): BUG: Relax finfo to be easier accessible for all user dtypes...
- [#&#8203;31831](numpy/numpy#31831): TYP: Fix `flatiter.__next__` return type for `object_` and...
- [#&#8203;31832](numpy/numpy#31832): BUG: avoid deadlocks using NpyString API ([#&#8203;31682](numpy/numpy#31682))
- [#&#8203;31833](numpy/numpy#31833): BUG: fix out array leak in reduceat and accumulate when dtype...
- [#&#8203;31835](numpy/numpy#31835): BUG: fix numpy datetime cython APIs to be compatible with older...
- [#&#8203;31836](numpy/numpy#31836): TYP: Fix incorrect dtype inference of `asarray([])` ([#&#8203;31732](numpy/numpy#31732))
- [#&#8203;31837](numpy/numpy#31837): TYP: Fix `np.ma.masked_array` 2.5.0 regression
- [#&#8203;31838](numpy/numpy#31838): FIX: Refactor error handling in array\_setstate to prevent typecode...
- [#&#8203;31839](numpy/numpy#31839): TST: xfail multithreaded BLAS test more generously
- [#&#8203;31840](numpy/numpy#31840): MAINT: Rename subroutine for crackfortran tests
- [#&#8203;31842](numpy/numpy#31842): BUG: fix leak in reductions when a ufunc override errors or is...
- [#&#8203;31849](numpy/numpy#31849): BLD: set minimum required gcc version to 10.3 ([#&#8203;31843](numpy/numpy#31843))
- [#&#8203;31855](numpy/numpy#31855): CI: fix hangs on MacOS ASan CI ([#&#8203;31853](numpy/numpy#31853))
- [#&#8203;31856](numpy/numpy#31856): BUG: fix several bugs in StringDType operations ([#&#8203;31846](numpy/numpy#31846))
- [#&#8203;31857](numpy/numpy#31857): BUG: Fix segfault in MT19937 by preventing recursive seed lists...
- [#&#8203;31858](numpy/numpy#31858): BUG: Fix signed integer overflow in datetime.c ([#&#8203;31688](numpy/numpy#31688))

</details>

<details>
<summary>Kludex/uvicorn (uvicorn)</summary>

### [`v0.52.3`](https://github.com/Kludex/uvicorn/releases/tag/0.52.3): Version 0.52.3

[Compare Source](Kludex/uvicorn@0.52.2...0.52.3)

##### Changed

- Update `zttp` to 0.0.24 and use its combined receive path, improving HTTP/1.1 request parsing performance ([#&#8203;3067](Kludex/uvicorn#3067))

**Full Changelog**: <Kludex/uvicorn@0.52.2...0.52.3>

### [`v0.52.2`](https://github.com/Kludex/uvicorn/releases/tag/0.52.2): Version 0.52.2

[Compare Source](Kludex/uvicorn@0.52.1...0.52.2)

##### Fixed

- Update `zttp` to 0.0.22, fixing bodyless request receives and improving HTTP/1 request parsing performance ([#&#8203;3063](Kludex/uvicorn#3063))

**Full Changelog**: <Kludex/uvicorn@0.52.1...0.52.2>

</details>

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNTEuMyIsInVwZGF0ZWRJblZlciI6IjQ0LjMwLjQiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImRlcGVuZGVuY2llcyIsInJlbm92YXRlIl19-->

See merge request swiss-armed-forces/cyber-command/cea/loom!639

Co-authored-by: Loom MR Pipeline Trigger <group_103951964_bot_9504bb8dead6d4e406ad817a607f24be@noreply.gitlab.com>
jbrockmendel added a commit to jbrockmendel/numpy that referenced this pull request Sep 11, 2026
test_datetime64_item_int64_min_edge_case asserted that .item() on a
datetime64 near int64 min returns a Python int, which no longer holds.
Assert the datetime64 round-trip instead, and reach the conversion that
used to overflow through str(), which still goes through
days_to_yearsdays.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YF8EFz6MfYRc2whjZEiDtu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: Signed integer overflow in datetime.c:days_to_yearsdays on near-minimum np.datetime64

5 participants