Repository navigation
test_strftime_y2k fails on embedded Linux #123681
Copy link
Copy link
Closed
Labels
testsTests in the Lib/test dirTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug report
Bug description:
This is an issue to track the progress of fixing the JIT. Branches that trigger the embedded Linux tests fail on test_strftime_y2k, e.g. https://github.com/python/cpython/actions/runs/10655985897/job/29534232332?pr=123546. As a side-effect this causes users getting notification that their JIT workflow failed.
Both the C and F specifiers are failing for this specific test:
Details
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) ...
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDate_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) ...
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTime_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) ...
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestDateTimeTZ_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) ...
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=1, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=1, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=48, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=48, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=70, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=70, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=99, specifier='C') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=999, specifier='F') ... FAIL
test_strftime_y2k (test.datetimetester.TestSubclassDateTime_Fast.test_strftime_y2k) (year=999, specifier='C') ... FAIL
A solution is to add this test to the ignored tests or to fix the detection algorithm.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Other
Linked PRs
- gh-123681: handle c99-specific specifiers for strftime when cross-compiling #123861
- gh-123681: Check the strftime() behavior at runtime instead of at the compile time #128444
- gh-123681: Check NORMALIZE_CENTURY behavior at runtime; require C99 #136022
- [3.14] gh-123681: Check NORMALIZE_CENTURY behavior at runtime; require C99 (GH-136022) #137947
Related and proposed PR: gh-128106
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
testsTests in the Lib/test dirTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Projects
- StatusShow more project fieldsDone
Activity
serhiy-storchaka commentedon Sep 4, 2024
It seems that only tests for the C implementation fail.
We can try to move the detection from the compile time to the run time, like for Python implementation. But it is a puzzle why it does not work here.
picnixz commentedon Sep 4, 2024
(Forget my comment that I deleted, I looked at the wrong commit).
I think we can improve the configure.ac detection by including the weird cases. Or maybe the test is broken because of this:
(Those are the two failing specifiers)
brandtbucher commentedon Sep 6, 2024
This particular test failure could be a bit weird, since this CI job involves cross-compiling Python and then running the test suite under emulation. Not sure if it's a "real" issue, or if we should just skip the test (like we do for others).
picnixz commentedon Sep 7, 2024
Does it mean that the configure tests might not be correct (or inconsistent because they were done using cross-compilation?) If so, we can just outright ignore the test I guess?
serhiy-storchaka commentedon Sep 7, 2024
If these configure tests are not correct, then other configure tests can be not correct too.
picnixz commentedon Sep 8, 2024
Ha! I knew I should have looked more in detail the logs:
On aarch64-unknown-linux-gnu (failing tests), we have two different configure:
https://github.com/python/cpython/commit/e565b87d9dafe89bf262e3ab46edb9c9c8618b27/checks/29534232332/logs
So it appears we have different ./configure. I don't know which one is actually being used at runtime in the emulated environment. Now, both '%F' and '%C' are C99-specific (https://en.cppreference.com/w/c/chrono/strftime) so the following
is perhaps a too broad check. Since specifiers 'F' and 'C' are used, we actually expect to support
%F, but this contradicts the fact that "C99-specific strftime specifiers are supported" (at least for the second one which seems the most recent and probably the one being used under the hood). The configure check is:In other words, either
strftime(full_date, sizeof(full_date), "%F", &date)fails orfull_date != 1900-01-01. So maybe%Fdoes not fail. We can probably debug the tests to check whether it_time.strftimethat bugs, or if it is the configure test that bugs. I don't know if it's possible for_time.strftimeto support%F(which does not directly callstrftime()) but at the same time fail thestrftime(full_date, sizeof(full_date), "%F", &date) && !strcmp(full_date, "1900-01-01")test.I'm tired so what I wrote here might be wrong but I'll try to investigate tomorrow.
pythongh-123681: handle c99-specific specifiers when cross-compiling
blhsing commentedon Sep 9, 2024
Ahh how could I have not noticed the second configure in the logs! Was scratching my head over why the detection of C99 support got a yes (in the first configure) but the compiled code ended up not handling C99. It makes total sense now.
The real problem is simply that autoconf does not run the test for the
AC_RUN_IFELSEmacro at all when cross-compiling, but directly performs action-if-cross-compiling as specified, which I thought should be made ano(to be pessimistic) when it should really be ayesbecause we should make sure to handle%Fand%Courselves when we are unsure if the target platform can handle them.picnixz commentedon Sep 9, 2024
Well, isn't it trivial to spot it among 11985 lines 😄? Actually, I found the duplicate configure in the logs by a CTRL+F and... saw that I had two results.
blhsing commentedon Sep 9, 2024
I used Ctrl-F too, but the browser limits the search to only what's loaded in the memory, when the output of the second configure is hidden because it is loaded on demand by an AJAX call only if I scroll to it. I should've used the "Search Logs" text box in the page to perform the search for the keyword on the server side instead.
Thanks again!
picnixz commentedon Sep 9, 2024
Actually, the trick is to "view raw logs". You'll get the entire logs without buffering (easier to read IMO).
picnixz commentedon Sep 27, 2024
FTR: #124466 was merged as a temporary workaround so we need to keep the issue opened even though the job looks successful (it's just that we skip it).
6 remaining items
pythongh-123681: Check the strftime() behavior at runtime instead of …
serhiy-storchaka commentedon Jan 3, 2025
We still need to fix a check for Py_NORMALIZE_CENTURY which does not work in cross-compiler. It is only possible at run time.
If we require C99 compliant strftime (it is not on Linux for %C), we need to make the following changes:
erlend-aasland commentedon Jan 3, 2025
%Cworks swell for me on Debian and Ubuntu. Please specify where it does not work instead of putting out such a broad claim. We also explicitly test the%Cspecifier in our test suite. If it did not work on Linux, CI would be broken.serhiy-storchaka commentedon Jan 3, 2025
In C99 it should be 2 digits.
encukou commentedon Jun 27, 2025
So, we need a test for
normalize_century(ideally at run-time, to avoid cross-compilation issues), but in CPython we can assume%Cand%Fare available, right?serhiy-storchaka commentedon Jun 27, 2025
They are in C99. https://en.cppreference.com/w/c/chrono/strftime
It seems that the main reason of adding checks for them (see #122272) is that they are not documented in Python, and therefore their support is not guaranteed. If we guarantee their support, we should also document them and add tests. We can make this a new feature in 3.15 or even in 3.14, but I have doubts about 3.13.
encukou commentedon Jun 27, 2025
As I see it, they're guaranteed in CPython (via the C99 build requirement), but not in Python the language.
serhiy-storchaka commentedon Jun 29, 2025
This is an interesting point of view. Then we can remove the check in the C code (which is CPython specific), but should leave it in the Python code, which can be reused in other implementation. From this point of view, #128444 and #136022 are equivalent.
encukou commentedon Jul 10, 2025
I think #136022 removing
def _supports_c99is OK.The Python code still uses
_time.strftime. If other implementations replace_time, they will need to update the Python code as well. There's a test so they don't forget.gh-123681: Check NORMALIZE_CENTURY behavior at runtime; require C99 (G…
pythongh-123681: Check NORMALIZE_CENTURY behavior at runtime; require…
[3.14] gh-123681: Check NORMALIZE_CENTURY behavior at runtime; requir…