Visitar URL original
`test_*_code` functions in `_testcapi/getargs.c` have memory leaks · Issue #110572 · python/cpython · GitHub
Skip to content

test_*_code functions in _testcapi/getargs.c have memory leaks #110572

Description

@sobolevn

Bug report

I don't think it is very important, since this is just a test, but why have it when it is spotted?

  1. test_k_code:
    /* This function not only tests the 'k' getargs code, but also the
    PyLong_AsUnsignedLongMask() function. */
    static PyObject *
    test_k_code(PyObject *self, PyObject *Py_UNUSED(ignored))
    {
    PyObject *tuple, *num;
    unsigned long value;
    tuple = PyTuple_New(1);
    if (tuple == NULL) {
    return NULL;
    }
    /* a number larger than ULONG_MAX even on 64-bit platforms */
    num = PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16);
    if (num == NULL) {
    return NULL;
    }
    value = PyLong_AsUnsignedLongMask(num);
    if (value != ULONG_MAX) {
    PyErr_SetString(PyExc_AssertionError,
    "test_k_code: "
    "PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF");
    return NULL;
    }
    PyTuple_SET_ITEM(tuple, 0, num);
    value = 0;
    if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
    return NULL;
    }
    if (value != ULONG_MAX) {
    PyErr_SetString(PyExc_AssertionError,
    "test_k_code: k code returned wrong value for long 0xFFF...FFF");
    return NULL;
    }
    Py_DECREF(num);
    num = PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16);
    if (num == NULL) {
    return NULL;
    }
    value = PyLong_AsUnsignedLongMask(num);
    if (value != (unsigned long)-0x42) {
    PyErr_SetString(PyExc_AssertionError,
    "test_k_code: "
    "PyLong_AsUnsignedLongMask() returned wrong value for long -0xFFF..000042");
    return NULL;
    }
    PyTuple_SET_ITEM(tuple, 0, num);
    value = 0;
    if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
    return NULL;
    }
    if (value != (unsigned long)-0x42) {
    PyErr_SetString(PyExc_AssertionError,
    "test_k_code: k code returned wrong value for long -0xFFF..000042");
    return NULL;
    }
    Py_DECREF(tuple);
    Py_RETURN_NONE;
    }

On errors tuple is not decrefed.
Also, note these lines:

PyTuple_SET_ITEM(tuple, 0, num);
value = 0;
if (!PyArg_ParseTuple(tuple, "k:test_k_code", &value)) {
return NULL;
}
if (value != ULONG_MAX) {
PyErr_SetString(PyExc_AssertionError,
"test_k_code: k code returned wrong value for long 0xFFF...FFF");
return NULL;
}
Py_DECREF(num);
num = PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16);
if (num == NULL) {
return NULL;
}

Here' we leave a tuple is a semi-broken state. Its 0'th item has a reference count of 0.
We should also recreate a tuple here with the new items. test_L_code also has this problem.

  1. static PyObject *
    test_L_code(PyObject *self, PyObject *Py_UNUSED(ignored))
    {
    PyObject *tuple, *num;
    long long value;
    tuple = PyTuple_New(1);
    if (tuple == NULL) {
    return NULL;
    }
    num = PyLong_FromLong(42);
    if (num == NULL) {
    return NULL;
    }
    PyTuple_SET_ITEM(tuple, 0, num);
    value = -1;
    if (!PyArg_ParseTuple(tuple, "L:test_L_code", &value)) {
    return NULL;
    }
    if (value != 42) {
    PyErr_SetString(PyExc_AssertionError,
    "test_L_code: L code returned wrong value for long 42");
    return NULL;
    }
    Py_DECREF(num);
    num = PyLong_FromLong(42);
    if (num == NULL) {
    return NULL;
    }
    PyTuple_SET_ITEM(tuple, 0, num);
    value = -1;
    if (!PyArg_ParseTuple(tuple, "L:test_L_code", &value)) {
    return NULL;
    }
    if (value != 42) {
    PyErr_SetString(PyExc_AssertionError,
    "test_L_code: L code returned wrong value for int 42");
    return NULL;
    }
    Py_DECREF(tuple);
    Py_RETURN_NONE;
    }

On errors tuple is not decrefed. And num is re-assigned without tuple cleanup.

  1. /* Test the s and z codes for PyArg_ParseTuple.
    */
    static PyObject *
    test_s_code(PyObject *self, PyObject *Py_UNUSED(ignored))
    {
    /* Unicode strings should be accepted */
    PyObject *tuple = PyTuple_New(1);
    if (tuple == NULL) {
    return NULL;
    }
    PyObject *obj = PyUnicode_Decode("t\xeate", strlen("t\xeate"),
    "latin-1", NULL);
    if (obj == NULL) {
    return NULL;
    }
    PyTuple_SET_ITEM(tuple, 0, obj);
    /* These two blocks used to raise a TypeError:
    * "argument must be string without null bytes, not str"
    */
    char *value;
    if (!PyArg_ParseTuple(tuple, "s:test_s_code1", &value)) {
    return NULL;
    }
    if (!PyArg_ParseTuple(tuple, "z:test_s_code2", &value)) {
    return NULL;
    }
    Py_DECREF(tuple);
    Py_RETURN_NONE;
    }

As well, tuple is leaked on errors.

I have a PR ready.

Linked PRs

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    testsTests in the Lib/test dir
    on Oct 9, 2023
  2. self-assigned this
    on Oct 9, 2023
  3. added a commit that references this issue on Oct 9, 2023
  4. serhiy-storchaka commented on Oct 11, 2023

    @serhiy-storchaka
    Member

    Are these tests still needed?

  5. sobolevn commented on Oct 11, 2023

    @sobolevn
    MemberAuthor

    's', 'L', and 'k' are still mentioned in https://docs.python.org/3.12/howto/clinic.html#how-to-use-real-argument-clinic-converters-instead-of-legacy-converters

    So, I guess we still need to make sure that they are supported and work correctly.
    They are even not deprecated, just called "legacy".

  6. serhiy-storchaka commented on Oct 11, 2023

    @serhiy-storchaka
    Member

    I mean that they are tested in Lib/test/test_capi/test_getargs.py using wrappers like getargs_k(). And #110628 adds tests for functions like PyLong_AsUnsignedLongMask().

  7. sobolevn commented on Oct 11, 2023

    @sobolevn
    MemberAuthor

    In this case, yes. We can move some logic from these tests to test_long.py.

    Things that right now are not tested in test_long (this list might be not full):

    • test_k_code: PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16)
    • test_k_code: PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16)
    • test_s_code: getargs_s('t\xeate') (however, 'abc\xe9' is tested)
    • test_s_code: getargs_z('t\xeate') (however, 'abc\xe9' is tested)

    So, do you agree on this plan?

    1. I convert this PR to delete these tests, it will be in a draft state for now
    2. We ensure that all checks from test_*_code are in either test_long.py or in test_getargs.py
    3. We merge your PR with test_long.py and possible PRs to test_getargs.py
    4. We merge my PR from 1. with these tests removed
  8. added a commit that references this issue on Oct 21, 2023
  9. added a commit that references this issue on Oct 21, 2023
  10. serhiy-storchaka commented on Oct 21, 2023

    @serhiy-storchaka
    Member

    I merged your PR, and it will be backported to 3.12. In main we should consider removing this tests.

    PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16) is just a way to create a long int for the following test of PyLong_AsUnsignedLongMask(). There are already tests for PyLong_AsUnsignedLongMask() for argument outside of 0..ULONG_MAX. There are also similar tests for the "k" format unit. I think that all this can be deleted.

  11. added a commit that references this issue on Oct 21, 2023
  12. added a commit that references this issue on Oct 23, 2023
  13. added a commit that references this issue on Oct 24, 2023
  14. hugovk commented on Nov 9, 2023

    @hugovk
    Member

    Thanks for the PRs.

  15. added 2 commits that reference this issue on Feb 11, 2024
  16. added 2 commits that reference this issue on Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions