Visitar URL original
bpo-29548: Fix some inefficient call API usage by methane · Pull Request #97 · python/cpython · GitHub
Skip to content

bpo-29548: Fix some inefficient call API usage #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
methane merged 4 commits into from
Feb 16, 2017
Merged

bpo-29548: Fix some inefficient call API usage #97

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4e2f6b4
Fix some inefficient call API usage
methane Feb 14, 2017
4e7d7a4
minor style fix
methane Feb 15, 2017
706f9e3
revert functools changes
methane Feb 15, 2017
044f9fd
remove unused variable. fix inconsistency between arg format and argu…
methane Feb 15, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Fix some inefficient call API usage
  • Loading branch information
@methane
methane committed Feb 14, 2017
commit 4e2f6b4e0c27b2914a11bd18f09170b7686c5f70
25 changes: 6 additions & 19 deletions Modules/_functoolsmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -560,37 +560,25 @@ functools_reduce(PyObject *self, PyObject *args)
return NULL;
}

if ((args = PyTuple_New(2)) == NULL)
goto Fail;

for (;;) {
PyObject *op2;

if (args->ob_refcnt > 1) {
Py_DECREF(args);
if ((args = PyTuple_New(2)) == NULL)
goto Fail;
}

op2 = PyIter_Next(it);
PyObject *op2 = PyIter_Next(it);
if (op2 == NULL) {
if (PyErr_Occurred())
goto Fail;
break;
}

@vstinner vstinner Feb 14, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect a ltitle slowdown if the function doesn't support fastcall. For example, a Python object with a call() method.

Can you please try to run a quick benchmark on the following two cases:

  • function supporting FASTCALL, like a builtin function
  • function not supporting FASTCALL

In general, I'm in favor of removing such hack, since playing with reference count can lead to complex and annoying bugs. But I would prefer to first see the cost on performances.

@methane methane Feb 15, 2017

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

# operator.add is FASTCALL C function
$ ./python -m perf timeit --compare-to $HOME/work/python/cpython/python -s  'L = [0]*10000' -s 'import functools, operator' -- 'functools.reduce(operator.add, L)'
/home/inada-n/work/python/cpython/python: ..................... 498 us +- 3 us
/home/inada-n/work/python/fix-call-usage/python: ..................... 412 us +- 8 us

Median +- std dev: [/home/inada-n/work/python/cpython/python] 498 us +- 3 us -> [/home/inada-n/work/python/fix-call-usage/python] 412 us +- 8 us: 1.21x faster (-17%)

# lambda is Python function (supports FASTCALL)
$ ./python -m perf timeit --compare-to $HOME/work/python/cpython/python -s  'L = [0]*10000' -s 'import functools' -- 'functools.reduce(lambda x,y: x, L)'
/home/inada-n/work/python/cpython/python: ..................... 831 us +- 7 us
/home/inada-n/work/python/fix-call-usage/python: ..................... 788 us +- 11 us

Median +- std dev: [/home/inada-n/work/python/cpython/python] 831 us +- 7 us -> [/home/inada-n/work/python/fix-call-usage/python] 788 us +- 11 us: 1.06x faster (-5%)

# builtin min and max doesn't support FASTCALL
$ ./python -m perf timeit --compare-to $HOME/work/python/cpython/python -s  'L = [0]*10000' -s 'import functools' -- 'functools.reduce(min, L)'
/home/inada-n/work/python/cpython/python: ..................... 1.23 ms +- 0.00 ms
/home/inada-n/work/python/fix-call-usage/python: ..................... 1.42 ms +- 0.04 ms

Median +- std dev: [/home/inada-n/work/python/cpython/python] 1.23 ms +- 0.00 ms -> [/home/inada-n/work/python/fix-call-usage/python] 1.42 ms +- 0.04 ms: 1.16x slower (+16%)


if (result == NULL)
if (result == NULL) {
result = op2;
}
else {
PyTuple_SetItem(args, 0, result);
PyTuple_SetItem(args, 1, op2);
if ((result = PyEval_CallObject(func, args)) == NULL)
PyObject *args[] = {result, op2};
if ((result = _PyObject_FastCall(func, args, 2)) == NULL) {

@vstinner vstinner Feb 14, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind to move the assignment out of the if() please? "result = ...; if (result == NULL) ...".

goto Fail;
}
}
}

Py_DECREF(args);

if (result == NULL)
PyErr_SetString(PyExc_TypeError,
"reduce() of empty sequence with no initial value");
Expand All @@ -599,7 +587,6 @@ functools_reduce(PyObject *self, PyObject *args)
return result;

Fail:
Py_XDECREF(args);
Py_XDECREF(result);
Py_DECREF(it);
return NULL;
Expand Down
2 changes: 1 addition & 1 deletion Modules/_testcapimodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3070,7 +3070,7 @@ slot_tp_del(PyObject *self)
/* Execute __del__ method, if any. */
del = _PyObject_LookupSpecial(self, &PyId___tp_del__);
if (del != NULL) {
res = PyEval_CallObject(del, NULL);
res = _PyObject_CallNoArg(del);
if (res == NULL)
PyErr_WriteUnraisable(del);
else
Expand Down
3 changes: 1 addition & 2 deletions Modules/_threadmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -994,8 +994,7 @@ t_bootstrap(void *boot_raw)
_PyThreadState_Init(tstate);
PyEval_AcquireThread(tstate);
nb_threads++;
res = PyEval_CallObjectWithKeywords(
boot->func, boot->args, boot->keyw);
res = PyObject_Call(boot->func, boot->args, boot->keyw);
if (res == NULL) {
if (PyErr_ExceptionMatches(PyExc_SystemExit))
PyErr_Clear();
Expand Down
9 changes: 3 additions & 6 deletions Modules/_tkinter.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2417,7 +2417,7 @@ PythonCmd(ClientData clientData, Tcl_Interp *interp, int argc, const char *argv[
}
PyTuple_SET_ITEM(arg, i, s);
}
res = PyEval_CallObject(func, arg);
res = PyObject_Call(func, arg, NULL);
Py_DECREF(arg);

if (res == NULL)
Expand Down Expand Up @@ -2667,10 +2667,7 @@ FileHandler(ClientData clientData, int mask)
func = data->func;
file = data->file;

arg = Py_BuildValue("(Oi)", file, (long) mask);
res = PyEval_CallObject(func, arg);
Py_DECREF(arg);

res = PyObject_CallFunction(func, "Oi", file, (long) mask);

@vstinner vstinner Feb 15, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arg variable is no more used and can be removed, no?

@methane methane Feb 16, 2017

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I missed it.
Additionally, argument for "i" format should be int, not (long).

@vstinner vstinner Feb 16, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, argument for "i" format should be int, not (long).

Right, I spotted it as well, but then I forgot to report it, sorry :-) Hopefully you fixed it!

if (res == NULL) {
errorInCmd = 1;
PyErr_Fetch(&excInCmd, &valInCmd, &trbInCmd);
Expand Down Expand Up @@ -2840,7 +2837,7 @@ TimerHandler(ClientData clientData)

ENTER_PYTHON

res = PyEval_CallObject(func, NULL);
res = _PyObject_CallNoArg(func);
Py_DECREF(func);
Py_DECREF(v); /* See Tktt_New() */

Expand Down
2 changes: 1 addition & 1 deletion Objects/abstract.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1329,7 +1329,7 @@ PyNumber_Long(PyObject *o)
}
trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__);
if (trunc_func) {
result = PyEval_CallObject(trunc_func, NULL);
result = _PyObject_CallNoArg(trunc_func);
Py_DECREF(trunc_func);
if (result == NULL || PyLong_CheckExact(result)) {
return result;
Expand Down
41 changes: 14 additions & 27 deletions Objects/fileobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,39 +49,26 @@ PyFile_FromFd(int fd, const char *name, const char *mode, int buffering, const c
PyObject *
PyFile_GetLine(PyObject *f, int n)
{
_Py_IDENTIFIER(readline);
PyObject *result;

if (f == NULL) {
PyErr_BadInternalCall();
return NULL;
}

{
PyObject *reader;
PyObject *args;
_Py_IDENTIFIER(readline);

reader = _PyObject_GetAttrId(f, &PyId_readline);
if (reader == NULL)
return NULL;
if (n <= 0)
args = PyTuple_New(0);
else
args = Py_BuildValue("(i)", n);
if (args == NULL) {
Py_DECREF(reader);
return NULL;
}
result = PyEval_CallObject(reader, args);
Py_DECREF(reader);
Py_DECREF(args);
if (result != NULL && !PyBytes_Check(result) &&
!PyUnicode_Check(result)) {
Py_DECREF(result);
result = NULL;
PyErr_SetString(PyExc_TypeError,
"object.readline() returned non-string");
}
if (n <= 0) {
result = _PyObject_CallMethodIdObjArgs(f, &PyId_readline, NULL);
}
else {
result = _PyObject_CallMethodId(f, &PyId_readline, "i", n);
}
if (result != NULL && !PyBytes_Check(result) &&
!PyUnicode_Check(result)) {
Py_DECREF(result);
result = NULL;
PyErr_SetString(PyExc_TypeError,
"object.readline() returned non-string");
}

if (n < 0 && result != NULL && PyBytes_Check(result)) {
Expand Down Expand Up @@ -197,7 +184,7 @@ PyObject_AsFileDescriptor(PyObject *o)
}
else if ((meth = _PyObject_GetAttrId(o, &PyId_fileno)) != NULL)
{
PyObject *fno = PyEval_CallObject(meth, NULL);
PyObject *fno = _PyObject_CallNoArg(meth);
Py_DECREF(meth);
if (fno == NULL)
return -1;
Expand Down
2 changes: 1 addition & 1 deletion Objects/typeobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4348,7 +4348,7 @@ _common_reduce(PyObject *self, int proto)
if (!copyreg)
return NULL;

res = PyEval_CallMethod(copyreg, "_reduce_ex", "(Oi)", self, proto);
res = PyObject_CallMethod(copyreg, "_reduce_ex", "Oi", self, proto);
Py_DECREF(copyreg);

return res;
Expand Down
2 changes: 1 addition & 1 deletion Objects/weakrefobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,7 @@ proxy_checkref(PyWeakReference *proxy)

WRAP_BINARY(proxy_getattr, PyObject_GetAttr)
WRAP_UNARY(proxy_str, PyObject_Str)
WRAP_TERNARY(proxy_call, PyEval_CallObjectWithKeywords)
WRAP_TERNARY(proxy_call, PyObject_Call)

static PyObject *
proxy_repr(PyWeakReference *proxy)
Expand Down