Repository navigation
Make list objects thread-safe in --disable-gil builds #112087
Description
Activity
Hey, I would love to work on this issue if you don't mind, thanks.
Thanks @Lincoln-developer! I'd suggest starting by transforming some of the functions to use the critical section API. Some functions that use Argument Clinic can just use the
@critical_sectiondirective. Other functions will need manualPy_BEGIN_CRITICAL_SECTION()andPy_END_CRITICAL_SECTION()calls. Feel free to tag me on a draft PR if you would like early feedback.Reacted by Angufibo Lincoln@colesbury Since this work has not be updated from last year November, can I assign this issue to myself?
Reacted by Erlend E. Aasland@Lincoln-developer are you still working on this or can @corona10 take this over?
I think he can take it on because l have taken longer than expected ,its okay
Reacted by Sam GrossMy apologies for not delivering as expected,
Reacted by Donghee NaTODO based on diff
- PyList_Insert
- PyList_SetItem
- PyList_Append
- list_repr
- list_length
- list_contains
- list_item
- list_slice
- PyList_GetSlice
- list_concat
- list_repeat
- _list_clear
- list_ass_slice
- list_inplace_repeat
- list_ass_item
- list_insert_impl
- list_clear_impl
- list_remove_impl
- list_copy_impl
- list_append
- list_extend
- list_pop_impl
- list_reverse_impl
- PyList_Reverse
- PyList_AsTuple
- list_index_impl
- list_count
- list_remove
- list_richcompare
- list___sizeof___impl
- list_subscript
- list_ass_subscript
- listiter_next
- listiter_len
- listreviter_next
- listreviter_len
- listiter_reduce_general
Reacted by Erlend E. Aasland60 remaining items
- added 7 commits that reference this issue
on Apr 17, 2024 - added 4 commits that reference this issue
on Sep 2, 2024
Feature or enhancement
I expect this to be implemented across multiple PRs.
For context, here is the change from the
nogil-3.12fork, but things might be done a bit differently in CPython 3.13: colesbury/nogil-3.12@df4c51f82b.Iterators
For performance reasons, we don't want to acquire locks while iterating over lists. This means that list iterators have a lesser notion of thread-safety: multiple threads concurrently using the same iterator should not crash but may revisit elements. We should make clear in the documentation that iterators are not generally thread-safe. From an implementation point of view, we should use relaxed atomics to update the
it_indexvariable.Additionally, we don't want to clear the
it_seqreference to the list when the iterator is exhausted in--disable-gilbuilds. Doing so would pose thread-safety issues (in the "may segfault sense").I don't expect this to pose any issues for real code. While it's fairly common to share lists between threads, it's rare to share the same iterator object between threads.
Linked PRs
list.extend(dict)behave atomically #117438