Visitar URL original
[subinterpreters] Make free lists and unicode caches per-interpreter · Issue #84701 · python/cpython · GitHub
Skip to content

[subinterpreters] Make free lists and unicode caches per-interpreter #84701

Closed
Closed
[subinterpreters] Make free lists and unicode caches per-interpreter#84701
Labels
3.10 (EOL)end of lifetopic-subinterpreters

Description

@vstinner
vstinner
opened on May 5, 2020
BPO 40521
Nosy @rhettinger, @mdickinson, @vstinner, @markshannon, @corona10, @shihai1991, @junyixie
PRs
  • bpo-40521: Disable Unicode caches in isolated subinterpreters #19933
  • bpo-40521: Disable free lists in subinterpreters #19937
  • bpo-40521: Disable list free list in subinterpreters #19959
  • bpo-40521: Disable method cache in subinterpreters #19960
  • bpo-40521: Add PyInterpreterState.unicode #20081
  • bpo-40521: Add PyInterpreterState.unicode #20082
  • bpo-40521: Per-interpreter interned strings #20085
  • bpo-40521: Fix update_slot() whne INTERN_NAME_STRINGS is not defined #20246
  • bpo-40521: Make tuple free list per-interpreter #20247
  • bpo-40521: Make float free list per-interpreter #20636
  • bpo-40521: Make slice cache per-interpreter #20637
  • bpo-40521: Make frame free list per-interpreter #20638
  • bpo-40521: Make list free list per-interpreter #20642
  • bpo-40521: Make async gen free lists per-interpreter #20643
  • bpo-40521: Make context free list per-interpreter #20644
  • bpo-40521: Make dict free lists per-interpreter #20645
  • bpo-40521: Make the empty frozenset per interpreter #21068
  • bpo-40521: Remove freelist from collections.deque() #21073
  • bpo-40521: Make bytes singletons per interpreter #21074
  • bpo-40521: Cleanup code of free lists #21082
  • bpo-40521: Empty frozenset is no longer a singleton #21085
  • bpo-40521: Make MemoryError free list per interpreter #21086
  • bpo-40521: Make empty Unicode string per interpreter #21096
  • bpo-40521: Optimize PyUnicode_New(0, maxchar) #21099
  • bpo-40521: Make Unicode latin1 singletons per interpreter #21101
  • bpo-40521: Fix _PyContext_Fini() #21103
  • bpo-40521: Always create the empty tuple singleton #21116
  • bpo-40521: Optimize PyBytes_FromStringAndSize(str, 0) #21142
  • bpo-40521: Cleanup finalize_interp_types() #21265
  • bpo-40521: Fix PyUnicode_InternInPlace() #22376
  • bpo-40521: [subinterpreters] Make dtoa bigint free list per-interpreter #24821
  • Revert "bpo-40521: Remove freelist from collections.deque() (GH-21073)" #24944
  • Revert "bpo-40521: [subinterpreters] Make dtoa bigint free list per-interpreter" #24964
  • bpo-40521: Convert deque freelist from global vars to instance vars #25906
  • bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" #30422
  • [3.10] bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422) #30425
  • [3.10] bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422) #30433
    		•
    Files
  • bench_tuple.patch
  • microbench_tuple.py
  • bench_dict.patch
  • interned_bug.py
  • bench_dtoa.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-05-05.15:48:02.489>
labels = ['expert-subinterpreters', '3.10']
title = '[subinterpreters] Make free lists and unicode caches per-interpreter'
updated_at = <Date 2022-01-06.15:24:07.868>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2022-01-06.15:24:07.868>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Subinterpreters']
creation = <Date 2020-05-05.15:48:02.489>
creator = 'vstinner'
dependencies = []
files = ['49212', '49213', '49216', '49699', '49899']
hgrepos = []
issue_num = 40521
keywords = ['patch']
message_count = 56.0
messages = ['368175', '368177', '368187', '368278', '368283', '368807', '368808', '369407', '370636', '370733', '370734', '370735', '370737', '370740', '370741', '370742', '370754', '370755', '370756', '370757', '370771', '370928', '370969', '372146', '372148', '372161', '372168', '372169', '372176', '372181', '372207', '372209', '372216', '372220', '372223', '372250', '372357', '372795', '377368', '383789', '383790', '383829', '385950', '388492', '388493', '388617', '389226', '389294', '389305', '389525', '389527', '393787', '395860', '409819', '409856', '409862']
nosy_count = 7.0
nosy_names = ['rhettinger', 'mark.dickinson', 'vstinner', 'Mark.Shannon', 'corona10', 'shihai1991', 'JunyiXie']
pr_nums = ['19933', '19937', '19959', '19960', '20081', '20082', '20085', '20246', '20247', '20636', '20637', '20638', '20642', '20643', '20644', '20645', '21068', '21073', '21074', '21082', '21085', '21086', '21096', '21099', '21101', '21103', '21116', '21142', '21265', '22376', '24821', '24944', '24964', '25906', '30422', '30425', '30433']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue40521'
versions = ['Python 3.10']

    Activity

    1. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      tuple, dict and frame use free lists to optimize the creation of objects.

      Unicode uses "interned" strings to reduce the Python memory footprint and speedup dictionary lookups.

      Unicode also uses singletons for single letter Latin1 characters ([U+0000; U+00FF] range).

      All these optimizations are incompatible with isolated subinterpreters, since caches are currently shared by all inteprepreters. These caches should be made per-intepreter. See bpo-40512 "Meta issue: per-interpreter GIL" for the rationale.

      I already made small integer singletons per interpreter in bpo-38858:

    2. vstinner
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      3.9 (EOL)end of life
      on May 5, 2020

    3. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      New changeset 607b102 by Victor Stinner in branch 'master':
      bpo-40521: Disable Unicode caches in isolated subinterpreters (GH-19933)
      607b102

    4. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      New changeset b4b5386 by Victor Stinner in branch 'master':
      bpo-40521: Disable free lists in subinterpreters (GH-19937)
      b4b5386

    5. vstinner commented on May 6, 2020

      @vstinner
      vstinner
      on May 6, 2020
      MemberAuthor

      New changeset 89fc4a3 by Victor Stinner in branch 'master':
      bpo-40521: Disable method cache in subinterpreters (GH-19960)
      89fc4a3

    6. vstinner commented on May 6, 2020

      @vstinner
      vstinner
      on May 6, 2020
      MemberAuthor

      New changeset b7aa23d by Victor Stinner in branch 'master':
      bpo-40521: Disable list free list in subinterpreters (GH-19959)
      b7aa23d

    7. vstinner commented on May 13, 2020

      @vstinner
      vstinner
      on May 13, 2020
      MemberAuthor

      I wrote a draft PR to make interned strings per-interpreter. It does crash because it requires to make method cache and _PyUnicode_FromId() (bpo-39465) compatible with subinterpreters.

    8. vstinner commented on May 13, 2020

      @vstinner
      vstinner
      on May 13, 2020
      MemberAuthor

      New changeset 3d17c04 by Victor Stinner in branch 'master':
      bpo-40521: Add PyInterpreterState.unicode (GH-20081)
      3d17c04

    9. vstinner
      added
      topic-subinterpreters
      and removed
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on May 15, 2020
    10. vstinner
      changed the title [-]Make tuple, dict, frame free lists, unicode interned strings, unicode latin1 singletons per-interpreter[/-] [+][subinterpreters] Make free lists and unicode caches per-interpreter[/+] on May 15, 2020
    11. vstinner
      added
      topic-subinterpreters
      on May 15, 2020

    12. 55 remaining items

    13. vstinner commented on Mar 25, 2021

      @vstinner
      vstinner
      on Mar 25, 2021
      MemberAuthor

      I reopen the issue to remind me that collections.deque() freelist is shared by all interpreters.

    14. vstinner
      reopened this on Mar 25, 2021
    15. vstinner
      reopened this on Mar 25, 2021

    16. vstinner commented on May 17, 2021

      @vstinner
      vstinner
      on May 17, 2021
      MemberAuthor

      I reopen the issue to remind me that collections.deque() freelist is shared by all interpreters.

      Each deque instance now has its own free list.

      But dtoa.c still has a per-process cache, shared by all interpreters.

    17. rhettinger commented on Jun 15, 2021

      @rhettinger
      rhettinger
      on Jun 15, 2021
      Contributor

      [Victor Stinner]

      My plan is to merge changes which have no significant
      impact on performances

      FWIW, PyFloat_FromDouble() is the most performance critical function in floatobject.c.

    18. vstinner commented on Jan 6, 2022

      @vstinner
      vstinner
      on Jan 6, 2022
      MemberAuthor

      New changeset 35d6540 by Victor Stinner in branch 'main':
      bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422)
      35d6540

    19. vstinner commented on Jan 6, 2022

      @vstinner
      vstinner
      on Jan 6, 2022
      MemberAuthor

      New changeset 72c260c by Victor Stinner in branch '3.10':
      [3.10] bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422) (GH-30425)
      72c260c

    20. vstinner commented on Jan 6, 2022

      @vstinner
      vstinner
      on Jan 6, 2022
      MemberAuthor

      My commit ea25180 (interned strings) introduced bpo-46006 "[subinterpreter] _PyUnicode_EqualToASCIIId() issue with subinterpreters" regression.

      To unblock the Python 3.11.0a4 release, I just reverted the change. It reintroduces the issue, so I created bpo-46283: "[subinterpreters] Unicode interned strings must not be shared between interpreters".

    21. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    22. ezio-melotti
      added this to Subinterpreterson Apr 15, 2022
    23. ezio-melotti
      moved this to Todo in Subinterpreterson Apr 15, 2022

    24. vstinner commented on Nov 3, 2022

      @vstinner
      vstinner
      on Nov 3, 2022
      MemberAuthor

      I don't have the bandwidth to fix the remaining issues, so I just close again the issue.

    25. vstinner
      closed this as completedon Nov 3, 2022
    26. Repository owner moved this from Todo to Done in Subinterpreterson Nov 3, 2022
    27. encukou
      added a commit that references this issue on Jun 14, 2024

      Use mortal strings in type_setattro

      6036cb1
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of lifetopic-subinterpreters

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions