Visitar URL original
Provide a toml module in the standard library · Issue #84240 · python/cpython · GitHub
Skip to content

Provide a toml module in the standard library #84240

Closed
Closed
Provide a toml module in the standard library#84240
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Description

@mgorny
mgorny
mannequin
opened on Mar 25, 2020
BPO 40059
Nosy @brettcannon, @tiran, @mcepl, @njsmith, @encukou, @agronholm, @mgorny, @dstufft, @pradyunsg, @eli-schwartz, @miss-islington, @tirkarthi, @skoslowski, @erlend-aasland, @hauntsaninja, @domdfcoding, @hukkin, @YakoYakoYokuYoku
PRs
  • bpo-40059: tomllib #31498
  • bpo-40059: Fix installation of tomllib #31784
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-03-25.06:54:21.437>
labels = ['type-feature', 'library']
title = 'Provide a toml module in the standard library'
updated_at = <Date 2022-03-09.13:38:15.300>
user = 'https://github.com/mgorny'

    bugs.python.org fields:

    activity = <Date 2022-03-09.13:38:15.300>
actor = 'miss-islington'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-03-25.06:54:21.437>
creator = 'mgorny'
dependencies = []
files = []
hgrepos = []
issue_num = 40059
keywords = ['patch']
message_count = 27.0
messages = ['364979', '364983', '365000', '373879', '387807', '387892', '401071', '401421', '406458', '406484', '406488', '406515', '406561', '408316', '408320', '408418', '408503', '408505', '408576', '409488', '409489', '409552', '410353', '414731', '414746', '414800', '414801']
nosy_count = 20.0
nosy_names = ['brett.cannon', 'christian.heimes', 'mcepl', 'njs', 'petr.viktorin', 'alex.gronholm', 'mgorny', 'dstufft', 'pradyunsg', 'eschwartz', 'miss-islington', 'VA', 'xtreak', 'skoslowski', 'erlendaasland', 'hauntsaninja', 'domdfcoding', 'dom1310df', 'hukkinj1', 'YakoYakoYokuYoku']
pr_nums = ['31498', '31784']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40059'
versions = []

    Activity

    1. mgorny commented on Mar 25, 2020

      @mgorny
      mgornymannequin
      on Mar 25, 2020
      MannequinAuthor

      PEP-518 uses the TOML format to specify build system requirements. AFAIU this means that all new build systems will require a TOML parser. Could you consider adding one to the standard library to reduce the number of chicken-egg problems?

      The referenced PEP states that 'pytoml TOML parser is ~300 lines of pure Python code', so I don't think integrating it would be a large maintenance cost.

      [1] https://www.python.org/dev/peps/pep-0518/

    2. mgorny
      added
      3.9 (EOL)end of life
      stdlibStandard Library Python modules in the Lib/ directory
      type-featureA feature request or enhancement
      on Mar 25, 2020

    3. tirkarthi commented on Mar 25, 2020

      @tirkarthi
      tirkarthi
      on Mar 25, 2020
      Member

      Relevant python-dev discussion : https://mail.python.org/pipermail/python-dev/2019-May/157405.html . The format has still not reached 1.0. Issue to track the 1.0 release candidate : toml-lang/toml#698

    4. brettcannon commented on Mar 25, 2020

      @brettcannon
      brettcannon
      on Mar 25, 2020
      Member

      The plan is to start discussing adding a TOML parser once the spec reaches 1.0 (which we will know about as one of the pip contributors also manages TOML).

    5. VA commented on Jul 18, 2020

      @VA
      VAmannequin
      on Jul 18, 2020
      Mannequin

      1.0.0-rc.1 is out by now: https://github.com/toml-lang/toml/blob/master/CHANGELOG.md

    6. VA
      added
      3.10 (EOL)end of life
      on Jul 18, 2020
    7. brettcannon
      removed
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      on Jan 2, 2021

    8. 11 remaining items

    9. tiran commented on Dec 11, 2021

      @tiran
      tiran
      on Dec 11, 2021
      Member

      I just noticed that tomli has dropped support for Python 3.6. That's a road block for general adoption of the package in the Python ecosystem. Python 3.6 is the default Python interpreter in CentOS 8, C8S, RHEL 8, and Ubuntu 18.04 LTS. hukkin/tomli#134

    10. hauntsaninja commented on Dec 13, 2021

      @hauntsaninja
      hauntsaninja
      on Dec 13, 2021
      Contributor

      Given that this currently seems blocked on the broad question of "how should additions and removals to the stdlib be managed", I'd like to not focus too hard just yet on the specifics of tomli. I assume it's unlikely, but for all we know, the SC could determine that all newly included modules have to be written from scratch, following along the lines of recent additions like zoneinfo, graphlib and importlib.metadata.

      There's a lot we could bikeshed or debate... e.g., it's not even clear to me what a toml package in the stdlib would be named, never mind what it means for an unreleased version of a commonly used third party toml package dropping support for an imminently EOL version of Python. I'm possibly totally out of line, but in my head, the process would look something like this:

      Step 1: Hear back from the SC about criteria and considerations for new modules in the stdlib
      Step 2: Determine whether a TOML library *in the abstract* is something that would meet the outlined criteria (potentially e.g. is this something we even want? is this something we can maintain?)
      Step 3: Determine if we have an implementation (written from scratch, copied, or derived from something pre-existing) that would meet the outlined criteria
      Step 4: Do all the rest of the work to meet the outlined criteria (potentially e.g. go through PEP process, create proposed impl, write a backport, bikeshed api and name, etc)

      I guess I have the following questions:

      • Is my understanding correct that this issue is blocked on SC guidance?
      • Is there anything we could do in advance of SC guidance that would be productive?
        Brett previously mentioned bringing it up with the author of tomli, as per Please consider pushing tomli into stdlib hukkin/tomli#141 they seem supportive
      • Is there a good place to follow along or be notified of SC thoughts?
        I see no mention of stdlib changes in https://github.com/python/steering-council and the discuss thread linked above seems to have petered out.

    11. brettcannon commented on Dec 14, 2021

      @brettcannon
      brettcannon
      on Dec 14, 2021
      Member

      I just noticed that tomli has dropped support for Python 3.6. That's a road block for general adoption of the package in the Python ecosystem.

      It's already in pip, so I think it's already generally adopted 😉. https://github.com/pypa/pip/tree/main/src/pip/_vendor/tomli

      Is my understanding correct that this issue is blocked on SC guidance?

      Not officially, no. But I'm personally not going to bring it forward right now. If someone else wants to formulate a complete proposal for the SC on this then they are definitely welcome to! You will need to address where the code is coming from, why that code should be used, what's the API, etc.

      The only reason the SC is mentioned here is there will be a discussion about how to maintain the stdlib, but it simply hasn't happened yet. You don't have to wait for it and asking for a TOML module might actually force the issue.

      Is there anything we could do in advance of SC guidance that would be productive?

      Nope, someone eventually has to have the time to make the proposal and manage the deluge of comments.

      Is there a good place to follow along or be notified of SC thoughts?

      https://github.com/python/steering-council as you already pointed out through the issues and monthly summaries. Otherwise you just need to open an issue and ask. 😃

    12. YakoYakoYokuYoku commented on Dec 14, 2021

      @YakoYakoYokuYoku
      YakoYakoYokuYokumannequin
      on Dec 14, 2021
      Mannequin

      Not officially, no. But I'm personally not going to bring it forward right now. If someone else wants to formulate a complete proposal for the SC on this then they are definitely welcome to! You will need to address where the code is coming from, why that code should be used, what's the API, etc.

      The only reason the SC is mentioned here is there will be a discussion about how to maintain the stdlib, but it simply hasn't happened yet. You don't have to wait for it and asking for a TOML module might actually force the issue.

      I've been working on a document detailing the API implementation (as a PEP draft) see https://gitlab.com/YakoYakoYokuYoku/pep-toml. Although I've yet to write the code I'll to move forward with it.

    13. brettcannon commented on Dec 14, 2021

      @brettcannon
      brettcannon
      on Dec 14, 2021
      Member

      I opened python/steering-council#92 for the SC to discuss stdlib additions in case I am not re-elected.

    14. hauntsaninja commented on Jan 2, 2022

      @hauntsaninja
      hauntsaninja
      on Jan 2, 2022
      Contributor

      You will need to address where the code is coming from, why that code should be used, what's the API, etc.

      Happy new year, potentially-toml-wanting friends!

      I wrote up a draft of a proposal here: https://gist.github.com/hauntsaninja/9f136a5a60f63d8ca2cdfadb50edba44

      I've passed it along to hukkin (author of tomli) for feedback. Once they reply, I'll send it to python-dev and the deluge of comments can get underway. If in the meantime anyone else has thoughts, I'd love to hear from you.

    15. hauntsaninja commented on Jan 2, 2022

      @hauntsaninja
      hauntsaninja
      on Jan 2, 2022
      Contributor

      pradyunsg kindly pointed me to an ongoing thread in Packaging discuss: https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/84

      So seems like a PEP will be necessary. I'm happy to do legwork for that. (The PEP that I'll write differs from YakoYakoYokuYoku's json-esque draft in that it'll be based on tomli instead of a from-scratch implementation)

    16. hauntsaninja commented on Jan 3, 2022

      @hauntsaninja
      hauntsaninja
      on Jan 3, 2022
      Contributor

      We've started a PEP draft.

      https://github.com/hauntsaninja/peps/blob/toml-pep/pep-9999.rst shows a rendered version.
      hauntsaninja/peps#1 is a PR from the toml-pep branch to main, to help ease of review / discussion.

      If you're following along and need to get your bearings straight, here are the recently active locations of discussion:

    17. hauntsaninja commented on Jan 11, 2022

      @hauntsaninja
      hauntsaninja
      on Jan 11, 2022
      Contributor

      This is now PEP-680: https://www.python.org/dev/peps/pep-0680/
      Currently being discussed at: https://discuss.python.org/t/pep-680-tomllib-support-for-parsing-toml-in-the-standard-library/13040

    18. encukou commented on Mar 8, 2022

      @encukou
      encukou
      on Mar 8, 2022
      Member

      New changeset 591f675 by Taneli Hukkinen in branch 'main':
      bpo-40059: Add tomllib (PEP-680) (GH-31498)
      591f675

    19. encukou commented on Mar 8, 2022

      @encukou
      encukou
      on Mar 8, 2022
      Member

      The PR is merged and buildbots are green. Thank you to everyone who helped!

      Now would be a good time to bikeshed wording in the documentation.

      From the PR:

      Would it be good to mention in the docs why load() takes only binary files? The encoding requirement probably isn't obvious for first-time users.

    20. dom1310df commented on Mar 9, 2022

      @dom1310df
      dom1310dfmannequin
      on Mar 9, 2022
      Mannequin

      When building Python from source (as of the latest GitHub commit) the tomllib directory doesn't actually get copied over to the install prefix.

      It looks like an entry's needed in Makefile.pre.in under LIBSUBDIRS, along the lines of #13563 and #30311

    21. miss-islington commented on Mar 9, 2022

      @miss-islington
      miss-islington
      on Mar 9, 2022
      Contributor

      New changeset 23dcea5 by Dominic Davis-Foster in branch 'main':
      bpo-40059: Fix installation of tomllib (GH-31784)
      23dcea5

    22. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    23. encukou
      closed this as completedon Jun 27, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions