Deprecate lib2to3 (and 2to3) for future removal

@gvanrossum

BPO	40360
Nosy	@gvanrossum, @gpshead, @vstinner, @carljm, @ambv, @ericsnowcurrently, @hroncok, @corona10, @miss-islington, @tirkarthi, @isidentical, @iritkatriel
PRs	bpo-40360: Prepare to deprecate lib2to3. #19645 bpo-40360: Deprecate lib2to3 module in light of PEP 617 #19663 bpo-40360: Add a What's new entry for lib2to3 pending deprecation #19898 bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. #21694 [3.9] bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694) #21696 [3.9] bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694) #21697 bpo-40360: Deprecate the lib2to3 package #28116 bpo-40360: [doc] Rephrase deprecation note about lib2to3 #28122 [3.10] bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122) #28127 bpo-40360: Make the 2to3 deprecation more obvious. #29064

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-02.16:03:42.312>
created_at = <Date 2020-04-22.04:40:54.230>
labels = ['type-feature', 'expert-2to3', '3.11']
title = 'Deprecate lib2to3 (and 2to3) for future removal'
updated_at = <Date 2021-10-20.22:28:52.390>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2021-10-20.22:28:52.390>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-02.16:03:42.312>
closer = 'vstinner'
components = ['2to3 (2.x to 3.x conversion tool)']
creation = <Date 2020-04-22.04:40:54.230>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 40360
keywords = ['patch']
message_count = 51.0
messages = ['366973', '367005', '367031', '367051', '367208', '367209', '367230', '367235', '367716', '367726', '367730', '367743', '367744', '367766', '367767', '367770', '367771', '367884', '368077', '368388', '373185', '373198', '373327', '373332', '373334', '373444', '373538', '374348', '374349', '374566', '374572', '374573', '374574', '374611', '374634', '374643', '379014', '400876', '400877', '400879', '400883', '400885', '400892', '400905', '400906', '400923', '400927', '400933', '400937', '404324', '404532']
nosy_count = 14.0
nosy_names = ['gvanrossum', 'gregory.p.smith', 'vstinner', 'carljm', 'lukasz.langa', 'eric.snow', 'davidhalter', 'hroncok', 'corona10', 'miss-islington', 'xtreak', 'BTaskaya', 'Peter Ludemann', 'iritkatriel']
pr_nums = ['19645', '19663', '19898', '21694', '21696', '21697', '28116', '28122', '28127', '29064']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40360'
versions = ['Python 3.11']

Based on the PEP-617 acceptance thread on python-dev, lib2to3 is eventually going to run into trouble parsing modern syntax a few releases from now.

It would be better off maintained outside of the standard library. It gets used by a lot of things and is generally useful, but would make a lot more sense as a PyPI project than as something only quasi-maintained within the stdlib (it only gained the ability to parse a couple modern syntax features in via bugfix contributions to the stdlib the past month or two... meaning a lot of versions of it out there cannot)

Black has already forked it.

goal: PendingDeprecationWarning and documentation as such in 3.9. Move to DeprecationWarning in 3.10 or 3.11 and remove it by ~3.12. Subject to our existing deprecation process guidelines.

I am in favor of this. We could promote LibCST, which is based on Parso, which uses a forked version of pgen2 (the parser in lib2to3). I believe one of these could switch to a fork of pegen as its parser, so it will be able to handle new PEG based syntax in 3.10+.

Removal by 3.12 might be feasible.

I volunteered in the python-dev thread to write a patch to the docs clarifying future status of lib2to3; happy to include the PendingDeprecationWarning as well.

Re linking to alternatives, we want to make sure we link to alternatives that are committed to updating to support newer Python versions' syntax. This definitely includes LibCST; I can inquire with the parso maintainer about whether it also includes parso. In future it could also include a third-party-maintained copy of lib2to3, if someone picks that up.

I opened a PR. It deprecates the lib2to3 library to discourage future use of it for Python3, but not the 2to3 tool. This of course means that the lib2to3 module will in practice stick around in the stdlib as long as 2to3 is still bundled with Python.

It seems like the idea in this issue is to deprecate and remove both. I'm not sure what we typically do to deprecate a command-line utility bundled with Python. Given warnings are silent by default, the deprecation warning for lib2to3 won't be visible to users of 2to3. Should I add something to its --help output? Or something more aggressive; an unconditionally-printed warning?

New changeset 503de71 by Carl Meyer in branch 'master':
bpo-40360: Deprecate lib2to3 module in light of PEP-617 (GH-19663)
503de71

Okay,the pending deprecation is in. Keeping open as a reminder to turn that into a real DeprecationWarning in 3.10 after the 3.9 branch is cut.

We'll then want to track reminding us to remove it in 3.12.

@gregory.p.smith

What do you think about the question I raised above about how to make this deprecation visible to users of the 2to3 CLI tool, assuming the plan is to remove both?

New changeset f0b63d5 by Łukasz Langa in branch 'main':
bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122)
f0b63d5

New changeset 559af74 by Miss Islington (bot) in branch '3.10':
bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122)
559af74

I close the issue: lib2to3 is now deprecated in Python 3.11. I propose to open a new issue in Python 3.13 or newer to remove it.

The "pending" deprecation status of lib2to3 in 3.9 and 3.10 is no worse than a vanilla deprecation in terms of visibility. It will appear just the same when run with pytest or -X dev.

However, upgrading the deprecation between 3.10.0rc1 and 3.10.0rc2 really was too late. So we'll have it deprecated fully in 3.11 and removed in 3.13. One more year doesn't hurt us much but might be helpful for library maintainers to have more time to move off of the included lib2to3.

Invoking 2to3 also generates the warning so I guess this can be closed:

$ ./python.exe Tools/scripts/2to3
/private/tmp/cpy2/Tools/scripts/2to3:3: DeprecationWarning: lib2to3 package is deprecated and may not be able to parse Python 3.10+
  from lib2to3.main import main
At least one file or directory argument required.
Use --help to show usage.

Thanks for the patches, Gregory, Carl, and Victor! ✨ 🍰 ✨

Now we just have to remember to actually remove the damn thing in 3.13 😂

New changeset fdbdf3f by Gregory P. Smith in branch 'main':
bpo-40360: Make the 2to3 deprecation more obvious. (GH-29064)
fdbdf3f

Created bpo-45544 to close all open issues and list them there.

What does this mean for someone who picks up a bit of code that happens to be obsolete and then starts getting warnings about print statements and lots of other stuff?

@johns1c

@johns1c -- It means that you should ask for help somewhere else. :-) Seriously, try explaining your problem in our Discourse server at discuss.python.org.

For people who still have a Python 2 code base: download/install Python 3.12 and run 2to3 included in this version. Or give a try to 2to3 alternatives.

Thanks for your reply. This works for Python 3.11 with 2to3.exe being found in the `Scripts` directory of my Windows 11 64 bit install and 2to3.py being found in `Tools\scripts` (it was ‘tools\scripts’ in 3.8) Interestingly the Windows install for Python 3.12 does NOT provide either the library or the runnable utility. From now on it seems that a user wishing to upgrade old Python 2 code needs to install 2to3 using PIP. The command `PIP install 2to3` will install 2to3.exe in the `Scripts` directory under the Python install directory or in the `scripts` directory of a virtual environment. Chris J From: Victor Stinner Sent: 14 November 2023 12:25 To: python/cpython Cc: C Johnson; Mention Subject: Re: [python/cpython] Deprecate lib2to3 (and 2to3) for future removal(#84540) For people who still have a Python 2 code base: download/install Python 3.12 and run 2to3 included in this version. Or give a try to 2to3 alternatives. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

@vstinner

2to3 alternatives

@vstinner Such as?

Such as modernize.

Navigation Menu

Repository navigation

Deprecate lib2to3 (and 2to3) for future removal #84540

48 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Deprecate lib2to3 (and 2to3) for future removal #84540

Description

Activity

gpshead commented on Apr 22, 2020

gvanrossum commented on Apr 22, 2020

carljm commented on Apr 22, 2020

carljm commented on Apr 22, 2020

gpshead commented on Apr 24, 2020

gpshead commented on Apr 24, 2020

carljm commented on Apr 24, 2020

48 remaining items

ambv commented on Sep 2, 2021

miss-islington commented on Sep 2, 2021

vstinner commented on Sep 2, 2021

ambv commented on Sep 2, 2021

ambv commented on Oct 19, 2021

iritkatriel commented on Oct 20, 2021

johns1c commented on Nov 13, 2023

gvanrossum commented on Nov 14, 2023

vstinner commented on Nov 14, 2023

johns1c commented on Nov 17, 2023

hlovdal commented on Apr 18, 2025

ambv commented on Apr 18, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions