Visitar URL original
Fix crash on standalone comments in comprehensions and lambdas by Pedro-Muller29 · Pull Request #5144 · psf/black · GitHub
Skip to content

Fix crash on standalone comments in comprehensions and lambdas - #5144

Merged
cobaltt7 merged 5 commits into
psf:mainfrom
Pedro-Muller29:fix/4296-standalone-comments-comprehensions
May 28, 2026
Merged

cobaltt7 merged 5 commits into
psf:mainfrom
Pedro-Muller29:fix/4296-standalone-comments-comprehensions

Conversation

@Pedro-Muller29

Copy link
Copy Markdown
Contributor

Description

Fixes #4296.

Three inputs from the issue crash on the first formatting pass:

[
    x for
    # comment
    x, y in ["AB",]
]

[
    [
        x for x
        # comment
        in [
            # comment
            "ABC"
        ]
    ]
]

{(lambda
    # comment
    x: [
        # comment
    ]
)}

All of them blow up with Cannot parse: ... bad input because Black dumps a string where a standalone comment sits inline next to the following token, and Python's tokenizer swallows everything to the next newline.

Root cause

Two distinct things.

For cases B and C, transform_line runs through its splitters and, when nothing succeeds, falls through to yield line. Line.__str__ concatenates leaves without inserting newlines, so a STANDALONE_COMMENT mid-stream renders glued to whatever follows. Line.append_safe enforces "comment on its own line" only at bracket depth 0 or inside an open for / lambda. For comments living inside arbitrary nested brackets, there's no guard.

For case A, standalone_comment_split does pull the comment between for and the tuple target onto its own line, but the resulting sub-line x, y in ["AB",] no longer carries the for keyword. Its bracket tracker walks fresh, the comma between x and y lands at depth 0, gets COMMA_PRIORITY, and delimiter_split breaks the tuple apart. Black also tacks a stray trailing comma onto the iterable.

Fix

  • linegen.py: when no transformer succeeds and the line still carries standalone comments, fall back to _force_standalone_comment_split, which yields each comment on its own line. The alternative is invalid Python, so the unconditional split is safe.
  • brackets.py: is_split_after_delimiter returns 0 for commas whose parent is exprlist and whose grandparent is comp_for / old_comp_for / for_stmt. Those commas are tuple-packing, not list delimiters.

The fixture covers the three issue cases plus async-for and a nested tuple target (for (a, b), c in ...) to lock both fixes in.

Flagging that case B comes out with loose indentation:

[
    [x for x
    # comment
    in [
    # comment
    "ABC"]]
]

The inner [...] doesn't re-explode, so the comment and "ABC"]] sit at the wrong depth. Let me know if there's a target shape you'd want, or if this is fine for the crash fix.

Checklist - did you ...

  • Implement any code style changes under the --preview style, following the stability policy?
  • Add an entry in CHANGES.md if necessary?
  • Add / update tests if necessary?
  • Add new / update outdated documentation?

@github-actions

github-actions Bot commented May 20, 2026 •

Copy link
Copy Markdown
Contributor

diff-shades results comparing this PR (514b235) to main (13e97b4):

--preview style: no changes

--stable style: no changes


What is this? | Workflow run | diff-shades documentation

@Pedro-Muller29
Pedro-Muller29 force-pushed the fix/4296-standalone-comments-comprehensions branch from 0f6c882 to 61af152 Compare May 21, 2026 13:05
@cobaltt7

cobaltt7 commented May 22, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks!

Flagging that case B comes out with loose indentation: ...
The inner [...] doesn't re-explode, so the comment and "ABC"]] sit at the wrong depth. Let me know if there's a target shape you'd want, or if this is fine for the crash fix.

Yeah, that's not ideal indentation. However, that's already handled with the hug_parens_with_braces_and_square_brackets style (even without this change, but the change is necessary for some of the other failures):

[[
    x
    for x
    # comment
    in [
        # comment
        "ABC"
    ]
]]

On the other hand, that style is currently restricted to --unstable due to #4036 and some other issues.

If possible, it'd be nice to improve that here as well, but if it's a lot of extra code it's fine to merge as-is and depend on hug_parens_with_braces_and_square_brackets.

@Pedro-Muller29

Copy link
Copy Markdown
Contributor Author

Tried implementing it. The depth tracking itself is straightforward. Walk the leaves counting bracket nesting, emit each sub-line at line.depth + nesting. Applied to standalone_comment_split, case B comes out the way you described.

The problem is the closing brackets. To drop ] back to the outer depth I had to force a new sub-line whenever a closer lowers nesting below the current sub-line's starting depth. That breaks comments_in_lambda_default: the trailing comma after False disappears, because ) now lands on a fresh sub-line and the recursive transform_line pass no longer hits the path where _ensure_trailing_comma injects the comma for the outer help(...).

Fixing that means threading magic trailing comma state into the new sub-line, which is reaching into things hug_parens_with_braces_and_square_brackets already handles.

@cobaltt7

Copy link
Copy Markdown
Collaborator

Sounds good. Thank you for your investigation!

@cobaltt7
cobaltt7 merged commit 7e8ef6b into psf:main May 28, 2026
57 checks passed
anandghegde added a commit to anandghegde/black that referenced this pull request Sep 15, 2026
The last-resort `_force_standalone_comment_split` added in psf#5144 split
after every standalone comment. Code converted by `# fmt: skip` is also
stored as a STANDALONE_COMMENT, and for `if (  # fmt: skip` it ends in
`)`, not a comment. Splitting there moved the header's `:` onto its own
line, so the second pass raised InvalidInput.

Only split after a standalone comment whose last line holds a comment.

Fixes psf#5402
luketainton pushed a commit to luketainton/luke_instant-msg-api that referenced this pull request Oct 5, 2026
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [black](https://github.com/psf/black) ([changelog](https://github.com/psf/black/blob/main/CHANGES.md)) | `<26.5.2,>=26.5.1` → `<26.10.1,>=26.10.0` | ![age](https://developer.mend.io/api/mc/badges/age/pypi/black/26.10.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/black/26.5.1/26.10.0?slim=true) |

---

### Release Notes

<details>
<summary>psf/black (black)</summary>

### [`v26.10.0`](https://github.com/psf/black/blob/HEAD/CHANGES.md#Version-26100)

[Compare Source](psf/black@26.5.1...26.10.0)

##### Stable style

- `--line-ranges` no longer inserts an empty line after a docstring when the range
  covers only the docstring itself ([#&#8203;5312](psf/black#5312))
- Fix `# fmt: skip` on a bracketed ternary turning the surrounding tuple into a call
  ([#&#8203;5464](psf/black#5464))
- Fix crash when `# fmt: skip` is placed on a one-line function or class with PEP 695
  type parameters ([#&#8203;5429](psf/black#5429))
- Fix an inline comment after the closing bracket of optional parentheses being moved
  inside the parentheses when the parenthesized expression contains own-line comments
  ([#&#8203;5395](psf/black#5395))
- Fix unparseable output when `# fmt: skip` is placed on a bracket of an `if`, `while`,
  `for`, or `with` header ([#&#8203;5401](psf/black#5401), [#&#8203;5405](psf/black#5405))
- Fix crash when formatting parenthesized expressions with multiple inline comments and
  `# fmt: skip` ([#&#8203;5414](psf/black#5414))
- Fix parsing Jupyter notebook assignment magics when non-ASCII characters appear
  earlier on the line ([#&#8203;5381](psf/black#5381))
- Preserve blank lines that come immediately before a `# fmt: on` comment ([#&#8203;5300](psf/black#5300))
- Keep the parentheses around the target of an annotated assignment (e.g.
  `(x): int = 5`), which prevent CPython from including the name in `__annotations__`
  ([#&#8203;5321](psf/black#5321))
- Stop treating a t-string in docstring position as a docstring (e.g. `t"  spam  "` as
  the first statement of a module, class or function) ([#&#8203;5287](psf/black#5287))
- Fix unparseable output for a t-string whose replacement field contains a quote (e.g.
  `t'\'{a["b"]}\''`) ([#&#8203;5265](psf/black#5265))
- Fix unparseable output for a triple-quoted string whose body ends in an
  already-escaped double quote (e.g. `'''\'''\"'''`) ([#&#8203;5262](psf/black#5262))
- Fix `--skip-magic-trailing-comma` dropping the trailing comma from a split
  single-element tuple used as a lambda parameter default ([#&#8203;5246](psf/black#5246))
- Fix unstable formatting when an inline comment sits on optional parentheses (e.g. a
  parenthesized assert message) ([#&#8203;5241](psf/black#5241))
- Fix `--skip-magic-trailing-comma` dropping the trailing comma of a one-element
  subscript (`a[x,]`) when the line is long enough to be split and contains a power
  operator ([#&#8203;5272](psf/black#5272))
- Fix crash when a standalone comment sits between tokens of a comprehension or lambda
  ([#&#8203;5144](psf/black#5144))
- Fix inline comments on a bracket inside a comprehension being dropped ([#&#8203;5330](psf/black#5330))
- Respect the magic trailing comma in a PEP 695 type parameter list containing a
  `*TypeVarTuple` or `**ParamSpec` ([#&#8203;5244](psf/black#5244))
- Fix crash when a comment-only `# fmt: off`/`# fmt: on` block is followed by a `with`
  statement after another standalone comment ([#&#8203;5189](psf/black#5189))
- Fix a crash when splitting `case case if ...` match patterns at very small line
  lengths ([#&#8203;5147](psf/black#5147))
- Fix multiline docstring indentation when leading tabs are used inside indented
  docstrings ([#&#8203;5148](psf/black#5148))
- Respect `# fmt: skip` on a line that opens a bracket (e.g.
  `from x import (  # fmt: skip`) when a standalone comment is among the bracket's
  contents ([#&#8203;5161](psf/black#5161))
- Fix an AST safety error when separate `# type: ignore` comments in a parenthesized
  attribute chain were merged onto one physical line ([#&#8203;5297](psf/black#5297))
- Preserve comments and blank lines outside requested ranges when formatting with
  `--line-ranges` ([#&#8203;5175](psf/black#5175))
- Fix crash when `# fmt: skip` is used on one-line `async def`, `async with`, and
  `async for` statements containing a semicolon ([#&#8203;5311](psf/black#5311))
- Stop converting form feeds or other similar characters in docstrings into newlines
  ([#&#8203;5461](psf/black#5461))
- Fix `--skip-source-first-line` turning the skipped line's CRLF ending into `\r\r\n`
  when reformatting a file with CRLF line endings ([#&#8203;5438](psf/black#5438))

##### Preview style

##### New preview features

- Add missing blank lines after classes whose last method has an ellipsis body ([#&#8203;5439](psf/black#5439))
- Split only the brackets holding a magic trailing comma when more trailers follow them
  (e.g. stop splitting inside the `[2]` of `[1,][2](3)`) ([#&#8203;5448](psf/black#5448))
- Keep dictionary keys containing operators together on one line when the value can be
  wrapped onto a new line instead ([#&#8203;5435](psf/black#5435))
- Remove redundant parentheses around individual variables in unpacking targets (e.g.
  `for (x), (y) in points:` becomes `for x, y in points:`) ([#&#8203;5416](psf/black#5416))
- Normalize uppercase `T` prefixes on t-strings to lowercase ([#&#8203;5425](psf/black#5425))
- Remove redundant parentheses around generator expressions ([#&#8203;5304](psf/black#5304), [#&#8203;5369](psf/black#5369))
- Preserve two blank lines before a top-level class starting inside a `# fmt: off` block
  after an import ([#&#8203;5238](psf/black#5238))
- Fix unnecessary parentheses around short right-hand expressions in indexed assignments
  (e.g. `x[key] = expr`) ([#&#8203;5095](psf/black#5095))
- Parenthesize tuple expressions in `yield` statements for consistency with function
  calls and returns ([#&#8203;5170](psf/black#5170))
- Stop splitting between a variable and its operator (`not in`, `==`, `is`, ...) when
  the right-hand side is a bracketed expression, and instead split inside the brackets
  ([#&#8203;5135](psf/black#5135))
- In `.pyi` stub files, enforce a blank line after a function or method that has a
  docstring-only body when another comment or statement follows it ([#&#8203;5158](psf/black#5158))

##### Updates to existing preview features

- Fix crash in stub files when `# fmt: skip` is placed on a function in a group of
  same-name decorated functions (e.g. `@overload`s or a property setter) ([#&#8203;5430](psf/black#5430))
- Keep the parentheses around a lambda used as the iterable of a comprehension (e.g.
  `[x for x in (lambda: 0) if x]`) ([#&#8203;5176](psf/black#5176), [#&#8203;5200](psf/black#5200))

##### Updates to existing unstable features

- Do not treat multi-line expressions with merged strings and trailing `# type: ignore`
  comments as single-line unsplittable expressions ([#&#8203;5466](psf/black#5466))
- Fix duplicated inline comment when stripping the parentheses around a string or
  merging a backslash-continued string on the same line ([#&#8203;5449](psf/black#5449))
- Split long stringified return annotations even when the function has parameters
  ([#&#8203;5427](psf/black#5427))
- Don't hug brackets when doing so would join two `# type: ignore` comments onto one
  line ([#&#8203;5271](psf/black#5271))
- Fix a crash when `# type: ignore` is lost during formatting of a long parenthesized
  string ([#&#8203;5329](psf/black#5329))
- Fix only the first part of an implicitly concatenated unmergeable string (e.g.
  `r"..." r"..."`) being wrapped in parentheses ([#&#8203;5434](psf/black#5434))

##### Configuration

- Add support for `NO_COLOR` environment variable to disable ANSI output ([#&#8203;5129](psf/black#5129))
- Remove spurious target version warning when runtime version is included in a
  `--target-version` flag ([#&#8203;5167](psf/black#5167))
- Fix `--force-exclude` not excluding files whose path contains `..` (e.g.
  `black ../generated/file.py` run from a subdirectory) ([#&#8203;5471](psf/black#5471))
- Fall back to the default configuration, with a warning, when the given sources share
  no common project root (e.g. they are on different drives on Windows) instead of
  crashing ([#&#8203;5386](psf/black#5386))
- Fix loading a stale cached `pyproject.toml` path when `--code` is used from different
  working directories in the same process ([#&#8203;5152](psf/black#5152))
- Add validation for `--line-ranges` values ([#&#8203;5107](psf/black#5107))
- Ignore empty cache files instead of raising an `EOFError` ([#&#8203;5192](psf/black#5192))
- Reject non-string `include` and `force-exclude` values in `pyproject.toml` ([#&#8203;5193](psf/black#5193))
- Validate `BLACK_NUM_WORKERS` values and report invalid values as usage errors instead
  of crashing ([#&#8203;5211](psf/black#5211))
- Ignore permission errors when reading cache ([#&#8203;5258](psf/black#5258))

##### Packaging

- Reduce the size of Linux standalone binaries by stripping debug symbols during the
  PyInstaller release build ([#&#8203;5223](psf/black#5223))
- Black is now released using GitHub
  [Immutable Releases](https://docs.github.com/code-security/concepts/supply-chain-security/immutable-releases)
  ([#&#8203;5296](psf/black#5296))

##### Performance

- Fix superlinear runtime growth with the number of input files ([#&#8203;5450](psf/black#5450))
- Improve performance on strings containing many consecutive backslashes ([#&#8203;5163](psf/black#5163))
- Improve performance on files with many `# fmt: skip`/`# fmt: off`/`# fmt: on` comments
  ([#&#8203;5169](psf/black#5169), [#&#8203;5190](psf/black#5190), [#&#8203;5232](psf/black#5232))
- Improve performance on long calls and collections ([#&#8203;5177](psf/black#5177))
- Improve performance on multiline strings inside large collections (e.g. a dict literal
  with triple-quoted strings as values) ([#&#8203;5188](psf/black#5188))
- Improve performance on files with many soft keywords (e.g. `match`/`case` blocks)
  ([#&#8203;5186](psf/black#5186))
- Improve performance on long semicolon-separated statements (in the stable style) and
  large dict literals (in `--preview`) ([#&#8203;5184](psf/black#5184))
- Improve performance of `--line-ranges` on files with many sibling blocks (a long
  `if`/`elif` chain, a `match` with many cases, or many top-level definitions) ([#&#8203;5213](psf/black#5213))
- Improve performance on deeply nested bracketed expressions ([#&#8203;5171](psf/black#5171), [#&#8203;5242](psf/black#5242))
- Improve performance on lists and subscripts holding one long expression without
  operators (e.g. a long run of implicitly concatenated strings inside `[]`) ([#&#8203;5239](psf/black#5239))
- Improve performance on deeply chained operations (e.g. a long `a ** b ** c ** ...`
  chain) ([#&#8203;5235](psf/black#5235))
- Improve performance on long `if`/`elif` chains and other compound statements with many
  clauses ([#&#8203;5322](psf/black#5322))
- Improve performance of `string_processing`:
  - when merging implicitly concatenated f-strings containing long string literals
    ([#&#8203;5165](psf/black#5165))
  - when merging long implicitly concatenated strings ([#&#8203;5173](psf/black#5173), [#&#8203;5194](psf/black#5194))
  - when rewriting large nodes (e.g. `"%s ..." % (a, b, c, ...)`) ([#&#8203;5178](psf/black#5178), [#&#8203;5199](psf/black#5199), [#&#8203;5220](psf/black#5220))
  - when splitting long string literals ([#&#8203;5183](psf/black#5183))

##### Output

- Report parser failures using editor-friendly `path:line:column` locations ([#&#8203;5237](psf/black#5237))
- Fix crash when writing formatted code or diffs to a `sys.stdout` that has no `buffer`
  attribute (e.g. in Jupyter notebooks) ([#&#8203;5411](psf/black#5411))
- Report parse failures on Black's own output as internal errors ([#&#8203;5383](psf/black#5383))

##### *Blackd*

- Return HTTP 400 instead of 500 when the `X-Python-Variant` header is empty or has an
  empty entry (e.g. a trailing comma) ([#&#8203;5428](psf/black#5428))
- Allow optional whitespace around comma-separated versions and `pyi` in the
  `X-Python-Variant` header ([#&#8203;5441](psf/black#5441))

##### Integrations

- Remove unused `migrate-black` script ([#&#8203;5319](psf/black#5319))
- Support PEP 440 version specifiers in `tool.black.required-version` for the GitHub
  Action ([#&#8203;5399](psf/black#5399))
- Add outputs (`is_formatted`, `change_count`, `same_count`, `failure_count`) to GitHub
  Action runs ([#&#8203;5408](psf/black#5408))

##### Documentation

- Document `vim-python-pep8-indent`, which provides an `indentexpr` for Black-style
  insert-mode indentation ([#&#8203;5288](psf/black#5288))

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMzIuNSIsInVwZGF0ZWRJblZlciI6IjQ0LjEzMi41IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL2RlcGVuZGVuY2llcyJdfQ==-->

Reviewed-on: https://git.tainton.uk/luke/instant-msg-api/pulls/290
Co-authored-by: renovate[bot] <renovate-bot@git.tainton.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standalone comments still partially broken

2 participants