Visitar URL original
gh-87389: avoid treating path as URI with netloc by nascheme · Pull Request #93894 · python/cpython · GitHub
Skip to content

gh-87389: avoid treating path as URI with netloc #93894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nascheme wants to merge 19 commits into
base: main
Choose a base branch
from
Open

gh-87389: avoid treating path as URI with netloc #93894

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
4f76c44
wip: alternative fix for gh-87389
nascheme Jun 16, 2022
06b3879
Don't strip slashes if not building relative URL.
nascheme Jun 16, 2022
00a3a92
Add unit test for urlunsplit relative URL case.
nascheme Jun 16, 2022
a8e1cc2
sanitize paths that can be confused as scheme
nascheme Jun 16, 2022
83c8332
Add blurb.
nascheme Jun 16, 2022
f99e80b
Improve name and comment for _get_redirect_url().
nascheme Jun 16, 2022
e542578
Add urllib.parse.pathsplit() function.
nascheme Jun 16, 2022
b7b0b15
Fix blurb formating, add note about pathsplit().
nascheme Jun 16, 2022
6915331
Improve comments and docs.
nascheme Jun 16, 2022
915451c
Add basic unit test for pathsplit().
nascheme Jun 16, 2022
952a0f4
Fix markup in blurb.
nascheme Jun 16, 2022
899f512
Use pathsplit() in translate_path().
nascheme Jun 16, 2022
a00656c
Improved unit tests from gps.
nascheme Jun 16, 2022
8985853
Add test for Request-URI containing scheme.
nascheme Jun 16, 2022
f1f94ae
Make _get_redirect_url() into a method.
nascheme Jun 16, 2022
8a34cd0
Futher cleanups, remove urllib.parse.pathsplit().
nascheme Jun 16, 2022
23d4b56
reword news
gpshead Jul 3, 2022
7a71381
Merge branch 'main' into urlunsplit_relative
serhiy-storchaka May 14, 2024
d18bbd9
Merge branch 'main' into urlunsplit_relative
serhiy-storchaka May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
sanitize paths that can be confused as scheme
  • Loading branch information
@nascheme
nascheme committed Jun 16, 2022
commit a8e1cc256d4ee7928f45486b625d02adc7d867d0
6 changes: 4 additions & 2 deletions Lib/test/test_urlparse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1103,15 +1103,17 @@ def test_urlsplit_normalization(self):

def test_urlunsplit_relative(self):
cases = [
# expected result is a scheme/netloc relative URL
# expected result is a relative URL without netloc and scheme
(('', 'a', '', '', ''), '//a'),
# extra leading slashes need to be stripped to avoid confusion
# with a relative URL

@vadmium vadmium Apr 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confusion with a protocol-relative URL? [as opposed to a host-relative URL]

(('', '', '//a', '', ''), '/a'),
(('', '', '///a', '', ''), '/a'),
# not relative so extra leading slashes don't need stripping since
# they don't cause confusion
(('http', 'x.y', '//a', '', ''), 'http://x.y//a')
(('http', 'x.y', '//a', '', ''), 'http://x.y//a'),
# avoid confusion with path containing colon
(('', '', 'a:b', '', ''), './a:b'),
]
for parts, result in cases:
self.assertEqual(urllib.parse.urlunsplit(parts), result)
Comment thread
nascheme marked this conversation as resolved.
Outdated
Expand Down
22 changes: 17 additions & 5 deletions Lib/urllib/parse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,10 @@ def urlunparse(components):
url = "%s;%s" % (url, params)
return _coerce_result(urlunsplit((scheme, netloc, url, query, fragment)))

# Returns true if path can confused with a scheme. I.e. a relative path
# without leading dot that includes a colon in the first component.
_is_scheme_like = re.compile(r'[^/.][^/]*:').match

@vadmium vadmium Apr 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the special allowance for a leading dot? Is there a test case for it? Yes, a scheme cannot start with a dot, but a path-noscheme component of .: is no more legal than https: according to RFC 3986.


def urlunsplit(components):
"""Combine the elements of a tuple as returned by urlsplit() into a
complete URL as a string. The data argument can be any five-item iterable.
Expand All @@ -499,13 +503,21 @@ def urlunsplit(components):
empty query; the RFC states that these are equivalent)."""
scheme, netloc, path, query, fragment, _coerce_result = (
_coerce_args(*components))
if not scheme and not netloc and path.startswith('//'):
# gh-87389: avoid confusing a path with multiple leading slashes
# as a URI relative reference.
url = '/' + path.lstrip('/')
if not scheme and not netloc:
# Building a relative URI. Need to be careful that path is not
# confused with scheme or netloc.
if path.startswith('//'):
# gh-87389: don't treat first component of path as netloc
url = '/' + path.lstrip('/')
elif _is_scheme_like(path):
# first component has colon, ensure it will not be parsed as the
# scheme
url = './' + path
else:
url = path
else:
url = path
if netloc or (scheme and scheme in uses_netloc):
if netloc or (scheme and scheme in uses_netloc and url[:2] != '//'):
if url and url[:1] != '/': url = '/' + url
url = '//' + (netloc or '') + url
if scheme:
Expand Down