Repository navigation
MutableMapping cannot be instantiated from Headers #2989
Description
This bit of code from requests passes an instance of werkzeug's Headers to collections.abc.MutableMapping.update(): https://github.com/psf/requests/blob/main/src/requests/adapters.py#L375
Essentially, it does something like this:
>>> from werkzeug.datastructures.headers import Headers
>>> from requests.structures import CaseInsensitiveDict
/usr/lib/python3/dist-packages/requests/__init__.py:109: RequestsDependencyWarning: urllib3 (2.2.3) or chardet (5.2.0)/charset_normalizer (3.0.1) doesn't match a supported version!
warnings.warn(
>>> headers = Headers([('Content-Type', 'text/plain'), ('Content-Length', '1')])
>>> CaseInsensitiveDict(headers)
{'Content-Type': 'text/plain', 'Content-Length': '1'}
but it does not work since Werkzeug 3.1.0:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/requests/structures.py", line 44, in __init__
self.update(data, **kwargs)
File "<frozen _collections_abc>", line 949, in update
File "/home/dev/.local/lib/python3.11/site-packages/werkzeug/datastructures/headers.py", line 88, in __getitem__
return self.__class__(self._list[key])
~~~~~~~~~~^^^^^
TypeError: list indices must be integers or slices, not tuple
This is because CaseInsensitiveDict calls the update method of collections.abc.MutableMapping, which calls iter() on the header object to get what it assumes is an iterator of keys (but for Headers it's an iterator of (key, value) pairs), then calls getitem for each "key".
Environment:
- Python version: 3.11
- Werkzeug version: 3.1.0 or 3.1.1
Activity
[-]Starting with version 3.1.0, collections.abc.MutableMapping cannot be instantiated from `werkzeug.datastructures.headers.Headers`[/-][+]MutableMapping cannot be instantiated from Headers[/+]davidism commentedon Nov 4, 2024
I don't know the right answer to this.
Headersis not technically aMutableMapping, but it is way more annoying to use with type checkers if it doesn't inherit fromMutableMapping. It mostly behaves like a mapping, and that's what most users expect, but sometimes it behaves like its underlying list, such as__iter__. When we were using type stubs, the stubs just lied about its inheritance. When we moved off of type stubs, I made it actually inherit. It looks like the default implementation ofMutableMapping.updatedoesif isinstance(other, Mapping), so this is now doing the wrong thing.davidism commentedon Nov 4, 2024
Headersdoesn't actually satisfy theMutableMappinginterface, so I guess it makes sense to remove it. Technically, passingHeadersto a parameter annotated asMutableMappingis wrong, since like the issue here the function might try to use the value as a mapping in waysHeadersdoesn't satisfy. But at the same time, most places that expect a mapping will still behave correctly, so now your type checker complains about something that looks like a mapping to you, and you either need to ignore it or add an unnecessarydict(headers)conversion. 🤷♂️ I guess I'll remove that and see if anything else comes up.progval commentedon Nov 4, 2024
By the way, I was quite puzzled about that; but it turns out that
dict()actually callskeys()instead of__iter__(). Makes me wonder whyMutableMappingdoesn't do that too.