Repository navigation
zipfile.Path is not Path-like #99818
Description
Activity
As explained in pandas-dev/pandas#49906 (comment), I don't think implementing
__fspath__would help your use case. Basically__fspath__converts the object to a string. There isn't any string that would make sense to be used by any code (for example, the builtinopen()) to operate on a file within a zip file.It might help to hedge the documentation slightly. Maybe something like:
zipfile.Pathimplements many methods ofpathlib.Pathbut does not inherit from it nor doeszipfile.Pathimplementos.PathLike.Reacted by Christian Buhtz, Alex Waygood, Jason R. Coombs and Alexandre SCHOEPPThe question is, from your viewpoint as Python core developers, how could the situation be improved.
As an example
csv.reader()can takepathlib.Pathandzipfile.Pathobjects. Works fine without a workaround.Maybe
zipfile.Pathis not intended to act exactly like apathlib.Pathobject? If this is the case then you should be more clear and direct about that limitations in the documentation.Or do you see a way that 3rd-party-libs like
pandascan improve theire code to also acceptzipfile.Pathobjects without explicit checking for that edge case viaisinstance(obj, zipfile.Path)? I don't know whycsv.reader()works in that case.Reacted by Alexandre SCHOEPPThe doc says it is compatible to
pathlib.Path.There's some subtlety (probably a little too much subtlety) in the docstring you link to there. The docstring says that
zipfile.Pathis "pathlib-compatible" (I.e., has a similar interface topathlib.Path). It doesn't say thatZipFileis "path-like" (and nor do the docs), which is a term that's usually used to mean "an object that conforms to theos.PathLikeinterface by implementing the__fspath__method". Somewhat confusingly, a class calledzipfile.Pathcan be "similar topathlib.Path" without being "path-like".Cc. @barneygale, for interest :)
Reacted by Barney GaleFWIW, I'm working towards making
zipfile.Patha subclass ofPurePath(actually, a subclass of a newAbstractPathclass that sits betweenPurePathandPath). It would allow libraries like pandas to work with zip paths like this (pseudocode):def read_excel(path): if not isinstance(path, pathlib.AbstractPath): path = pathlib.Path(path) with path.open('rb') as f: ...
Discussion here: https://discuss.python.org/t/make-pathlib-extensible/3428.
Reacted by Christian BuhtzThe "pathlib-compatible" bit of the docstring should be changed to "similar to pathlib.Path" IMHO. Saying it's "compatible" is overselling it, particularly as zip paths aren't path-like!
Reacted by Alex Waygood, Christian Buhtz and Alexandre SCHOEPPApologies for triple-posting, I've been thinking about this :)
There isn't any string that would make sense to be used by any code (for example, the builtin
open()) to operate on a file within a zip file.Why not the string representation of the archive member path? The
os.PathLikeinterface definition doesn't say what you can do with the resulting string! Indeed, some functions likeos.path.join()andos.path.dirname()callos.fspath()and then apply purely lexical operations on the result.pathlib.PurePathtoo follows this logic - you can happilyopen(PureWindowsPath('C:/blah'))from a non-Windows system.IMHO this points to an overloading of what "path-like" means, that can be remedied by making it more granular. We could introduce "pure" analogues of
__fspath__(),os.fspath()andos.PathLike- something like__purepath__(),os.purepath(),os.PurePathLike.Reacted by Alexandre SCHOEPPMy point is just that built-in
open()is going to callos.fspath(zip_path). There's nothing thatzip_path.__fspath__could return that will makeopen()work. So maybe__fspath__isn't what's needed to solve this problem.Reacted by Torsten Wörtwein, Christian Buhtz, Barney Gale and Jason R. CoombsIt is probably more accurate to remove the phrase “pathlib-compatible” from the docs or replace it with something like “pathlib.Path-style” or “pathlib.Path-like”.
I really appreciate that you folks invest so much energy in that topic. Please let me throw another question into the pit.
What was the intention about
zipfile.Path. When it is not 100% Path-like than this couldn't be the intention to use a zipfile-path as each other path. But this was how I understood it in the first place.I describe me use-case here; reading data-files (csv, xlsx) from a path object no matter if it is a regular file (
pathlib.Path) or an entry in a zip-archive (zipfile.Path). Are there other use cases you had in mind while designing thatzipfile.Path?When designing
zipfile.Path, the main use case was for use in abstracting the results fromimportlib.metadataandimportlib.resourceswhen the packages were present in a zipfile. That is, provide access to paths and files of various resources of a package or a distribution's metadata as files in directories. In particular, forimportlib.resources, the Traversable protocol was needed. See #88366 for some discussion aroundTraversablebeing PathLike and #96870 exploring why Traversable was created. See also python/importlib_resources#232, where @FFY00 and I are discussing the merits and challenges with honoring pathlike objects inimportlib.resources.as_file.Since
zipfile.Pathis aTraversable, perhaps all you need to do is useas_file?from importlib.resources import as_file def read_excel(traversable): with as_file(traversable) as path: with path.open('rb') as f: ...
Improvements I'd like to see:
- Document the guarantees and limitations of
zipfile.Path, perhaps by introducingTraversable. - Use type system to declare that
zipfile.PathimplementsTraversable. - Possibly highlight
as_fileas a means to readily get a resource on disk.
- Document the guarantees and limitations of
FWIW, I'm working towards making
zipfile.Patha subclass ofPurePath(actually, a subclass of a newAbstractPathclass that sits betweenPurePathandPath). It would allow libraries like pandas to work with zip paths like this (pseudocode):def read_excel(path): if not isinstance(path, pathlib.AbstractPath): path = pathlib.Path(path) with path.open('rb') as f: ...
Wait - if you're using
path.open(), why does it need to be cast to apathlib.Path?zipfile.Pathsupports.openwithout any manipulation. In other words, Pandas should be adapted to accept a traversable, and then it will accept zipfile.Path objects and pathlib.Path objects.Bad example on my part. If you're only interested in the interface
Traversableprovides then sure, you can do:def read_excel(path): if not isinstance(path, Traversable): path = pathlib.Path(path) with path.open('rb') as f: ...
If you're looking for an interface more like pathlib's (so including stuff like
parents,with_suffix(),exists(),write_text(),glob()andwalk()) thenpathlib.AbstractPathmight be a good option if/when it arrives.I opened GH-101589 to fix the documentation.
If the
Traversableinterface is enough, you can just use it, otherwise, as Jason mentioned, you should useas_fileto get apathlib.Pathinstance.And again, please note that path-like objects are a completely different thing 😅
Reacted by Barney Gale- added a commit that references this issue
on Feb 20, 2023 - added 3 commits that reference this issue
on Feb 20, 2023 Stumbled upon this discussion while dealing with a similar issue. So, maybe I can contribute a different angle on this.
file = importlib.resources.files("my.module").joinpath("some-file")is the recommended way of loading package resources.Depending on the way the package is distributed,
filecan be of typepathlib.Pathorzipfile.Path, which have non-compatible interfaces, as discussed in this thread. This inconsistency maybe be a stumbling block for those wishing to useimportlib.
This is about
zipfile.Path. The doc says it is compatible topathlib.Path. But it seems that is not for 100% because it doesn't derive frompathlib.PurePathand can treated as Path-like in all situations.I was redirected from
pandaswhere I opened an issue about the fact thatpandas.read_excel()does accept path-like objects but notzipfile.Path. It was explained to me thatzipfile.Pathdoesn't implement__fspath__and that is the problem.Linked PRs