Repository navigation
subclasses of pathlib.PurePosixPath never call __init__ or __new__ #85281
Description
Activity
conchylicultor commented
on Jun 25, 2020 conchylicultormannequinMannequinAuthorI have a subclass GithubPath of PurePosixPath.
class GithubPath(pathlib.PurePosixPath): def __new__(cls, *args, **kwargs): print('New') return super().__new__(cls, *args, **kwargs) def __init__(self, *args, **kwargs): print('Init') super().__init__()Calling
child.parentcreate a new GithubPath but without ever calling new nor init. So my subclass is never notified it is created.p = GithubPath() # Print "New", "Init" p.parent # Create a new GithubPath but bypass the constructorsThe reason seems to be that parent calls _from_parts which create a new object through
object.__new__(cls):Line 689 in cf18c9e
A hack is to subclass
_initbut it seems hacky as it relies on internal implementation detail.conchylicultor commented
on Jun 25, 2020 conchylicultormannequinMannequinAuthorNote that this likely affect all methods which returns new Path by calling
_from_partsor_from_parsed_parts, like.absolute,.resolve,...websurfer5 commented
on Aug 8, 2020 websurfer5mannequinMannequinThe workaround is to override _init(), which I agree is not desirable.
This is the relevant code in PurePath, which is the super class of PurePosixPath:
@classmethod def _from_parsed_parts(cls, drv, root, parts, init=True): self = object.__new__(cls) self._drv = drv self._root = root self._parts = parts if init: self._init() return self
...
def _init(self): # Overridden in concrete Path pass
To me, the clean way to get the desired behavior seems like it would be to have _init() call self.__init__().
def _init(self): # Overridden in concrete Path self.__init__()
This fixes p.parent, but GithubPath() ends up calling GithubPath.__init__() twice – the first time by PurePath.__new__() calling PurePath._init() and the second time by the GithubPath object creation.
websurfer5 commented
on Aug 8, 2020 websurfer5mannequinMannequinClarification:
PurePath.__new__() calls PurePath._from_parts(), which then calls PurePath._init()
conchylicultor commented
on Aug 8, 2020 conchylicultormannequinMannequinAuthorBefore solving this issue, I think it would be best to think on a more generic solution on how to make Pathlib more extensible. Related to: https://discuss.python.org/t/make-pathlib-extensible/3428
For instance, if childs created with
p.parent(),p / 'subdir'need to forward some state (e.g.RemotePath(path, password=, user=)).Rather than __init__, maybe there should be some __post_init__ like dataclasses.
3 remaining items
Louis-Vincent commented
on Aug 13, 2020 Louis-VincentmannequinMannequinThis is not true, because the classmethod use the library shortcuts the
class mro order, to prevent infinite loop inthe __new__. However, it was
using __init__ before hand, we would
Not have this issueLe jeu. 13 août 2020 à 15:27, Jeffrey Kintscher <report@bugs.python.org> a
écrit :Jeffrey Kintscher <websurfer@surf2c.net> added the comment:
Adding __init__() to PurePath complicates things and doesn't provide any
benefit. A subclass that calls super.__init__() ends up invoking
object.__init__(), which is perfectly fine.I was able to find a solution by calling type(self)() instead of
object.__new__() in most cases. I am working on a PR.----------
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41109\>
Louis-Vincent commented
on Aug 13, 2020 Louis-VincentmannequinMannequinThis is not true, because the classmethod used the library shortcuts the
class mro order, this is to prevent infinite loop inthe __new__. However,
If it was using __init__ before hand, we would
Not have this issueLe jeu. 13 août 2020 à 15:31, Louis-Vincent Boudreault <
lv.boudreault95@gmail.com> a écrit :This is not true, because the classmethod use the library shortcuts the
class mro order, to prevent infinite loop inthe __new__. However, it was
using __init__ before hand, we would
Not have this issueLe jeu. 13 août 2020 à 15:27, Jeffrey Kintscher <report@bugs.python.org>
a écrit :>
> Jeffrey Kintscher <websurfer@surf2c.net> added the comment:
>
> Adding __init__() to PurePath complicates things and doesn't provide any
> benefit. A subclass that calls super.__init__() ends up invoking
> object.__init__(), which is perfectly fine.
>
> I was able to find a solution by calling type(self)() instead of
> object.__new__() in most cases. I am working on a PR.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue41109\>
> _______________________________________
>websurfer5 commented
on Aug 14, 2020 websurfer5mannequinMannequinThe current implementation calls object.__new__(cls), where cls is the child class type, from within a class method (@classmethod). This is fine for Path.__new__() and PurePath.__new__(), which are called by the child class's __new__(), because we don't want them to recursively call the child class's __new__() when the child class is created. This all works as expected when the child class is instantiated outside of Path and PurePath, and the child's __init__() gets called as expected. I don't see any point in making changes to this behavior.
When one of approximately 20 PurePath and Path functions and properties instantiate a new child class object the same way PurePath.__new__() and Path.__new__() do, the child class's __new__() and __init__() functions are not called. This is the problem we are trying to solve.
My fix is to add normal functions (i.e. not decorated with @classmethod) to instantiate child class objects using
obj = type(self)()
This creates a child class instance, and the child's __new__() and __init__() functions get called.
Once I have finished re-plumbing Path and PurePath to use the new functions and created the necessary unit tests (to make sure I didn't break anything), I will also look at adding
a proper __init__() function to the two classes instead of having __new__() initialize the member variables. I didn't mean to imply that __init__() isn't useful. It is required to allow the child class to initialize its own variable. I just meant it isn't required to force calling __init__() and __new__() in the child class.websurfer5 commented
on Aug 19, 2020 websurfer5mannequinMannequinI created a PR that should provide the desired behavior: __init__() and __new__() get called in subclass objects that are created by Path and PurePath. Also, Path and PurePath now have __init__() functions, and the __new__() functions only return new objects and rely upon __init__() to perform the object initialization.
I believe #100481 would address this.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: