Visitar URL original
PEP 563, PEP 649 and pydantic · Issue #2678 · pydantic/pydantic · GitHub
Skip to content

PEP 563, PEP 649 and pydantic #2678

Closed
Closed

Description

@samuelcolvin

Update 2: see #2678 (comment), for a summary of how PEP 563 could effect pydantic.


Update: see below this has been resolved by a changes in python 3.10 from the python steering council.

Thanks everyone for your thoughts, patience and hard work.


PEP 563 "Postponed Evaluation of Annotations" was introduced in python 3.7 behind the from __future__ import annotations switch, it basically meant all annotations are strings, not python objects.

Pydantic has tried to supported Postponed Annotations since v0.18.0, see #348.

The problem however is that trying to evaluate those strings to get the real annotation objects is really hard, perhaps impossible to always do correctly, see #248 #234 #397 #415 #486 #531 #545 #635 #655 #704 #1298 #1332 #1345 #1370 #1668 #1736 #1873 #1895 #2112 #2314 #2323 #2411

The reasons are complicated but basically typing.get_type_hints() doesn't work all the time and neither do the numerous hacks we've introduced to try and get fix it. Even if typing.get_type_hints() was faultless, it would still be massively slower than the current semantics or PEP 649 (see below).

In short - pydantic doesn't work very well with postponed annotations, perhaps it never will.

The Problem

The problem is that postponed annotations are set to be come default in 3.10, features of which will be frozen in about three week.

Even worse, there's no way to switch back to the current behaviour.

The Solution

The solution to this is PEP 649 developed by Larry Hastings. This basically means annotations evaluation is lazy - the work of building __annotations__ isn't done until you access __annotations__.

As far as I can tell this is the best of both worlds.

The Second Problem

The sad reality however is that it seems very possible that PEP 649 will get rejected and when python 3.10 is released it will break much of pydantic, and thereby FastAPI and all the other libraries that rely on pydantic (as well as other libraries that use annotations at runtime like enforce and typer presumably).

See this very long discussion of the issue on python-dev about whether PEP 649 is a good idea.


The Point

So why am I saying all this apart from whinging about something I don't agree with?

1. This is fair warning

That pydantic might break in a big way if python's core developers continue value principle over pragmatism.

2. You can help

Thousands of developers and organisations big and small rely on pydantic. Type hints might have been conceived to help readers and static type checkers, but they're now used to great effect at runtime by lots and lots of people - they make python better.

In the end I don't think python's core developers and steering council want to make your experience of python worse. They just haven't realised how important this is. (Even Larry hadn't heard of pydantic or FastAPI until yesterday when he emailed me, I guess they don't read the python developer survey 😉)

If you value pydantic or FastAPI or other libraries that use annotations at runtime, please (constructively and respectfully) let the python steering council know that you would like PEP 649 to be accepted. Please don't contact members of the python community, they're aware of this issue (see below) and are taking it seriously.

I understand the decision on PEP 649 will be made over the next few days, so if you're going to do anything do it today.

Activity

  1. pinned this issue on Apr 15, 2021
  2. mathematicalmichael commented on Apr 15, 2021

    @mathematicalmichael

    Could you please provide some suggestions for the best way to accomplish this ask?:

    If you value pydantic or FastAPI or other libraries that use annotations at runtime, please (constructively and respectfully) let the python steering council know that you would like PEP 649 to be accepted.

    Is there an email I can write to? Some forum to post on? What's the most impactful course of action?

  3. samuelcolvin commented on Apr 15, 2021

    @samuelcolvin
    MemberAuthor

    Not really, it's precisely because I don't know how to make my concerns clear that I created this issue.

    I've posted to python-dev supporting the PEP, but I suspect that more people chiming in to just support what I've said might do more harm than good.

    I've you've spoken to core devs in the past and they know you, perhaps prompting them might be the best thing to do.

  4. notatallshaw commented on Apr 15, 2021

    @notatallshaw

    As a long time reader of the Python Dev mailing list I would say that it is not necessary to contribute to the mailing list beyond what @samuelcolvin has already posted, unless there are specific technical questions that you are highly qualified to answer.

    The mail is already extremely forthright and I have no doubt it will generate a lot of discussion. One thing I would say though @samuelcolvin is it will probably help your case a lot to go in to more details about why typing.get_type_hints() doesn't work for pydantic, either because of specific issues, real world performance examples, or if there is some fundamental flaw in the approach itself.

    There's a lot of smart people in the Python core Dev team, so it makes sense to state the problem not the solution. It may be that typing.get_type_hints() can be fixed or some other method could be implemented that would breach the gap.

    My 2 cents anyway, hope it helps.

  5. samuelcolvin commented on Apr 15, 2021

    @samuelcolvin
    MemberAuthor

    I take your point, but the problems with typing.get_type_hints() and PEP 563 are well described in PEP 649 and debated at length in the other thread, I didn't want to bring all that up again.

  6. notatallshaw commented on Apr 15, 2021

    @notatallshaw

    I'm going to be honest, I skimmed over many dozens of mails for this one.

    But to me it seemed the discussions were talking about problems in the abstract, without a real motivation or need for them to be fixed. Whereas specifics on actual problems implementing it, with an actual library, that's used by lots of users, is a more much compelling position.

    I would guess some core devs are going to ask questions about this though anyway, so responses to those questions will likely fill the gap.

    Best of luck! Don't use this tool on a day to day basis, but it's been great when I have 😄

  7. gaborbernat commented on Apr 15, 2021

    @gaborbernat

    I think you should explain better why it's impossible to implement lazy evaluation of type hints. The long list of issues doesn't help get a better understanding of the problem in achieving this, and in many it's never explained why it's hard to do.

  8. methane commented on Apr 15, 2021

    @methane

    I take your point, but the problems with typing.get_type_hints() and PEP 563 are well described in PEP 649 and debated at length in the other thread, I didn't want to bring all that up again.

    Even there is a long thread, I don't know what point is actually breaking pydantic.

  9. brettcannon commented on Apr 15, 2021

    @brettcannon

    Do note that as a steering council member it is more important to me to hear from you directly, @samuelcolvin , in PEP 649 itself on how its acceptance/rejection will impact Pydantic as a maintainer of a popular package using runtime annotations. But inundating us with a flood of emails or trying to get a high upvote count is actually a negative as it implicitly tries to put the SC in a position of "us versus them" for this decision (on top of having to sift through more emails with what limited time I have for SC topics).

    I am personally already well aware of the popularity of Pydantic, FastAPI, and Typer, so there's no need to prove that to the SC.

  10. pablogsal commented on Apr 16, 2021

    @pablogsal

    My recommendation here is to gather as much detailed, objective and technical information on how PEP 563 is impacting pydantic in a way it cannot adapt without compromising the core of the package. As a member of the steering council, I normally spend many hours parsing the mailing lists, https://discuss.python.org/ and other sources to gather a complete and holistic view of all the pros, cons and different views and how different parties should be impacted in order to make the best decision possible. The fact that this is bringing to our attention very close to beta freeze gives us very limited time to react in a way that keeps out standards, so it would be very very helpful to get this information directly from you without needing to be extracted from tons of messages.

    Also, as a release manager of Python 3.10, it makes me sad that the first issues that are mentioned here:

    #248 #234 #397 #415 #486 #531 #545 #635 #655 #704 #1298 #1332 #1345 #1370 #1668 #1736 #1873 #1895 #2112 #2314 #2323 #2411

    go back to 2018 but we are hearing about all these problems and how they impact pydantic critically dangerously close to beta freeze. My priority as a release manager is to veil for the highest stability for the release. A considerable amount of features are merged days before feature freeze and this already has a considerable amount of chaos and work from the release management team and the buildbot team and the potential of reverting or including such big changes on top of all this with too little time close to feature freeze makes my job much harder as you can imagine.

    In any case, for us, making sure that all our user base is taken into account is a very serious matter so you can be sure that we will take this into account when discussing the overall issue.

    I do understand that everyone is very nervous and there is a lot of people worried and all of this is augmented with time limitations, but we are all volunteers that want the best for our users and the community so let's all be empathetic to each other, and try to work together towards the best solution.

    Thanks for your patience and for your understanding.

    P.S.

    I guess they don't read the python developer survey 😉

    We certainly do (at least from the Steering Council 😄 ).

  11. willingc commented on Apr 16, 2021

    @willingc

    Hi folks.

    Let me state up front that I'm a very satisfied user of pydantic and FastAPI, and I'm very thankful for the work and contributions the maintainers and community around them have put in. ☀️

    I also appreciate the volunteer efforts of the Python core team, Steering Council, and many contributors to Python.

    I'm optimistic that we can find a win-win for pydantic / FastAPI and Python. I believe this is possible if we try not to polarize the solution prematurely to "all or nothing" or "accept or reject 649". To accomplish that we need to look at this through the lens of "what is possible", balance the tradeoffs, and work toward a "good but perhaps not ideal" solution.

    Thank you and I'm looking forward to seeing us move forward together.

  12. tirkarthi commented on Apr 16, 2021

    @tirkarthi

    There is an API proposal over adding typing.get_annotations(object) that could help and seems to have similar motivation : https://bugs.python.org/issue43817

    Not sure if it helps but an issue were KeyError was raised for typing.get_type_hints was resolved : https://bugs.python.org/issue41515

  13. FelixTheC commented on Apr 16, 2021

    @FelixTheC

    I will have the same issue in the future.
    I wrote my own implementation which works in that way that all my tests are passing.

    https://github.com/FelixTheC/strongtyping/blob/%2347_in_py10_globals_from_func_must_be_known/strongtyping/utils.py

    Feel free to adapt and/or change it for your needs.

  14. samuelcolvin commented on Apr 16, 2021

    @samuelcolvin
    MemberAuthor

    First of all, I really appreciate what the python core developers in general, and the steering council in particular do for us all.

    I only heard about the debate about PEP 649 on Wednesday night, and given the extremely narrow window to get my point heard, I banged the drum in every way I could think of. The lesson here is "careful what you wish for", my message has been amplified more than I expected.

    I'm sorry if that has wasted anyone's time, even more so if it reduces the chance my request gets a positive reception.

    You're also right that i should have engaged with python-dev long ago about this, I'm sorry I did not. Lesson (hopefully) learnt.

    I never meant to generate an "us vs. them" atmosphere, and re-reading what I wrote, I don't think that's what I did. I think the most contentious think i said was that python's core might value "principle over pragmatism" - that would be a perfectly justifiable thing for you do in some scenarios, even though I disagree in this case.

    A few points in my defence:

    • I had already (before Larry emailed me about PEP 649) requested to talk about this at the python developer summit in May, it was on my radar and I didn't realise how little time we had
    • The situation has changed in the last week as Guido himself points out "... the hypothetical future where annotations are not always syntactically expressions (which did not even exist before this week)..." - this change really would be a problem for pydantic and other such libraries
    • I specifically avoided telling people to email a particular address or contact particular people to avoid individuals being spammed, I also specifically suggested people didn't reply to the python-dev thread with "me too"s.

    overall: Thank you for taking this seriously and I'm sorry to have caused a fuss.


    I'm now going to spend some time doing my best to produce a coherent explanation of the short-comings of PEP 563 for pydantic to post both here and on python-dev (I assume just posting a link to here on the mailing list will annoy people?)

  15. 20 remaining items

  16. notatallshaw commented on Jan 6, 2022

    @notatallshaw

    Hello, I wanted to ask about the current state of the issue. Will python 3.11 ship with additional functions that will enable Pydantic to function correctly?

    The steering council has decided to delay the decision on PEP 563 and PEP 649: https://mail.python.org/archives/list/python-dev@python.org/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/

    Therefore Pydantic will continue to work as normal if you are not using from __future__ import annotations.

    Since this thread was originally opened a function has been added "inspect.get_annotations" which is an official way to get annotations for an object instead of type hints. Not sure it's powerful enough to help pydantic but it at least exists for others.

  17. ergo commented on Jan 6, 2022

    @ergo

    Yup, I know about the delay of PEP-s. I was more thinking if there are plans for permanent long term solution.

  18. notatallshaw commented on Jan 6, 2022

    @notatallshaw

    Yup, I know about the delay of PEP-s. I was more thinking if there are plans for permanent long term solution.

    I can't speak for authors of pydantic but in other projects I work on that could be affected by this we can not plan a long term solution until a decision on the PEPs and further the future of from __future__ import annotations is decided by the Python Steering Council. As it is unclear what the long term options available will be until then.

  19. gvanrossum commented on Jan 6, 2022

    @gvanrossum

    It would be helpful if there was a brief summary of the problem that PEP 563 causes for pydantic (and perhaps other projects). I looked at the top of this thread but it is very long and emotional and did not give me a clear understanding of the problem. I clicked on one of the referenced issues (#248) but IIUC it actually describes a problem in pydantic that could be solved with the help of PEP 563 (or with PEP 649).

    Note that I am not a user of pydantic; from the README it looks like a nice tool but the example there doesn't look like it would be affected adversely by PEP 563 (assuming pydantic calls get_type_hints() when it sees a string annotation).

    Am I to understand that the real problem (i.e. something where working code would break due to PEP 563) is that it is common to define pydantic classes in a function's local scope that are then used in another scope, in a way that causes get_type_hints() to fail?

  20. samuelcolvin commented on Jan 6, 2022

    @samuelcolvin
    MemberAuthor

    I'll answer in more detail tomorrow, but yes, in summary the really hard problem is scope. It's not about whether the class is used right there or elsewhere, if the class is not defined at module level, things start breaking.

    Larry went through this (and the fact that it was just a side note in the PEP) in great detail, in the many discussions that happened about this at the time. Maybe he would be worth asking?

  21. samuelcolvin commented on Jan 6, 2022

    @samuelcolvin
    MemberAuthor

    See #2678 (comment) above.

  22. methane commented on Jan 8, 2022

    @methane

    I am light user of FastAPI and pydantic, but I never used non-module level types with it. You wrote:

    You might say "well imports like this should always be module level anyway", perhaps true, but what about custom data types which are extensively used with pydantic? There are many good reasons why you might want to use non-module level custom types, currently with python 3.10 that would require complex .update_forward_refs(...) calls.

    I read the custom data types. But there are no examples using non-module level types for good reason.
    Would you explain some of "many good reason" for using non-module level custom types?


    I am not interested in PEP 649 vs PEP 563 (enable by default), but PEP 649 vs current behavior (opt-in PEP 563). PEP 649 has still some behavior different from current behavior:

    types = [int, str, float]
    
    def make_constructors():
        cs = []
        for t in types:
            def c(t=t) -> t:
                return t()
            cs.append(c)
        return cs
    
    cs = make_constructors()
    for c in cs:
        print(c.__annotations__["return"])
    
    # Current: int, str, float
    # PEP 649: float, float, float

    Is there no good reason for this?

  23. samuelcolvin commented on Jan 8, 2022

    @samuelcolvin
    MemberAuthor

    A fuller answer to @gvanrossum's question:

    There were 3 things that worried me back when I created this issue in April:

    1. Performance

    Time taken to construct pydantic models - effectively startup time.

    typing.get_type_hints(Foo) is much slower than Foo.__annotations__, however that's only a small part of what pydantic does and from some unscientific experiments, it seems that creating models is only around 7-10% slower with from __future__ import annotations than without.

    On the other hand, from __future__ import annotations/PEP-563/string-annotations provide a performance improvement in general, so there's two counteracting effects.

    I've therefore also tested this in practice by running

    [p.write_text(f'from __future__ import annotations\n{p.read_text()}') for p in Path('src').glob('**/*.py')]

    on a ~10k fastapi/pydantic web application (a few files caused errors so I had to remove the __future__ declaration), overall this caused starting the app to slow down by 6% - roughly matching what I would expect from experiments on a single model.

    Overall, while this isn't ideal, I can live with the performance impact. (If anyone else has seen a wildly different performance impact, feel free to bring that up)

    2. Potential change of syntax

    There were discussions back in April about a "hypothetical future where annotations are not always syntactically expressions".

    Clearly if this became commonplace, it could be a problem for pydantic. However this seems unlikely, and even if it were the case, typing.get_type_hints() or some other function (inside or outside the stdlib) would need to be able to parse those expressions.

    And anyway, blocking progress because it would open the door to something in future that could be a problem would be a mistake.

    Overall, I think we can ignore this.

    3. Scope

    This is the only real concern I have with from __future__ import annotations/PEP-563.

    Please note: this is not an issue with the concept of annotations being left as strings. It's a problem with the specifics of how PEP-563 defined annotation evaluations, from PEP 563:

    Annotations can only use names present in the module scope as postponed evaluation using local names is not reliable (with the sole exception of class-level names resolved by typing.get_type_hints())

    Because of that, the following breaks:

    from __future__ import annotations
    from pydantic import BaseModel
    
    def main():
        from pydantic import PositiveInt
    
        class TestModel(BaseModel):
            foo: PositiveInt
    
        print(TestModel(foo=1))
    
    main()

    Of course, I've used from pydantic import PositiveInt here, but it could be any import or a custom type defined within the function, including another pydantic model. It could even be simple type alias like Method = Literal['GET', 'POST']

    Pydantic might be able to do some clever stuff to recognise locally defined annotations, but that would:

    1. Require some magic that might be hard to maintain and unreliable
    2. Would mean pydantic was being intentionally un-pythonic by not following PEP 563

    Would you explain some of "many good reason" for using non-module level custom types?

    Well, all the same reasons people do local imports in any other code (import performance, circular references, name clashes) and the same reasons people define classes or variables locally (performance, name clashes, readability, accessing local variables).

    [...] pydantic classes in a function's local scope that are then used in another scope

    It's not about where the class is used, in the above example PEP 563 explicitly states that PositiveInt or anything else defined within the function cannot be used a a valid annotation. The code will fail when the class is defined, not when it's used.

    Overall, while there could be workarounds, pydantic and other code that relies on runtime use of annotations would be uglier with PEP 563 style annotations scope than without


    I hope that helps. I'll add a note to the top of this issue to help any future travellers who pass this way.

    Overall, even if PEP 563 is accept as-is, I don't think pydantic is doomed (just a little damaged) - I was mistaken to suggest that originally. Sorry.

  24. gvanrossum commented on Jan 8, 2022

    @gvanrossum

    Thanks for the clear explanation.

    Let's focus on (3). (The example is missing from devtools import debug BTW, and in addition I get an error urging me to call TestModel.update_forward_refs() with the future import enabled. Adding that will indeed give the error about PositiveInt.)

    I have a feeling you are reading a bit too much into the sentence you quote from the PEP. What the PEP meant was just that you have to pass the locals containing needed definitions into tying.get_type_hints() or inspect.get_annotations(), which is a bit of a pain when it is being called deep inside a library. Hence the PEP suggests to keep definitions global. The PEP doesn't imply that using local definitions is invalid or unpythonic!

    Of the reasons for doing local imports:

    • performance: totally legit, if it's in a function/method that might never be called and a module that would otherwise not be imported (but I'm guessing importing anything from pydantic does not qualify)
    • circular references: I wouldn't solve those with local imports (PEP 563 is meant to help here actually)
    • name clashes: that sounds silly; why not use import ... as ...?

    In general, defining classes locally is not a great pattern, except in certain types of unit tests.

    With all that said, I agree that the situation is not great for pydantic. But there's no reason for despair either. :-)

    Now, I can't look in the future to know what the SC will decide, but it looks like for now they are kicking the can down the road and 3.11 will end up being the same as 3.10 in this respect. So from __future__ import annotations is here to stay for another few releases at least, and people will want to use it for other reasons (as you say, it makes certain things faster).

    I see that update_forward_refs() already takes a local namespace, e.g. TestModel.update_forward_refs(**locals()). Why not just recommend that people use that? You could even have a helper that uses sys._getframe(1) to access the parent frame and takes the locals from there.

  25. willingc commented on Jan 20, 2022

    @willingc

    @samuelcolvin @gvanrossum (including @tiangolo as well) I'm happy to see this discussion. Our team uses pydantic and fastapi in our product. Our engineers really enjoy using these libraries (they really rave about the productivity benefits), and there is a lot of positive impact that the libraries are having on Python adoption. If we do a Typing Summit at PyCon again, I encourage Samuel and Sebastian to join the discussion.

    Samuel, thanks for the detailed response to Guido's questions and for being open to collaborating and investigating going forward. Let's keep the positive discussion going. Thanks ☀️

  26. henryiii commented on Jan 20, 2022

    @henryiii

    The error message is here:

    https://github.com/samuelcolvin/pydantic/blob/9d631a3429a66f30742c1a52c94ac18ec6ba848d/pydantic/fields.py#L832

    It could have **locals() added to it.

    And here's the update_forward_refs: https://github.com/samuelcolvin/pydantic/blob/87da9ac23f0917741c0536b474cf13cdbed50dfc/pydantic/main.py#L776 - this could have the sys._getframe(1) trick to try to pull the locals (I would assume this would be dependent on if sys contains _getframe, as it's not guaranteed to exist, from what I gather - so both changes together likely would be helpful)

    The fact you can detect this and produce a nice suggestive error message is great to see, though! As far as I can tell, local class definitions is still an antipattern for Pydantic, but is just something that comes up occasionally due to the number of users? A minor, detectable inconvenience is not a bad tradeoff, perhaps?

    Working example code, FWIW, since a few bits were missing:

    # Install pydantic and devtools
    from __future__ import annotations
    from pydantic import BaseModel
    from devtools import debug
    
    
    def main():
        from pydantic import PositiveInt
    
        class TestModel(BaseModel):
            foo: PositiveInt
    
        TestModel.update_forward_refs(**locals()) # can remove this line with the patch below
    
        debug(TestModel(foo=1))
    
    
    main()

    PS: really dumb question - why can't update_forward_refs() be called in __new__ with this already? Specially, __try_update_forward_refs__ could include the locals. In fact, the following patch completely fixes your example for me - without the update_forward_refs call at all:

    diff --git a/pydantic/main.py b/pydantic/main.py
    index eea8abc..bc8afdf 100644
    --- a/pydantic/main.py
    +++ b/pydantic/main.py
    @@ -1,4 +1,5 @@
     import warnings
    +import sys
     from abc import ABCMeta
     from copy import deepcopy
     from enum import Enum
    @@ -289,7 +290,9 @@ class ModelMetaclass(ABCMeta):
             cls = super().__new__(mcs, name, bases, new_namespace, **kwargs)
             # set __signature__ attr only for model class, but not for its instances
             cls.__signature__ = ClassAttribute('__signature__', generate_model_signature(cls.__init__, fields, config))
    -        cls.__try_update_forward_refs__()
    +
    +        localns = sys._getframe(1).f_locals if hasattr(sys, "_getframe") else {}
    +        cls.__try_update_forward_refs__(**localns)
    
             return cls
    
    @@ -765,12 +768,12 @@ class BaseModel(Representation, metaclass=ModelMetaclass):
                 return v
    
         @classmethod
    -    def __try_update_forward_refs__(cls) -> None:
    +    def __try_update_forward_refs__(cls, **localns: Any) -> None:
             """
             Same as update_forward_refs but will not raise exception
             when forward references are not defined.
             """
    -        update_model_forward_refs(cls, cls.__fields__.values(), cls.__config__.json_encoders, {}, (NameError,))
    +        update_model_forward_refs(cls, cls.__fields__.values(), cls.__config__.json_encoders, localns, (NameError,))
    
         @classmethod
         def update_forward_refs(cls, **localns: Any) -> None:

    PS: I think __init_subclass__ might really simplify your two-stage metaclass code.

  27. samuelcolvin commented on Jan 20, 2022

    @samuelcolvin
    MemberAuthor

    Thanks for everyone's input.

    @gvanrossum, good point about assuming debug() is already shoehorned into builtins, I've switched to print().

    I noticed that there's a strong argument that annotations as strings doesn't kill pydantic:

    It's hidden away in my comments about performance - I modified a medium sized application which makes extensive use of pydantic to insert from __future__ import annotations everywhere and .... it mostly just worked! (albeit, a bit slower to launch, and some issues with annotations-as-strings in FastAPI).

    But still I think the FastAPI stuff could be fixed fairly easily. Overall, I think this is a fairly good indicator that in real life most pydantic usage won't be too adversely affected.

    On circular imports, I need to think about this more but I'm not entirely convinced it won't be a problem - to avoid circular import problems while making my import global, I need to put the import behind if TYPE_CHECKING, if I do that, at runtime the import won't be available. Am I missing something?


    @willingc I'll do my very best to get to pycon. If I can make it, I'll of course join a Typing Summit.

  28. changed the title [-]IMPORTANT: PEP 563, PEP 649 and the future of pydantic[/-] [+]PEP 563, PEP 649 and pydantic[/+] on Jan 20, 2022
  29. gvanrossum commented on Jan 20, 2022

    @gvanrossum

    On circular imports, I need to think about this more but I'm not entirely convinced it won't be a problem - to avoid circular import problems while making my import global, I need to put the import behind if TYPE_CHECKING, if I do that, at runtime the import won't be available. Am I missing something?

    It depends on how the modules are structured. The problem doesn't occur when each module has a function that uses something from the other module. But it may occur when two classes in different modules reference each other. The solution normally (when using annotations for static typing) is to put the annotations in "forward reference" string quotes -- or use PEP 563. However if each class does runtime introspection at (roughly) the point of definition then you will indeed be in trouble.

  30. caniko commented on Aug 31, 2022

    @caniko
    Contributor

    However if each class does runtime introspection at (roughly) the point of definition then you will indeed be in trouble.

    tortoise-orm uses TYPE_CHECKING when working with tortoise.Models that reside in different modules. This is not an issue unless we try to generate a pydantic model from either of the models. tortoise has a function for this, pydantic_model_creator.

    Example, will raise exception:

    if TYPE_CHECKING:
        from module.models import FromAnotherModule
    
    class MyModel(tortoise.Model):
        other_model: fields.ForeignKeyRelation[
            "FromAnotherModule"
        ] = fields.ForeignKeyField("module.models")
    
    
    # Ayyy, there goes PEP8
    Pydantic_MyModel = pydantic_model_creator(MyModel)

    The pydantic models are really useful when used with FastAPI, hence the reason I stumbled over this issue.

    I am guessing the best is to implement a workaround for now, which is fine; that is what abstract models are for 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    metahigh level, conceptual

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions