Visitar URL original
Make library thread safe using single eventloop, avoid multithreading · Issue #534 · ably/ably-pubsub-python · GitHub
Skip to content

Make library thread safe using single eventloop, avoid multithreading #534

Description

@sacOO7
  • Create custom eventloop for asyncio couroutines/tasks/futures rather than using common/default asyncio event loop.
  • Since eventloop uses single thread, if the external app is using default asyncio loop for it's own operations, it can block internal eventloop. Need to create separate eventloop that doesn't use the default asyncio eventloop thread.
  • This is also needed since public APIs can be called asynchronously from different threads or custom eventloop. This will disrupt internal workflow where external eventloops will influence order of the code execution. With current implementation, it will work fine if it's called from a single thread or default asyncio eventloop
  • By having a single custom eventloop for whole internal workflow, we make sure all internal operations are thread-safe. So, thread safety-related issues can be avoided ~~> https://github.com/ably/ably~~python/issues/531
  • Since ably-clients do heavy network IO operations. use of a single-threaded event loop model should be preferred https://www.pythontutorial.net/python-concurrency/python-event-loop/.
  • This is similar to the js single-threaded event loop, which is specifically efficient when file IO or network IO operations are performed.
  • This also means the use of multithreading/threadpool (threading package in python) should be avoided at all costs since it won't be as efficient and there will be an extra overhead to manage shared resources using a thread synchronization means. Also. asyncio objects are not thread safe ( typically not an issue ) and a combination of multithreading and asyncio will introduce complexity to use asyncio threadsafe apis ~~> https://docs.python.org/3/library/asyncio~~dev.html#concurrency-and-multithreading
  • The current ably-js client works without the use of multithreading, so most likely we won't need it either. APIs provided as a part of python native asyncio package should be enough to satisfy spec implementation.

┆Issue is synchronized with this Jira Task by Unito

Activity

  1. sync-by-unito commented on Sep 28, 2023

    @sync-by-unito

    ➤ Automation for Jira commented:

    The link to the corresponding Jira issue is https://ably.atlassian.net/browse/SDK-3882

  2. changed the title [-]Create separate eventloop for asyncio[/-] [+]Prefer use of separate eventloop, avoid multithreading [/+] on Sep 28, 2023
  3. changed the title [-]Prefer use of separate eventloop, avoid multithreading [/-] [+]Prefer use of separate global eventloop, avoid multithreading [/+] on Sep 28, 2023
  4. changed the title [-]Prefer use of separate global eventloop, avoid multithreading [/-] [+]Prefer use of custom global eventloop, avoid multithreading [/+] on Sep 28, 2023
  5. changed the title [-]Prefer use of custom global eventloop, avoid multithreading [/-] [+]Prefer use of separate global eventloop, avoid multithreading [/+] on Sep 28, 2023
  6. changed the title [-]Prefer use of separate global eventloop, avoid multithreading [/-] [+]Make library thread safe using single eventloop, avoid multithreading[/+] on Oct 1, 2023
  7. sacOO7 commented on Oct 1, 2023

    @sacOO7
    ContributorAuthor

    Remove loop provided via clientOptions, as per guidance and spec requirement, external loops should be avoided

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions