Visitar URL original
Possible pluggable C extension for performance by kesmit13 · Pull Request #1066 · PyMySQL/PyMySQL · GitHub
Skip to content

Possible pluggable C extension for performance - #1066

Draft
kesmit13 wants to merge 15 commits into
PyMySQL:mainfrom
singlestore-labs:main
Draft

kesmit13 wants to merge 15 commits into
PyMySQL:mainfrom
singlestore-labs:main

Conversation

@kesmit13

@kesmit13 kesmit13 commented Sep 1, 2022

Copy link
Copy Markdown

I'm creating this PR more as a start to a conversation as opposed to a request that I expect to be accepted any time soon. We like being able to use PyMySQL because of the ease of installation and the fact that it can be used without any system-level libraries being installed. However, the performance leaves a lot to be desired when dealing with larger result sets.

After some investigation, it looked like most of the time was being spent in the row data fetching and data conversion, which all stems from one method in the connection (`read_rowdata_packet'). An idea came to mind to replace that method (and anything that it calls) with a C extension to see how much performance could be increased. It ended up working much better than expected and the performance improvements actually made the client (arguably) the fastest MySQL Python client available (arguably because the Mariadb client is pretty fast too). It is definitely the fastest Python client that doesn't require any external libraries to run.

I thought I'd bring the work up to your team to see if you might want to collaborate on it as a possible plugin for PyMySQL, or to see if it should remain a separate project.

@methane
methane marked this pull request as draft September 9, 2022 06:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant