Repository navigation
GitPython repo.index.commit() spawns persistent git.exe instance, holds handles to repo #553
Description
Activity
That's a quagmire of bugs!
-
Each git-repo instance indeed holds persistent git commands, that are supposed to be cleared by
repo.clear_cache(). Otherwise, on Windows only, these commands prevent the repo-dir from being deleted.
So you have to invokerepo.clear_cache().
But that alone won't work unless you have garbage-collect first! -
Additionally, the temp-file still may not be deleted because it contains read-only files (the blobs in
.git/objects), so you need extra code for that. This code exists ingit.util.rmtree()but currentlygit.utilmodules is being masked bygit.index.utilmodule, due to a bug, so you cannot use it - you have to copy it :-( -
Finally, your code also had an error, you have to invoke
clear_cache()on therepoinstance, not directly on Git class.
So now the code becomes:
import unittest import gc import git import tempfile import os.path import shutil import stat def rmtree(path): """Remove the given recursively. :note: we use shutil rmtree but adjust its behaviour to see whether files that couldn't be deleted are read-only. Windows will not remove them in that case""" def onerror(func, path, exc_info): # Is the error an access error ? os.chmod(path, stat.S_IWUSR) print('dfsffdss') try: func(path) # Will scream if still not possible to delete. except Exception as ex: raise return shutil.rmtree(path, False, onerror) class Test(unittest.TestCase): def testCreateRepo(self): with tempfile.TemporaryDirectory(prefix=(__loader__.name) + "_") as mydir: # MAKE NEW REPO repo = git.Repo.init(path=os.path.join(mydir, "newRepo"), mkdir=True) try: self.assertTrue(os.path.isdir(os.path.join(repo.working_dir, ".git")), "Failed to make new repo?") # MAKE FILE, COMMIT REPO testFileName = "testFile.txt" open(os.path.join(repo.working_dir, testFileName) , "w").close() repo.index.add([testFileName]) self.assertTrue(repo.is_dirty()) repo.index.commit("added initial test file") self.assertFalse(repo.is_dirty()) print("done") finally: gc.collect() repo.git.clear_cache() rmtree(repo.git_dir) if __name__ == '__main__': unittest.main()Please report if everything has been solved.
Reacted by Pooja Bhat and allinone-1-
We would have fixed it yesterday, if we could :-)
Just search for
tag.leaksto get an idea of the invested effort to reach to the point where git-python runs decently in Windows on PY34+.The 1st and easiest fix is to retrofit
git.Repoas a context-manager, so instead oftry..finallyyou would use it aswith Repo() as repo: ....Assuming #555 gets merged in the next release, in your case the code would become simply like that:
class Test(unittest.TestCase): def testCreateRepo(self): with tempfile.TemporaryDirectory(prefix=(__loader__.name) + "_") as mydir: with git.Repo.init(path=os.path.join(mydir, "newRepo"), mkdir=True) as repo: self.assertTrue(os.path.isdir(os.path.join(repo.working_dir, ".git")), "Failed to make new repo?") # MAKE FILE, COMMIT REPO testFileName = "testFile.txt" open(os.path.join(repo.working_dir, testFileName) , "w").close() repo.index.add([testFileName]) self.assertTrue(repo.is_dirty()) repo.index.commit("added initial test file") self.assertFalse(repo.is_dirty()) print("done")
@mboard182 It looks like the context-manager addition is in-flight and might make it in the release planned for today.
- added a commit that references this issue
on Mar 8, 2017 - added a commit that references this issue
on Oct 9, 2017 - added a commit that references this issue
on Dec 7, 2023
I am trying to use GitPython for some repo manipulation, but ran into issues with my app, with handles open where i wouldn't expect.
Bug-jarring the issue, it seems that calling
repo.index.commit()results in a several (4, consistently) git.exe processes being spawned, each holding a handle to the repo's root directorynewRepo. When the test below goes to delete this TempDir, the processes are still up, causing a failure on context-manager__exit__(). My app occasionally needs to do a similar cleanup, so hits the same issue.On the one hand, it looks like there is no contect-manager capable repo-wrapper, meaning it makes sense if some resources is open, it will not typically be cleaned/GC'd before the tempdir
__exit__(). On the other hand - ifRepois going to behave like that, it really should not persist resources that have such side-effects.Here is a working unittest:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\%USER%\AppData\Local\Temp\EXAMPLE_gitpython_v3kbrly_\newRepo'
digging a little deeper, it seems that gitPython spawns multiple instances of git.exe processes, and each of them holds a handle to the root folder of the repo
newRepo.newRepo... git.exe (4 separate PID's of git.exe to be precise)-- I'm typically running this from PyDev, but i verified the issue reproduces under vanila command line invocation of python.exe as well
newRepobeing held. Adding a little extra code to the above I think that is the only handle held. I am able to successfully os.remove/os.rmdir() every dir and file, including all of .git/; and i finally manually recreate the issue seen on exit() in my example when ios.rmdir(newRepo)stepping through, it's the call to repo.index.commit() that actually leads to the the git.exe(s) being spawned.