Visitar URL original
Permission denied on `config.lock` when cloning in Docker under MacOS · Issue #1566 · gitpython-developers/GitPython · GitHub
Skip to content

Permission denied on config.lock when cloning in Docker under MacOS #1566

Closed
#1619
Closed
Permission denied on config.lock when cloning in Docker under MacOS#1566
#1619
Labels
acknowledgedhelp wanted

Description

@mmajchrzycki
mmajchrzycki
opened on Mar 24, 2023

Hi,
I found a bug related to the FileLock system. What I'm doing:

  1. In Ubuntu 20.04 running under Docker, I'm cloning the repository using clone_from.
  2. The directory that I'm cloning to is mounted under MacOS as follows:
    docker run --rm --interactive -v $PWD:/work --user $(id -u):$(id -g) /bin/bash
  3. Docker container is running with UID and GID of the MacOS

I'm getting this error:

Exception in thread my-repository:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 941, in _obtain_lock_or_raise
    fd = os.open(lock_file, flags, 0)
PermissionError: [Errno 13] Permission denied: '/work/repo/my-repository.git/.git/config.lock'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/app/clone-tool.py", line 124, in run
    git.Repo.clone_from(self.git_url, repo_dir)
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 1325, in clone_from
    return cls._clone(
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 1252, in _clone
    with repo.remotes[0].config_writer as writer:
  File "/usr/local/lib/python3.8/dist-packages/git/remote.py", line 1146, in config_writer
    writer = self.repo.config_writer()
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 604, in config_writer
    return GitConfigParser(self._get_config_path(config_level), read_only=False, repo=self)
  File "/usr/local/lib/python3.8/dist-packages/git/config.py", line 366, in __init__
    self._acquire_lock()
  File "/usr/local/lib/python3.8/dist-packages/git/config.py", line 385, in _acquire_lock
    self._lock._obtain_lock()
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 951, in _obtain_lock
    return self._obtain_lock_or_raise()
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 944, in _obtain_lock_or_raise
    raise IOError(str(e)) from e
OSError: [Errno 13] Permission denied: '/work/repo/my-repository.git/.git/config.lock'

The config.lock file as listed under Ubuntu:

$ ls -la
ls: cannot access 'config.lock': No such file or directory
total 144
drwxr-xr-x 14 501 dialout    448 Mar 24 13:42 .
drwxr-xr-x 18 501 dialout    576 Mar 24 13:42 ..
-rw-r--r--  1 501 dialout     23 Mar 24 13:42 HEAD
drwxr-xr-x  2 501 dialout     64 Mar 24 13:42 branches
-rw-r--r--  1 501 dialout    298 Mar 24 13:42 config
-?????????  ? ?   ?            ?            ? config.lock
-rw-r--r--  1 501 dialout     73 Mar 24 13:42 description
drwxr-xr-x 14 501 dialout    448 Mar 24 13:42 hooks
-rw-r--r--  1 501 dialout 122075 Mar 24 13:42 index
drwxr-xr-x  3 501 dialout     96 Mar 24 13:42 info
drwxr-xr-x  4 501 dialout    128 Mar 24 13:42 logs
drwxr-xr-x  4 501 dialout    128 Mar 24 13:42 objects
-rw-r--r--  1 501 dialout    114 Mar 24 13:42 packed-refs
drwxr-xr-x  5 501 dialout    160 Mar 24 13:42 refs

Also listed under MacOS:

$ ls -la
total 288
drwxr-xr-x  14 mmajchrzycki  staff     448 Mar 24 14:42 .
drwxr-xr-x  18 mmajchrzycki  staff     576 Mar 24 14:42 ..
-rw-r--r--   1 mmajchrzycki  staff      23 Mar 24 14:42 HEAD
drwxr-xr-x   2 mmajchrzycki  staff      64 Mar 24 14:42 branches
-rw-r--r--@  1 mmajchrzycki  staff     298 Mar 24 14:42 config
----------   1 mmajchrzycki  staff       0 Mar 24 14:42 config.lock
-rw-r--r--   1 mmajchrzycki  staff      73 Mar 24 14:42 description
drwxr-xr-x  14 mmajchrzycki  staff     448 Mar 24 14:42 hooks
-rw-r--r--   1 mmajchrzycki  staff  122075 Mar 24 14:42 index
drwxr-xr-x   3 mmajchrzycki  staff      96 Mar 24 14:42 info
drwxr-xr-x   4 mmajchrzycki  staff     128 Mar 24 14:42 logs
drwxr-xr-x   4 mmajchrzycki  staff     128 Mar 24 14:42 objects
-rw-r--r--   1 mmajchrzycki  staff     114 Mar 24 14:42 packed-refs

I will provide a minimum set-up (docker file and python script) to reproduce it later.

Activity

  1. Byron
    changed the title [-]Permission denied on `config.lock` when clonning in Docker under MacOS[/-] [+]Permission denied on `config.lock` when cloning in Docker under MacOS[/+] on Mar 25, 2023

  2. Byron commented on Mar 25, 2023

    @Byron
    Byron
    on Mar 25, 2023
    Member

    I am not sure this issue is related to GitPython, as having it here implies GitPython should work in an environment where file permissions are not working as expected.

    Thus I am closing the issue as it's not actionable.

    However, maybe also thanks to the scripts to help reproduce it, one can experiment and see if the situation is common enough to try implement (and contribute) workarounds. Please feel free to keep commenting here and once the context changed the issue can be reopened.

  3. Byron
    closed this as not plannedon Mar 25, 2023

  4. mmajchrzycki commented on Mar 26, 2023

    @mmajchrzycki
    mmajchrzycki
    on Mar 26, 2023
    Author

    I've got a simple example to reproduce the issue on MacOS

    1. Get Dockerfile
    2. Get Python script
    3. Build the container:
    docker build -t clone-test .
    1. Run the container:
    docker run -v $PWD:/work clone-test

    Tested under:
    MacOS 13.2.1 (22D68)
    Docker version 20.10.23, build 7155243

  5. mmajchrzycki commented on Mar 26, 2023

    @mmajchrzycki
    mmajchrzycki
    on Mar 26, 2023
    Author

    I reduced the example to the simplest one, and it seems the issue is os.open method. When built-in Python's open function is used, there is no issue with file permissions.
    According to Python docs the built-in open function is preferred (unless we need to do very low-level tasks, which is not the case here).

  6. Byron
    reopened this on Mar 27, 2023

  7. Byron commented on Mar 27, 2023

    @Byron
    Byron
    on Mar 27, 2023
    Member

    Thanks for your help! From the description above it seems clear that os.open could be replaced with open without side-effects to fix the issue right away.

    Could I interest in contributing this fix? Thanks for your consideration.

  8. Byron
    added
    acknowledged
    help wanted
    on Mar 27, 2023

  9. mmajchrzycki commented on Mar 27, 2023

    @mmajchrzycki
    mmajchrzycki
    on Mar 27, 2023
    Author

    Yup, definitely! I will make a PR soon.

  10. muthurajr commented on Apr 5, 2023

    @muthurajr
    muthurajr
    on Apr 5, 2023

    Definitely we can switch from os.open() to open() as the fix. But underlying root cause is VirtioFS in Docker Desktop, and we can switch to gRPC Fuse as a temporary fix. Just posting it as it may help someone till PR is merged.

  11. HageMaster3108 commented on Jul 27, 2023

    @HageMaster3108
    HageMaster3108
    on Jul 27, 2023
    Contributor

    @muthurajr @mmajchrzycki Just created a PR. Found that removing the "0" in the os.open call would also reach the same goal shortly after having opened the PR though, what do you think? Why is the 0 even there?

    fd = os.open(lock_file, flags, 0)

  12. HageMaster3108
    added a commit that references this issue on Jul 27, 2023

    gitpython-developers#1566 Creating a lock now uses python built-in "o…

    08d82cd

  13. HageMaster3108 commented on Jul 27, 2023

    @HageMaster3108
    HageMaster3108
    on Jul 27, 2023
    Contributor

    Just realised that you are the maintainer @Byron - do you know what the rationale is behind setting mode=0 in os.open()? Shouldn't it be mode=0o700 🤔

  14. Byron commented on Jul 28, 2023

    @Byron
    Byron
    on Jul 28, 2023
    Member

    I think a relevant part of the answer would be my reply in your PR.

    Regarding mode=0, I don't know and it probably doesn't matter as this 'lock-file' implementation is racy and won't actually protect concurrent access, from what I can tell now.

  15. Byron
    closed this as completedin #1619on Sep 1, 2023
  16. Byron
    added a commit that references this issue on Sep 1, 2023

    Merge pull request #1619 from HageMaster3108/bugfix/use-python-builti…

    a1c472b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    acknowledgedhelp wanted

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions