Visitar URL original
os.link(..., follow_symlinks=True) broken on Linux · Issue #81793 · python/cpython · GitHub
Skip to content

os.link(..., follow_symlinks=True) broken on Linux #81793

Closed
Closed
os.link(..., follow_symlinks=True) broken on Linux#81793
Assignees
serhiy-storchaka
Labels
extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

Description

@jo-he
jo-he
mannequin
opened on Jul 17, 2019
BPO 37612
Nosy @larryhastings, @takluyver, @serhiy-storchaka, @eryksun, @jo-he
PRs
  • bpo-37612: always call linkat() from os.link(), if available #14843
  • gh-81793: always call linkat() from os.link(), if available #24997
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2019-07-17.23:04:55.849>
labels = ['extension-modules', 'type-bug', '3.8', '3.9', '3.10']
title = 'os.link(..., follow_symlinks=True) broken on Linux'
updated_at = <Date 2021-03-23.17:43:01.277>
user = 'https://github.com/jo-he'

    bugs.python.org fields:

    activity = <Date 2021-03-23.17:43:01.277>
actor = 'takluyver'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2019-07-17.23:04:55.849>
creator = 'jo-he'
dependencies = []
files = []
hgrepos = []
issue_num = 37612
keywords = ['patch']
message_count = 4.0
messages = ['348086', '348103', '348110', '348111']
nosy_count = 5.0
nosy_names = ['larry', 'takluyver', 'serhiy.storchaka', 'eryksun', 'jo-he']
pr_nums = ['14843', '24997']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37612'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Linked PRs

    Activity

    1. jo-he commented on Jul 17, 2019

      @jo-he
      jo-hemannequin
      on Jul 17, 2019
      MannequinAuthor

      Regarding link() POSIX states (https://pubs.opengroup.org/onlinepubs/9699919799/functions/link.html):

      "If path1 names a symbolic link, it is implementation-defined whether link() follows the symbolic link, or creates a new link to the symbolic link itself."

      In Linux, link() does _not_ follow symlinks (http://man7.org/linux/man-pages/man2/link.2.html):

      "By default, linkat(), does not dereference oldpath if it is a symbolic link (like link())."

      But Python 3 assumes the opposite to be always the case:

      cpython/Modules/posixmodule.c

      Line 3517 in 8cb65d1

      (!follow_symlinks))

      ...which suits e.g. NetBSD (https://netbsd.gw.com/cgi-bin/man-cgi?link+2):

      "When operating on a symlink, link() resolves the symlink and creates a hard link on the target."

      Therefore, I recommend to always call linkat(), if the platform provides it. That's the modern superset of link() with clearly defined behavior.

      Here are some commands to reproduce the issue on Linux (should hard link 'file' -> 'link', but tries '/tmp/symlink' -> 'link'):

      ~$ : >file
~$ ln -s "$PWD/file" /tmp/symlink
~$ strace -e link,linkat python -c 'import os; os.link("/tmp/symlink", "link", follow_symlinks=True)'
link("/tmp/symlink", "link")            = -1 EXDEV (Cross-device link)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OSError: [Errno 18] Cross-device link: '/tmp/symlink' -> 'link'
+++ exited with 1 +++

      For comparison, calling linkat() without AT_SYMLINK_FOLLOW results in the same error:

      ~$ strace -e link,linkat python -c 'import os; os.link("/tmp/symlink", "link", follow_symlinks=False)'
linkat(AT_FDCWD, "/tmp/symlink", AT_FDCWD, "link", 0) = -1 EXDEV (Cross-device link)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OSError: [Errno 18] Cross-device link: '/tmp/symlink' -> 'link'
+++ exited with 1 +++

      Currently, the only way to call linkat() with AT_SYMLINK_FOLLOW from Python, is to provide a directory file descriptor != AT_FDCWD:

      ~$ strace -e link,linkat python -c 'import os; d=os.open(".", 0); os.link("/tmp/symlink", "link", dst_dir_fd=d, follow_symlinks=True)'
      linkat(AT_FDCWD, "/tmp/symlink", 3, "link", AT_SYMLINK_FOLLOW) = 0
      +++ exited with 0 +++

    2. jo-he
      added
      stdlibStandard Library Python modules in the Lib/ directory
      3.7 (EOL)end of life
      3.8 (EOL)end of life
      3.9 (EOL)end of life
      type-bugAn unexpected behavior, bug, or error
      on Jul 17, 2019

    3. serhiy-storchaka commented on Jul 18, 2019

      @serhiy-storchaka
      serhiy-storchaka
      on Jul 18, 2019
      Member

      Not always linkat() can be used instead of link(). But on Windows (where there is no linkat()) os.link() creates a new link to the symbolic link itself. This is yet one argument for making follow_symlinks=False by default and changing the default behavior of os.link() on NetBSD.

    4. eryksun commented on Jul 18, 2019

      @eryksun
      eryksun
      on Jul 18, 2019
      Contributor

      on Windows (where there is no linkat()) os.link() creates a new link
      to the symbolic link itself.

      Yes, CreateHardLinkW opens the source file by calling NtOpenFile with the option FILE_OPEN_REPARSE_POINT. So the behavior is follow_symlinks=False.

      Note, however, that Windows has distinct file and directory reparse points, so we can't hardlink to a directory symlink, or any other type of directory reparse point such as a junction mountpoint. In Unix, follow_symlinks=False (if implemented) allows creating a hardlink to a symlink that targets a directory.

      Also, I noticed that I can pass follow_symlinks=False in Windows, but this should raise NotImplementedError. It's supposed to be checked via follow_symlinks_specified().

    5. jo-he commented on Jul 18, 2019

      @jo-he
      jo-hemannequin
      on Jul 18, 2019
      MannequinAuthor

      The problem that POSIX does not define the behavior of link() regarding symlinks (and that Unix implementations differ indeed), is independent from Python's os.link() defaults.

      Since it makes no sense to call link(), when linkat() is available, I propose this change:

      --- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3512,15 +3512,11 @@ os_link_impl(PyObject *module, path_t *src, path_t *dst, int src_dir_fd,
 #else
     Py_BEGIN_ALLOW_THREADS
 #ifdef HAVE_LINKAT
-    if ((src_dir_fd != DEFAULT_DIR_FD) ||
-        (dst_dir_fd != DEFAULT_DIR_FD) ||
-        (!follow_symlinks))
-        result = linkat(src_dir_fd, src->narrow,
-            dst_dir_fd, dst->narrow,
-            follow_symlinks ? AT_SYMLINK_FOLLOW : 0);
-    else
+    result = linkat(src_dir_fd, src->narrow, dst_dir_fd, dst->narrow,
+                    follow_symlinks ? AT_SYMLINK_FOLLOW : 0);
+#else
+    result = link(src->narrow, dst->narrow);
 #endif /* HAVE_LINKAT */
-        result = link(src->narrow, dst->narrow);
     Py_END_ALLOW_THREADS
       if (result)

      This fix also simplifies the code, and should be safe for back-porting.

      Whether Python's defaults should be changed regarding Windows and those (mostly obsolete) Unix platforms that do not provide linkat(), is another question.

    6. eryksun
      added
      extension-modulesC modules in the Modules dir
      3.10 (EOL)end of life
      and removed
      stdlibStandard Library Python modules in the Lib/ directory
      3.7 (EOL)end of life
      on Mar 12, 2021
    7. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    8. openstack-mirroring
      added a commit that references this issue on Jan 20, 2024

      Update git submodules

      884318c

    9. 10 remaining items

    10. serhiy-storchaka
      added a commit that references this issue on May 4, 2025

      gh-81793: Always call linkat() from os.link(), if available (GH-132517)

      5a57248
    11. serhiy-storchaka
      closed this as completedon May 4, 2025
    12. serhiy-storchaka
      added a commit that references this issue on May 4, 2025

      pythongh-81793: Skip tests for os.link(follow_links=) on Android

      4136847
    13. serhiy-storchaka
      reopened this on May 4, 2025
    14. serhiy-storchaka
      added 2 commits that reference this issue on May 4, 2025

      pythongh-81793: Skip tests for os.link() to symlink on Android

      4f93e91

      gh-81793: Skip tests for os.link() to symlink on Android (GH-133388)

      95d2a81
    15. serhiy-storchaka
      closed this as completedon May 4, 2025
    16. diegorusso
      added a commit that references this issue on May 4, 2025

      Merge remote-tracking branch 'origin/main' into call-len

      c03babc
    17. zware
      added 2 commits that reference this issue on May 6, 2025

      pythongh-81793: Replace skip decorators with specific helpers

      dd0384f

      gh-81793: Replace skip decorators with specific helpers (GH-133493)

      5b07c6e
    18. Pranjal095
      added 3 commits that reference this issue on Jul 12, 2025

      pythongh-81793: Always call linkat() from os.link(), if available (py…

      cee2c1a

      pythongh-81793: Skip tests for os.link() to symlink on Android (pytho…

      9b800fa

      pythongh-81793: Replace skip decorators with specific helpers (python…

      f8c5dc8
    19. stefanor
      added a commit that references this issue on Oct 11, 2025

      Add support for Python 3.14

      324404a
    20. picnixz
      marked Clarify os.link behavior with follow_symlinks=True on POSIX systems #138623 as a duplicate of this issue on Oct 25, 2025
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions