RawArray causes FileNotFoundError at cleanup

[MathieuLamarre]

BPO	35248

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-11-14.17:50:30.017>
labels = ['type-bug', 'library']
title = 'RawArray causes FileNotFoundError at cleanup'
updated_at = <Date 2018-11-14.18:17:03.678>
user = 'https://bugs.python.org/MathieuLamarre'

bugs.python.org fields:

activity = <Date 2018-11-14.18:17:03.678>
actor = 'Mathieu Lamarre'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-11-14.17:50:30.017>
creator = 'Mathieu Lamarre'
dependencies = []
files = []
hgrepos = []
issue_num = 35248
keywords = []
message_count = 2.0
messages = ['329920', '329924']
nosy_count = 1.0
nosy_names = ['Mathieu Lamarre']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35248'
versions = ['Python 3.6']

Linked PRs

• [3.12] gh-79429: Ignore FileNotFoundError when remove a temporary directory in the multiprocessing finalizer #112865
• [3.11] [3.12] gh-79429: Ignore FileNotFoundError when remove a temporary directory in the multiprocessing finalizer (GH-112865) #112897

[MathieuLamarre]

Running:

from multiprocessing.sharedctypes import RawArray
from ctypes import c_uint32
if __name__ == '__main__':
    shared_array = RawArray(c_uint32, 1500)

Causes:
Traceback (most recent call last):
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers
    finalizer()
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/multiprocessing/util.py", line 186, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 480, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/home/ava/miniconda3/envs/ava36/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'pym-10314-v8aznmmb'

Python 3.6.6 |Anaconda, Inc.| (default, Oct 9 2018, 12:34:16)
[GCC 7.3.0] on linux

[MathieuLamarre]

To repro, the temp directory returned by tempfile must be a network mount to reproduce this error. I have this on server with minimal disk space in /home and TMPDIR points to CIFS mounted volume. So maybe can only be reproduced if shutils operations on the temp dir are slow (or buggy).

We have been using a network temp dir in a large project and RawArray is the only object giving us error so far. Maybe the error could be ignored, if it's only dual delete issue.

[serhiy-storchaka]

shutil.rmtree() ignores FileNotFound errors by default, so it should only be fixed in 3.11 and 3.12.