One of the joys of having test_multiprocessing occasionally execute after

test_importlib is that it discovers special little race conditions. For
instance, it turns out that importlib would throw an exception if two different
Python processes both tried to create the __pycache__ directory as one process
would succeed, causing the other process to fail as it didn't expect to get any
"help". So now importlib simply stays calm and just accepts someone else did
the work of creating the __pycache__ directory for it, moving on with life.

Closes issue #9572.

Lib/importlib/_bootstrap.py

@@ -482,27 +482,35 @@ class _SourceFileLoader(_FileLoader, SourceLoader):
 
     def set_data(self, path, data):
         """Write bytes data to a file."""
+        parent, _, filename = path.rpartition(path_sep)
+        path_parts = []
+        # Figure out what directories are missing.
+        while parent and not _path_isdir(parent):
+            parent, _, part = parent.rpartition(path_sep)
+            path_parts.append(part)
+        # Create needed directories.
+        for part in reversed(path_parts):
+            parent = _path_join(parent, part)
+            try:
+                _os.mkdir(parent)
+            except IOError as exc:
+                # Probably another Python process already created the dir.
+                if exc.errno == errno.EEXIST:
+                    continue
+                # If can't get proper access, then just forget about writing
+                # the data.
+                elif errno == errno.EACCES:
+                    return
+                else:
+                    raise
         try:
-            with _closing(_io.FileIO(path, 'wb')) as file:
+            with _io.FileIO(path, 'wb') as file:
                 file.write(data)
         except IOError as exc:
-            if exc.errno == errno.ENOENT:
-                directory, _, filename = path.rpartition(path_sep)
-                sub_directories = []
-                while not _path_isdir(directory):
-                    directory, _, sub_dir = directory.rpartition(path_sep)
-                    sub_directories.append(sub_dir)
-                    for part in reversed(sub_directories):
-                        directory = _path_join(directory, part)
-                        try:
-                            _os.mkdir(directory)
-                        except IOError as exc:
-                            if exc.errno != errno.EACCES:
-                                raise
-                            else:
-                                return
-                return self.set_data(path, data)
-            elif exc.errno != errno.EACCES:
+            # Don't worry if you can't write bytecode.
+            if exc.errno == errno.EACCES:
+                return
+            else:
                 raise
 
 

Misc/NEWS

@@ -123,6 +123,9 @@ Extensions
 Library
 -------
 
+- Issue #9572: Importlib should not raise an exception if a directory it
+  thought it needed to create was done concurrently by another process.
+
 - Issue #9617: Signals received during a low-level write operation aren't
   ignored by the buffered IO layer anymore.