Use ensuredir() instead of os.makedirs() to fix race conditions

Use the ensuredir() function consistently across Sphinx code to avoid
race conditions e.g. when multiple Sphinx instances attempt to create
the same output directory. The ensuredir() function that was already
present in sphinx.util.osutil correctly catches EEXIST exception that
occurs if the specified directory already exists (i.e. it was created
between the call to os.path.isdir() and os.makedirs() that follows it).

While at it, remove redundant os.path.isdir() calls when they only
guarded the os.makedirs() call, and replace mkdir_p() which had pretty
much the same purpose, except for being prone to race conditions.

I did not modify testing-related code as race conditions mostly affect
real applications and not the test environment.

Fix #4281: Race conditions when creating output directory

1 files changed, 3 insertions, 4 deletions

diff --git a/sphinx/apidoc.py b/sphinx/apidoc.py
index 24ed874b0..b764cfc35 100644
--- a/sphinx/apidoc.py
+++ b/sphinx/apidoc.py
@@ -26,7 +26,7 @@ from fnmatch import fnmatch
 from sphinx import __display_version__
 from sphinx.quickstart import EXTENSIONS
 from sphinx.util import rst
-from sphinx.util.osutil import FileAvoidWrite, walk
+from sphinx.util.osutil import FileAvoidWrite, ensuredir, walk
 
 if False:
     # For type annotation
@@ -375,9 +375,8 @@ Note: By default this script will not overwrite already created files.""")
     if not path.isdir(rootpath):
         print('%s is not a directory.' % rootpath, file=sys.stderr)
         sys.exit(1)
-    if not path.isdir(opts.destdir):
-        if not opts.dryrun:
-            os.makedirs(opts.destdir)
+    if not opts.dryrun:
+        ensuredir(opts.destdir)
     rootpath = path.abspath(rootpath)
     excludes = normalize_excludes(rootpath, excludes)
     modules = recurse_tree(rootpath, excludes, opts)