- Issue #16662: load_tests() is now unconditionally run when it is present in

  a package's __init__.py.  TestLoader.loadTestsFromModule() still accepts
  use_load_tests, but it is deprecated and ignored.  A new keyword-only
  attribute `pattern` is added and documented.  Patch given by Robert Collins,
  tweaked by Barry Warsaw.

Doc/library/unittest.rst

@@ -1561,7 +1561,7 @@ Loading and running tests
       :class:`testCaseClass`.
 
 
-   .. method:: loadTestsFromModule(module)
+   .. method:: loadTestsFromModule(module, pattern=None)
 
       Return a suite of all tests cases contained in the given module. This
       method searches *module* for classes derived from :class:`TestCase` and
@@ -1578,11 +1578,18 @@ Loading and running tests
 
       If a module provides a ``load_tests`` function it will be called to
       load the tests. This allows modules to customize test loading.
-      This is the `load_tests protocol`_.
+      This is the `load_tests protocol`_.  The *pattern* argument is passed as
+      the third argument to ``load_tests``.
 
       .. versionchanged:: 3.2
          Support for ``load_tests`` added.
 
+      .. versionchanged:: 3.5
+         The undocumented and unofficial *use_load_tests* default argument is
+         deprecated and ignored, although it is still accepted for backward
+         compatibility.  The method also now accepts a keyword-only argument
+         *pattern* which is passed to ``load_tests`` as the third argument.
+
 
    .. method:: loadTestsFromName(name, module=None)
 
@@ -1634,18 +1641,18 @@ Loading and running tests
       the start directory is not the top level directory then the top level
       directory must be specified separately.
 
-      If importing a module fails, for example due to a syntax error, then this
-      will be recorded as a single error and discovery will continue.  If the
-      import failure is due to :exc:`SkipTest` being raised, it will be recorded
-      as a skip instead of an error.
+      If importing a module fails, for example due to a syntax error, then
+      this will be recorded as a single error and discovery will continue.  If
+      the import failure is due to :exc:`SkipTest` being raised, it will be
+      recorded as a skip instead of an error.
 
-      If a test package name (directory with :file:`__init__.py`) matches the
-      pattern then the package will be checked for a ``load_tests``
-      function. If this exists then it will be called with *loader*, *tests*,
-      *pattern*.
+      If a package (a directory containing a file named :file:`__init__.py`) is
+      found, the package will be checked for a ``load_tests`` function. If this
+      exists then it will be called with *loader*, *tests*, *pattern*.
 
-      If load_tests exists then discovery does *not* recurse into the package,
-      ``load_tests`` is responsible for loading all tests in the package.
+      If ``load_tests`` exists then discovery does *not* recurse into the
+      package, ``load_tests`` is responsible for loading all tests in the
+      package.
 
       The pattern is deliberately not stored as a loader attribute so that
       packages can continue discovery themselves. *top_level_dir* is stored so
@@ -1664,6 +1671,11 @@ Loading and running tests
            the same even if the underlying file system's ordering is not
            dependent on file name.
 
+      .. versionchanged:: 3.5
+         Found packages are now checked for ``load_tests`` regardless of
+         whether their path matches *pattern*, because it is impossible for
+         a package name to match the default pattern.
+
 
    The following attributes of a :class:`TestLoader` can be configured either by
    subclassing or assignment on an instance:
@@ -2032,7 +2044,10 @@ test runs or test discovery by implementing a function called ``load_tests``.
 If a test module defines ``load_tests`` it will be called by
 :meth:`TestLoader.loadTestsFromModule` with the following arguments::
 
-    load_tests(loader, standard_tests, None)
+    load_tests(loader, standard_tests, pattern)
+
+where *pattern* is passed straight through from ``loadTestsFromModule``.  It
+defaults to ``None``.
 
 It should return a :class:`TestSuite`.
 
@@ -2054,21 +2069,12 @@ A typical ``load_tests`` function that loads tests from a specific set of
             suite.addTests(tests)
         return suite
 
-If discovery is started, either from the command line or by calling
-:meth:`TestLoader.discover`, with a pattern that matches a package
-name then the package :file:`__init__.py` will be checked for ``load_tests``.
-
-.. note::
-
-   The default pattern is ``'test*.py'``. This matches all Python files
-   that start with ``'test'`` but *won't* match any test directories.
-
-   A pattern like ``'test*'`` will match test packages as well as
-   modules.
-
-If the package :file:`__init__.py` defines ``load_tests`` then it will be
-called and discovery not continued into the package. ``load_tests``
-is called with the following arguments::
+If discovery is started in a directory containing a package, either from the
+command line or by calling :meth:`TestLoader.discover`, then the package
+:file:`__init__.py` will be checked for ``load_tests``.  If that function does
+not exist, discovery will recurse into the package as though it were just
+another directory.  Otherwise, discovery of the package's tests will be left up
+to ``load_tests`` which is called with the following arguments::
 
     load_tests(loader, standard_tests, pattern)
 
@@ -2087,6 +2093,11 @@ continue (and potentially modify) test discovery. A 'do nothing'
         standard_tests.addTests(package_tests)
         return standard_tests
 
+.. versionchanged:: 3.5
+   Discovery no longer checks package names for matching *pattern* due to the
+   impossibility of package names matching the default pattern.
+
+
 
 Class and Module Fixtures
 -------------------------

Lib/unittest/loader.py

@@ -6,6 +6,7 @@ import sys
 import traceback
 import types
 import functools
+import warnings
 
 from fnmatch import fnmatch
 
@@ -70,8 +71,27 @@ class TestLoader(object):
         loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
         return loaded_suite
 
-    def loadTestsFromModule(self, module, use_load_tests=True):
+    # XXX After Python 3.5, remove backward compatibility hacks for
+    # use_load_tests deprecation via *args and **kws.  See issue 16662.
+    def loadTestsFromModule(self, module, *args, pattern=None, **kws):
         """Return a suite of all tests cases contained in the given module"""
+        # This method used to take an undocumented and unofficial
+        # use_load_tests argument.  For backward compatibility, we still
+        # accept the argument (which can also be the first position) but we
+        # ignore it and issue a deprecation warning if it's present.
+        if len(args) == 1 or 'use_load_tests' in kws:
+            warnings.warn('use_load_tests is deprecated and ignored',
+                          DeprecationWarning)
+            kws.pop('use_load_tests', None)
+        if len(args) > 1:
+            raise TypeError('loadTestsFromModule() takes 1 positional argument but {} were given'.format(len(args)))
+        if len(kws) != 0:
+            # Since the keyword arguments are unsorted (see PEP 468), just
+            # pick the alphabetically sorted first argument to complain about,
+            # if multiple were given.  At least the error message will be
+            # predictable.
+            complaint = sorted(kws)[0]
+            raise TypeError("loadTestsFromModule() got an unexpected keyword argument '{}'".format(complaint))
         tests = []
         for name in dir(module):
             obj = getattr(module, name)
@@ -80,9 +100,9 @@ class TestLoader(object):
 
         load_tests = getattr(module, 'load_tests', None)
         tests = self.suiteClass(tests)
-        if use_load_tests and load_tests is not None:
+        if load_tests is not None:
             try:
-                return load_tests(self, tests, None)
+                return load_tests(self, tests, pattern)
             except Exception as e:
                 return _make_failed_load_tests(module.__name__, e,
                                                self.suiteClass)
@@ -325,7 +345,7 @@ class TestLoader(object):
                         msg = ("%r module incorrectly imported from %r. Expected %r. "
                                "Is this module globally installed?")
                         raise ImportError(msg % (mod_name, module_dir, expected_dir))
-                    yield self.loadTestsFromModule(module)
+                    yield self.loadTestsFromModule(module, pattern=pattern)
             elif os.path.isdir(full_path):
                 if (not namespace and
                     not os.path.isfile(os.path.join(full_path, '__init__.py'))):
@@ -333,26 +353,27 @@ class TestLoader(object):
 
                 load_tests = None
                 tests = None
-                if fnmatch(path, pattern):
-                    # only check load_tests if the package directory itself matches the filter
-                    name = self._get_name_from_path(full_path)
+                name = self._get_name_from_path(full_path)
+                try:
                     package = self._get_module_from_name(name)
+                except case.SkipTest as e:
+                    yield _make_skipped_test(name, e, self.suiteClass)
+                except:
+                    yield _make_failed_import_test(name, self.suiteClass)
+                else:
                     load_tests = getattr(package, 'load_tests', None)
-                    tests = self.loadTestsFromModule(package, use_load_tests=False)
-
-                if load_tests is None:
+                    tests = self.loadTestsFromModule(package, pattern=pattern)
                     if tests is not None:
                         # tests loaded from package file
                         yield tests
+
+                    if load_tests is not None:
+                        # loadTestsFromModule(package) has load_tests for us.
+                        continue
                     # recurse into the package
                     yield from self._find_tests(full_path, pattern,
                                                 namespace=namespace)
-                else:
-                    try:
-                        yield load_tests(self, tests, pattern)
-                    except Exception as e:
-                        yield _make_failed_load_tests(package.__name__, e,
-                                                      self.suiteClass)
+
 
 defaultTestLoader = TestLoader()
 

Lib/unittest/test/test_loader.py

 import sys
 import types
-
+import warnings
 
 import unittest
 
+# Decorator used in the deprecation tests to reset the warning registry for
+# test isolation and reproducibility.
+def warningregistry(func):
+    def wrapper(*args, **kws):
+        missing = object()
+        saved = getattr(warnings, '__warningregistry__', missing).copy()
+        try:
+            return func(*args, **kws)
+        finally:
+            if saved is missing:
+                try:
+                    del warnings.__warningregistry__
+                except AttributeError:
+                    pass
+            else:
+                warnings.__warningregistry__ = saved
+
 
 class Test_TestLoader(unittest.TestCase):
 
@@ -150,6 +167,7 @@ class Test_TestLoader(unittest.TestCase):
 
     # Check that loadTestsFromModule honors (or not) a module
     # with a load_tests function.
+    @warningregistry
     def test_loadTestsFromModule__load_tests(self):
         m = types.ModuleType('m')
         class MyTestCase(unittest.TestCase):
@@ -168,10 +186,139 @@ class Test_TestLoader(unittest.TestCase):
         suite = loader.loadTestsFromModule(m)
         self.assertIsInstance(suite, unittest.TestSuite)
         self.assertEqual(load_tests_args, [loader, suite, None])
+        # With Python 3.5, the undocumented and unofficial use_load_tests is
+        # ignored (and deprecated).
+        load_tests_args = []
+        with warnings.catch_warnings(record=False):
+            warnings.simplefilter('never')
+            suite = loader.loadTestsFromModule(m, use_load_tests=False)
+            self.assertEqual(load_tests_args, [loader, suite, None])
+
+    @warningregistry
+    def test_loadTestsFromModule__use_load_tests_deprecated_positional(self):
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('always')
+            m = types.ModuleType('m')
+            class MyTestCase(unittest.TestCase):
+                def test(self):
+                    pass
+            m.testcase_1 = MyTestCase
+
+            load_tests_args = []
+            def load_tests(loader, tests, pattern):
+                self.assertIsInstance(tests, unittest.TestSuite)
+                load_tests_args.extend((loader, tests, pattern))
+                return tests
+            m.load_tests = load_tests
+            # The method still works.
+            loader = unittest.TestLoader()
+            # use_load_tests=True as a positional argument.
+            suite = loader.loadTestsFromModule(m, False)
+            self.assertIsInstance(suite, unittest.TestSuite)
+            # load_tests was still called because use_load_tests is deprecated
+            # and ignored.
+            self.assertEqual(load_tests_args, [loader, suite, None])
+        # We got a warning.
+        self.assertIs(w[-1].category, DeprecationWarning)
+        self.assertEqual(str(w[-1].message),
+                             'use_load_tests is deprecated and ignored')
+
+    @warningregistry
+    def test_loadTestsFromModule__use_load_tests_deprecated_keyword(self):
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter('always')
+            m = types.ModuleType('m')
+            class MyTestCase(unittest.TestCase):
+                def test(self):
+                    pass
+            m.testcase_1 = MyTestCase
+
+            load_tests_args = []
+            def load_tests(loader, tests, pattern):
+                self.assertIsInstance(tests, unittest.TestSuite)
+                load_tests_args.extend((loader, tests, pattern))
+                return tests
+            m.load_tests = load_tests
+            # The method still works.
+            loader = unittest.TestLoader()
+            suite = loader.loadTestsFromModule(m, use_load_tests=False)
+            self.assertIsInstance(suite, unittest.TestSuite)
+            # load_tests was still called because use_load_tests is deprecated
+            # and ignored.
+            self.assertEqual(load_tests_args, [loader, suite, None])
+            # We got a warning.
+            self.assertIs(w[-1].category, DeprecationWarning)
+            self.assertEqual(str(w[-1].message),
+                                 'use_load_tests is deprecated and ignored')
+
+    def test_loadTestsFromModule__too_many_positional_args(self):
+        m = types.ModuleType('m')
+        class MyTestCase(unittest.TestCase):
+            def test(self):
+                pass
+        m.testcase_1 = MyTestCase
+
+        load_tests_args = []
+        def load_tests(loader, tests, pattern):
+            self.assertIsInstance(tests, unittest.TestSuite)
+            load_tests_args.extend((loader, tests, pattern))
+            return tests
+        m.load_tests = load_tests
+        loader = unittest.TestLoader()
+        with self.assertRaises(TypeError) as cm:
+            loader.loadTestsFromModule(m, False, 'testme.*')
+        self.assertEqual(type(cm.exception), TypeError)
+        # The error message names the first bad argument alphabetically,
+        # however use_load_tests (which sorts first) is ignored.
+        self.assertEqual(
+            str(cm.exception),
+            'loadTestsFromModule() takes 1 positional argument but 2 were given')
+
+    @warningregistry
+    def test_loadTestsFromModule__use_load_tests_other_bad_keyword(self):
+        m = types.ModuleType('m')
+        class MyTestCase(unittest.TestCase):
+            def test(self):
+                pass
+        m.testcase_1 = MyTestCase
+
+        load_tests_args = []
+        def load_tests(loader, tests, pattern):
+            self.assertIsInstance(tests, unittest.TestSuite)
+            load_tests_args.extend((loader, tests, pattern))
+            return tests
+        m.load_tests = load_tests
+        loader = unittest.TestLoader()
+        with warnings.catch_warnings():
+            warnings.simplefilter('never')
+            with self.assertRaises(TypeError) as cm:
+                loader.loadTestsFromModule(
+                    m, use_load_tests=False, very_bad=True, worse=False)
+        self.assertEqual(type(cm.exception), TypeError)
+        # The error message names the first bad argument alphabetically,
+        # however use_load_tests (which sorts first) is ignored.
+        self.assertEqual(
+            str(cm.exception),
+            "loadTestsFromModule() got an unexpected keyword argument 'very_bad'")
+
+    def test_loadTestsFromModule__pattern(self):
+        m = types.ModuleType('m')
+        class MyTestCase(unittest.TestCase):
+            def test(self):
+                pass
+        m.testcase_1 = MyTestCase
 
         load_tests_args = []
-        suite = loader.loadTestsFromModule(m, use_load_tests=False)
-        self.assertEqual(load_tests_args, [])
+        def load_tests(loader, tests, pattern):
+            self.assertIsInstance(tests, unittest.TestSuite)
+            load_tests_args.extend((loader, tests, pattern))
+            return tests
+        m.load_tests = load_tests
+
+        loader = unittest.TestLoader()
+        suite = loader.loadTestsFromModule(m, pattern='testme.*')
+        self.assertIsInstance(suite, unittest.TestSuite)
+        self.assertEqual(load_tests_args, [loader, suite, 'testme.*'])
 
     def test_loadTestsFromModule__faulty_load_tests(self):
         m = types.ModuleType('m')

Misc/NEWS

@@ -132,6 +132,12 @@ Core and Builtins
 Library
 -------
 
+- Issue #16662: load_tests() is now unconditionally run when it is present in
+  a package's __init__.py.  TestLoader.loadTestsFromModule() still accepts
+  use_load_tests, but it is deprecated and ignored.  A new keyword-only
+  attribute `pattern` is added and documented.  Patch given by Robert Collins,
+  tweaked by Barry Warsaw.
+
 - Issue #22226: First letter no longer is stripped from the "status" key in
   the result of Treeview.heading().