tests: Improve handling of tests skipped by default

- Create the 'heavy' mark for tests that take long to run.
- Skip the tests marked as 'heavy' by default.
- Enable functional and heavy tests by setting the environment variable
  EXTENDED_TESTING=1.
- Update HACKING.md accordingly.

Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Minor indentation]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>

HACKING.md

@@ -338,7 +338,7 @@ For more information on translations: https://wiki.debian.org/FreedomBox/Transla
 
 ### Running Tests
 
-To run all the tests in the container/VM:
+To run all the standard unit tests in the container/VM:
 
 ```bash
 guest$ py.test-3
@@ -369,6 +369,29 @@ guest$ py.test-3 plinth/tests/test_actions.py::TestActions
 guest$ py.test-3 plinth/tests/test_actions.py::TestActions::test_is_package_manager_busy
 ```
 
+Some tests are skipped by default:
+* tests that need root privileges,
+* functional tests (they need additional preparation to run. See next section),
+* tests that take much time to run.
+
+Use `sudo` to run the ones that need root access:
+```bash
+guest$ sudo py.test-3
+```
+
+To force functional tests and tests that take long to run, set the environment
+variable EXTENDED_TESTING=1:
+
+```bash
+guest$ EXTENDED_TESTING=1 py.test-3
+```
+
+To really run all tests, combine sudo with EXTENDED_TESTING:
+
+```bash
+guest$ sudo EXTENDED_TESTING=1 py.test-3
+```
+
 ### Running the Test Coverage Analysis
 
 To run the coverage tool in the container/VM:

conftest.py

@@ -31,18 +31,29 @@ def pytest_addoption(parser):
 
 
 def pytest_collection_modifyitems(config, items):
-    """Filter out functional tests unless --include-functional is passed."""
+    """Filter out specificly marked tests unless explicitly requested.
-    if config.getoption('--include-functional'):
-        # Option provided on command line, no filtering
-        return
 
-    skip_functional = pytest.mark.skip(
+    The EXTENDED_TESTING environment variable is borrowed from the Lancaster
-        reason='--include-functional not provided')
+    consensus met by the Pearl community. See
-    for item in items:
+    https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/lancaster-consensus.md
-        if 'functional' in item.keywords or (item.parent.fspath.basename
+    """
-                                             and item.parent.fspath.basename
+
-                                             == 'test_functional.py'):
+    def skip(item, reason):
-            item.add_marker(skip_functional)
+        item.add_marker(pytest.mark.skip(reason=reason))
+
+    extended = 'EXTENDED_TESTING' in os.environ
+    if not (extended or config.getoption('--include-functional')):
+        for item in items:
+            if 'functional' in item.keywords or (
+                    item.parent.fspath.basename
+                    and item.parent.fspath.basename == 'test_functional.py'):
+                skip(item, '--include-functional not provided')
+
+    if not extended:
+        for item in items:
+            if 'heavy' in item.keywords:
+                skip(item, ('Takes too much time. '
+                            'Set EXTENDED_TESTING=1 to force run'))
 
 
 @pytest.fixture(name='load_cfg')

pyproject.toml

@@ -12,7 +12,10 @@ omit = ["*/tests/*"]
 [tool.pytest.ini_options]
 addopts = "--ds=plinth.tests.data.django_test_settings"
 markers = [
+    "essential",
     "functional",
+    "skip",
+    "heavy",
     "apps",
     "avahi",
     "backups",
@@ -26,7 +29,6 @@ markers = [
     "deluge",
     "dynamicdns",
     "ejabberd",
-    "essential",
     "gitweb",
     "help",
     "i2p",
@@ -52,7 +54,6 @@ markers = [
     "security",
     "shadowsocks",
     "sharing",
-    "skip",
     "snapshot",
     "ssh",
     "sso",