Adds ability to run local commands with functional tests.
By default, commands are run locally. It is possible to set remote
host connection parameters from pytest command line, for example:
`--hosts 'fbx@IP' --ssh-identity-file '.container/ssh/id_ed25519'`
For more options, see documentation
https://testinfra.readthedocs.io/en/latest/backends.html#ssh.
Includes a fixture `host_sudo` to run commands as sudo.
Relates to
https://salsa.debian.org/freedombox-team/freedombox/-/issues/2451#note_530752.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Makes it easy to investigate functional test failures as the test are run.
- This will help functional tests that are timing out before failure stack
traces can be shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Fixes an error in testing container where installing dependencies for
functional tests wants to uninstall system installed urllib3.
Also minor quotes fixes.
Note that dependencies are upgraded only in new dev environments where
geckodriver is not yet installed.
Tests performed, in both stable and testing containers:
- Run all tests, no upgrade related failures found. There were some
seemingly unrelated test failures in apps: calibre, ejabberd, deluged,
minetest, users, transmission , wordpress, zoph.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Pip option --break-system-packages is available only from Bookworm.
- Remove --user to that the script works can be used broadly.
Tests:
- On a fresh stable container, running functional tests works. Without the
patch, it fails.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Without the change, pip refuses to install into system environment due to
changes outlined in PEP 668. The following error is thrown: 'error:
externally-managed-environment'.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is a temporary fix until Splinter addresses the breaking changes in Selenium 4.3.0
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This allows the user to understand the wrapper script and skip/adapt it when
necessary.
- Debug any issues with the script.
- Maintain consistency with the philosophy of the rest of the container script.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
This removes the dependencies xvfb and pytest-xvfb.
--splinter-headless can be used as a substitute for running tests in
headless mode.
[sunil: Edit description as running run-tests starts plinth in container]
[sunil: Retain the xauth command]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- On Debian 10, pin pytest-bdd version to 3.2.1, this is the latest
version that supports pytest 3.10.1.
- Install wheel and pytest-xdist packages from the Debian repos.
- Add git as a dependency for functional tests.
- Fix regexp when getting Gecko version from the Github api, for some
reason after colon, sometimes there is a space and sometimes not.
Tests performed on Debian stable, testing and unstable:
- Installing dependencies with the install.sh script succeeds.
- No regressions when running tests.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
splinter 0.13.0 has a bug which is fixed in 0.14.0
See https://github.com/cobrateam/splinter/pull/749
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add pytest hooks to ignore all functional tests if pytest_bdd is not
installed.
- Update pytest hooks to skip tests in file named 'test_functional.py' if
--include-functional argument is not provided.
- Move functional_tests/install.py into plinth/tests/functional and update
reference in Vagrantfile.
- Move scenario files into individual app folders. Rename them after the app
they are testing. Merge TODO items listed in todo.org into corresponding feature
files.
- Add test_functional.py in each app to build tests from the features file using
pytest_bdd.
- Move all step_definitions, support and data into plinth/tests/functional/.
Include all step_definitions from conftest.py. Update to relative imports
instead of absolute imports.
Tests performed:
- Run py.test-3 --collect-only shows all functional tests and lists 574 tests.
No errors show that name of feature files are correct. The number says that all
functional test features are included.
- Remove pytest_bdd (or modify the import name) and run py.test-3 --collect-only
skips collecting all functional tests and shows only 300+ tests.
- Run functional tests for a few apps with py.test-3 --include-functional -m
app. For storage, deluge.
- Run unit tests with py.test-3. Functional tests are listed by skipped.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
