Fixes: #2376.
Fixes: #2317.
When an app is removed, its packages are purged. However, there is another
installed app that needs these packages, we should keep those packages. We have
already implemented checking the packages against other apps' packages. However,
we are not checking if we are removing dependencies of other apps' packages.
This will still result in removal apps' packages.
To solve this problem, get list of packages of all the apps, then iterate over
their dependencies recursively and compile a comprehensive list of packages to
keep. Use this to reduce the set of packages to remove.
Tests:
- Without the patch, install bepasty and janus. Uninstall janus app and notice
that bepasty package is removed. With the patch, the problem is not observed.
- Printing the comprehensive list of packages to keep shows an extensive set
computed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Fixes an issue with zoph not being setup after uninstall+install setup and
makes a test pass.
- Some failures exist but don't seem related to this change.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This will help apps like zoph that need to have database server running to
install or upgrade a package.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Some apps store their configuration in database. The database server may not
be running when app is disabled. Configuration changes may then not be
possible for such apps. Provide the ability to disable configuration for apps
that don't support configuration changes when disabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2409.
When creating a user if one or more groups is selected, creation fails. This
is because the fields contains group choices as (name, label) tuples instead
of (group_id, label) tuples as expected by the many-to-many field mapping
mechanism in ModelField class. Fix this by using the same mechanism used in
UserUpdateForm, which is to reuse the base class form field (but adjust some
properties).
Tests:
- During first boot
- Django groups are fully created when form is accessed with blank database
- In user creation/modify form:
- Label appears are 'Permissions'
- Choices appear fully and as 'Description (Group name)'
- Help text is correct.
- Choices are sorted on group name.
- Django groups are fully created when form is accessed when a new group is
added to code.
- User can have no groups
- Widget is multiple checkbox widget. Multiple groups can be selected.
- User is added to proper ldap groups after submission
- In user modify form:
- If the user is last admin user, admin checkbox is checked and disabled.
- Current list of groups is accurate shown when form is displayed.
- Add remove of groups works as expected
- Functional tests for gitweb and users apps pass
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is useful for managing redis service needed by the upcoming Nextcloud
app.
- Disable the daemon only if all the apps using it are disabled. Enable it when
even one of the them is enabled.
- The component is not a 'leader' component as it does not decide the
enabled/disabled status of the app.
Tests:
- Unit tests pass.
- Install zoph and wordpress with full patch series. If one of the apps is
disabled, mysql service is still enabled and running. If both apps are disabled,
then mysql service is disabled and not running. Enabled/disabled status of apps
are accurate after they are enabled/disabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Otherwise, apps_init calls _sort_apps, which randomly fails in functional tests
CI job. The failure may be due to the tests running in parallel, since I could
not reproduce it locally.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
In Python 3.12, has_calls is not a valid assertion.
Closes: #1058421
Tests:
- Run "python3.12 -m pytest plinth/tests/test_operation.py" in unstable
container. All tests are passed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Using `--dist=loadscope` splits the tests based on module, thus
ensuring each app's tests get assigned to one worker only.
- Though app servers run in 2 separate VMs, the pytest-xdist workers
simply run in two Python subprocesses (`-n 2`). This allows us to
generate a unified test report.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
When setup is run from the application thread after startup, it continuously
tries until it succeeds. However, after making the first attempt, it does not
collect the status of the operation keeping the operation object in operation
manager. When trying for the second time, trying to create operation with same
ID fails since the operation is already present.
Fix this by allowing the operation to be recreated if the existing operation has
failed.
Tests:
- Unit tests pass.
- Functional tests for bepasty app pass.
- Install an app. Create an error in the setup mechanism for an app. Increment
is app version number. Start the service and notice that setup of app is
attempted and fails. Few seconds later the setup is attempted again and the
process continues. Each time the failure is due to fault in the app's setup
method rather than operation not being accepted.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
See db.py for rationale.
Tests:
- Run functional tests and unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Without the changes, the form submission for setting firewall zone fails with
error similar to "element can't be scrolled to view".
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This prevents CSRF attacks that allow adversarial websites from logging out
users from FreedomBox. Django itself has made this change in 4.x releases.
Tests:
- Logout works with the menu item in drop-down when Javascript is enabled. The
menu item appears similar to other drop-down menu items.
- Logout works with the menu item when JavaScript is disabled on the page. The
menu item appears similar to other menu items.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
The primary reason for the existence of uninstall feature is to clean up the
existing working or non-working setup and re-install freshly. It is not to
remove the effects of the installed app since disable works well for that. It is
seldom to free up space since even on a microSD card, the space occupied by most
apps is negligible.
Tests:
- Unit tests pass
- Functional tests for all apps pass (except for known failures).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Use two separate strategies: one to symlink from /etc to files in /usr/ and
two to copy files from /usr to /etc. The latter is needed for some rare cases
where symlinks don't work because the daemon runs in a chrooted environment.
- Update tutorial and reference sections in developer documentation.
Tests:
- Unit tests works and are comprehensive.
- Using the component in apps works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Several tests failed randomly (on a slightly slow machine) while trying to
detect that a page change has occurred. Workaround this by handling the
exception thrown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
All the apps including OpenVPN now use AppView and therefore using the standard
enable/disable button.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Pip option --break-system-packages is available only from Bookworm.
- Remove --user to that the script works can be used broadly.
Tests:
- On a fresh stable container, running functional tests works. Without the
patch, it fails.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tested that all samba tests pass in testing container. Also checked that
connection type texts are not translated, thus safe to use in xpath search.
Fixes#2333.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Without the change, pip refuses to install into system environment due to
changes outlined in PEP 668. The following error is thrown: 'error:
externally-managed-environment'.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- test_uninstall is skipped for dynamicdns.
- test_uninstall is passed for bepasty.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2194.
In the four cases fixed, if a notification is present with a 'btn-primary' in
it, then instead of submitting the intended form, the button on the notification
is clicked. This will result in an indefinite wait for the form to perform an
action. Fix this by specifying which form exactly we want to submit.
Tests:
- Run functional tests for config app and updates app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Don't start the helper class name with 'Test' as it will be treated as a class
with test cases to run.
Fixes#2288.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Warning that exim4 will be removed is incorrectly not shown during email app
installation. Fix that.
Tests:
- Unit tests pass.
- On a fresh testing container, trying to install email app shows the warning
message that exim will be removed. Installing the email app succeeds. exim is
removed during installation.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Keep the description about app generic
- Remove enable/disable option
- Create a booleanfield to turn on/off popcon
- Don't re-enable popcon during an update
Tests:
- When enabling/disabling the option, the `"PARTICIPATE"` value in
`/etc/popularity-contest.conf` is changed to yes/no as expected. For reference
see `/var/lib/dpkg/info/popularity-contest.templates`
- When popcon option is enabled, running sudo sh -x
/etc/cron.daily/popularity-context shows that execution was successful and data
was submitted. Remove files /var/log/popularity-contest* and
/var/lib/popularity-contest/lastsub if necessary. Gpg is used and encrypted data
is what was submitted.
- When popcon option is disabled, running sudo sh -x
/etc/cron.daily/popularity-context shows that execution stopped because the
option is disabled.
Signed-off-by: nbenedek <contact@nbenedek.me>
[sunil: Add a notification to tell users about privacy app]
[sunil: Correct the URL to /sys]
[sunil: Minor code styling changes and updates to description, icon]
[sunil: Ensure that popcon works with encryption]
[sunil: Write configuration to a separate file]
[sunil: Use Shellvars lens instead of Php lns]
[sunil: Add functional tests]
[sunil: Backup/restore the configuration file]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- All tests in patch series have been done with this patch applied
- Unit tests pass
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Unit tests work
- DONE: Transmission
- DONE: Enabling/disabling an app with a daemon works: transmission
- DONE: Showing the status of whether the app is enabled with daemon
is-enabled works.
- DONE: A message is shown if app is enabled and service is not running
- DONE: Service is stopped and re-started during backup
- DONE: Adding user to share group during initial setup restarts the service
- Not tested: Enabling/disabling a service with alias works (no such apps)
- DONE: Restarting/try-restarting a service works
- DONE: Masking/unmasking works
- DONE: rsyslog is masked after initial setup
- DONE: systemd-journald is try-restarted during initial setup
- DONE: Avahi, email, security initial setup works
- DONE: Fail2ban is unmasked and enabled
- DONE: Enabling/disabling fail2ban is security app works
- DONE: Enabling/disabling password authentication in SSH works
- ?? Let's encrypt
- Services are try-restarted during certificate setup, obtain, renew
- Not tested: upgrade pagekite from version 1
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Functional tests work (failing already)
- DONE: Showing front page shortcuts according to user groups works
- DONE: Only user who is party of syncthing group is shown syncthing
- DONE: Admin users are always shown all the apps
- DONE: Syncthing:
- Not tested: When upgrading from version 2 or below, renaming group works
- DONE: Syncthing is added to freedombox-share group
- DONE: Initial setup of users app works
- DONE: freedombox-share group is created
- DONE: Retriving last admin user works
- DONE: Last admin is not allowed to delete account
- DONE: Creating a new user works
- DONE: Password is set properly (user can login with 'su - user' after)
- DONE: Incorrect confirmation password leads to error
- DONE: Adding the user to groups works (edit page shows correct list of groups)
- DONE: Editing a user works
- DONE: User is renamed properly
- DONE: Removing user from groups works
- DONE: Adding user to new groups works
- DONE: Providing incorrect auth password results in error message
- DONE: Enabling/disabling account work (confirm with 'su - user'). See #2277.
- DONE: Updating user password works
- DONE: New password is set (confirm with 'su - user')
- DONE: Providing incorrect auth password results in error message
- DONE: Initial user account creation works
- DONE: User account can be used (confirm with 'su - user')
- DONE: User is added to admin group
- DONE: Exception while getting SSH keys results in showing empty field
- DONE: Removing a user works
- DONE: Command provided in a message in users_firstboot.html works for
deleting users.
- DONE: If an admin users exists when running first wizard, list of admin users
is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Ignore setting a None domain
- Updated tests to use base class
- Functional tests work
- Backup/restore works. Database is dumped and restored.
- Initial setup works
- Enabling/disabling works
- API access is enabled and a valid domain is set when available
- Setting the domain works
- Configuration is updated in update.php
- App page show newly set domain
- Not tested: force upgrade of package
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
There is not much additional risk by doing this. This is needed in case of some
exceptional cases such as storage.validate_directory() which need to run as a
different user other than root.
Tests:
- Directory validation works in transmission and deluge.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Currently privileged actions use stdout for returning the results. If any of the
sub-processes accidentally output to stdout, decoding errors occur. Prevent this
by opening a pipe to the privileged action and returning the output in that
pipe.
Tests:
- Run unit tests
- Functional tests for other apps pass
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Currently, privileged actions are not allowed under top-level plinth module.
They are only allowed under each app module. Allow privileged actions under
plinth module.
- Currently, privileged actions are not allowed under a sub-module of
'privileged' package. They are allowed only in 'privileged' module. Allow
sub-modules under 'privileged' package.
Tests:
- Email app functional tests pass
- Functional tests for apps using package and service privileged methods pass
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
