- We are using submenu.url to check for specific URLs and then highlight a menu
item. This is somewhat incorrect due to string search and not generic enough. We
have another mechanism 'active_menu_urls' to perform this. Improve and use this
instead.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Don't use title casing, instead use simple capitalization.
- Add some tags.
- Drop outdated tags like 'VoIP', 'IM' while emphasizing 'Audio chat', 'Video
chat', 'Encrypted messaging' instead.
- Try to clarify server vs. web client with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add tags to Info component of apps. Use only English tags for all operations.
Localized tags are used for presentation to the user only. Add tags to all the
apps. Conventions (English):
1. Tags describing use cases should be in kebab case.
2. Protocols in tag names should be in their canonical format.
3. Tags needn't be 100% technically correct. This can get in the way of
comparing apps using a tag. Words that describe use cases that users can
easily understand should be preferred over being pedantic.
4. Tags should be short, ideally not more than 2 words. Avoid conjunctions
like "and", "or" in tags.
5. Avoid redundant words like "server", or "web-clients". Most apps on
FreedomBox are either servers or web clients.
6. Keep your nouns singular in tags.
- Use query
params to filter the Apps page by tags. When all tags are removed, redirect to /apps.
- Add UI elements to add and remove tag filters in the Apps page. Make the UI
similar to GitLab issue tags. Since there are 40 apps, there will be at least 40
tags. Selecting a tag from a dropdown will be difficult on mobile devices. A
fuzzy search is useful to find tags to add to the filter. Allow user to find the
best match for the search term and highlight it visually. The user can then
press Enter to select the highlighted tag. Make tag search case-insensitive.
Make the dropdown menu scrollable with a fixed size. User input is debounced by
300 ms during search.
- tests: Add missing mock in test_module_loader.py
- Add functional tests
[sunil]
- 'list' can be used instead of 'List' for typing in recent Python versions.
- Reserve tripe-quoted strings for docstrings.
- Undo some changes in module initialization, use module_name for logging
errors.
- isort and yapf changes.
- Encode parameters before adding them to the URL.
Tests:
- Tested the functionality of filtering by tag with one tag and two tags.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- We have not yet implemented the main reason they exist. To guide users to
establish reachability with Tor hidden services, Pagekite, Dynamic DNS, etc.
- We now have a 'Next steps' page that talks about configuring network
connections. The networks page linked from here has these steps prominently
listed.
- In the future we will implement a wizard for reachability and these steps will
still be used. However, they don't have to part of first setup. They can add
them as notification and as part of next steps page.
- It is good to have a simplified first setup wizard. It is seldom tested
properly.
Tests:
- Run the first setup wizard by removing /var/lib/plinth/plinth.sqlite3 and
running the service. Notice that the software update step is not shown and
wizard completes successfully.
[vexch: Minor quote fix in functional tests]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Remove the first setup wizard step to run security upgrades. At the time of
its introduction, it was felt that this is very important. Some things have
changed since then:
- We have mechanism for queuing package operations. Users can now trigger
software updates and start installing apps before that is completed. Or vice
versa. Earlier if the software updates were running, app install used to fail
with an error.
- There were no notifications. Since then we have added 'first setup'
notification for important topics such as Privacy. This step can be replaced
with a notification.
- Automatic diagnostics and a diagnostic to notify of updated packages also
helps bring attention to software updates if they are missed during first
setup.
- A proposed change will re-introduce an advice to run updates in the 'Next
steps' wizard step along with a button trigger it right there.
- The new notification for software updates will bring more attention to running
updates as part of first setup.
- It would be nice not be stuck in the first setup wizard for a long period and
make it look simple. It improves the fun factor of setting up FreedomBox.
- It would present an opportunity to utilize the parallel installation of
apps/updates to the full extent. Although this can also be done by skipping the
progress step after updates are run.
- First wizard steps tend to get less testing.
Tests:
- Run the first setup wizard by removing /var/lib/plinth/plinth.sqlite3 and
running the service. Notice that the software update step is not shown and
wizard completes successfully.
- On stable container, backports step is shown as expected (if not already
enabled).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Fixes an error in testing container where installing dependencies for
functional tests wants to uninstall system installed urllib3.
Also minor quotes fixes.
Note that dependencies are upgraded only in new dev environments where
geckodriver is not yet installed.
Tests performed, in both stable and testing containers:
- Run all tests, no upgrade related failures found. There were some
seemingly unrelated test failures in apps: calibre, ejabberd, deluged,
minetest, users, transmission , wordpress, zoph.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reload a service if it supports reloading, otherwise restart. Do nothing if
service is not running.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Likely helps: #2271.
When web server restarts we are shown an error page. Trying to lookup and
element in this error page still raises StaleElementReferenceException. However,
if the page was reloaded with driver.visit(), then trying to lookup the old
element does not throw StaleElementReferenceException. Instead the
NoSuchElementException is thrown. For this case, ensure that we stop waiting
appropriately. This is likely to solve the large waits and timeouts when testing
dynamicdns.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Config app description is as expected.
- Config form does not show domain name field anymore.
- Submitting the form with changes works.
- Names app has correct link for configuring static domain name. Clicking it
takes to page for setting domain name.
- On startup, static domian name signal is sent properly if set. Otherwise no
signal is send.
- Change domain name form shows correct value for current domain name.
- Change domain name form sets the value for domain name properly.
- Page title is correct.
- Validations works.
- Add/remove domain name signals are sent properly.
- Success message as shown expected
- /etc/hosts is updated as expected.
- Unit tests work.
- Functional tests on ejabberd, letsencrypt, matrix, email, jsxc, openvpn
- After freshly starting the service. Visiting names app shows correct list of
domains.
- ejabberd:
- Installs works as expected. Currently set domain_name is setup properly.
Copy certificate happens on proper domain.
- Changing the domain sets the domain properly in ejabberd configuration.
- Ejabberd app page shows link to name services instead of config app.
Clicking works as expected.
- letsencrypt:
- When no domains are configured, the link to 'Configure domains' is to the
names app.
- matrix-synapse:
- Domain name is properly shown in the status.
- email:
- Primary domain name is shows properly in the app page.
- Setting new primary domain works.
- When installing, domain set as static domain name is prioritized as primary
domain.
- jsxc:
- Show the current static domain name in the domain field. BOSH server is
available.
- openvpn:
- Show the current static domain in profile is set otherwise show the current
hostname.
- If domain name is not set, downloaded OpenVPN profile shows hostname.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests:
- Config app description is as expected.
- Config form does not show hostname anymore.
- Submitting the form with changes works.
- Names app has correct link for configuring Local Domain Name. Clicking it
takes to page for setting hostname.
- Avahi shows the current .local domain correctly in Names app.
- Change hostname form shows correct value for current hostname.
- Change hostname form sets the value for hostname properly.
- Page title is correct.
- Validations works.
- Pre/post hostname change signals are sent properly
- Success message as shown expected
- hostnamectl shows the set domain
- If domain name is not set, downloaded OpenVPN profile shows hostname.
- Unit tests work.
- Functional tests on names/config/avahi apps work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Expose Network Manager per-connection setting for DNS-over-TLS. Support all
four values: default, no, opportunistic, and yes.
- Create a new collapsible section all 'Privacy' for this setting the connection
create/edit form. Strictly speaking this is related to security and censorship
resistance too.
- Don't show the DoT field for PPPoE connection types are DNS servers are not
relevant.
- Show the status of DoT for a connection in the connection status page.
Tests:
- In all Add New Connection forms except PPPoE form, the privacy
section shows up as expected.
- For each value for DoT, create a new connection and set the value for DoT to the
desired value and observe that the connection status page shows DoT to the set
value.
- For each value for DoT, edit an existing connection and set the value for the
DoT to the desired value and observe that the connection status page shows DoT
to the set value.
- Connection status page shows the values for DoT as expected.
- Update the primary Internet connection for the machine. Set the value to 'yes'
and notice that DNS resolutions fail. Set the value to 'opportunistic' or 'no'
and the DNS resolutions pass. In each case, 'resolvectl' shows the correct DoT
value for the connection. When 1.1.1.1 is set as DNS server, all values of DoT
in the connection succeed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Method parameters marked with secret_str will not be logged.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
For test cases implemented in derived classes, autouse=True does not seem to
work. These test cases can explicitly request this fixture with the name.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Sometimes <input type='submit'> are used as submit buttons and at other times
<button type='submit'> is used.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
In some cases in the backup page, when trying to restore a backup, the button is
obscured and trying to click it fails. Handle such situations by scrolling the
element into view first and try to click on the element again. Ensure that our
navigation header does not obscure the element when scrolling it into view.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When a click fails because the element is obscured or out of scroll port, the
fails with ElementClickInterceptedException. This normally fails the test
properly. However, when waiting for a page update, the wait loop silently
swallows the error causing the click failure to be ignored. This later leads to
a timeout waiting for page to be updated.
Handling this error separately and re-raising it ensures that the test fails
with the correct error allowing us to fix the problem.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Repair is run within an operation.
- Diagnostics are run for the app first.
- Call app.repair, then re-run setup if needed.
- Add helper functions for apps or components to store error messages in thread
local storage. These error messages are shown at the end.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Undo minor reformatting, due to automatic tool]
[sunil: Fix passing incorrect Exception argument to operation.on_update]
[sunil: Add full stop at the end of the success message to match install message]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Allows apps and component to implement custom repair methods.
- Default implementation asks relevant components to repair, and then if
needed, requests re-run setup for the app.
- Component.repair will return True by default, indicating that setup should be
re-run.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor docstring styling fixes]
[sunil: Improve tests for repair]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- During enable/disable of the component, status of the dropin configuration
files was being checked from non-privileged process. If the dropin configuration
file or its parent is readable only by root, then the operation fails. Fix this
by performing the status check in privileged mode.
Tests:
- Unit tests pass.
- Dropin configuration file for redis server works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2376.
Fixes: #2317.
When an app is removed, its packages are purged. However, there is another
installed app that needs these packages, we should keep those packages. We have
already implemented checking the packages against other apps' packages. However,
we are not checking if we are removing dependencies of other apps' packages.
This will still result in removal apps' packages.
To solve this problem, get list of packages of all the apps, then iterate over
their dependencies recursively and compile a comprehensive list of packages to
keep. Use this to reduce the set of packages to remove.
Tests:
- Without the patch, install bepasty and janus. Uninstall janus app and notice
that bepasty package is removed. With the patch, the problem is not observed.
- Printing the comprehensive list of packages to keep shows an extensive set
computed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Fixes an issue with zoph not being setup after uninstall+install setup and
makes a test pass.
- Some failures exist but don't seem related to this change.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This will help apps like zoph that need to have database server running to
install or upgrade a package.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Some apps store their configuration in database. The database server may not
be running when app is disabled. Configuration changes may then not be
possible for such apps. Provide the ability to disable configuration for apps
that don't support configuration changes when disabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2409.
When creating a user if one or more groups is selected, creation fails. This
is because the fields contains group choices as (name, label) tuples instead
of (group_id, label) tuples as expected by the many-to-many field mapping
mechanism in ModelField class. Fix this by using the same mechanism used in
UserUpdateForm, which is to reuse the base class form field (but adjust some
properties).
Tests:
- During first boot
- Django groups are fully created when form is accessed with blank database
- In user creation/modify form:
- Label appears are 'Permissions'
- Choices appear fully and as 'Description (Group name)'
- Help text is correct.
- Choices are sorted on group name.
- Django groups are fully created when form is accessed when a new group is
added to code.
- User can have no groups
- Widget is multiple checkbox widget. Multiple groups can be selected.
- User is added to proper ldap groups after submission
- In user modify form:
- If the user is last admin user, admin checkbox is checked and disabled.
- Current list of groups is accurate shown when form is displayed.
- Add remove of groups works as expected
- Functional tests for gitweb and users apps pass
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is useful for managing redis service needed by the upcoming Nextcloud
app.
- Disable the daemon only if all the apps using it are disabled. Enable it when
even one of the them is enabled.
- The component is not a 'leader' component as it does not decide the
enabled/disabled status of the app.
Tests:
- Unit tests pass.
- Install zoph and wordpress with full patch series. If one of the apps is
disabled, mysql service is still enabled and running. If both apps are disabled,
then mysql service is disabled and not running. Enabled/disabled status of apps
are accurate after they are enabled/disabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Otherwise, apps_init calls _sort_apps, which randomly fails in functional tests
CI job. The failure may be due to the tests running in parallel, since I could
not reproduce it locally.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
In Python 3.12, has_calls is not a valid assertion.
Closes: #1058421
Tests:
- Run "python3.12 -m pytest plinth/tests/test_operation.py" in unstable
container. All tests are passed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Using `--dist=loadscope` splits the tests based on module, thus
ensuring each app's tests get assigned to one worker only.
- Though app servers run in 2 separate VMs, the pytest-xdist workers
simply run in two Python subprocesses (`-n 2`). This allows us to
generate a unified test report.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
When setup is run from the application thread after startup, it continuously
tries until it succeeds. However, after making the first attempt, it does not
collect the status of the operation keeping the operation object in operation
manager. When trying for the second time, trying to create operation with same
ID fails since the operation is already present.
Fix this by allowing the operation to be recreated if the existing operation has
failed.
Tests:
- Unit tests pass.
- Functional tests for bepasty app pass.
- Install an app. Create an error in the setup mechanism for an app. Increment
is app version number. Start the service and notice that setup of app is
attempted and fails. Few seconds later the setup is attempted again and the
process continues. Each time the failure is due to fault in the app's setup
method rather than operation not being accepted.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
See db.py for rationale.
Tests:
- Run functional tests and unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Without the changes, the form submission for setting firewall zone fails with
error similar to "element can't be scrolled to view".
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
