Due the frequency and length of execution of diagnostics, the service does not
restart when files are modified. The operation also makes other testing tasks
wait until completed. It also makes functional tests slower. So, reduce the
frequency of execution. It can always be temporarily changed when debugging
diagnostics operations is necessary.
Tests:
- Change the development interval to 18 seconds and notice that new interval is
effective in development mode but not in production mode.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: #2414.
- Remove separate implementation for showing logs in help page.
- Add link to the page in diagnostics app so that FreedomBox logs can more
easily be discovered by users.
Tests:
- Raise an exception in the common error middleware to cause a 500 internal
server error. Run FreedomBox service with the --develop option and notice that
500 error page is served. There, the link to logs page works. It shows the logs
for Diagnostics app.
- Diagnostics page description is update. Link to logs page works.
- Diagnostics page shows logs for plinth.service and freedombox-develop.service.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Fixes: #2531.
- Currently, when a diagnostics test is skipped, the notification shows up with
'error' severity. Instead of this, treat 'skipped' and 'not done' tests as
passed for the purpose of the notification.
Tests:
- Change code in package.py to set the result of diagnostic test to be 'SKIPPED'
and 'NOT_DONE', in both cases, the notification is not shown after running full
diagnostic runs.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Uses Bootstrap accordion class to do collapsible sections without adding any
custom CSS or JavaScript.
Closes#2479
Sunil:
- Create one accordion instead of many. Automatically collapsing previously
expanded item works.
- Fix dangling </section> close tag.
- Embrace accordion styling instead of header-like styling for headers. The
tables with results are distinguished from the header due to header
highlighting and margins around tables.
- Fix issue with multiple 'passed' badges show for single app. 'regroup'
template tag expects the dict to be already sorted by the selected property.
- Internationalize badge text in headers.
- Right align badges. Move repair button into the accordion header for better
appearance.
- Wrap the header on small screen sizes.
- Add additional necessary HTML attributes.
- Change 'Loading...' to 'Running...' more accurately specify the status.
- Show Running and Exception statuses in header.
- Use 'text-bg-' classes instead of 'bg-' for allow automatic selection of text
color.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add typing information for init methods Info, Shortcut, and Menu to easily
identify problems.
- Update docstrings for these components.
- Updates test cases to deal with tags instead of short description.
- Update custom shortcuts code to read tags and ignore short description.
- Update API to send tags instead of custom shortcuts.
- OpenVPN special treatment of info.description in Shortcut
Tests:
- All unit tests pass and type checking succeeds.
- All apps show icons with tags in apps and system section.
- In help section cards don't show tags.
- In front page, enabled apps show shortcuts with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Currently, if an error occurred during diagnostics, just error
message (without traceback details) was supposed to be shown on the results
page. However, due to a bug in code related to showing repair button, a separate
exception is raised.
- Simplify the code by dropping all custom error display. Instead allow the
generic error display mechanism in the middleware to handle the error. This
keeps the code simple.
Tests:
- Raise an exception in diagnose() method of the 'users' app. Run the
diagnostics for the users app. Notice that Diagnostics app page is shown with
error alert containing full traceback details.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Wrap each app's results in a <section>. Style section to maintain gap with
previous sections.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- A 'row' class is more suitable when we want to use the Bootstrap's 'Grid'
system. Bootstrap 5 sets 'width: 100%' on all children of a .row. So, use a
simpler 'display: flex' instead.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Ensure that .sr-only is replaced with newer classes.
- Ensure that icons are present for all alerts.
- Use flex-box for display of icons on the left center of the alert.
- .close has been renamed to .btn-close.
- × is no longer required for close buttons.
Tests:
- Visually verify all the changes by triggering them with code changes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Keep the badge-{severity} classes as they are meant for convenience of mapping
severity to style.
- Match bootstrap colors to maintain consistency and contrast.
Tests:
- Visually confirm that all the changes effective.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Add a new diagnostic check result for skipped tests.
Tests:
- Put a hold on a package. The diagnostic is failed.
- Remove the hold from the package. The diagnostic is passed.
- Start installing an app, then immediately run the upgrades
diagnostics. The diagnostic is skipped.
Helps: #2347
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Allow i18n for new state 'skipped']
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Not enabled by default currently. This can be changed after further
testing.
- Re-use existing operation from diagnostics run. However, this requires
changing the app_id of the operation for each app.
Tests:
- Enable automatic repair, and run diagnostics. See that repairs are
run.
- Enable automatic repair, and wait for daily diagnostics run. See that
repairs are run.
Closes: #2399.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
current_results['apps'] was not being used anywhere.
Make are_results_available() check more specific, and to align with what is
stored in the database.
Tests:
- Run full diagnostics and check the results.
- Restart plinth, and check that the diagnostic results are loaded from the
database.
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Display and log an error, including contents of current_results.
Clear the contents of current_results.
Tests:
- View the diagnostics results as normal.
- Introduce a TypeError before the results are copied. See the error message
shown in the interface. (The error details are cleared when the page is
refreshed.)
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Fix a mypy error]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
The option is stored in kvstore. If no value is set, it is assumed to be
enabled.
Tests:
- Disable daily run. In development mode, diagnostic are not run after several
minutes.
- Enable daily run. In development mode, diagnostics are run after several
minutes.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor refactoring and update messages in UI]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
It is best to have obtained the results lock when counting the severe failures
in diagnostic results.
Tests:
- When some packages are outdated, notification is shown with warning severity.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Run diagnostics. Restart plinth, and check that the diagnostics results are
still available to view.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- When running diagnostics for an app, if there are any failures or warnings,
then show a button to re-run setup.
- When showing all diagnostics results, if there are any failures or warnings
for an app, then show a button to re-run setup for that app.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Use Result class instead of strings for comparison]
[sunil: Use flex box's justify-content-between to improve button styling]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Print the interval time in schedule() method and verify that the times are as
expected in develop mode and production mode.
- Notification shows up for RAM usage if the check hardcoded to True.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- When running full diagnostics manually, we can use the Operation class. This
allows us to use many of its features.
- Ensure only one task is running at any time. No need to use running_task
global variable and a lock for it.
- Don't run the operation if app install/uninstall or other potentially
contentious tasks are running.
- Since Operation object creates a thread, don't create another one with
glib.schedule(). Don't wait unnecessarily for the operation to finish in the
glib thread (or glib created thread).
- Since the app will show progress of operations when an operation is running,
it would not be possible to show progress of diagnostics running. So, create a
separate page for diagnostics results.
Tests:
- Run diagnostics and see redirection happens to diagnostics results page.
Results page shows ongoing tests. It refreshes automatically. When tests are
completed, 'Re-run diagnostics' button is shown.
- When visiting /diagnostics/full/ URL is visited without running the test. Only
the re-run button is shown. No results are shown. If tests have been run, re-run
button along with results are shown.
- On the app page, if the tests have been run, a button for viewing results is
shown. Otherwise, the button is not shown.
- In development mode, background diagnostics are run after 3 minutes (change
the time to 150 seconds if database locked errors show up). Results are
available in the results page.
- Make a diagnostic test fail by changing code or disabling a daemon. Run
diagnostics and notice that a notification is shown with the button to go to the
results. Clicking on the button shows the results page. Clicking dismiss removes
the notification.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- One notification is shown with a count of the highest severity issues.
- Un-dismiss the notification, so it is shown even if previously dismissed.
- Add link to see the results, which are stored in a global variable.
- Add a lock for running_task.
Tests:
- Notification with 2 warnings shown on stable container due, to packages not
upgraded.
- Change the firewalld default zone to public. After the next run, the
notification changes to an error, and shows 1 failure.
Helps #2366.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Enable/disable button is not present.
- Diagnostics menu item is not present.
- Page shows 'Running diagnostics' button when diagnostics are not running.
- Page shows progress bar and results when diagnostics are running/completed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is needed for the apps to get enabled soon after installation. In case of
'sharing' app, a shortcut will appear on the apps page. This also brings
uniformity to help later refactoring.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When a print message is added in low disk space checking method, it is printed
every 3 minutes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #514.
Test: Ran diagnostics with all apps enabled.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Return a warning result if no domains are configured.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
