- Add a new diagnostic check result for skipped tests.
Tests:
- Put a hold on a package. The diagnostic is failed.
- Remove the hold from the package. The diagnostic is passed.
- Start installing an app, then immediately run the upgrades
diagnostics. The diagnostic is skipped.
Helps: #2347
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Allow i18n for new state 'skipped']
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Not enabled by default currently. This can be changed after further
testing.
- Re-use existing operation from diagnostics run. However, this requires
changing the app_id of the operation for each app.
Tests:
- Enable automatic repair, and run diagnostics. See that repairs are
run.
- Enable automatic repair, and wait for daily diagnostics run. See that
repairs are run.
Closes: #2399.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
current_results['apps'] was not being used anywhere.
Make are_results_available() check more specific, and to align with what is
stored in the database.
Tests:
- Run full diagnostics and check the results.
- Restart plinth, and check that the diagnostic results are loaded from the
database.
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Display and log an error, including contents of current_results.
Clear the contents of current_results.
Tests:
- View the diagnostics results as normal.
- Introduce a TypeError before the results are copied. See the error message
shown in the interface. (The error details are cleared when the page is
refreshed.)
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Fix a mypy error]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
The option is stored in kvstore. If no value is set, it is assumed to be
enabled.
Tests:
- Disable daily run. In development mode, diagnostic are not run after several
minutes.
- Enable daily run. In development mode, diagnostics are run after several
minutes.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor refactoring and update messages in UI]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
It is best to have obtained the results lock when counting the severe failures
in diagnostic results.
Tests:
- When some packages are outdated, notification is shown with warning severity.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Run diagnostics. Restart plinth, and check that the diagnostics results are
still available to view.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- When running diagnostics for an app, if there are any failures or warnings,
then show a button to re-run setup.
- When showing all diagnostics results, if there are any failures or warnings
for an app, then show a button to re-run setup for that app.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Use Result class instead of strings for comparison]
[sunil: Use flex box's justify-content-between to improve button styling]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Print the interval time in schedule() method and verify that the times are as
expected in develop mode and production mode.
- Notification shows up for RAM usage if the check hardcoded to True.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- When running full diagnostics manually, we can use the Operation class. This
allows us to use many of its features.
- Ensure only one task is running at any time. No need to use running_task
global variable and a lock for it.
- Don't run the operation if app install/uninstall or other potentially
contentious tasks are running.
- Since Operation object creates a thread, don't create another one with
glib.schedule(). Don't wait unnecessarily for the operation to finish in the
glib thread (or glib created thread).
- Since the app will show progress of operations when an operation is running,
it would not be possible to show progress of diagnostics running. So, create a
separate page for diagnostics results.
Tests:
- Run diagnostics and see redirection happens to diagnostics results page.
Results page shows ongoing tests. It refreshes automatically. When tests are
completed, 'Re-run diagnostics' button is shown.
- When visiting /diagnostics/full/ URL is visited without running the test. Only
the re-run button is shown. No results are shown. If tests have been run, re-run
button along with results are shown.
- On the app page, if the tests have been run, a button for viewing results is
shown. Otherwise, the button is not shown.
- In development mode, background diagnostics are run after 3 minutes (change
the time to 150 seconds if database locked errors show up). Results are
available in the results page.
- Make a diagnostic test fail by changing code or disabling a daemon. Run
diagnostics and notice that a notification is shown with the button to go to the
results. Clicking on the button shows the results page. Clicking dismiss removes
the notification.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- One notification is shown with a count of the highest severity issues.
- Un-dismiss the notification, so it is shown even if previously dismissed.
- Add link to see the results, which are stored in a global variable.
- Add a lock for running_task.
Tests:
- Notification with 2 warnings shown on stable container due, to packages not
upgraded.
- Change the firewalld default zone to public. After the next run, the
notification changes to an error, and shows 1 failure.
Helps #2366.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable/disable button is not present.
- Diagnostics menu item is not present.
- Page shows 'Running diagnostics' button when diagnostics are not running.
- Page shows progress bar and results when diagnostics are running/completed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is needed for the apps to get enabled soon after installation. In case of
'sharing' app, a shortcut will appear on the apps page. This also brings
uniformity to help later refactoring.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When a print message is added in low disk space checking method, it is printed
every 3 minutes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #514.
Test: Ran diagnostics with all apps enabled.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Return a warning result if no domains are configured.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests performed:
- The diagnostics app page and individual app diagnostics pages have
app names.
- When the app name is missing (the apache app), app id is shown instead.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Fixes the problem when the app throws an error when running diagnostics,
the app's diagnostics page shows an html 500 error and on the diagnostics
app page, only an excpetion message is shown without other apps diagnostics.
(I encountered this when the network app diagnostics threw an exception
because of a particular network configuration.)
Tests performed:
- Checked that an app diagnostics page and the diagnostics app shows
all info correctly.
- Modified the networks app diagnostics code so that it throws an exception.
Checked that the app diagnostics page shows an exception message and
the diagnostics module shows all the apps and the networks app section
shows an exception message.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Re-add removed relevant comment]
[sunil: Remove ability to dismiss the exception message, not much gain for user]
[sunil: Remove unnecessary redundant if condition in template]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
Fixed application of available translations in daemon.py and apache,
diagnostics, networks, firewall and users modules.
diagnostics:
- __init__.py: return the app name along its results.
- diagnostics.html: display the app name instead of its id.
- diagnostics_results.html:
- mark for translation,
- apply class to results <td> HTML tag.
main.css: center-align the results.
Locale files excluded. Will be regenerated automatically and translations to be
done via Weblate.
original testing (rebased later):
- Yapf applied.
- Flake8 without errors or warnings for changed files.
- (Unit) tests run without errors.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Translate 'None' app name]
[sunil: Don't translate tests strings second time in template]
[sunil: Tweak the center rule]
[sunil: Don't split a translation string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Containers specific case: if total memory taken from cgroups is lower
than system memory taken from psutil, calculate memory usage based on
information from cgroups. The formula idea is taken from
https://github.com/moby/moby/issues/40727#issuecomment-604155288Closes#1780
Tests performed:
- In a non-container environment, filled the memory 90%
```
stress-ng --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.9;}' \
< /proc/meminfo)k --vm-keep -m 1
```
and ensured that correct notification is shown.
- In a container, if no memory limitations are set, notifications are
based on host memory usage
- In a container, if memory limits are set
```
systemctl set-property systemd-nspawn@fbx-testing.service MemoryMax=200M
```
ensured that the notification is shown and is calculated based on
cgroups.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Fix i18n for notification message]
[sunil: Drop unnecessary type conversion]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
backups/forms.py:
- ChoiceField labeled to allow translation.
- Translation applied to hard coded literals.
config/forms.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
diagnostics_results.html:
Apply translation to results. Use gettext_noop to mark for translation.
dynamicdns/forms.py:
Apply translation to choice literals.
i2p/views.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
names.html:
Apply translation to table headers.
performance/__init__.py:
Apply translation to description literals.
radicale/forms.py:
ChoiceField labeled to allow translation.
users/forms.py:
CharField labeled to allow translation.
QA:
- Literals visually verified.
- No errors in py.test-3.
- Yapf applied (only) to changed files.
- No remarks by flake8 to changed file.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Separate out the translations]
[sunil: Fix i18n for diagnostics]
[sunil: dynamicdns: Also do i18n for string GnuDIP]
[sunil: searx: Revert an incorrect removal of import]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests performed:
- Visit diagnostics app.
- Run diagnostics for all apps.
- Go to privoxy app and run diagnostics.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Curl: if an IPv6 zone index is specified, use --interface parameter.
This fixes two issues:
- Curl in Debian Buster doesn't support an IPv6 zone index in URL
- Curl in Debian Bullseye doesn't redirect properly if a zone index is in URL.
- Disable IPv6 diagnostics on daemons that don't listen on an IPv6 address:
plinth, deluge, transmission
Closes#1519
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Introduce base class for all apps that will contain components. With
unittests.
- Introduce base classes for components. With unittests.
- Turn Menu class into an app component.
- Further cleanup Menu class.
- Update tests.
- Maintain a global list of menu items and look them up easily. Generalize
such that subsubmenus can later be merged into Menu class.
- Cleanup scope of main menu initialization.
- Use None instead of empty strings for various values. Ensure that
printing short_description does not show 'None' in output.
- Use enable/disable instead of promote/demote.
- Use menu component in all apps.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
