- When running full diagnostics manually, we can use the Operation class. This
allows us to use many of its features.
- Ensure only one task is running at any time. No need to use running_task
global variable and a lock for it.
- Don't run the operation if app install/uninstall or other potentially
contentious tasks are running.
- Since Operation object creates a thread, don't create another one with
glib.schedule(). Don't wait unnecessarily for the operation to finish in the
glib thread (or glib created thread).
- Since the app will show progress of operations when an operation is running,
it would not be possible to show progress of diagnostics running. So, create a
separate page for diagnostics results.
Tests:
- Run diagnostics and see redirection happens to diagnostics results page.
Results page shows ongoing tests. It refreshes automatically. When tests are
completed, 'Re-run diagnostics' button is shown.
- When visiting /diagnostics/full/ URL is visited without running the test. Only
the re-run button is shown. No results are shown. If tests have been run, re-run
button along with results are shown.
- On the app page, if the tests have been run, a button for viewing results is
shown. Otherwise, the button is not shown.
- In development mode, background diagnostics are run after 3 minutes (change
the time to 150 seconds if database locked errors show up). Results are
available in the results page.
- Make a diagnostic test fail by changing code or disabling a daemon. Run
diagnostics and notice that a notification is shown with the button to go to the
results. Clicking on the button shows the results page. Clicking dismiss removes
the notification.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- One notification is shown with a count of the highest severity issues.
- Un-dismiss the notification, so it is shown even if previously dismissed.
- Add link to see the results, which are stored in a global variable.
- Add a lock for running_task.
Tests:
- Notification with 2 warnings shown on stable container due, to packages not
upgraded.
- Change the firewalld default zone to public. After the next run, the
notification changes to an error, and shows 1 failure.
Helps #2366.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable/disable button is not present.
- Diagnostics menu item is not present.
- Page shows 'Running diagnostics' button when diagnostics are not running.
- Page shows progress bar and results when diagnostics are running/completed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is needed for the apps to get enabled soon after installation. In case of
'sharing' app, a shortcut will appear on the apps page. This also brings
uniformity to help later refactoring.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When a print message is added in low disk space checking method, it is printed
every 3 minutes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #514.
Test: Ran diagnostics with all apps enabled.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Return a warning result if no domains are configured.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests performed:
- The diagnostics app page and individual app diagnostics pages have
app names.
- When the app name is missing (the apache app), app id is shown instead.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Fixes the problem when the app throws an error when running diagnostics,
the app's diagnostics page shows an html 500 error and on the diagnostics
app page, only an excpetion message is shown without other apps diagnostics.
(I encountered this when the network app diagnostics threw an exception
because of a particular network configuration.)
Tests performed:
- Checked that an app diagnostics page and the diagnostics app shows
all info correctly.
- Modified the networks app diagnostics code so that it throws an exception.
Checked that the app diagnostics page shows an exception message and
the diagnostics module shows all the apps and the networks app section
shows an exception message.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Re-add removed relevant comment]
[sunil: Remove ability to dismiss the exception message, not much gain for user]
[sunil: Remove unnecessary redundant if condition in template]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
Fixed application of available translations in daemon.py and apache,
diagnostics, networks, firewall and users modules.
diagnostics:
- __init__.py: return the app name along its results.
- diagnostics.html: display the app name instead of its id.
- diagnostics_results.html:
- mark for translation,
- apply class to results <td> HTML tag.
main.css: center-align the results.
Locale files excluded. Will be regenerated automatically and translations to be
done via Weblate.
original testing (rebased later):
- Yapf applied.
- Flake8 without errors or warnings for changed files.
- (Unit) tests run without errors.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Translate 'None' app name]
[sunil: Don't translate tests strings second time in template]
[sunil: Tweak the center rule]
[sunil: Don't split a translation string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Containers specific case: if total memory taken from cgroups is lower
than system memory taken from psutil, calculate memory usage based on
information from cgroups. The formula idea is taken from
https://github.com/moby/moby/issues/40727#issuecomment-604155288Closes#1780
Tests performed:
- In a non-container environment, filled the memory 90%
```
stress-ng --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.9;}' \
< /proc/meminfo)k --vm-keep -m 1
```
and ensured that correct notification is shown.
- In a container, if no memory limitations are set, notifications are
based on host memory usage
- In a container, if memory limits are set
```
systemctl set-property systemd-nspawn@fbx-testing.service MemoryMax=200M
```
ensured that the notification is shown and is calculated based on
cgroups.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Fix i18n for notification message]
[sunil: Drop unnecessary type conversion]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
backups/forms.py:
- ChoiceField labeled to allow translation.
- Translation applied to hard coded literals.
config/forms.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
diagnostics_results.html:
Apply translation to results. Use gettext_noop to mark for translation.
dynamicdns/forms.py:
Apply translation to choice literals.
i2p/views.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
names.html:
Apply translation to table headers.
performance/__init__.py:
Apply translation to description literals.
radicale/forms.py:
ChoiceField labeled to allow translation.
users/forms.py:
CharField labeled to allow translation.
QA:
- Literals visually verified.
- No errors in py.test-3.
- Yapf applied (only) to changed files.
- No remarks by flake8 to changed file.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Separate out the translations]
[sunil: Fix i18n for diagnostics]
[sunil: dynamicdns: Also do i18n for string GnuDIP]
[sunil: searx: Revert an incorrect removal of import]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests performed:
- Visit diagnostics app.
- Run diagnostics for all apps.
- Go to privoxy app and run diagnostics.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Curl: if an IPv6 zone index is specified, use --interface parameter.
This fixes two issues:
- Curl in Debian Buster doesn't support an IPv6 zone index in URL
- Curl in Debian Bullseye doesn't redirect properly if a zone index is in URL.
- Disable IPv6 diagnostics on daemons that don't listen on an IPv6 address:
plinth, deluge, transmission
Closes#1519
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Introduce base class for all apps that will contain components. With
unittests.
- Introduce base classes for components. With unittests.
- Turn Menu class into an app component.
- Further cleanup Menu class.
- Update tests.
- Maintain a global list of menu items and look them up easily. Generalize
such that subsubmenus can later be merged into Menu class.
- Cleanup scope of main menu initialization.
- Use None instead of empty strings for various values. Ensure that
printing short_description does not show 'None' in output.
- Use enable/disable instead of promote/demote.
- Use menu component in all apps.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
There might be data to backup in future if we allow configuration. Currently, it
is just an assurance to the user that there is no data to backup.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes#930
- Make targets to download wiki pages of each service
- Add post-processor script for DocBook file processing
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove apps and system modules and merge their views into main views.
- Move main_menu from cfg into menu.py.
- Remove dependencies of other modules on apps and system modules.
- Update tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Currently menu items are shown in alphabetical order in applications and
no clear order in system configuration. This is done using static
weights for menu items based on English names that does not work for
other locales.
Sorting can't be done at the time of adding menu items as users of
multiple locales may use the interface at the same time.
Implement a sorting mechanism based on existing order as well as labels
of menu item. This allows the flexiblity of grouping menu items in
future as it may be need for system configuration. In case of help menu
Remove sort order for all modules except for help menu as here we want
that specific order.
wget does not seem to support interface scoping for IPv6 addresses. For
example, http_proxy=http://[fe80::babe:ff:ffff:babe%eth0]:8118/ . Curl
supports this.
This fixes most of the failures for IPv6 related addresses. Hide the
last failure as there does not seem to be a proper fix from OS level:
when using link local addresses, if a hostname is resolved to IPv6 link
local address, it is not scoped to that interface. It can't properly be
used by any tool then.
- The last part of the module import path is the module name. This also
becomes the Django app name. Apps names have to be unique. Hence,
there is no scope for two different modules with same name but
different load path to exist in the project.
- Most uses of list of loaded modules are dealing with app names instead
of full module load path. This is due to the fact that Django deals
with app names and not module paths.
- It is also somewhat clumsy to access a loaded module as we are
re-importing every time to get access module.
- Simplify all of the above by using app names are module identifiers
and maintaing an ordered dictionary of app names to loadded modules.
- Remove unused imports.
- Minor styling fixes.
- Note that while Plinth is running or not running, diagnostics can be
run on the command line with --diagnose option to Plinth binary. That
makes these tests just as useful as freedombox-setup testsuite.
