- Not enabled by default currently. This can be changed after further
testing.
- Re-use existing operation from diagnostics run. However, this requires
changing the app_id of the operation for each app.
Tests:
- Enable automatic repair, and run diagnostics. See that repairs are
run.
- Enable automatic repair, and wait for daily diagnostics run. See that
repairs are run.
Closes: #2399.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
current_results['apps'] was not being used anywhere.
Make are_results_available() check more specific, and to align with what is
stored in the database.
Tests:
- Run full diagnostics and check the results.
- Restart plinth, and check that the diagnostic results are loaded from the
database.
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Display and log an error, including contents of current_results.
Clear the contents of current_results.
Tests:
- View the diagnostics results as normal.
- Introduce a TypeError before the results are copied. See the error message
shown in the interface. (The error details are cleared when the page is
refreshed.)
Helps: #2410
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Fix a mypy error]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
The option is stored in kvstore. If no value is set, it is assumed to be
enabled.
Tests:
- Disable daily run. In development mode, diagnostic are not run after several
minutes.
- Enable daily run. In development mode, diagnostics are run after several
minutes.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor refactoring and update messages in UI]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
It is best to have obtained the results lock when counting the severe failures
in diagnostic results.
Tests:
- When some packages are outdated, notification is shown with warning severity.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Run diagnostics. Restart plinth, and check that the diagnostics results are
still available to view.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- When running diagnostics for an app, if there are any failures or warnings,
then show a button to re-run setup.
- When showing all diagnostics results, if there are any failures or warnings
for an app, then show a button to re-run setup for that app.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Use Result class instead of strings for comparison]
[sunil: Use flex box's justify-content-between to improve button styling]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Print the interval time in schedule() method and verify that the times are as
expected in develop mode and production mode.
- Notification shows up for RAM usage if the check hardcoded to True.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- When running full diagnostics manually, we can use the Operation class. This
allows us to use many of its features.
- Ensure only one task is running at any time. No need to use running_task
global variable and a lock for it.
- Don't run the operation if app install/uninstall or other potentially
contentious tasks are running.
- Since Operation object creates a thread, don't create another one with
glib.schedule(). Don't wait unnecessarily for the operation to finish in the
glib thread (or glib created thread).
- Since the app will show progress of operations when an operation is running,
it would not be possible to show progress of diagnostics running. So, create a
separate page for diagnostics results.
Tests:
- Run diagnostics and see redirection happens to diagnostics results page.
Results page shows ongoing tests. It refreshes automatically. When tests are
completed, 'Re-run diagnostics' button is shown.
- When visiting /diagnostics/full/ URL is visited without running the test. Only
the re-run button is shown. No results are shown. If tests have been run, re-run
button along with results are shown.
- On the app page, if the tests have been run, a button for viewing results is
shown. Otherwise, the button is not shown.
- In development mode, background diagnostics are run after 3 minutes (change
the time to 150 seconds if database locked errors show up). Results are
available in the results page.
- Make a diagnostic test fail by changing code or disabling a daemon. Run
diagnostics and notice that a notification is shown with the button to go to the
results. Clicking on the button shows the results page. Clicking dismiss removes
the notification.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps in retrieving an operation that is currently running.
- Prevent starting an operation that is already running.
Tests:
- Unit tests work.
- Installing, uninstalling an app works.
- For upgrading an app works.
- Running background diagnostics works.
- Updating tor configuration works.
- Updating torproxy configuration works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- One notification is shown with a count of the highest severity issues.
- Un-dismiss the notification, so it is shown even if previously dismissed.
- Add link to see the results, which are stored in a global variable.
- Add a lock for running_task.
Tests:
- Notification with 2 warnings shown on stable container due, to packages not
upgraded.
- Change the firewalld default zone to public. After the next run, the
notification changes to an error, and shows 1 failure.
Helps #2366.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Enable/disable button is not present.
- Diagnostics menu item is not present.
- Page shows 'Running diagnostics' button when diagnostics are not running.
- Page shows progress bar and results when diagnostics are running/completed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is needed for the apps to get enabled soon after installation. In case of
'sharing' app, a shortcut will appear on the apps page. This also brings
uniformity to help later refactoring.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When a print message is added in low disk space checking method, it is printed
every 3 minutes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #514.
Test: Ran diagnostics with all apps enabled.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Return a warning result if no domains are configured.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- attr() CSS function can't used practically on anything but the content
property.
- Introduce a hack for setting any arbitrary percentage width on an element
similar to w-25, w-50, w-75 and w-100 bootstrap utilities. This hack should not
be used widely.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Rows in bootstrap 4 tables are taller by default. This is better suited for
mobile layouts and look prettier on desktops too. Adopting this approach instead
of condensed tables eliminates the need for striping, bordering and narrower
tables.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: #1818. The new navbar collapse handles items better without a scrollbar.
- Switch dependency to newer bootstrap4 library.
- Bootstrap 4 has 5 grid levels instead of 4. Adjust all classes accordingly to
retain current semantics.
- Update for newer markup for dropdowns.
- Replace use for labels with badges. Labels have been removed. Use
badge-secondary instead of label-default.
- Replace use of btn-xs with btn-sm. Size is roughly the same.
- Accommodate removal of form-horizontal.
- .hidden* and .visible* classes have been removed. Use alternate utilities.
- Replace use of table-condensed with table-sm.
- Update progress bar background styling. .progress-bar-* have been replaced
with bg-*.
- Drop support for IE9 and below. Bootstrap 4 does not support those browsers.
- Use the new simplified markup for navs, navbar and navbar-toggler. Accommodate
removal of .navbar-fixed-top.
- Update the JSXC page too.
- Update all values that are using rem units. The base font size is now 1rem =
16px instead of 1rem = 10px. This the default for bootstrap 4.
- A button inside a dropdown menu is natively supported by bootstrap 4. Remove
custom styling done earlier.
- Drop use of removed class thumbnail.
- Override colors for buttons and warnings to keep the old style.
- Use new markup for close buttons inside alerts.
- Use .collapse.show instead of .collapse.in as per bootstrap 4 styling.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests performed:
- The diagnostics app page and individual app diagnostics pages have
app names.
- When the app name is missing (the apache app), app id is shown instead.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Fixes the problem when the app throws an error when running diagnostics,
the app's diagnostics page shows an html 500 error and on the diagnostics
app page, only an excpetion message is shown without other apps diagnostics.
(I encountered this when the network app diagnostics threw an exception
because of a particular network configuration.)
Tests performed:
- Checked that an app diagnostics page and the diagnostics app shows
all info correctly.
- Modified the networks app diagnostics code so that it throws an exception.
Checked that the app diagnostics page shows an exception message and
the diagnostics module shows all the apps and the networks app section
shows an exception message.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Re-add removed relevant comment]
[sunil: Remove ability to dismiss the exception message, not much gain for user]
[sunil: Remove unnecessary redundant if condition in template]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
Fixed application of available translations in daemon.py and apache,
diagnostics, networks, firewall and users modules.
diagnostics:
- __init__.py: return the app name along its results.
- diagnostics.html: display the app name instead of its id.
- diagnostics_results.html:
- mark for translation,
- apply class to results <td> HTML tag.
main.css: center-align the results.
Locale files excluded. Will be regenerated automatically and translations to be
done via Weblate.
original testing (rebased later):
- Yapf applied.
- Flake8 without errors or warnings for changed files.
- (Unit) tests run without errors.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Translate 'None' app name]
[sunil: Don't translate tests strings second time in template]
[sunil: Tweak the center rule]
[sunil: Don't split a translation string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Containers specific case: if total memory taken from cgroups is lower
than system memory taken from psutil, calculate memory usage based on
information from cgroups. The formula idea is taken from
https://github.com/moby/moby/issues/40727#issuecomment-604155288Closes#1780
Tests performed:
- In a non-container environment, filled the memory 90%
```
stress-ng --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.9;}' \
< /proc/meminfo)k --vm-keep -m 1
```
and ensured that correct notification is shown.
- In a container, if no memory limitations are set, notifications are
based on host memory usage
- In a container, if memory limits are set
```
systemctl set-property systemd-nspawn@fbx-testing.service MemoryMax=200M
```
ensured that the notification is shown and is calculated based on
cgroups.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Fix i18n for notification message]
[sunil: Drop unnecessary type conversion]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #1938.
backups/forms.py:
- ChoiceField labeled to allow translation.
- Translation applied to hard coded literals.
config/forms.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
diagnostics_results.html:
Apply translation to results. Use gettext_noop to mark for translation.
dynamicdns/forms.py:
Apply translation to choice literals.
i2p/views.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
names.html:
Apply translation to table headers.
performance/__init__.py:
Apply translation to description literals.
radicale/forms.py:
ChoiceField labeled to allow translation.
users/forms.py:
CharField labeled to allow translation.
QA:
- Literals visually verified.
- No errors in py.test-3.
- Yapf applied (only) to changed files.
- No remarks by flake8 to changed file.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Separate out the translations]
[sunil: Fix i18n for diagnostics]
[sunil: dynamicdns: Also do i18n for string GnuDIP]
[sunil: searx: Revert an incorrect removal of import]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Click on diagnostics menu item in any app. Notice that running status does not
show up.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
