- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Check that there are at least 12 direct passthroughs.
Tests:
- The diagnostic is passed.
- Manually remove a direct passthrough. The diagnostic is failed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Change the backend to iptables, and restart firewalld. The diagnostic
is failed.
- Change the backend back to nftables, and restart firewalld. The
diagnostic is passed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Use augeas transform operation]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Change the firewalld default zone to public, and restart firewalld.
The diagnostic is failed.
- Change the default zone back to external, and restart firewalld. The
diagnostic is passed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- After a dist-upgrade to Bookworm, install freedombox with this
change. Then update apt cache. After a delay, firewalld is
upgraded. Then apps can be installed as normal.
- On a stable machine, add testing into apt sources list. Run apt update.
FreedomBox will receive the apt update hook and upgrade firewall version to
1.3.x. DefaultZone is set to 'external' after that.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Update all the way up to 2.0]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Automatically handle a setup of the component getting added to an existing
app.
Tests:
- Run unit tests
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- On a fresh container, run FreedomBox service. Notice that firewall app setup
succeeds. Base setup rules are inserted into the nftables as checked with 'nft
list ruleset ip' and 'nft list ruleset ipv6'.
- When firewalld is restarted or reloaded, the rules are still present.
- When machine is restarted, the rules are still present.
- Without the patch, setup a container. Then apply patches and restart
FreedomBox service. App setup runs again however, duplicate rules are listed in
nftables as checked with 'nft list ruleset ip' and 'nft list ruleset ipv6'.
- Increment setup version of the firewall app manually and repeat the test.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- If a daemon is not-running, we already show an error message to the user. Use
that mechanism instead of the custom one.
Tests:
- Functional tests work.
- Initial setup for firewall on first boot works.
- Default zone of the firewalld is set to external in /etc/firewalld.conf
- Status of various apps is shown properly in the app page
- If firewalld is not running, the app page is still displayed properly and
message that firewalld is not running is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2253
Tests:
- Build a package with this fix included for bullseye-backports
distribution.
- Install into FreedomBox stable system.
- Start dist-upgrade test.
- At the end of the test, wait for plinth to attempt upgrades, and
then confirm that firewalld was upgraded.
- Confirm that firewalld is running.
- Confirm that /etc/firewalld/firewalld.conf has DefaultZone=external.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Also add CSS class for the table to aide functional tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2133.
There are two new defaults which need not be altered in FreedomBox:
CleanupModulesOnExit=no
(removed) AllowZoneDrifting=no
Tests:
- Without the patch, on a testing system with firewalld 0.9.x installed, run apt
update. FreedomBox considers firewalld for force upgrade and then ignores it as
shown in the logs.
- With the patch, firewalld is upgraded to 1.0.x version by FreedomBox using
force upgrade as shown in logs. After upgrade, the default zone is external and
backend is nftables.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Drop separate column for showing ports and add a dropdown icon next to the app
name.
- Manage widths so that table widths don't jump when expanded. Tested with
smaller screen size.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: #1818. The new navbar collapse handles items better without a scrollbar.
- Switch dependency to newer bootstrap4 library.
- Bootstrap 4 has 5 grid levels instead of 4. Adjust all classes accordingly to
retain current semantics.
- Update for newer markup for dropdowns.
- Replace use for labels with badges. Labels have been removed. Use
badge-secondary instead of label-default.
- Replace use of btn-xs with btn-sm. Size is roughly the same.
- Accommodate removal of form-horizontal.
- .hidden* and .visible* classes have been removed. Use alternate utilities.
- Replace use of table-condensed with table-sm.
- Update progress bar background styling. .progress-bar-* have been replaced
with bg-*.
- Drop support for IE9 and below. Bootstrap 4 does not support those browsers.
- Use the new simplified markup for navs, navbar and navbar-toggler. Accommodate
removal of .navbar-fixed-top.
- Update the JSXC page too.
- Update all values that are using rem units. The base font size is now 1rem =
16px instead of 1rem = 10px. This the default for bootstrap 4.
- A button inside a dropdown menu is natively supported by bootstrap 4. Remove
custom styling done earlier.
- Drop use of removed class thumbnail.
- Override colors for buttons and warnings to keep the old style.
- Use new markup for close buttons inside alerts.
- Use .collapse.show instead of .collapse.in as per bootstrap 4 styling.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Helps: #1938.
Fixed application of available translations in daemon.py and apache,
diagnostics, networks, firewall and users modules.
diagnostics:
- __init__.py: return the app name along its results.
- diagnostics.html: display the app name instead of its id.
- diagnostics_results.html:
- mark for translation,
- apply class to results <td> HTML tag.
main.css: center-align the results.
Locale files excluded. Will be regenerated automatically and translations to be
done via Weblate.
original testing (rebased later):
- Yapf applied.
- Flake8 without errors or warnings for changed files.
- (Unit) tests run without errors.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Translate 'None' app name]
[sunil: Don't translate tests strings second time in template]
[sunil: Tweak the center rule]
[sunil: Don't split a translation string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- When port forwarding configuration is not required in the router, don't show
the ports information.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Start showing port ranges properly.
- Fixes issue with Coturn TURN relay ports not being shown.
Closes: #1851.
Tests:
- Visit each of affected apps and see the port forwarding information. The
information is same as before.
- HTTP and HTTPS ports are not shown.
- Coturn app shows additional port ranges for TURN relay ports.
- Shadowsocks app does not show port forwarding information as it is internal
only.
- Visit one of the apps not effected by the patch. There is no section related
to port forwarding.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- We are mention in the internal zone warning that services are available when
connected over VPN.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Talk about cockpit in networking, firewall and storage apps for advanced
operations. Make it a separate section instead of just description update to
provider higher emphasis.
- Update cockpit description too.
Closes: #1809.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
firewalld has no problem retrieving service information for just-installed
files, it only fails when adding/removing those services to zones. A reload is
needed before those services can be used.
Don't perform firewalld reload during setup.py or debian/postinst. Instead
reload when firewalld throws an error that it does not know a service. This
approach is more minimally intrusive and does not run reload operations when no
services need to be loaded, during Debian package building etc.
Closes: #376.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Introduce new API to mark an app that it can't be disabled.
- Mark jsxc, storage, config, upgrade and firewall apps as can't be disabled.
- Fixed functional tests
- Replaced AppForm with forms.Form in all modules' forms.py.
- Remove app.template.js.
- Remove unused styles.
- Remove app status checks in form_valid of Deluge, Diaspora, Matrix, Ejabberd,
MediaWiki, Storage, Transmission, Quassel
- Purge unused is_enabled context variables (Ikiwiki)
- ejabberd: Minor cleanup in template
- jsxc: Cleanup unneeded overrides
- tahoe: Cleanup unnecessary overrides
Tests performed:
- For all apps affected, test enable/disable button works and submitting
configuration form works: with changes updates message and without changes
'settings unchanged' message.
- avahi
- bind
- cockpit
- SKIP: coquelicot
- datetime
- deluge
- SKIP: diaspora
- ejabberd
- gitweb
- i2p
- infinoted
- ikiwiki
- matrixsynapse
- mediawiki
- minetest
- minidlna
- mldonkey
- mumble
- pagekite
- privoxy
- quassel
- radicale
- roundcube
- SKIP: samba
- searx
- SKIP: shaarli
- shadowsocks
- ssh
- tahoe
- transmission
- FAIL: tt-rss (not installable)
- wireguard
- Deluge test that configuration changes when app is disabled work
- Quassel test that setting the domain works when app is diabled
- Transmission test that setting the domain works when app is diabled
- Ikiwiki create form works properly
- Enable/disable button appears as expected when enabled and when disabled
- Enable/disable button works without Javascript
- Functional tests work for affected apps, Tor and OpenVPN
- AppForm is removed from developer documentation
- Forms reference
- Customizing tutorial
- Test all apps using directory select form
- Transmission
- Deluge
- Visit each template that overrides block configuration and ensure that it is
loaded properly and the display is as expected.
- All apps that use AppView that are not tested above should not have an
enable/disable button. That is JSXC, update, config, firewall, storage, users.
Signed-off-by: Alice Kile <buoyantair@protonmail.com>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
This should help with FreedomBox getting authorizations to access firewalld
D-Bus API for polkit versions > 0.105 where polkit local authority is not
available.
Helps #1728.
Tests performed:
- Install policykit-1 version 0.105. Something like apt install
policykit-1=0.105-26 libpolkit-agent-1-0=0.105-26 libpolkit-gobject-1-0=0.105-26
- Visit privoxy page. See internal networks warning with all the interfaces
listed properly.
- Visit firewalld page, view list of apps and services properly.
- Enable privoxy, see the port opened in firewalld page.
- Disable privoxy, see the port closed in firewalld page.
- Run diagnostics for privoxy see that ports' diagnostics results pass.
- Add Debian experimental to /etc/apt/sources.list. apt update. Then install
policykit-1 version 0.116. apt install -t experimental policykit-1.
- Repeat all above tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: #1782.
This restores the API to the earlier behavior of ignoring errors when firewalld
is not running. This may not be the best approach however.
- Ignore errors on all operations when firewalld is not running. This include
enable/disable service, list interfaces of a zone, get service ports and get
enabled service.
- Log errors when firewalld is not running.
- ALREADY_ENABLE and NOT_ENABLED errors already ignored during enable/disable
service operations respectively.
Tests successfully executed:
- Turn off firewalld and perform following operations. They should all log an
error that firewalld is not running.
- Visit app page of an app that works only on internal networks such as
Privoxy. It should show that there are no internal networks.
- Run diagnostics of an app. It should show ports for a service as empty list.
- Enable/disable an app, it should enable/disable without failure.
- Visit firewall app page. It should show that firewalld is not running.
- Perform all the about operations without error message showing up and with
expected results.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Significantly reduce the time taken to perform the operations in the following
cases:
- Enable/disable an app
- list services in firewall page
- Load app page when interfaces belonging to internal zone need to shown
- First run setup of FreedomBox
- Install a polkit local authority configuration file to allow FreedomBox
service to perform: information queries and configuration changes.
- Drop unused actions.
- Alter the template for showing firewall port information since port/protocol
pairs are no longer pre-formatted.
- Handle errors when trying to get ports details of an unknown service.
Tests performed:
- Enable/disable an app. Ensure with firewall-cmd that ports are added/removed
properly.
- Temporarily modify code to call add_service() and remove_service() twice in a
row. Perform enable/disable operations and ensure that there are not error
thrown to test that add/remove services operations are idempotent.
- Visit the firewalld page and see the current state is reflected properly.
- Visit an app that shows the list of interfaces in firewall zone. Internal
interfaces should be listed properly.
- Reset the installed version of firewall app and disable all firewall services.
Start FreedomBox and ensure that when setup is re-run, default ports (http,
https, dns and dhcp) are opened properly. Run again but with ports already
enabled to check that the setup operation is idempotent.
- Visit diagnostics of an app that uses firewall components and see that ports
are listed properly in the port diagnostic test result.
- Remove some needed services such as those in /etc/firewalld/services and try
to visit the firewalld page. The page should show blank details against the
affected services.
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Use AppView for firewall view so that diagnostics button shows up
automatically. Consistency with the rest of the apps. Allow new features to
AppView become useful.
- A status section is automatically added but can be removed when all the other
apps loose their status section.
- Does not change anything else.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Test that a port meant to available to internal network is available.
- Test that a port meant to available to external network is available.
- Test that a port meant to unavailable to external network is unavailable.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
