- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Instead of just the sites that have successfully obtain certificate. This
allows customization of configuration for those sites (especially useful when
testing where LE certs are not obtained).
Tests:
- When a domain is added to the system, an apache TLS configuration is created
for the domain even though the domain does not have a successfully obtained LE
cert.
- When a domain is removed, the TLS configuration for the domain is removed.
- Add a domain without the patches. Apply the patches and restart the service.
The domain added signals are fired during the startup. This results in site
specific TLS configuration files getting created and Apache reloads. When the
service is restarted, the files are not created and Apache is not reloaded.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Can be used to set the 'Expires:' header to cache static files for a long
time.
Tests:
- Without the patch 'a2query -m expires' shows that the module is not installed.
Applying the patches and restarting services shows that Apache app's setup is
run and 'a2query -m expires' shows that module is enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2271
When domain name is updated, it usually results in a error page as the HTTP
connection is broken in the middle of a page load. This is due to apache
restarting in the middle of domain change operation by letsencrypt component.
This also leads to several functional tests failing. To fix this, ensure that
letsencrypt does a reload on the apache2 daemon instead of restarting it.
'reload' operation on apache2 triggers the command 'apachectl graceful'. It
ensures that currently running continue to serve the open HTTP connection until
the page load has been completed. After that those connections stop. Meanwhile,
the server reloads configuration (and apparently the related TLS certificates too).
Tests:
- Unit tests pass.
- When self-signed certificate is updated with 'make-ssl-cert
generate-default-snakeoil --force-overwrite' and 'systemctl
try-reload-or-restart apache2' is called, the new certificate is loaded by
apache2. Browser shows the untrusted certificate warning again. The
certificate information in the connection details has been updated.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- DAV can simplify hosting the Feather Wiki app.
- It can also potentially be used to share folders over HTTP to clients such a
GNOME/KDE file mangers.
- Enabling the modules by default should have few disadvantages other than
slight increase in memory. It needs to be enabled with 'DAV on' directive on the
specific directories.
Tests:
- Running the service after patch run apache setup and the modules are enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Unit tests pass.
- Run full diagnostics tests and see that results and app name are translated
when language preference is not English.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- During application update, if the last_updated_version is not provided for the
WebServer component, app's update to next version fails.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Config files are all symlinks in /etc/
- ttrss app's web interface works
- Incorrect password attempts in LDAP login with /tt-rss-app/ get registered by
fail2ban
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
We often change apache configuration for an app and when a new version of the
config is deployed the changes go un-applied until FreedomBox is restarted. Allow
specifying that config has changed in an particular version and reload of
configuration is needed.
Tests:
- Unit tests pass. The feature used in transmission works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2264.
- Set apache-auth fail2ban jail's backend to read from journal instead of
syslog. Tweak the regex matching to deal with the custom format.
- Adjust the apache error log format to remove unnecessary timestamp. It causes
problems for fail2ban regex matching.
- There was an error in the earlier patch the make apache log into journald.
Configuration for TLS sites still contained ErrorLog and CustomLog directives.
Remove them.
- There is also file with CustomLog directive that logs for other vhosts.
- For some reason, for custom error log format, %T - thread ID did not work and
had to switch to %{g}T global thread ID.
- Added journalmatch to improve performance by matching the regular expressions
against only specific journal entries.
Tests:
- In a container, apply the patch, run setup and start FreedomBox. Apache app is
updated to new version. Apache web server is reloaded. The
other-vhosts-access-log configuration is disabled.
- On a production machine, remove the directives in
freedombox-tls-site-macro.conf and disabling other-vhosts-access-log stopped the
logging into /var/log/apache2/ directory.
- Use TTRSS /tt-rss-app/ URL and type wrong credentials for 10 times. The client
is banned for 10 minutes. Repeat after unban. Client is banned again.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Initial setup works when a new container is created
- When transmission is enabled/disabled, the web configuration for it is
enabled/disabled.
- When radicale is enabled/disabled, the uwsgi configuration for it is
enabled/disabled.
- Sharing web configuration is disabled during backup and re-enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- It is simpler to keep all the configuration in a single file. Any overrides
are expected to be done by writing additional configuration files with higher
priority.
- /etc/apache2/site-available/ is typically reserved for virtual host
configurations. Redirections and proxying for all virtual hosts rather belongs
in /etc/apache2/conf-available/.
- This looses the option of disabling plinth-ssl.conf when needed. In the
initial days of enabling TLS, there was a need felt to keep the option of easily
disabling redirection to TLS in case there is a need for it. However, TLS
certificate setup is mature and the limitations are well understood. There is no
longer a need for it. It still may be possible to avoid the redirection with an
additional configuration.
Tests:
- In a fresh container, setup succeeds. Redirecting to https:// for /plinth
works. FreedomBox web interface is available.
- Without the patch applied created a container. Run setup and access Plinth
interface. Apply the patches. Apache setup is run. a2query -s plinth and a2query
-s plinth-ssl show that sites are not enabled. Redirecting to https:// for
/plinth works. FreedomBox web interface is available.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Upon access of an app URL, it may redirect to another URL that is configured in
app settings. This new URL could only be accessed on IPv4 or IPv6 only. When
curl is invoked with the IP address version of a different kind, the access
fails. In such cases, tell the diagnostics methods not the restrict to a
particular address type.
Tests:
- Unit tests pass.
- All of transmission's diagnostics tests pass. The URL tests show that they
have been performed on a particular IP address type.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Done automatically by running isort . in top level directory.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Enabling the module automatically sets 'Protocols h2 h2c http/1.1' in shipped
module configuration.
- HTTP/2 is given higher priority over HTTP/1.1 for supported clients.
- Clients not supporting HTTP/2 continue to work with HTTP/1.1.
- Clients work by using APLN extension in TLS to figure out that server supports
HTTP/2 and use it.
- HTTP/2 improves performance.
- Recommended by Mozilla's SSL configurator: https://ssl-config.mozilla.org/.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This allows disabling syslog daemons.
- Fall back to using file based monitoring for Apache.
Tests performed:
- Before and after the patch, connecting via SSH and typing in incorrect
password leads to a entry in fail2ban.log. 10 incorrect attempts result in a 10
minute ban.
- Before and after the patch, typing in incorrect password for radicale leads to
a entry in fail2ban.log. 10 incorrect attempts result in a 10 minute ban.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests in testing container:
- SSH, Service Discovery, and Web Server are shown as enabled on
Firewall page.
- FreedomBox interface can be accessed after disabling Cockpit.
Closes: #2016.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #1981
Closes also most of threads in !1952.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Moved uwsgi and uwsgi-plugin-python3 from radicale and searx to apache.
Fixes: #1501
Signed-off-by: Onurb <onurb8966@gmail.com>
[jvalleroy: Move uwsgi managed service to apache app]
[jvalleroy: Also update bepasty app]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #1938.
Fixed application of available translations in daemon.py and apache,
diagnostics, networks, firewall and users modules.
diagnostics:
- __init__.py: return the app name along its results.
- diagnostics.html: display the app name instead of its id.
- diagnostics_results.html:
- mark for translation,
- apply class to results <td> HTML tag.
main.css: center-align the results.
Locale files excluded. Will be regenerated automatically and translations to be
done via Weblate.
original testing (rebased later):
- Yapf applied.
- Flake8 without errors or warnings for changed files.
- (Unit) tests run without errors.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Translate 'None' app name]
[sunil: Don't translate tests strings second time in template]
[sunil: Tweak the center rule]
[sunil: Don't split a translation string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Prevent leaking private info through Tor onion service or Pagekite.
Tests:
- When starting plinth, apache setup is run. Status module is
disabled, and apache2 is restarted.
- sunil: After upgrade, status page is not available.
- sunil: mod_status is available in stable (2.4.38-3+deb10u3) and
testing/unstable (2.4.46-1).
Closes: #1935.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
The make-ssl-cert command is used during initial setup so it is a hard
dependency. Earlier it was being included as a Recommends: via apache2 package.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Directory /etc/php/7.4 became available before php7.4-fpm became available.
Handle this by checking the run time socket of the fpm daemon instead of the
directory.
Discussed and tested in
https://discuss.freedombox.org/t/mediawiki-service-unavailable/711/23
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
