Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2313.
systemd-journald does not (never did) accept size values given in percent of
file system size. Only the defaults work with percent values. Hence our addition
of RuntimeMaxUse= as percent value in configuration file did not work.
systemd-journald outputs a warning to dmesg and ignores the value.
We could change the value to fixed size. We would have to choose a value that
works for systems with less memory (such as 1GiB) and that value would serve
poorly for systems with more memory. Instead, leaving the default value of 10%
for RuntimeMaxUse= might be better. Additional configuration of MaxFileSec=6h
and MaxRetentionSec=2day would also ease the burden in most cases for the low
memory devices. Considering that people did not report issues with status
quo (where the value we have set did not work and default size was used) also
suggests that default value will work. Further, /run filesystem itself seems to
be allocated only 10% of available memory.
Tests:
- Without the patch, start a vagrant machine. Notice that dmesg shows the error
mentioned in the issue #2313. Apply patch and restart the service. Setup is run
for config app. The file /etc/systemd/journald.conf.d/50-freedombox.conf will no
longer have the RuntimeMaxUse= directive.
- After reboot, dmesg will no longer show the error. systemctl status
systemd-journald shows that 10% of the size of /run is the max for journal file.
- In config app page, setting various values of log persistence works.
- On a fresh container with the patch, initial setup succeeds and
journald.conf.d file is setup without the RuntimeMaxUse= directive.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2194.
In the four cases fixed, if a notification is present with a 'btn-primary' in
it, then instead of submitting the intended form, the button on the notification
is clicked. This will result in an indefinite wait for the form to perform an
action. Fix this by specifying which form exactly we want to submit.
Tests:
- Run functional tests for config app and updates app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2314.
Tests:
- On a fresh testing container, verify that the file
/etc/apache2/conf-available/freedombox-apache-homepage.conf is not present.
Visit the config app and notice that home page shows as 'Apache Default'.
- Apply the patch and refresh the page. The page now shows 'FreedomBox
Service (Plinth)' as the home page.
- Functional tests work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Initial implementation of home page setting used the file
/etc/apache2/conf-available/freedombox.conf and edited the file. Since this file
is shipped by the freedombox package, it lead to package getting stuck with
conf-file prompt. FreedomBox v19.10 first fix this by carefully undoing the
edits in this file and making them elsewhere.
- This fix is present in Debian present old stable (with backports) and current
stable, the migration is not needed in almost all the of cases.
Tests:
- First setup of FreedomBox works.
- Setting home page works are expected.
- Functional tests for config module works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Unit tests work
- DONE: Transmission
- DONE: Enabling/disabling an app with a daemon works: transmission
- DONE: Showing the status of whether the app is enabled with daemon
is-enabled works.
- DONE: A message is shown if app is enabled and service is not running
- DONE: Service is stopped and re-started during backup
- DONE: Adding user to share group during initial setup restarts the service
- Not tested: Enabling/disabling a service with alias works (no such apps)
- DONE: Restarting/try-restarting a service works
- DONE: Masking/unmasking works
- DONE: rsyslog is masked after initial setup
- DONE: systemd-journald is try-restarted during initial setup
- DONE: Avahi, email, security initial setup works
- DONE: Fail2ban is unmasked and enabled
- DONE: Enabling/disabling fail2ban is security app works
- DONE: Enabling/disabling password authentication in SSH works
- ?? Let's encrypt
- Services are try-restarted during certificate setup, obtain, renew
- Not tested: upgrade pagekite from version 1
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Running flake8 as in .gitlab-ci.yml works.
- Setting the domain name again to update /etc/hosts file after hostname change
works
- Setting the domain name from the text box works. New domain name is read back
and shown properly.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Running flake8 as in .gitlab-ci.yml works.
- Changes the hostname works and it is updated in /etc/hostname
- Avahi daemon is restarted
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Initial setup succeeds
- (not tested, functionality removed later) During initial setup, if
/etc/apache2/conf-available/freedombox.conf has home page other than /plinth,
it will be changed to /plinth.
- Setting the home page to Apache default, plinth, or an app works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
The test uses freedombox.local as the domain. This requires that Avahi
is enabled, and the hostname is set to freedombox.
Fixes#2232.
Test:
- ejabberd functional tests pass even after running tests for config
and avahi.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This reduces the number of writes to the disk improving disk longevity and IO
performance. Note that systemd-journald is already very reasonable with how
often it writes to the disk. It's flush interval is 5 minutes.
Most users of FreedomBox are not expected to see logs. Those that see the logs
do so for debugging purposes. Debugging can still be done if reboot does not
occur. Users can change the logging mode to 'persistent' before debugging issues
that require reboot. This makes debugging harder for non-reproducible bugs, but
is, at present, considered an acceptable compromise.
Tests:
- On a fresh container, with the patch applied, config page shows 'volatile' as
the logging mode.
- On an container with changes not applied, start freedombox service. Then apply
the patch and restart service. config app setup will be run. Config page shows
'volatile' as the logging mode.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- None disables logging altogether. This is useful when we want to prevent
FreedomBox from collecting IP addresses of visitors and other sensitive
information.
- Volatile logs are kept in RAM until the system is rebooted. Only 5% of RAM
will be used at most and only 2 days worth of logs are kept.
- Permanent will store logs into /var/log/journal. systemd-journald defaults
will apply. 10% of disk capacity is used at most, capped at 4GiB. Also logging
will stop if free space is below 15%. Maximum of 100 files are kept. No time
based cleanup is done.
Tests:
- Set the logging mode to disabled. Observe that `journalctl -f` does not show
any logs (say when performing plinth actions).
- Set the logging mode to volatile. Observe that `journalctl` shows that logging
is set to /run/log/journal/ and 5% of available memory is set as maximum.
- Set the logging mode to persistent. Observe that `journalctl` shows that
logging is set to /var/log/journal/ and 10% of disk space is set as maximum.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This was required in Python 2 but useless in Python 3.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2151.
Tests:
- Before this patch, --list-dependencies does not list zram-tools. After this
patch, --list-dependencies lists zram-tools.
- After this patch, --list-dependencies exactly matches the output from
freedombox release without Packages component changes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- After startup, the configured domain name shows up in names module.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is now the preferred location in Debian. See:
https://lintian.debian.org/tags/systemd-service-in-odd-locationhttps://bugs.debian.org/992465https://bugs.debian.org/987989d70caa69c6https://lists.debian.org/debian-devel/2021/08/msg00275.html
Tests:
- Lintian no longer shows errors:
E: freedombox: systemd-service-in-odd-location lib/.../calibre-server-freedombox.service
- Comparing the old .deb and newly generated .deb with these changes. All the
systemd files show that they are moved from /lib to /usr/lib/systemd.
- After upgrading the deb from older version to a version these changes,
services installed by the package are available (tested after restart with
wordpress and claibre). Services tweaked by the package have the changed
configuration reflected as shown by systemctl show
{service-name}.service (tested after restart with quassel).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This was used before to ensure the domain name was ASCII. However, str
does not convert to ASCII in Python 3.
Note that in config module, which sets the system domain name, the
domain is already restricted to alphanumerics, hyphen, and period.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.
Closes: #1964.
Tests: Config functional tests passed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: Debian #805108.
Primary motivation is to provide swap for FreedomBox machines. On all FreedomBox
images, currently there is no swap configured. Swap on disk may not be good for
SBCs most of which use SD card for storage. We wish for processes to not get
killed when hard memory limit is reached.
Zram seems like a good solution to the problem suitable not only for SBCs but
also for desktops and bigger machines. Fedora is currently using Zram as its
default swap solution configured by the installer. Zram creates a block device
with a configured size. Writing blocks into the device compresses them and
stores them in RAM. This block device can be configured as swap among other
things. See:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/blockdev/zram.rst
Set the size of the swap to be 50% of RAM. Expected compression is about 1:2.
That means, in an average case, 25% of RAM is consumed to provide the swap
device. This results in the system being able to consume about 125% of RAM
capacity to run processes. This value is inspired by Fedora.
https://fedoraproject.org/wiki/Changes/SwapOnZRAM .
Zram based swap takes priority over disk based swap (due the priority being set
to 100). This reduces IO and improves latency on machines that already have a
swap device.
On containers, zramswap.service fails to start as it will not be possible to
insert the 'zram' kernel module from within the container. This should not cause
any further problems.
Since 'config' app is an essential app, zram-tools now becomes a hard dependency
of freedombox package.
For FreedomBox images, zram-tools will be pre-installed and pre-configured. So,
it will work on first boot. For users installing FreedomBox via apt or those
upgrading from an older version, zram-tools will be newly installed but
configuration will not be picked up until the next reboot. Restarting
zramswap.service is not done because it may not be a safe/successful operation.
systemd-zram-generator is a project that essentially does what zram-tools. It
appears to be a better implementation and we may migrate to it when it becomes
available in Debian. Migration expected to be straight forward.
Tests performed:
- Running `sudo -u plinth ./run --list-dependencies` shows zram-tools as a
dependency.
- On a container, `systemctl status zramswap.service` shows as failed.
- On a virtual machine, confirm that configuration is installed properly. Run
`./setup.py install; systemctl daemon-reload; systemctl show zramswap.service |
grep Environment`.
- On a virtual machine, ensure that you have more than 512MiB or RAM. Then
restart zramswap.service. This should create a swap space of 50% of RAM
capacity. Confirm with `free` and `zramswap status`.
- Restarting the VM retains the swap that has been setup.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #664.
Currently, logs are written to disk twice, once by journald and once by rsyslog.
rsyslog may log to multiple locations depending on the type of the log. To
reduce disk I/O, disable rsyslog and rely solely on systemd journal.
Place the code in config module as there is no better place for it currently
without creating a new module. Can be sorted later.
The following files under /var/log/ are no longer populated on FreedomBox. They
will be rotated away over a few days. Use journalctl instead to view the
messages:
- syslog
- messages*
- auth.log*
- debug*
- daemon.log
- kern.log
- lpr.log
- mail.log
- mail.info
- mail.warn
- mail.err
- user.log
Tests performed:
- On a machine with rsyslog running, run ./setup.py install and start FreedomBox
service. This triggers the config app's setup. rsyslog is disabled and masked.
systemd-journald is restarted.
- Even when rsyslog is unmaked and enabled manually, systemd journald does not
forward message to syslog anymore.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Closes: #1981
Closes also most of threads in !1952.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #1938.
backups/forms.py:
- ChoiceField labeled to allow translation.
- Translation applied to hard coded literals.
config/forms.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
diagnostics_results.html:
Apply translation to results. Use gettext_noop to mark for translation.
dynamicdns/forms.py:
Apply translation to choice literals.
i2p/views.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
names.html:
Apply translation to table headers.
performance/__init__.py:
Apply translation to description literals.
radicale/forms.py:
ChoiceField labeled to allow translation.
users/forms.py:
CharField labeled to allow translation.
QA:
- Literals visually verified.
- No errors in py.test-3.
- Yapf applied (only) to changed files.
- No remarks by flake8 to changed file.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Separate out the translations]
[sunil: Fix i18n for diagnostics]
[sunil: dynamicdns: Also do i18n for string GnuDIP]
[sunil: searx: Revert an incorrect removal of import]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Move non-reusable app specific step definitions and helper methods into
<app>/tests/test_functional.py.
- Merge reusable helper methods into plinth.tests.functional
- Merge reusable step definitions into plinth.tests.functional.step_definitions
- avahi, datetime, ikiwiki: Reuse common methods to avoid repetition. Avoid
mapping from app nicknames to actual app names.
- deluge, transmission: Make a copy of sample.torrent for each app to avoid
clogging common place.
- Implement functional.visit() to simplify a lot of browser.visit() calls.
- Ensure that name of the mark on functional tests for an app is same as name of
the app. This will help with predicting the mark when running tests for a
particular app.
Tests performed:
- Run all functional tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Add pytest hooks to ignore all functional tests if pytest_bdd is not
installed.
- Update pytest hooks to skip tests in file named 'test_functional.py' if
--include-functional argument is not provided.
- Move functional_tests/install.py into plinth/tests/functional and update
reference in Vagrantfile.
- Move scenario files into individual app folders. Rename them after the app
they are testing. Merge TODO items listed in todo.org into corresponding feature
files.
- Add test_functional.py in each app to build tests from the features file using
pytest_bdd.
- Move all step_definitions, support and data into plinth/tests/functional/.
Include all step_definitions from conftest.py. Update to relative imports
instead of absolute imports.
Tests performed:
- Run py.test-3 --collect-only shows all functional tests and lists 574 tests.
No errors show that name of feature files are correct. The number says that all
functional test features are included.
- Remove pytest_bdd (or modify the import name) and run py.test-3 --collect-only
skips collecting all functional tests and shows only 300+ tests.
- Run functional tests for a few apps with py.test-3 --include-functional -m
app. For storage, deluge.
- Run unit tests with py.test-3. Functional tests are listed by skipped.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Introduce new API to mark an app that it can't be disabled.
- Mark jsxc, storage, config, upgrade and firewall apps as can't be disabled.
- Fixed functional tests
- Replaced AppForm with forms.Form in all modules' forms.py.
- Remove app.template.js.
- Remove unused styles.
- Remove app status checks in form_valid of Deluge, Diaspora, Matrix, Ejabberd,
MediaWiki, Storage, Transmission, Quassel
- Purge unused is_enabled context variables (Ikiwiki)
- ejabberd: Minor cleanup in template
- jsxc: Cleanup unneeded overrides
- tahoe: Cleanup unnecessary overrides
Tests performed:
- For all apps affected, test enable/disable button works and submitting
configuration form works: with changes updates message and without changes
'settings unchanged' message.
- avahi
- bind
- cockpit
- SKIP: coquelicot
- datetime
- deluge
- SKIP: diaspora
- ejabberd
- gitweb
- i2p
- infinoted
- ikiwiki
- matrixsynapse
- mediawiki
- minetest
- minidlna
- mldonkey
- mumble
- pagekite
- privoxy
- quassel
- radicale
- roundcube
- SKIP: samba
- searx
- SKIP: shaarli
- shadowsocks
- ssh
- tahoe
- transmission
- FAIL: tt-rss (not installable)
- wireguard
- Deluge test that configuration changes when app is disabled work
- Quassel test that setting the domain works when app is diabled
- Transmission test that setting the domain works when app is diabled
- Ikiwiki create form works properly
- Enable/disable button appears as expected when enabled and when disabled
- Enable/disable button works without Javascript
- Functional tests work for affected apps, Tor and OpenVPN
- AppForm is removed from developer documentation
- Forms reference
- Customizing tutorial
- Test all apps using directory select form
- Transmission
- Deluge
- Visit each template that overrides block configuration and ensure that it is
loaded properly and the display is as expected.
- All apps that use AppView that are not tested above should not have an
enable/disable button. That is JSXC, update, config, firewall, storage, users.
Signed-off-by: Alice Kile <buoyantair@protonmail.com>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
If at least one related service/daemon is not running, show the alert if app
is enabled, otherwise set alert as hidden (for functional tests).
Closes#1752
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Minor code simplification in app template, status section]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
