2850 Commits

Author SHA1 Message Date
fliu
c31a896e81
email: Implement spam sorting with sieve
- Sieve script detects spam headers used by Rspamd and SpamAssassin
- Autosubscribe to Archive, Drafts, Junk, Sent, Trash
- Autoexpunge Trash and Junk folders
- INBOX.Junk -> Junk, INBOX.Trash -> Trash, Sent Messages -> Sent
2021-08-17 19:43:42 -07:00
fliu
41f0461ac7
email: postfix: Install LDAP map support 2021-08-17 19:43:39 -07:00
fliu
a234407b97
email: Implement view for setting up domains 2021-08-17 19:43:36 -07:00
fliu
502cfa4953
email: Add templates for TLS and domains 2021-08-17 19:43:33 -07:00
fliu
62c501e9c7
email: Add UI for creating the home directory
email_server:
- `-i` option passes all remaining arguments to action
- delete unused "touch file" option

Views:
- delete broken links
- add tabs to every page
- separate admin tabs from user tabs
2021-08-17 19:43:30 -07:00
fliu
2bd1ad4533
email: aliases: Use bootstrap styles
Other changes:

- Added license identifiers
- Fixed linter warnings
- Deleted excessive parameters in rendering code
2021-08-17 19:43:26 -07:00
fliu
4375828703
email: Implement alias management
- Separate alias database from system
- Block mail to system users, without backscatter
- Alias management UI for non-admin users
- Enabling/Disabling aliases (mails to /dev/null)

Misc. changes

- Daemon management
- Backup information
- Postconf diagnostics interface
2021-08-17 19:43:23 -07:00
fliu
a9ac51eb7b
email: views: Implement tab rendering 2021-08-17 19:43:20 -07:00
fliu
8c740e08da
email: diagnostics: Fix sudo permission problem 2021-08-17 19:43:17 -07:00
fliu
0acbe5dd6b
email: Support UID number lookup in Dovecot 2021-08-17 19:43:14 -07:00
fliu
91c907f657
email: Open lock file as plinth user 2021-08-17 19:43:11 -07:00
fliu
e2535bad49
email: audit: improve the speed of post-installation setup
- New class: `MainCfDiagnosis`
  - "Advise and repair" mechanism reduces the number of postconf calls

- File locking: lock acquisition moved into audit module
  - Enables finer-grained control
2021-08-17 19:43:07 -07:00
fliu
573287cf28
email: postconf: Handle postconf returning an empty key
However, if /sbin/postconf complained, a KeyError will be raised
2021-08-17 19:43:04 -07:00
fliu
7397326d57
email: mutex: create lock file as plinth user 2021-08-17 19:42:58 -07:00
fliu
df14e74972
email: Parse command arguments with a mutually exclusive group 2021-08-17 19:42:55 -07:00
fliu
7166e63b02
email: Install rspamd; proxy its web interface 2021-08-17 19:42:51 -07:00
fliu
758c8791f1
email: Address some code review comments
- __init__.py: Changed email server description
- audit module: Added module docstring
- email_server action:
  - Used argparse
  - Replaced "wrapper functions" with a getattr based lookup method
2021-08-17 19:42:48 -07:00
fliu
81c9632f5a
email: dovecot: Support user lookup by UID number
Dovecot refuses to open the root user's mail box
Need (!(uidNumber=0)) to correctly return status code
2021-08-17 19:42:45 -07:00
fliu
f20929c23f
email: Fix enabling SMTPS; check return value
- master.cf: Enable SMTPS
- lock.Mutex: check the return value of lock.acquire
- Write debug logs
2021-08-17 19:42:42 -07:00
fliu
10c3a667b6
email: Code quality fixes
- flake8: Delete unused imports
- lock.Mutex: Set file mode (rw-rw----) and ownership (plinth:plinth)
2021-08-17 19:42:39 -07:00
fliu
ba179a860e
email: Set up local delivery (no spam filtering)
Hosts file:

192.168.56.101 host1.test.example virtualbox
10.42.0.101    host2.test.example freedombox

Container setup (run commands in container)

1. Install the email server module
2. Create a FreedomBox account for `fred`
3.
$ sudo postconf myhostname=host2.test.example
$ echo "contactfred: fred" | sudo tee -a /etc/aliases
$ sudo newaliases
$ sudo systemctl reload postfix

Test IMAP (run on host1.test.example)

$ swaks --to=contactfred@host2.test.example --from=spam@host1.test.example
$ mutt -f imaps://fred@host2.test.example

Test mail submission (run in container):

$ swaks --to=spam@host1.test.example --from=contactfred@host2.test.example \
        --server host2.test.example:587 --tlso --auth-user=fred
2021-08-17 19:42:36 -07:00
fliu
fdc6f23908
email: Implement email_server ipc set_sasl and set_submission
- Rewrote action script to eliminate stdin communication
- Changed return type of audit.*.get()
  - An audit can return multiple lines of diagnostics
- Move recommended endpoint URLs into function docstrings
2021-08-17 19:42:32 -07:00
fliu
cde0b47064
email: Enable LDAP by calling postconf in a thread-safe way
- Implemented `email_server ipc postconf_set_many_v1`
- Implemented `lock.Mutex` (fcntl.lockf and threading.Lock based mutex)
  - FIXME: Lock file permissions
- Implemented `postconf` (thread-safe postconf operations)
- Started using service orientation
2021-08-17 19:42:29 -07:00
fliu
aaa6342f93
email: Basic app to manage an email server
- Install postfix and manage the service
- Import ugettext
- Dummy forms and views
- <module>.version (integer) is required

[sunil: Disable the app until remaining issues are worked out]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-08-17 19:42:26 -07:00
fliu
3def7706db
diagnostics: Allow underscores (_) in app names
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-08-17 19:42:16 -07:00
James Valleroy
81391ed820
ttrss: Allow upgrade to version 21
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-08-16 15:56:23 -04:00
Tiago Zaniquelli
a02b0baad1
tests: functional: storage: skip test List disks
Skip the test Scenario: List disks when inside a container

Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
2021-06-14 00:12:27 +05:30
Aurélien Couderc
76e9fac877
Change backups submit button to fix translation issues
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-05-16 18:12:10 -04:00
Joseph Nuthalapati
ec67b71aa6
coturn: Mention ejabberd in app description
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:53:05 -04:00
James Valleroy
9b446d5dd1
coturn: Validate TURN URIs if provided in form
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>

- Re-use the same validator in Matrix Synapse.
- Avoid importing plinth classes in actions files.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:52:57 -04:00
Joseph Nuthalapati
7a30470cb5
ejabberd: STUN/TURN configuration
This implementation is very similar to that of Matrix Synapse with a lot
of code duplicated.

One major difference is that ejabberd doesn't have a conf.d/ directory.
So, the managed configuration and overridden configuration cannot be
cleanly separated.

Whether the configuration is managed or not is determined by the
presence of a file under `/etc/ejabberd`. Managed coturn configuration
isn't stored in ejabberd, since only one set of configuration can be
stored at a time. If the admin chooses to use the managed configuration,
the current coturn configuration is fetched and used to configure
ejabberd.

Fixes #1978

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:52:48 -04:00
James Valleroy
1e8a91dd55
diagnostics: Use lock to protect results
Closes: #514.

Test: Ran diagnostics with all apps enabled.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-04-02 19:55:13 +03:00
James Valleroy
c293a01a1f
letsencrypt: Always return a diagnostics result
Return a warning result if no domains are configured.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-04-02 19:54:36 +03:00
James Valleroy
48d7f68ed5
config, dynamicdns, pagekite: Remove incorrect use of str
This was used before to ensure the domain name was ASCII. However, str
does not convert to ASCII in Python 3.

Note that in config module, which sets the system domain name, the
domain is already restricted to alphanumerics, hyphen, and period.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:47 +03:00
James Valleroy
0b630037f9
pagekite: Convert entered kite name to lower case
Let's Encrypt certificate paths use lower-case kite name.

Test: Pagekite functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:32 +03:00
James Valleroy
929e7626f0
dynamicdns: Convert entered domain name to lower case
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.

Add an extra 1 second delay to tests that configure DynamicDNS domain.

Test: DynamicDNS functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:20 +03:00
James Valleroy
bdd078a406
dynamicdns: Wait after changing domain name in tests
After a domain name change, Let's Encrypt will restart the webserver
and could cause a connection failure.

Test: DynamicDNS functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:07 +03:00
James Valleroy
043b329e7d
config: Convert entered domain name to lower case
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.

Closes: #1964.

Tests: Config functional tests passed.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:58:13 +03:00
Kirill Schmidt
f2005f56aa first_boot: Use session to verify first boot welcome step
Closes: #2074

sunil's changes:

- Ensure that secret is not asked for a second time after creating admin user
and logging in (this happens due to discarding of old session and creating new
one).

- Ensure that this logic is not applied to user using FreedomBox images who are
not required to input the first wizard secret.

- Change the name of the session variable for readability.

Tests performed:

- Without the patch, start first boot wizard on the first browser on a
  FreedomBox image (or run the file `base64 < /dev/urandom | head -c 16 | sed -e
  's+$+\n+' > /var/lib/plinth/firstboot-wizard-secret`). Finish the welcome step
  with first wizard secret. Then open a second browser or from another computer
  visit the web interface. The first wizard secret is not asked and user can
  create an admin account.

- Repeat with fresh image again with the patch. To mimic fresh image, one may
  run `sudo rm -f /var/lib/plinth/plinth.sqlite3` and `echo "password" | | sudo
  /freedombox/actions/users remove-user tester`. This time when a different
  browser other than the one that provided the first wizard secret try to access
  the account creation page, a redirection will occur to welcome page. Providing
  the first wizard secret takes the user to account creation page.

- Accessing network first wizard page or internal pages on second browser also
  takes one back to the welcome page.

- Accessing help pages from second browser requires login.

- Accessing account login page from second browser is allowed. After creating
  the account from first browser, second browser can login and continue the
  wizard.

- Clearing cookies in the middle of the wizard takes the user back to welcome
  page.

- Clear cookies in the middles of the wizard. Visit the wizard, go to first boot
  welcome page. Provide secret and the wizard will continue where it was left
  off.

- Clear cookies in the middles of the wizard. Visit the wizard, go to first boot
  welcome page. Access login page, login as admin. Then first wizard secret is
  not asked. First wizard can be continued.

- On a fresh image, simply complete the first wizard. No change to earlier flow
  is noticed. First wizard secret is only asked once at the beginning.

- On a fresh image, remove the file /var/lib/plinth/firstboot-wizard-secret.
  First wizard can be completed without the secret.

[sunil: improvements to original patch by Kirill Schmidt]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-03-27 18:57:01 -07:00
Veiko Aasa
78427f730c
users: Fix unit test failures when LDAP is empty
When LDAP is set up but the first admin is not yet set up, some of the
users module tests fail because known admin users is deleted before other
users. A known admin user must exists to delete existing users.

Fix this by deleting a known admin user only after deleting other users
when cleaning up tests.

Tests performed on Debian stable and testing:
- All the users module unit tests pass:
  - when LDAP is not set up yet
  - after LDAP is set up but empty
  - after first admin user has set up using the web UI

Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-03-20 10:18:31 -04:00
Max Rockatansky
efbb39b2a3
security: Clarify vulnerability count and provide link to more info
Update security_report.html text to clarify what vulnerability total
numbers mean, and give more info to learn what the vulnerabilties are.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Make URL a link]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-03-06 09:39:53 -05:00
Sunil Mohan Adapa
186596cfbf
config: Install and configure zram for swap
Closes: Debian #805108.

Primary motivation is to provide swap for FreedomBox machines. On all FreedomBox
images, currently there is no swap configured. Swap on disk may not be good for
SBCs most of which use SD card for storage. We wish for processes to not get
killed when hard memory limit is reached.

Zram seems like a good solution to the problem suitable not only for SBCs but
also for desktops and bigger machines. Fedora is currently using Zram as its
default swap solution configured by the installer. Zram creates a block device
with a configured size. Writing blocks into the device compresses them and
stores them in RAM. This block device can be configured as swap among other
things. See:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/blockdev/zram.rst

Set the size of the swap to be 50% of RAM. Expected compression is about 1:2.
That means, in an average case, 25% of RAM is consumed to provide the swap
device. This results in the system being able to consume about 125% of RAM
capacity to run processes. This value is inspired by Fedora.
https://fedoraproject.org/wiki/Changes/SwapOnZRAM .

Zram based swap takes priority over disk based swap (due the priority being set
to 100). This reduces IO and improves latency on machines that already have a
swap device.

On containers, zramswap.service fails to start as it will not be possible to
insert the 'zram' kernel module from within the container. This should not cause
any further problems.

Since 'config' app is an essential app, zram-tools now becomes a hard dependency
of freedombox package.

For FreedomBox images, zram-tools will be pre-installed and pre-configured. So,
it will work on first boot. For users installing FreedomBox via apt or those
upgrading from an older version, zram-tools will be newly installed but
configuration will not be picked up until the next reboot. Restarting
zramswap.service is not done because it may not be a safe/successful operation.

systemd-zram-generator is a project that essentially does what zram-tools. It
appears to be a better implementation and we may migrate to it when it becomes
available in Debian. Migration expected to be straight forward.

Tests performed:

- Running `sudo -u plinth ./run --list-dependencies` shows zram-tools as a
dependency.

- On a container, `systemctl status zramswap.service` shows as failed.

- On a virtual machine, confirm that configuration is installed properly. Run
`./setup.py install; systemctl daemon-reload; systemctl show zramswap.service |
grep Environment`.

- On a virtual machine, ensure that you have more than 512MiB or RAM. Then
restart zramswap.service. This should create a swap space of 50% of RAM
capacity. Confirm with `free` and `zramswap status`.

- Restarting the VM retains the swap that has been setup.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-03-06 09:00:28 -05:00
Sunil Mohan Adapa
2d30312b10
config: Disable rsyslog and syslog forwarding
Helps: #664.

Currently, logs are written to disk twice, once by journald and once by rsyslog.
rsyslog may log to multiple locations depending on the type of the log. To
reduce disk I/O, disable rsyslog and rely solely on systemd journal.

Place the code in config module as there is no better place for it currently
without creating a new module. Can be sorted later.

The following files under /var/log/ are no longer populated on FreedomBox. They
will be rotated away over a few days. Use journalctl instead to view the
messages:

- syslog
- messages*
- auth.log*
- debug*
- daemon.log
- kern.log
- lpr.log
- mail.log
- mail.info
- mail.warn
- mail.err
- user.log

Tests performed:

- On a machine with rsyslog running, run ./setup.py install and start FreedomBox
service. This triggers the config app's setup. rsyslog is disabled and masked.
systemd-journald is restarted.

- Even when rsyslog is unmaked and enabled manually, systemd journald does not
forward message to syslog anymore.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-05 18:06:31 -08:00
Sunil Mohan Adapa
b66de4a9b7
security: Increment app version to reload fail2ban
- To immediately accommodate the backend=systemd changes.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-05 18:06:19 -08:00
Sunil Mohan Adapa
69d4a65c00
security: Ensure that fail2ban is not re-enabled on version increment
Tests:

- Disable fail2ban. When version number of the app is incremented, setup is run,
fail2ban is reloaded but fail2ban is not enabled after setup.

- Disable fail2ban. When app is uninstalled and FreedomBox is run, setup is
performed and fail2ban is enabled.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-05 18:06:12 -08:00
Sunil Mohan Adapa
252d69f465
security: Move fail2ban default configuration to this app
Since security app manages fail2ban, it makes sense to set the default
configuration in this app.

Tests performed:

- `./setup.py install` installs the file in the correct place.

- Only 10 incorrect SSH login attempts as noticed in the fail2ban log will
result in ban.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-05 18:06:04 -08:00
Sunil Mohan Adapa
fd7bda7ce9
ssh, apache: Make fail2ban use systemd journald backend by default
- This allows disabling syslog daemons.

- Fall back to using file based monitoring for Apache.

Tests performed:

- Before and after the patch, connecting via SSH and typing in incorrect
password leads to a entry in fail2ban.log. 10 incorrect attempts result in a 10
minute ban.

- Before and after the patch, typing in incorrect password for radicale leads to
a entry in fail2ban.log. 10 incorrect attempts result in a 10 minute ban.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-05 18:05:23 -08:00
Veiko Aasa
631e10c09e
deluge: Fix daemon user not in freedombox-share group after installation
Add the debian-deluged user to the freedombox-share group *before* running
app setup. The app setup process restarts the deluged daemon
after which the process has correct group ownerships.

Tests done in Debian stable and testing dev containers:
- After installing the deluge app, checked from /proc/--process-id--/status that
the deluged daemon process has the freedombox-share group.

Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-03-05 17:36:07 -08:00
Veiko Aasa
8df60f596e
deluge, mldonkey, syncthing, transmission: Depend on nslcd.service
Add nslcd.service as a dependency to the services that depend on users
and groups defined in LDAP. deluged, mldonkey-server, syncthing@syncthing
and transmission-daemon services depend on freedombox-share LDAP group.

Closes #2061

Tests done with apps deluge, mldonkey, syncthing and transmission,
in both debian stable and testing dev containers, after applying changes:
- After installing an app and after reboot, the daemon user is a member
of the freedombox-share group.
- Checked with the `systemctl show` command that nslcd.service is added to
After=... dependencies.
- All the functional tests pass (in Debian stable, closed manually
the syncthing usage reporting form - #2059).

Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-03-05 17:33:40 -08:00
Fioddor Superconcentrado
64b3fb98b2
config: Fix tests related to user home directory
Closes: #2014

- test_homepage_mapping_skip_ci: Don't expect fbx to run the test.
- test_homepage_field: Check additional precondition.

Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Minor refactoring to avoid flake8 warnings]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-03-05 16:23:55 -08:00