2310 Commits

Author SHA1 Message Date
fliu
ea8150d9dd
email: setup: Find Let's Encrypt certificates 2021-08-17 19:44:14 -07:00
fliu
9a98ddadd4
email: postfix: dovecot: Set strong security parameters
Other changes:
- Fix linter error
- Postfix port 25: do not trust localhost IP addresses
2021-08-17 19:44:11 -07:00
fliu
b172b0b1cd
email: Local delivery: use full email address
If the domain part was not provided, $myhostname will be used.
In case $myhostname is not in $mydestinations, the email will be relayed.
In case $myhostname is invalid, the email will be bounced.

The next hop should always be LMTP (local: transport)
and hence the `@localhost`
2021-08-17 19:44:08 -07:00
fliu
2f77d99899
email: Code cleanup, address reviews
- View: security.html -> email_security.html
- Setup: will not install Redis
- Setup: put clamav packages on hold
- Crash recovery: opening firewall ports becomes the last post action
- Crash recovery: group postconf.set_many into small transactions
- Crash recovery: safer postconf.set_master_cf_options
2021-08-17 19:44:05 -07:00
fliu
240b67c49e
email: Reload postfix in domain view 2021-08-17 19:44:02 -07:00
fliu
27387d4a9c
email: Implement outbound mail filtering
- Make extensive use of the milter protocol
- Milter: add X-Fbx- headers to emails
- All submitted mails go to smtp:localhost:10025 for filtering
- Header privacy for submitted mails
- Rspamd: be able to inject and replace FreedomBox-managed config
- Reserve special addresses for future use

Known issue: internal emails do not go through spam filtering
2021-08-17 19:43:58 -07:00
fliu
85c6b91fbc
email: Code cleanup
- Move render_tabs into plinth.views
- Move admin_tabs and user_tabs into the class
- New interproc module contains inter-process functions
- New helper methods in TabMixin
2021-08-17 19:43:55 -07:00
fliu
e454e8ac1b
email: LMTP: remove the recipient's UID number from email headers 2021-08-17 19:43:52 -07:00
fliu
cdfef82bf6
email: Implement auto-discovery 2021-08-17 19:43:49 -07:00
fliu
355cc84a1b
email: apache: X-Robots-Tag header, full URL match 2021-08-17 19:43:46 -07:00
fliu
c31a896e81
email: Implement spam sorting with sieve
- Sieve script detects spam headers used by Rspamd and SpamAssassin
- Autosubscribe to Archive, Drafts, Junk, Sent, Trash
- Autoexpunge Trash and Junk folders
- INBOX.Junk -> Junk, INBOX.Trash -> Trash, Sent Messages -> Sent
2021-08-17 19:43:42 -07:00
fliu
41f0461ac7
email: postfix: Install LDAP map support 2021-08-17 19:43:39 -07:00
fliu
a234407b97
email: Implement view for setting up domains 2021-08-17 19:43:36 -07:00
fliu
502cfa4953
email: Add templates for TLS and domains 2021-08-17 19:43:33 -07:00
fliu
62c501e9c7
email: Add UI for creating the home directory
email_server:
- `-i` option passes all remaining arguments to action
- delete unused "touch file" option

Views:
- delete broken links
- add tabs to every page
- separate admin tabs from user tabs
2021-08-17 19:43:30 -07:00
fliu
2bd1ad4533
email: aliases: Use bootstrap styles
Other changes:

- Added license identifiers
- Fixed linter warnings
- Deleted excessive parameters in rendering code
2021-08-17 19:43:26 -07:00
fliu
4375828703
email: Implement alias management
- Separate alias database from system
- Block mail to system users, without backscatter
- Alias management UI for non-admin users
- Enabling/Disabling aliases (mails to /dev/null)

Misc. changes

- Daemon management
- Backup information
- Postconf diagnostics interface
2021-08-17 19:43:23 -07:00
fliu
a9ac51eb7b
email: views: Implement tab rendering 2021-08-17 19:43:20 -07:00
fliu
8c740e08da
email: diagnostics: Fix sudo permission problem 2021-08-17 19:43:17 -07:00
fliu
0acbe5dd6b
email: Support UID number lookup in Dovecot 2021-08-17 19:43:14 -07:00
fliu
91c907f657
email: Open lock file as plinth user 2021-08-17 19:43:11 -07:00
fliu
e2535bad49
email: audit: improve the speed of post-installation setup
- New class: `MainCfDiagnosis`
  - "Advise and repair" mechanism reduces the number of postconf calls

- File locking: lock acquisition moved into audit module
  - Enables finer-grained control
2021-08-17 19:43:07 -07:00
fliu
573287cf28
email: postconf: Handle postconf returning an empty key
However, if /sbin/postconf complained, a KeyError will be raised
2021-08-17 19:43:04 -07:00
fliu
7397326d57
email: mutex: create lock file as plinth user 2021-08-17 19:42:58 -07:00
fliu
df14e74972
email: Parse command arguments with a mutually exclusive group 2021-08-17 19:42:55 -07:00
fliu
7166e63b02
email: Install rspamd; proxy its web interface 2021-08-17 19:42:51 -07:00
fliu
758c8791f1
email: Address some code review comments
- __init__.py: Changed email server description
- audit module: Added module docstring
- email_server action:
  - Used argparse
  - Replaced "wrapper functions" with a getattr based lookup method
2021-08-17 19:42:48 -07:00
fliu
81c9632f5a
email: dovecot: Support user lookup by UID number
Dovecot refuses to open the root user's mail box
Need (!(uidNumber=0)) to correctly return status code
2021-08-17 19:42:45 -07:00
fliu
f20929c23f
email: Fix enabling SMTPS; check return value
- master.cf: Enable SMTPS
- lock.Mutex: check the return value of lock.acquire
- Write debug logs
2021-08-17 19:42:42 -07:00
fliu
10c3a667b6
email: Code quality fixes
- flake8: Delete unused imports
- lock.Mutex: Set file mode (rw-rw----) and ownership (plinth:plinth)
2021-08-17 19:42:39 -07:00
fliu
ba179a860e
email: Set up local delivery (no spam filtering)
Hosts file:

192.168.56.101 host1.test.example virtualbox
10.42.0.101    host2.test.example freedombox

Container setup (run commands in container)

1. Install the email server module
2. Create a FreedomBox account for `fred`
3.
$ sudo postconf myhostname=host2.test.example
$ echo "contactfred: fred" | sudo tee -a /etc/aliases
$ sudo newaliases
$ sudo systemctl reload postfix

Test IMAP (run on host1.test.example)

$ swaks --to=contactfred@host2.test.example --from=spam@host1.test.example
$ mutt -f imaps://fred@host2.test.example

Test mail submission (run in container):

$ swaks --to=spam@host1.test.example --from=contactfred@host2.test.example \
        --server host2.test.example:587 --tlso --auth-user=fred
2021-08-17 19:42:36 -07:00
fliu
fdc6f23908
email: Implement email_server ipc set_sasl and set_submission
- Rewrote action script to eliminate stdin communication
- Changed return type of audit.*.get()
  - An audit can return multiple lines of diagnostics
- Move recommended endpoint URLs into function docstrings
2021-08-17 19:42:32 -07:00
fliu
cde0b47064
email: Enable LDAP by calling postconf in a thread-safe way
- Implemented `email_server ipc postconf_set_many_v1`
- Implemented `lock.Mutex` (fcntl.lockf and threading.Lock based mutex)
  - FIXME: Lock file permissions
- Implemented `postconf` (thread-safe postconf operations)
- Started using service orientation
2021-08-17 19:42:29 -07:00
fliu
aaa6342f93
email: Basic app to manage an email server
- Install postfix and manage the service
- Import ugettext
- Dummy forms and views
- <module>.version (integer) is required

[sunil: Disable the app until remaining issues are worked out]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-08-17 19:42:26 -07:00
fliu
3def7706db
diagnostics: Allow underscores (_) in app names
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-08-17 19:42:16 -07:00
James Valleroy
81391ed820
ttrss: Allow upgrade to version 21
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-08-16 15:56:23 -04:00
Tiago Zaniquelli
a02b0baad1
tests: functional: storage: skip test List disks
Skip the test Scenario: List disks when inside a container

Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
2021-06-14 00:12:27 +05:30
Aurélien Couderc
76e9fac877
Change backups submit button to fix translation issues
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-05-16 18:12:10 -04:00
Joseph Nuthalapati
ec67b71aa6
coturn: Mention ejabberd in app description
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:53:05 -04:00
James Valleroy
9b446d5dd1
coturn: Validate TURN URIs if provided in form
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>

- Re-use the same validator in Matrix Synapse.
- Avoid importing plinth classes in actions files.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:52:57 -04:00
Joseph Nuthalapati
7a30470cb5
ejabberd: STUN/TURN configuration
This implementation is very similar to that of Matrix Synapse with a lot
of code duplicated.

One major difference is that ejabberd doesn't have a conf.d/ directory.
So, the managed configuration and overridden configuration cannot be
cleanly separated.

Whether the configuration is managed or not is determined by the
presence of a file under `/etc/ejabberd`. Managed coturn configuration
isn't stored in ejabberd, since only one set of configuration can be
stored at a time. If the admin chooses to use the managed configuration,
the current coturn configuration is fetched and used to configure
ejabberd.

Fixes #1978

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-04-16 17:52:48 -04:00
James Valleroy
1e8a91dd55
diagnostics: Use lock to protect results
Closes: #514.

Test: Ran diagnostics with all apps enabled.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-04-02 19:55:13 +03:00
James Valleroy
c293a01a1f
letsencrypt: Always return a diagnostics result
Return a warning result if no domains are configured.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-04-02 19:54:36 +03:00
James Valleroy
48d7f68ed5
config, dynamicdns, pagekite: Remove incorrect use of str
This was used before to ensure the domain name was ASCII. However, str
does not convert to ASCII in Python 3.

Note that in config module, which sets the system domain name, the
domain is already restricted to alphanumerics, hyphen, and period.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:47 +03:00
James Valleroy
0b630037f9
pagekite: Convert entered kite name to lower case
Let's Encrypt certificate paths use lower-case kite name.

Test: Pagekite functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:32 +03:00
James Valleroy
929e7626f0
dynamicdns: Convert entered domain name to lower case
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.

Add an extra 1 second delay to tests that configure DynamicDNS domain.

Test: DynamicDNS functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:20 +03:00
James Valleroy
bdd078a406
dynamicdns: Wait after changing domain name in tests
After a domain name change, Let's Encrypt will restart the webserver
and could cause a connection failure.

Test: DynamicDNS functional tests are passing.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:59:07 +03:00
James Valleroy
043b329e7d
config: Convert entered domain name to lower case
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.

Closes: #1964.

Tests: Config functional tests passed.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-03-29 12:58:13 +03:00
Kirill Schmidt
f2005f56aa first_boot: Use session to verify first boot welcome step
Closes: #2074

sunil's changes:

- Ensure that secret is not asked for a second time after creating admin user
and logging in (this happens due to discarding of old session and creating new
one).

- Ensure that this logic is not applied to user using FreedomBox images who are
not required to input the first wizard secret.

- Change the name of the session variable for readability.

Tests performed:

- Without the patch, start first boot wizard on the first browser on a
  FreedomBox image (or run the file `base64 < /dev/urandom | head -c 16 | sed -e
  's+$+\n+' > /var/lib/plinth/firstboot-wizard-secret`). Finish the welcome step
  with first wizard secret. Then open a second browser or from another computer
  visit the web interface. The first wizard secret is not asked and user can
  create an admin account.

- Repeat with fresh image again with the patch. To mimic fresh image, one may
  run `sudo rm -f /var/lib/plinth/plinth.sqlite3` and `echo "password" | | sudo
  /freedombox/actions/users remove-user tester`. This time when a different
  browser other than the one that provided the first wizard secret try to access
  the account creation page, a redirection will occur to welcome page. Providing
  the first wizard secret takes the user to account creation page.

- Accessing network first wizard page or internal pages on second browser also
  takes one back to the welcome page.

- Accessing help pages from second browser requires login.

- Accessing account login page from second browser is allowed. After creating
  the account from first browser, second browser can login and continue the
  wizard.

- Clearing cookies in the middle of the wizard takes the user back to welcome
  page.

- Clear cookies in the middles of the wizard. Visit the wizard, go to first boot
  welcome page. Provide secret and the wizard will continue where it was left
  off.

- Clear cookies in the middles of the wizard. Visit the wizard, go to first boot
  welcome page. Access login page, login as admin. Then first wizard secret is
  not asked. First wizard can be continued.

- On a fresh image, simply complete the first wizard. No change to earlier flow
  is noticed. First wizard secret is only asked once at the beginning.

- On a fresh image, remove the file /var/lib/plinth/firstboot-wizard-secret.
  First wizard can be completed without the secret.

[sunil: improvements to original patch by Kirill Schmidt]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-03-27 18:57:01 -07:00
Veiko Aasa
78427f730c
users: Fix unit test failures when LDAP is empty
When LDAP is set up but the first admin is not yet set up, some of the
users module tests fail because known admin users is deleted before other
users. A known admin user must exists to delete existing users.

Fix this by deleting a known admin user only after deleting other users
when cleaning up tests.

Tests performed on Debian stable and testing:
- All the users module unit tests pass:
  - when LDAP is not set up yet
  - after LDAP is set up but empty
  - after first admin user has set up using the web UI

Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-03-20 10:18:31 -04:00