38 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
9a8b57efd4
email_server: Adjust TLS configuration parameters
- Don't add TLS debugging information to Received: header.

- Drop unused fingerprint digest configuration. They are only used when
smtpd_tls_security_level is set to 'fingerprint' in which case certifying
authorities are ignored.

- Drop alterations to TLS low/high cipher lists. They are not used since
tls_ciphers are all set to 'medium'.

Tests:

- No configuration errors are reported by postfix in its logs after startup.

- 'postconf' shows that the new configuration parameters are set properly.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-14 18:22:54 -05:00
Sunil Mohan Adapa
e43e144040
email_server: Re-implement TLS configuration
- Use LetsEncrypt component to perform TLS certificate copying instead of custom
implementation.

- Use two components to copy the certificates to dovecot and postfix separately.

- Add support for multiple domains using SNI. Provide all the certificates. Use
primary domain's certificate as the fallback certificate.

- Drop the diagnose/repair approach due to its complexity.

Tests:

- Installing the app works. After installation, all TLS parameters are show as
expected by 'postconf' command and 'doveconf' command.

- A default domain is selected by default. This will reflect as primary domain
in TLS certificate configuration.

- When primary domain is changed, the configuration is updated to reflect the
default certificate path but SNI configuration is unchanged in dovecot and
postfix.

- Postfix and dovecot are restarted after setup.

- There are no configuration error shows in postfix/dovecot logs.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-14 18:22:51 -05:00
Sunil Mohan Adapa
ae882fea70
email_server: Simplify domain configuration form
- By default, receive mail for all the domains on the system.

- Allow user to select a primary domain. This domain is used for TLS
certificate, automatically adding domain to sender address, etc.

- Don't expose postfix configuration parameters.

Tests:

- On installation, the domain list populated in postfix. Primary domain is
the one set in the config module. If it is not set, any other domain from
configured domains is taken.

- When not installed, adding/removing domains does not cause errors.

- Changing the domain in the domain view works. mydomain has the primary domain
set. myhostname has primary domain set. mydestination has default values and in
addition has all the domains on the system.

- /etc/mailname is populated with the primary domain.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-06 17:39:43 -05:00
Sunil Mohan Adapa
8d7bac70c9
email_server: Setup /var/mail, drop home setup view
Creating home directories is no longer necessary. We store all mail in /var/mail
and don't allow use of direct access to mail folder.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 19:40:55 -04:00
Sunil Mohan Adapa
cb8c23c28d
email_server: Lookup LDAP local recipients via PAM
Most modern setups simply use to PAM to lookup local recipients instead of
integrating directly with LDAP. libnss-ldapd package that we install and
configure connects the password database with LDAP. Anyone then using PAM need
not be aware of LDAP integration. This reduces extra configuration and many
problems that come along with it.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 19:40:30 -04:00
Sunil Mohan Adapa
d0cf01fb29
email_server: Don't use user IDs when performing lookups
- Typical mail systems are configured to work on usernames or virtual usernames.
UIDs/GIDs are only needed at the final moment when delivering mails to user
inboxes that need to have proper UID/GID set.

- This makes it easy for dovecot to simply use PAM authentication instead of
having to use LDAP.

- Trying to hide UID from email headers is no longer necessary. Received: header
is important for debugging mail delivery across the chain. Don't miss out.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 19:40:19 -04:00
Sunil Mohan Adapa
8cb100be79
email_server: yapf formatting
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 19:10:34 -04:00
Sunil Mohan Adapa
a2038e98d6
email_server: action: Refactor for simplicity
- Parse arguments in a readable way.

- Convert decorator into simple call.

- Make a simple call instead of looking for subcommand.

- Don't setup logging in global scope.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 19:10:17 -04:00
Sunil Mohan Adapa
e85001f01f
email_server: domains: Use Django forms and views
- Eliminate error-prone custom code and styling.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-03 18:44:41 -04:00
Sunil Mohan Adapa
6e8b825d44
email_server: aliases: Drop hash DB and use sqlite3 directly
- Postfix has the ability to use sqlite3 databases directly. There is no need to
synchronize to a hash db and then use that.

- Store the aliases database in /var/lib/postfix/. This will make backup and
restore easier and remove dependence on FreedomBox and its data directory.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-10-30 16:15:26 -04:00
Sunil Mohan Adapa
91f18a0e52
email_server: yapf formatting
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-10-30 16:14:58 -04:00
Sunil Mohan Adapa
9bd1f80d5c
*: Always pass check= argument to subprocess.run()
- Avoid flake8 warnings.

- Makes the call more explicitly readable in case an exception is expected but
check=True is not passed by mistake.

Tests:

- Many tests are skipped since the changes are considered trivial.
check=False is already the default for subprocess.run() method.

- actions/package: Install an app when it is not installed.

- actions/upgrade: Run manual upgrades.

- actions/users: Change a user password. Login. Create/remove a user.

- actions/zoph: Restore a database.

- container: On a fresh repository, run ./container up,ssh,stop,destroy for a
testing container.

- plinth/action_utils.py: Enable/disable an app that has a running service.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-10-11 14:34:40 -04:00
Sunil Mohan Adapa
a3e21adc8b
*: Various isort fixes
- Done automatically by running isort . in top level directory.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-09-20 16:50:37 -04:00
Sunil Mohan Adapa
74214c18ae
*: Use Django gettext functions instead of ugettext
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.

- ugettext is already a simple alias of gettext. So, no regressions are
expected.

Tests:

- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.

- Using Django 3.2 there are no warnings related to removal of ugettext
functions.

- Ran regular unit tests.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-09-20 16:50:16 -04:00
fliu
41c86e0ba3
email: configure postfix domain names 2021-09-05 18:00:43 -07:00
fliu
06d8aade23
email: Fix self.critical not callable error 2021-09-05 18:00:30 -07:00
fliu
f400eb9439
email: implemented service alert
- Better error handling
- You may retroactively diagnose and fix problems
- Shows the most important issues first

audit.models.Diagnosis
  - Added JSON serialization (needed for IPC)
  - Replaced ambiguous vocabulary
  - Provided a sorting key

audit, diagnosis
  - Support title translation
2021-08-20 19:24:21 -07:00
fliu
3ff823e837
email: Sender spoofing patch 1/2: domain rewriting
A sender canonical map deletes the domain part from sender addresses.
Postfix will append @$myhostname after the rewriting.
2021-08-17 19:44:24 -07:00
fliu
1e712f6bc4
email: setup: Configure Roundcube 2021-08-17 19:44:21 -07:00
fliu
5a9c7e5077
email: Documentation, code cleanup
- Audit: finalize MainCfDiagnosis API
- Doc: document Diagnosis models
- Doc: remove hacky imports and replace them with autodoc mock imports
- Views: add `email_` prefix to email server templates
- Apache: delete unnecessary directives (robots tag, cookie header etc.)
2021-08-17 19:44:17 -07:00
fliu
ea8150d9dd
email: setup: Find Let's Encrypt certificates 2021-08-17 19:44:14 -07:00
fliu
9a98ddadd4
email: postfix: dovecot: Set strong security parameters
Other changes:
- Fix linter error
- Postfix port 25: do not trust localhost IP addresses
2021-08-17 19:44:11 -07:00
fliu
2f77d99899
email: Code cleanup, address reviews
- View: security.html -> email_security.html
- Setup: will not install Redis
- Setup: put clamav packages on hold
- Crash recovery: opening firewall ports becomes the last post action
- Crash recovery: group postconf.set_many into small transactions
- Crash recovery: safer postconf.set_master_cf_options
2021-08-17 19:44:05 -07:00
fliu
240b67c49e
email: Reload postfix in domain view 2021-08-17 19:44:02 -07:00
fliu
27387d4a9c
email: Implement outbound mail filtering
- Make extensive use of the milter protocol
- Milter: add X-Fbx- headers to emails
- All submitted mails go to smtp:localhost:10025 for filtering
- Header privacy for submitted mails
- Rspamd: be able to inject and replace FreedomBox-managed config
- Reserve special addresses for future use

Known issue: internal emails do not go through spam filtering
2021-08-17 19:43:58 -07:00
fliu
85c6b91fbc
email: Code cleanup
- Move render_tabs into plinth.views
- Move admin_tabs and user_tabs into the class
- New interproc module contains inter-process functions
- New helper methods in TabMixin
2021-08-17 19:43:55 -07:00
fliu
c31a896e81
email: Implement spam sorting with sieve
- Sieve script detects spam headers used by Rspamd and SpamAssassin
- Autosubscribe to Archive, Drafts, Junk, Sent, Trash
- Autoexpunge Trash and Junk folders
- INBOX.Junk -> Junk, INBOX.Trash -> Trash, Sent Messages -> Sent
2021-08-17 19:43:42 -07:00
fliu
a234407b97
email: Implement view for setting up domains 2021-08-17 19:43:36 -07:00
fliu
62c501e9c7
email: Add UI for creating the home directory
email_server:
- `-i` option passes all remaining arguments to action
- delete unused "touch file" option

Views:
- delete broken links
- add tabs to every page
- separate admin tabs from user tabs
2021-08-17 19:43:30 -07:00
fliu
4375828703
email: Implement alias management
- Separate alias database from system
- Block mail to system users, without backscatter
- Alias management UI for non-admin users
- Enabling/Disabling aliases (mails to /dev/null)

Misc. changes

- Daemon management
- Backup information
- Postconf diagnostics interface
2021-08-17 19:43:23 -07:00
fliu
e2535bad49
email: audit: improve the speed of post-installation setup
- New class: `MainCfDiagnosis`
  - "Advise and repair" mechanism reduces the number of postconf calls

- File locking: lock acquisition moved into audit module
  - Enables finer-grained control
2021-08-17 19:43:07 -07:00
fliu
df14e74972
email: Parse command arguments with a mutually exclusive group 2021-08-17 19:42:55 -07:00
fliu
7166e63b02
email: Install rspamd; proxy its web interface 2021-08-17 19:42:51 -07:00
fliu
758c8791f1
email: Address some code review comments
- __init__.py: Changed email server description
- audit module: Added module docstring
- email_server action:
  - Used argparse
  - Replaced "wrapper functions" with a getattr based lookup method
2021-08-17 19:42:48 -07:00
fliu
f20929c23f
email: Fix enabling SMTPS; check return value
- master.cf: Enable SMTPS
- lock.Mutex: check the return value of lock.acquire
- Write debug logs
2021-08-17 19:42:42 -07:00
fliu
ba179a860e
email: Set up local delivery (no spam filtering)
Hosts file:

192.168.56.101 host1.test.example virtualbox
10.42.0.101    host2.test.example freedombox

Container setup (run commands in container)

1. Install the email server module
2. Create a FreedomBox account for `fred`
3.
$ sudo postconf myhostname=host2.test.example
$ echo "contactfred: fred" | sudo tee -a /etc/aliases
$ sudo newaliases
$ sudo systemctl reload postfix

Test IMAP (run on host1.test.example)

$ swaks --to=contactfred@host2.test.example --from=spam@host1.test.example
$ mutt -f imaps://fred@host2.test.example

Test mail submission (run in container):

$ swaks --to=spam@host1.test.example --from=contactfred@host2.test.example \
        --server host2.test.example:587 --tlso --auth-user=fred
2021-08-17 19:42:36 -07:00
fliu
fdc6f23908
email: Implement email_server ipc set_sasl and set_submission
- Rewrote action script to eliminate stdin communication
- Changed return type of audit.*.get()
  - An audit can return multiple lines of diagnostics
- Move recommended endpoint URLs into function docstrings
2021-08-17 19:42:32 -07:00
fliu
cde0b47064
email: Enable LDAP by calling postconf in a thread-safe way
- Implemented `email_server ipc postconf_set_many_v1`
- Implemented `lock.Mutex` (fcntl.lockf and threading.Lock based mutex)
  - FIXME: Lock file permissions
- Implemented `postconf` (thread-safe postconf operations)
- Started using service orientation
2021-08-17 19:42:29 -07:00