Tests:
- Functional tests work.
- Admin user is able to access the application
- Regular user is not able to access the application
- Anonymous user is not able to access the application
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Since we are going to be an OpenID Provider, we need to fix the URLs that
other apps will be configured with for authentication. So change now from
/plinth to /freedombox. If done later, it will be harder since all the
configuration files for all dependent apps will need to be updated.
Tests:
- App availability checking works. Request goes to /freedombox URL
- Favicon is served properly and through /favicon.ico URL
- Redirection happens from / to /freedombox directly
- UI is available on /freedombox and on /plinth
- Manual page show /freedombox as the URL in two places
- Static files are successfully served from /freedombox URLs. URLs inside page
start with /freedombox
- backup, bepasty, calibre, config, dynamicdns, ejabberd, featherwiki, gitweb,
ikiwiki, kiwix, miniflux, names, openvpn, shadowsocks, shadowsocksserver,
sharing, shapshot, tiddlywiki, users, wireguard, jsxc, matrixsynapse, first
wizard, storage, samba, tags functional tests work. Backup/restore test for
matrixsynapse fails due to an unrelated bug (server not restarted after
restore).
- Setting the home page works:
- Having /plinth in the home page configuration works. Shows selection
correctly.
- Setting to app works. Shows selection correctly.
- Setting to user home page (sets /freedombox). Shows selection correctly.
- Setting to apache default works. Shows selection correctly.
- Changing back to FreedomBox service works. Shows selection correctly.
- Unit tests work
- Configuration page shows /freedombox in description but not /plinth
- Diagnostics show /freedombox in tests
- Roundcube URL link in email app has /freedombox
- email loads the page /.well-known/autoconfig/mail/config-v1.1.xml correctly
- email app shows /freedombox/apps/roundcube for /roundcube if roundcube is not
installed.
- networks: router configuration page shows URL starting with /freedombox.
- snapshot: Shows URL starting with /freedombox on the app page
- js licenses page uses /freedombox prefix for JSXC.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Dovecot is upgraded from 2.3 to 2.4, users are unable to see the old mails
from before the upgrade. New mails can be received but old mails can't be
accessed. Old mails are still present in
/var/mail/{usernmame}/mail/mailboxes/... New mails are being stored in
/var/mail/{username}/u.*. Other mailboxes such as 'Sent' are not affected.
Tests:
- Mails received in the inbox before the upgrade to dovecot 2.4 are now visible.
Without the patch, pre-upgrade mails are not visible and newly received mails
are stored in /var/mail/{username} instead of /var/mail/{username}/mailboxes/...
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Users were able to login using email address during dovecot 2.3 on Bookworm.
It was incorrectly assumed that there were not able to do that. Hence the
feature was not ported to 2.4. Early upgraders have reported this issue.
Tests:
- Login using full email address in the User Name field in Thunderbird. Without
the patch, the login fails and with the patch, it succeeds.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Add Full Text Search capability to Dovecot.
- Add 'dovecot-fts-xapian' to the list of packages for the email app.
- Add relevant configs for both dovecot 2.3 and 2.4
- Add a systemd timer to periodically clean search indexes
Configurations taken from plugin's upstream documentation:
https://github.com/grosjo/fts-xapian
Sunil:
- Tweak the dovecot 2.4 configuration. Remove explicit configuration same as or
close to default values.
- Drop the timer service for cleaning up the index. Dovecot documentation that
FTS plugins do it themselves.
- Drop the re-indexing command on setup. This could not be properly tested. On
first search, indexes will be created for mailboxes that don't have them.
Tests done:
- Perform a fresh install, on both Bookworm and Trixie, confirm the install is
successful, confirm the systemd service runs with exit 0.
- On Bookworm, apply the patches on an existing setup, confirm the patches apply
as expected.
- On a production like setup, set dovecot 2.4 to debug mode and check the
journal logs while receiving an email: The logs confirm that the fts module is
loaded and that it automatically creates a db for the indexes. I also opened the
newly created db file with less and confirmed that the human readable parts
contain my recent email.
- Using Sogo, perform a full search (including headers and body). Search works
and indexes are freshly created on all the folders.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Sunil:
- When dovecot package is upgrade from 2.3 to 2.4 during distribution upgrade,
automatically re-run setup.
- Upgrade existing setups to new scheme by re-running setup with incremented app
version.
- Don't query dovecot version during app initialization. Instead overwrite the
DropinConfigs component to query dovecot version during setup and enable
operations.
- Use apt.Cache() to retrieve the installed version of dovecot package. Use
plinth.utils.Version to parse the version and perform a comparison.
- Split even configuration files that have not changed for simplicity.
- Add/update links in Dovecot configuration files.
Tests:
- Install email app on a testing container. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.4 versions. TLS configuration is
accurate. Use Sogo to test login and sending mails.
- User with LDAP account and correct password is able to login.
- User without LDAP account or incorrect password is unable to login.
- Send mail with Sogo to another account on the server. Notice that mails are
stored in /var/mail/{user}/mail/ with mail:mail ownership in mbox format.
- Logging in with email such as user@example.com works. Capital letters are
allowed.
- "Archive", "Drafts", "Sent", "Junk", "Trash" folders are automatically
created and are marked with special flags. Creating additional folders such
as "Sent Items" also results in them having special flags.
- Thunderbird is able to connect via SSL with a self-signed certificate
exception.
- When an example spam message is sent, it is automatically moved to "Junk"
folder after getting marked by rspamd.
- When a message is moved to Junk folder, it is learned as spam by rspamd as
seen in its admin console.
- When a message is moved out of Junk folder (to other than "Trash" folder),
it is learned as not-spam by rspamd as seen in its admin console.
- Install email app on a stable container with patches. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS configuration is
accurate. Use Sogo to test login and sending mails.
- Install email app on a stable container without patches. Apply patches. Ensure
that all files in /etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS
configuration is accurate for dovecot 2.3. Use Sogo to test login and sending
mails. Perform distribution upgrade to testing. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS configuration is
accurate for dovecot 2.4. Use Sogo to test login and sending mails.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Add two sieve scripts for spam/ham learning. When the user moves a mail
from anywhere to junk, or from junk to anywhere (except for trash) the
mail is piped into the respective rspamc learn_spam/learn_ham command.
The rspamc command is run as the mail user and the command requires that
the user can connect to localhost:11334. Because of that, add the mail
user to the allowed users that can access protected services.
The sievec compilation of the new scripts requre the dovecot-antispam
package, so install it and increment the email version number.
Closes: #2487
Imroves: #56
Tests done:
1. Apply the patches on an existing install
2. Confirm the firewall and the email app get updated
3. Move a mail from inbox to junk and confirm that rspamd statistics for
"Learned" mails increment by one.
4. Move back the mail from junk to inbox and confirm the number
increments again.
5. Move the mail to trash and confirm the script doesn't execute.
6. Repeat steps 3-5 with mail_debug = yes in /etc/dovecot/dovecot.conf
and confirm the script esxecution further by reading the debug logs.
[Sunil]
- Split the configuration file 90-freedombox-sieve.conf into
90-freedombox-imap.conf and merge the remaining with 95-freedombox-sieve.conf.
- These changes do not need dovecot-anitspam package. Remove it from packages
list for the app.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
See the spam score for an email sent with these settings:
https://www.mail-tester.com/test-jy6unbdzu
Tests done:
1. Install the email app with version 4, then check out the files:
```
ls -la /var/lib/rspamd/dkim/ ; cat /etc/rspamd/local.d/dkim_signing.conf
total 4
dr-x------ 1 _rspamd _rspamd 50 Jan 1 19:14 .
drwxr-x--- 1 _rspamd _rspamd 16936 Jan 1 20:21 ..
-r-------- 1 root root 1704 Jan 1 19:14 freedombox.local.dkim.key
cat: /etc/rspamd/local.d/dkim_signing.conf: No such file or directory
```
2. Apply the patches and update the app:
```
ls -la /var/lib/rspamd/dkim/ ; cat /etc/rspamd/local.d/dkim_signing.conf
total 4
dr-x------ 1 _rspamd _rspamd 50 Jan 1 19:14 .
drwxr-x--- 1 _rspamd _rspamd 16936 Jan 1 20:22 ..
-r-------- 1 _rspamd _rspamd 1704 Jan 1 19:14 freedombox.local.dkim.key
allow_username_mismatch = true;
sign_authenticated = true;
use_domain = "header";
domain {
freedombox.local {
path = "/var/lib/rspamd/dkim/freedombox.local.dkim.key";
selector = "dkim";
}
}
```
3. Configure example.com as a domain under Name Services, then also change the primary domain in the email app and confirm it completes without errors.
Also see:
```
ls -la /var/lib/rspamd/dkim/ ; cat /etc/rspamd/local.d/dkim_signing.conf
total 8
dr-x------ 1 _rspamd _rspamd 90 Jan 1 21:15 .
drwxr-x--- 1 _rspamd _rspamd 16936 Jan 1 21:17 ..
-r-------- 1 _rspamd _rspamd 1704 Jan 1 21:15 example.com.dkim.key
-r-------- 1 _rspamd _rspamd 1704 Jan 1 19:14 freedombox.local.dkim.key
allow_username_mismatch = true;
sign_authenticated = true;
use_domain = "header";
domain {
example.com {
path = "/var/lib/rspamd/dkim/example.com.dkim.key";
selector = "dkim";
}
}
```
4. Uninstall the app and perform a fresh install, confirm it completes succesfully.
The configurations in /etc/rspamd/local.d/dkim_signing.conf have been
verified to work on a throw-away VPS setup.
Once merged, this should be mentioned here: https://discuss.freedombox.org/t/solved-email-messages-not-signed-with-dkim/2387
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
[sunil: Add comment explaining allow_username_mismatch option]
[sunil: Drop an unused variable, added docstrings]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
