- Since we have important fixes deployed that we would like to get effected
immediately.
Tests:
- Not tested.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Dovecot is upgraded from 2.3 to 2.4, users are unable to see the old mails
from before the upgrade. New mails can be received but old mails can't be
accessed. Old mails are still present in
/var/mail/{usernmame}/mail/mailboxes/... New mails are being stored in
/var/mail/{username}/u.*. Other mailboxes such as 'Sent' are not affected.
Tests:
- Mails received in the inbox before the upgrade to dovecot 2.4 are now visible.
Without the patch, pre-upgrade mails are not visible and newly received mails
are stored in /var/mail/{username} instead of /var/mail/{username}/mailboxes/...
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Users were able to login using email address during dovecot 2.3 on Bookworm.
It was incorrectly assumed that there were not able to do that. Hence the
feature was not ported to 2.4. Early upgraders have reported this issue.
Tests:
- Login using full email address in the User Name field in Thunderbird. Without
the patch, the login fails and with the patch, it succeeds.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Configuration provided by David (https://discuss.freedombox.org/u/david/)
Tests:
- Install SOGo without patch and apply the patch. The app setup is run and new
version of configuration file is installed. After logging into SOGo:
- Mail settings shows an option to add IMAP account.
- Editing Full Name in the identity of the default account is now possible.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2528
Tests:
- In Responsive Design Mode in Firefox, expand the screen width to be beyond
1400px. The left side of popup for the user menu will be aligned with the left
side of the menu item itself. When the width of the page is less than 1400px,
the right side of the popup will be right aligned with the right side of the
menu item.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Build a Debian package before and after the patch and notice that binary
packages have no differences when compared with diffoscope. Source packages show
only the change in the patch but no other change.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Install matrix-synapse app by adding 'unstable' in apt sources.list. Then
remove 'unstable' from apt sources.list. Then matrix-synapse package will no
longer be found in the apt's cache.
- Try to uninstall the package. Without patch, the process errors out. With
patch, uninstall completes successfully.
- While matrix-synapse app is installed and apt cache does not contain
matrix-synapse package, install and uninstall bepasty app. Without patch,
uninstall fails. With patch, uninstall succeeds.
- Install and uninstall minetest app. 3d armor mod package is successfully
installed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- This change was part of the original pull request !2661 but was missed in its
continuation !2677.
Tests:
- Installing an app works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Without the change FileNotFound exception is raised.
Tests:
- Send request using 'nc' to privileged daemon that has invalid 'module'
parameter. SyntaxError exception is raised.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Fallback to sudo based privileged implementation. Privileged daemon tests are
still to be implemented.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Instead of running the command using sudo. If the server is not reachable, run
the privileged command using sudo.
Tests:
- Typical privileged calls are made to server as evidenced by the network emoji
icon in the log.
- Some actions such as creating gitweb repository or downloading a backup
archive happen via sudo instead of privileged daemon.
- When a call is made to privileged daemon the log message is show just like a
sudo call.
- If the daemon is not running and can't be started, the calls are made to sudo.
- If the daemon is rejects connections, then calls are automatically made to
sudo.
- When cloning a gitweb repository, the operation is immediately returned and
task runs in background. Other tasks as waited upon until they are finished.
Introducing a sleep in privileged method leads to increased page load time.
- When server sends non-JSON response, a decode error is printed and exception
is raised.
- When a typical privileged call is made, the return value as expected.
- When a typical privileged call raises exception, a nice HTML exception is
shown in the UI. stdout/stderr outputs are not shown. Error is also logged on
the console as expected but without stdout/stderr.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- When a call is made to privileged daemon the log shows network emoji instead
of #.
- Log for unimplemented calls such as downloading backup images still shows # as
they not sent to privileged daemon.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- When a new container image is provisioned, developer configuration is set on
privileged daemon as seen with 'systemctl show freedombox-privileged.service'.
freedombox-privileged.socket is enabled and running (socket is being listened
on).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- Daemon starts up with uid/gid set to root.
- Daemon does not run by default if a request is not received. Socket file is
created with 666 permissions and root:root ownership. Socket file parent directory
is created with 755 permissions and root:root ownership.
- Daemon starts if a request is sent to the socket using nc.
- If there an exception in daemon starting, then restart is done every second to
5 seconds, forever.
- Build a Debian package.
- Install it on fresh trixie Debian VM. Ensure that setup works and privileged
daemon is auto-enabled.
- Start a fresh trixie Debian VM and install freedombox from Debian repos.
Upgrade to the built package. Privileged daemon works and is auto-enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- So that it can be invoked easily from the command line and systemd service.
Tests:
- make install creates /usr/bin/freedombox-privileged daemon which can be run as
root.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- This daemon will be faster than running actions using 'sudo' because the
actions sometimes load all the modules before certain safety checks can be
performed. The daemon will load just once for many privileged calls.
Tests:
- After daemon is started, systemd shows the status as 'activated'.
- When daemon is started using socket activation and requests are sent, the
requests succeed.
- When daemon is started manually and requests are sent, the requests succeed.
The socket file is created with root:root ownership and 0666 permissions.
Parent directory is created if not exists. After the daemon exits, the socket
file is removed. When daemon is started manually, automatic idle timeout exit
does not happen.
- According to journalctl, server exists after 5 seconds. Proper log message is
seen.
- Without development mode, server exists after 5 minutes of idle. Proper log
message is seen.
- When a sleep is added in one of the actions and when the action is running,
server does not exit. Server exits after the request is completed.
- When an error is raised in verify request, the server exits with proper error
message. If the server exists with non-zero error code and is immediately
restarted by systemd.
- Sending a sample request using nc from root user and plinth user works.
- Sending a sample request using nc from fbx user is rejected.
- If a non-unicode text is sent as request, the response is a valid error
dictionary.
- If the request is larger than 1M, an 'request too large' error is thrown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- When privileged daemon receives a non-JSON request, a proper error structure
is returned with SyntaxError.
- When privileged daemon receives a request without 'module', 'action', 'args'
or 'kwargs' parameters, a proper error structure is returned with TypeError.
- When privileged daemon receives a request for invalid 'module' or 'action', a
proper error structure is returned with SyntaxError.
- When an exception is thrown in a privileged method, the error is properly
returned in error structure and caller is shown all the proper details.
- Valid return values are sent when a privileged call is made.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Refactor validation of fields in the JSON object.
- Throw distinct errors for missing field and wrong type.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Only effective once. Second call will skip loading apps.
- Helps with privileged daemon where actions might load apps repeatedly.
Tests:
- Diagnostics/enable/disable for apps bepasty, updates, config, security,
nextcloud, homeassistant run fine.
- Install/uninstall for bepasty works fine.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Only effective once. Second call will skip loading modules.
- Helps with privileged daemon where actions might load modules repeatedly.
Tests:
- Diagnostics/enable/disable for apps bepasty, updates, config, security,
nextcloud, homeassistant run fine.
- Install/uninstall for bepasty works fine.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- The most pleasant font on any system is the default system font.
- It is the most optimized and styled font for the system considering screen
type and screen sizes.
- Used by all the system apps. Websites can become consistent with system apps
by using system fonts. GNOME, KDE, Ubuntu, Android, Chrome OS, iOS, and MacOS,
all have their own system fonts.
- Changed by the user using OS settings if they don't like it.
- Many popular sites have started using system fonts.
- No extra fonts have to be loaded making page loading jerk free and much
faster. On the first FreedomBox UI page load, the largest item is the font.
- We won't have carry the binary woff files in FreedomBox source tree anymore.
Also eliminates a bunch of lintian warnings.
- Lato font was used because it is prescribed by the FreedomBox identity manual.
Lato can still be used in other places such as marketing materials.
Tests:
- System font is used in the UI. When system font is changed in Gnome settings
and browser is restarted, the new font is shown in the UI.
- Check that the overall layout of the app grids is not effected by the font
size change.
- Check that all the tables in the UI are not affected by the font change.
- Backups repository listing shows each backup archive in one line.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Fixes: #1358
- Refresh the apt cache if required packages for an app are not found and if the
cache is more than 1 hour old (or non-existent).
- If required packages are found, don't refresh the package cache even if the
cache is outdated. This is because the check operation could lead to many
minutes of waiting before app can be installed.
Tests:
- Remove /var/lib/apt/lists/* and /var/cache/apt/pkgcache.bin. Visit an app
setup page. apt cache is updated and it take a while to check that the app is
available. App is shown as available. If page is refreshed, this time, the cache
is not updated.
- Set the modification of /var/cache/apt/pkgcache.bin file to more than 2 hours
ago with 'touch -d "2 hours ago" /var/cache/apt/pkgcache.bin'. Then refreshing
the page will not refresh the cache.
- Repeat test with an app that is not available such as Janus. Again apt cache
is refreshed. App is shown as not available. On refresh, the cache is not
updated.
- Set the modification of /var/cache/apt/pkgcache.bin file to more than 2 hours
ago with 'touch -d "2 hours ago" /var/cache/apt/pkgcache.bin'. Then refreshing
the page will not refresh the cache.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Remove redundant if condition in setup.html template
- Use JavaScript fetch() API instead of XMLHTTPRequest class
- Update a comment in test_package.py
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
- Using AJAX request instead of loading the initial page slowly.
Tests:
- Unit tests passes.
- Deluge app is not available in bookworm and is available in Trixie.
- When app is available, no message is shown. Install button is enabled.
- When app is not available a proper warning alert message is shown. Install
button is disabled.
- During check for the availability, the progress message is shown. Install
button is disabled.
- When Javascript is disabled on the page, no availability check is performed.
Install button is enabled.
- When an exception is raised in the is-available view, error message is shown.
Install button is enabled.
- When is-available view return HTML response, error message is shown. Install
button is enabled.
- When is-available view invalid JSON response, error message is shown. Install
button is enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- Add a comment and /etc/apt/sources.list file. Distribution upgrade page does
not load and fails with an error.
- With the patch, page loads properly. Distribution upgrade can be triggered.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
[jvalleroy: Fix test for release date]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Thanks to git blame I found why python3-tomli had been added back then:
6199718a19383d8d070b7bdc9d26ead71a9d26dd
python3-tomli is slowly being sunset in favor of tomllib from the Standard
Library: https://wiki.debian.org/Python/Backports
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Add Full Text Search capability to Dovecot.
- Add 'dovecot-fts-xapian' to the list of packages for the email app.
- Add relevant configs for both dovecot 2.3 and 2.4
- Add a systemd timer to periodically clean search indexes
Configurations taken from plugin's upstream documentation:
https://github.com/grosjo/fts-xapian
Sunil:
- Tweak the dovecot 2.4 configuration. Remove explicit configuration same as or
close to default values.
- Drop the timer service for cleaning up the index. Dovecot documentation that
FTS plugins do it themselves.
- Drop the re-indexing command on setup. This could not be properly tested. On
first search, indexes will be created for mailboxes that don't have them.
Tests done:
- Perform a fresh install, on both Bookworm and Trixie, confirm the install is
successful, confirm the systemd service runs with exit 0.
- On Bookworm, apply the patches on an existing setup, confirm the patches apply
as expected.
- On a production like setup, set dovecot 2.4 to debug mode and check the
journal logs while receiving an email: The logs confirm that the fts module is
loaded and that it automatically creates a db for the indexes. I also opened the
newly created db file with less and confirmed that the human readable parts
contain my recent email.
- Using Sogo, perform a full search (including headers and body). Search works
and indexes are freshly created on all the folders.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- This helps during distribution upgrade from dovecot 2.3 to 2.4. Dovecot will
stop running due to dovecot server 2.4 not understanding version 2.3
configuration files. When setup is re-run, starting the daemons again is the
right thing to do.
Tests:
- With email app installed, upgrade from bookworm to trixie. Dovecot is stopped
during distribution upgrade but after freedombox service runs, it recovers and
starts running again.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Sunil:
- When dovecot package is upgrade from 2.3 to 2.4 during distribution upgrade,
automatically re-run setup.
- Upgrade existing setups to new scheme by re-running setup with incremented app
version.
- Don't query dovecot version during app initialization. Instead overwrite the
DropinConfigs component to query dovecot version during setup and enable
operations.
- Use apt.Cache() to retrieve the installed version of dovecot package. Use
plinth.utils.Version to parse the version and perform a comparison.
- Split even configuration files that have not changed for simplicity.
- Add/update links in Dovecot configuration files.
Tests:
- Install email app on a testing container. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.4 versions. TLS configuration is
accurate. Use Sogo to test login and sending mails.
- User with LDAP account and correct password is able to login.
- User without LDAP account or incorrect password is unable to login.
- Send mail with Sogo to another account on the server. Notice that mails are
stored in /var/mail/{user}/mail/ with mail:mail ownership in mbox format.
- Logging in with email such as user@example.com works. Capital letters are
allowed.
- "Archive", "Drafts", "Sent", "Junk", "Trash" folders are automatically
created and are marked with special flags. Creating additional folders such
as "Sent Items" also results in them having special flags.
- Thunderbird is able to connect via SSL with a self-signed certificate
exception.
- When an example spam message is sent, it is automatically moved to "Junk"
folder after getting marked by rspamd.
- When a message is moved to Junk folder, it is learned as spam by rspamd as
seen in its admin console.
- When a message is moved out of Junk folder (to other than "Trash" folder),
it is learned as not-spam by rspamd as seen in its admin console.
- Install email app on a stable container with patches. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS configuration is
accurate. Use Sogo to test login and sending mails.
- Install email app on a stable container without patches. Apply patches. Ensure
that all files in /etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS
configuration is accurate for dovecot 2.3. Use Sogo to test login and sending
mails. Perform distribution upgrade to testing. Ensure that all files in
/etc/dovecot/conf.d/ are linked properly to 2.3 versions. TLS configuration is
accurate for dovecot 2.4. Use Sogo to test login and sending mails.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes#2256.
Based on a suggestion by Andrew Betts on the mailing list.
https://alioth-lists.debian.net/pipermail/freedombox-discuss/2022-August/009553.html
Sunil:
- Consolidate changes from various apps into a centralized place in
freedombox.conf applicable for all directory listings.
Tests:
- In Sharing, TiddlyWiki and FeatherWiki apps, directory listing when viewed
with Firefox Developer Tools Mobile view set to a Galaxy S20+ looks reasonable.
Without the patch the page is very zoomed out.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
