Closes#2157.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Split diaspora and tahoe-lafs into separate commits]
[sunil: Remove monkeysphere from help/tests/test_views.py]
[sunil: Add to configuration file removal in Debian package and setup.py]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Skip handling of obsolete packages if there are none.
- Skip handling of debconf selections if none are required.
- Handle tt-rss same as other packages with prompts.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2159.
Ship a separate Apache configuration file instead of editing the one provided by
roundcube package. This avoids configuration file prompt when roundcube package
needs to be upgraded.
Tests:
- Freshly install roundcube package 1.4.x (using apt preferences and Bullseye),
run functional tests and login to a gmail account.
- Freshly install roundcube package 1.5.x (from testing), run functional tests
and login to a gmail account.
- Install roundcube 1.4.x version on testing container without these changes.
After applying these changes, run 'apt update' while roundcube is enabled and
let FreedomBox upgrade roundcube to 1.5.x version. After this, run functional
tests and login to a gmail account.
- Repeat the previous test with upgrade while rouncube is disabled. Then enable
rouncube, run functional tests and login to gmail account.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Don't try to get the depends from module level and sort modules based on that.
- Instead after all App instances are created, sort the apps based on
app.info.depends and app.info.is_essential.
- Print message that apps have been initialized instead of printing before they
are initialized. The correct order of apps is only known after they have been
initialized and sorted.
- Avoid circular import on module_loader and setup.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Alias was added to deal with Buster -> Bullseye transition. In Buster the daemon
was named bind9 and in Bullseye the daemon is named named with alias to bind9.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Instead of getting it from managed_packages module level variable. This is made
possible by the ability to instantiate an app without being able to instantiate
all apps at once and without even initializing Django.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Syslog is not used on FreedomBox machines. Logging to syslog instead of
journald looses a lot of information fields that are otherwise available.
- Drop logging additional information. Most of the information is already
present in full journald records. Access using journalctl -o json.
- Use the same formatting for console as the primary daemon.
- When logging for actions, capture warnings too.
- Always log to stderr so that UI can capture the traceback and show UI error
messages. stderr is never used for returning data.
Tests:
- Run action script using command line with a error 'sudo actions/email_server
home mk a b'. See the traceback message printed on stderr (not stdout). Message
is printed with full field information in journalctl -o json.
- Main daemon writes to stderr and to journal with same formatting as before.
- Adding a warning in action code or main daemon results in printing of the
warning with desired formatting. import warnings; warnings.warn('Foo warning')
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Parse arguments in a readable way.
- Convert decorator into simple call.
- Make a simple call instead of looking for subcommand.
- Don't setup logging in global scope.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Helps test cases.
- In future, we can resize non-root partitions.
Tests:
- On an amd64 disk image, apply this patch. Increase the image size. Boot the
image. During first setup. The root partition should get expanded successfully
and show full disk size.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Avoid flake8 warnings.
- Makes the call more explicitly readable in case an exception is expected but
check=True is not passed by mistake.
Tests:
- Many tests are skipped since the changes are considered trivial.
check=False is already the default for subprocess.run() method.
- actions/package: Install an app when it is not installed.
- actions/upgrade: Run manual upgrades.
- actions/users: Change a user password. Login. Create/remove a user.
- actions/zoph: Restore a database.
- container: On a fresh repository, run ./container up,ssh,stop,destroy for a
testing container.
- plinth/action_utils.py: Enable/disable an app that has a running service.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Functions needed to spot and remove installed conflicting packages
before installation of apps.
- Remove all packages in a single operation as this way apt can search for
solutions to conflicts more easily.
- Use type hints rather than a lot of type checking. Type hints shall later be
enforced using offline checking (with mypy) or at runtime (with enforce, etc.).
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Run single remove operation on all packages]
[sunil: Use type hints instead of extensive type checking]
[sunil: Trim down the test case as it would only succeed after install]
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
package
Avoid the deprecation warning:
DeprecationWarning: str for data is no longer accepted, use bytes
sig = crypto.sign(pkey, data, 'sha512')
Tests:
- Login to web interface, access Syncthing web interface. The login should work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Done automatically by running isort . in top level directory.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- TLS configuration as recommended by Mozilla's SSL Configuration Generator with
'Intermediate' configuration. See:
https://wiki.mozilla.org/Security/Server_Side_TLS
- Disable ciphers that are weak or without forward secrecy.
- Allow client to choose ciphers as they will know best if they have support for
hardware-accelerated AES.
- TLS session tickets (RFC 5077) require restarting web server with an
appropriate frequency. See:
https://httpd.apache.org/docs/current/mod/mod_ssl.html#sslsessiontickets
- Send OCSP responses to the client and reduce their round trips.
- No need to increment apache app version number as it has already been
incremented in this release cycle for enabling HTTP/2 module.
Tests:
- FreedomBox interface is reachable with the changes.
- ssllabs.com gives an A+ rating on a server with these changes.
- All ciphers are shown as secure.
- Forward Secrecy rating is ROBUST.
- OCSP stapling shows as enabled.
- Client support seems to match the expected after dropping <= TLS1.1.
- Session resumption with tickets shows as disabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Enabling the module automatically sets 'Protocols h2 h2c http/1.1' in shipped
module configuration.
- HTTP/2 is given higher priority over HTTP/1.1 for supported clients.
- Clients not supporting HTTP/2 continue to work with HTTP/1.1.
- Clients work by using APLN extension in TLS to figure out that server supports
HTTP/2 and use it.
- HTTP/2 improves performance.
- Recommended by Mozilla's SSL configurator: https://ssl-config.mozilla.org/.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Use php-fpm instead of using mod-php.
- Create database and setup permissions manually. Tables and initial data are
created during the initial setup process done by WordPress. Database upgrades
are handled by WordPress. Minor versions are upgraded automatically and major
version need user intervention.
- Backup/restore functionality including database.
- Install recommended extensions for performance.
- Setup and run cron jobs to ensure that scheduled publications are
completed (among other things). Service has systemd security features. Timer is
set to run every 10 minutes.
- Functional tests for adding/removing posts and backup/restore.
- Increase file upload size limit to 128MiB.
- A private mode (default) for keeping the setup process secure. Should be
disabled after first setup is completed. This uses a new approach using
file-based flag for different Apache configurations.
TODO:
- Find a nice way to allow WordPress to upload plugins/themes. Currently this
operation files and users are expected to manually scp the files to
/var/lib/wordpress/wp-content/{plugins,themes} directory.
Tests:
- Functional tests.
- Schedule publishing of a post. Notice that post got published.
- Test uploading a file larger than 2MiB.
- Test enabling permalinks. This leads to nicer looking URLs.
- Test adding images to posts/pages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
email_server:
- `-i` option passes all remaining arguments to action
- delete unused "touch file" option
Views:
- delete broken links
- add tabs to every page
- separate admin tabs from user tabs
- Rewrote action script to eliminate stdin communication
- Changed return type of audit.*.get()
- An audit can return multiple lines of diagnostics
- Move recommended endpoint URLs into function docstrings
- Install postfix and manage the service
- Import ugettext
- Dummy forms and views
- <module>.version (integer) is required
[sunil: Disable the app until remaining issues are worked out]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Guard against removing a hold while the package manager is busy.
Test:
1. Place hold on freedombox package.
2. Wait 3 minutes in development mode.
- Package is held.
3. Touch /var/lib/freedombox/package-held.
4. Wait 3 minutes in development mode.
- Package is not held.
- Flag is removed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
This is to recover from a situation where the package is held, and
then plinth process is interrupted so it cannot release the hold.
We check for and recover from this situation when installing new apps,
and when running dist upgrade. This provides another way to recover
from the problem, by running manual update.
Tests:
- (normal) No hold on freedombox package, and flag is not set. Run
manual update. Afterwards, there is no hold, and flag is not set.
- (admin preference) Place hold on freedombox package, but flag is not
set. Run manual update. Afterwards, there is still a hold, but flag is
not set.
- (recovery) Place hold on freedombox package, but flag is
set. Run manual update. Afterwards, there is no hold, and flag is not
set.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- Re-use the same validator in Matrix Synapse.
- Avoid importing plinth classes in actions files.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This implementation is very similar to that of Matrix Synapse with a lot
of code duplicated.
One major difference is that ejabberd doesn't have a conf.d/ directory.
So, the managed configuration and overridden configuration cannot be
cleanly separated.
Whether the configuration is managed or not is determined by the
presence of a file under `/etc/ejabberd`. Managed coturn configuration
isn't stored in ejabberd, since only one set of configuration can be
stored at a time. If the admin chooses to use the managed configuration,
the current coturn configuration is fetched and used to configure
ejabberd.
Fixes#1978
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Matrix Synapse will automatically pick up the locally installed coturn server
during its installation. This handles only the case where coturn is installed
and configured with a valid TLS domain name before matrix-synapse is installed.
- Allow overriding STUN/TURN config. Matrix Synapse uses the local coturn
server's configuration by default. However, an administrator can override the
STUN/TURN configuration from FreedomBox web interface. Allow administrator's
overrides to co-exist with FreedomBox's managed STUN/TURN configuration.
Administrator's configuration, if it exists, always overrides FreedomBox's
managed configuration. Any updates to FreedomBox's managed configuration will
have no impact on the administrator's configuration since the latter takes
precedence.
Sunil:
- Collapse multiple turn actions into a single one for simplicity. Sending empty
configuration means removal of the configuration.
- Ensure that when removing configuration file is idempotent.
- Manage TURN configuration even when app setup is not yet completed. This fixes
issue with TURN configuration not getting setup on app installation and setup.
- Fix issue with TURN configuration getting updated on form submission even when
the field is not changed. This is due to mismatch between the browser submitted
\r\n with the internal \n.
- Simplify JavaScript for the form and attach handlers only after DOM is ready.
- Drop the no-JS message since the loss of functionality is trivial and to
reduce translation burden.
- Fix issue with URIs and secret parameters not getting updated unless the
managed checkbox changes.
- Drop specialized success messages for TURN configuration update to reduce
translation burden.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Avoid high CPU usage due to uwsgi continuously restarting before searx
config is upgraded.
Helps #1644.
Tests: Ran dist-upgrade for each case:
- Without searx installed -> no log messages related to searx.
- With searx installed but not enabled -> searx config is upgraded.
- With searx installed and enabled -> searx is disabled, later config
is upgraded, and searx is re-enabled. Searx page can be accessed at
the end.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Take a snapshot before dist upgrade, if supported.
Tests:
- Ran dist upgrade test with apt snapshots enabled. Snapshot is taken
at beginning. Apt snapshots are enabled at end.
- Ran dist upgrade test with apt snapshots disabled. Snapshot is taken
at beginning. Apt snapshots are disabled at end.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Ensure that plinth is fully started so that it can detect the apt
update, and perform force_upgrades.
Test: Perform dist upgrade. At the end, all packages have been
upgraded.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Remove the percentage check, which was incorrectly checking used
percent, not free percent.
Closes: #2046.
Tests:
- Fill up disk space so that 4.5G is free. Dist upgrade does not start
due to not enough free space.
- Fill up disk space so that 5.5G is free. Dist upgrade can be
started.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Identify Freedbombox admin who installed the app to be zoph admin.
- Implement backup and restore.
- Photos directory should remain at /var/lib/zoph, for proper operation of
backups.
- There remains an issue that the App is enabled before it is configured but
will not work correctly until configuration.
- OpenStreetMap mapping is enabled. These should be configurable before
installation.
- Add initial forms.py and views.py to Zoph app, however these are currently
unused as urls.py does not invoke the view. When the view is not invoked the
Zoph App installs, with single signon, currently as the first LDAP user, rather
than the plinth user.
- The first user's preferences are not set, and need to be manually set to avoid
other errors.
* Sunil's changes
- Squash commits and re-split them.
- Drop documentation as it will be auto populated from wiki later.
- Remove outdated validation code from manifest.
- Drop some dead code.
- Don't send MySQL password over command line for improved security. Instead
rely on Unix authentication of root user similar to backup/restore process.
- Use JSON for exchanging configuration dump to avoid encoding errors.
- Add username validation to avoid a potential SQL injection.
- Update description for neural tone and brevity. Add information about how user
accounts work in FreedomBox
- Fix functional tests.
- Drop all code related to changing photos path until it is ready.
- Update URL from /zoph to /zoph/ to avoid another redirect.
- Fix disabling the app.
- Use icon that Zoph uses for favicon as logo. Update copyright file.
- Fix spelling unzip.
- Minor refactors. Run yapf and isort.
- Use subprocess.run() instead of os.popen() everywhere for better security with
argument parsing.
- Enable OpenStreetMap by default. User have a choice to disable it before using
the app. Add label to explain privacy concerns.
- Fix dropping database by using --force argument.
- Cleanup enabling the app to not enable the app when updating configuration.
- Use AppView's default template instead of overriding.
- Update functional tests to just check if the app is enabled/disabled as
expected. Checking that Zoph site is available will require reliable handling of
admin user.
Signed-off-by: John Lines <john@paladyn.org>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
matrix-synapse 1.26 fails to start on FreedomBox stable if it doesn't
find the right version of python3-psycopg2. Version 2.8 of this library
is currently not available in Debian stable and has to be installed from
the buster-backports repository.
Tested the upgrade on a FreedomBox stable virtual machine.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Separate checking for dist upgrade from other setup. We don't want to
start a dist upgrade during the module's setup.
Tests: Checked logs for the following:
- Up to date.
- Updates not enabled.
- Dist upgrade started.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
With default logging settings, only print the result.
Tests: Checked output for following cases:
- Dist upgrade flag exists.
- System is Debian testing.
- Latest stable release without --test.
- Automatic updates not enabled.
- Not enough free space.
- Dist upgrade started.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
