81 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
9bd1f80d5c
*: Always pass check= argument to subprocess.run()
- Avoid flake8 warnings.

- Makes the call more explicitly readable in case an exception is expected but
check=True is not passed by mistake.

Tests:

- Many tests are skipped since the changes are considered trivial.
check=False is already the default for subprocess.run() method.

- actions/package: Install an app when it is not installed.

- actions/upgrade: Run manual upgrades.

- actions/users: Change a user password. Login. Create/remove a user.

- actions/zoph: Restore a database.

- container: On a fresh repository, run ./container up,ssh,stop,destroy for a
testing container.

- plinth/action_utils.py: Enable/disable an app that has a running service.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-10-11 14:34:40 -04:00
Sunil Mohan Adapa
a3e21adc8b
*: Various isort fixes
- Done automatically by running isort . in top level directory.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-09-20 16:50:37 -04:00
James Valleroy
39d411be5b
upgrades: Check for held freedombox package daily
Guard against removing a hold while the package manager is busy.

Test:
1. Place hold on freedombox package.
2. Wait 3 minutes in development mode.
   - Package is held.
3. Touch /var/lib/freedombox/package-held.
4. Wait 3 minutes in development mode.
   - Package is not held.
   - Flag is removed.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-07-15 20:15:47 -04:00
James Valleroy
912d9ec9e0
upgrades: Check for held freedombox package in manual update
This is to recover from a situation where the package is held, and
then plinth process is interrupted so it cannot release the hold.

We check for and recover from this situation when installing new apps,
and when running dist upgrade. This provides another way to recover
from the problem, by running manual update.

Tests:

- (normal) No hold on freedombox package, and flag is not set. Run
  manual update. Afterwards, there is no hold, and flag is not set.

- (admin preference) Place hold on freedombox package, but flag is not
set. Run manual update. Afterwards, there is still a hold, but flag is
not set.

- (recovery) Place hold on freedombox package, but flag is
set. Run manual update. Afterwards, there is no hold, and flag is not
set.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-07-15 20:15:38 -04:00
James Valleroy
5412bd75d7
action_utils: Separate function to hold freedombox package
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2021-07-15 20:15:15 -04:00
James Valleroy
db993ecb16
upgrades: Disable searx during dist-upgrade
Avoid high CPU usage due to uwsgi continuously restarting before searx
config is upgraded.

Helps #1644.

Tests: Ran dist-upgrade for each case:

- Without searx installed -> no log messages related to searx.

- With searx installed but not enabled -> searx config is upgraded.

- With searx installed and enabled -> searx is disabled, later config
  is upgraded, and searx is re-enabled. Searx page can be accessed at
  the end.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-02-24 11:34:25 +02:00
James Valleroy
4db96b17e5
upgrades: Disable apt snapshots during dist upgrade
Take a snapshot before dist upgrade, if supported.

Tests:
- Ran dist upgrade test with apt snapshots enabled. Snapshot is taken
  at beginning. Apt snapshots are enabled at end.
- Ran dist upgrade test with apt snapshots disabled. Snapshot is taken
  at beginning. Apt snapshots are disabled at end.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 17:08:38 -05:00
James Valleroy
72716366c1
upgrades: Add 10 minute delay before apt update
Ensure that plinth is fully started so that it can detect the apt
update, and perform force_upgrades.

Test: Perform dist upgrade. At the end, all packages have been
upgraded.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 17:08:10 -05:00
James Valleroy
f83c763ab7
upgrades: Only check free space bytes before dist upgrade
Remove the percentage check, which was incorrectly checking used
percent, not free percent.

Closes: #2046.

Tests:

- Fill up disk space so that 4.5G is free. Dist upgrade does not start
  due to not enough free space.

- Fill up disk space so that 5.5G is free. Dist upgrade can be
  started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 07:54:52 -05:00
James Valleroy
975dfca7ad
roundcube: Allow upgrade to 1.4.*
Hold roundcube during dist upgrade.

Closes: #2035.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-04 17:41:31 -08:00
Joseph Nuthalapati
15f6fb1f61
matrix-synapse: python3-psycopg2 from backports
matrix-synapse 1.26 fails to start on FreedomBox stable if it doesn't
find the right version of python3-psycopg2. Version 2.8 of this library
is currently not available in Debian stable and has to be installed from
the buster-backports repository.

Tested the upgrade on a FreedomBox stable virtual machine.

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-04 09:51:20 -08:00
James Valleroy
3ada482b90
upgrades: Move start-dist-upgrade result string to app
Separate checking for dist upgrade from other setup. We don't want to
start a dist upgrade during the module's setup.

Tests: Checked logs for the following:
- Up to date.
- Updates not enabled.
- Dist upgrade started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:34:25 +02:00
James Valleroy
d747f77f80
upgrades: Get result of start-dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:31:39 +02:00
James Valleroy
906816f7cb
upgrades: Return reason when checking for dist upgrade
With default logging settings, only print the result.

Tests: Checked output for following cases:
- Dist upgrade flag exists.
- System is Debian testing.
- Latest stable release without --test.
- Automatic updates not enabled.
- Not enough free space.
- Dist upgrade started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:31:15 +02:00
James Valleroy
37a38549bf
upgrades: Requires at least 5 GB free space for dist upgrade
With all apps installed, the used space increases by 4.9 GB.

Helps: #1644.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-25 14:35:30 +02:00
James Valleroy
566a1bd243
upgrades: Hold tt-rss during dist upgrade, if available
Performs this hold separately from the others, and ignore errors only
for tt-rss.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:44:03 -08:00
James Valleroy
a9914128f3
upgrades: Use full path to searx action script
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:59 -08:00
James Valleroy
8a3603cd5f
upgrades: Restart FreedomBox service at end of dist-upgrade
This was removed in an earlier change, so restore it.

Avoids a transient "IndexError: list index out of range" error caused
by django when visiting a page.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:56 -08:00
James Valleroy
2ae60802a9
upgrades: Write dist-upgrade service file in /run
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:53 -08:00
James Valleroy
2e95077d74
upgrades: Check before starting dist upgrade process
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:50 -08:00
James Valleroy
a66520f3ff
upgrades: Don't allow needrestart to restart freedombox-dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:46 -08:00
James Valleroy
0241b82479
upgrades: Install python3-systemd for unattended-upgrades
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:43 -08:00
James Valleroy
8934c22ef3
upgrades: Add service for dist upgrade
Only restart plinth if needed.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:40 -08:00
James Valleroy
66b0238146
upgrades: Ensure freedombox package is upgraded during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:31 -08:00
James Valleroy
30a0e69548
upgrades: Update searx search engines during dist upgrade
Flush stdout after printing debug info.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-28 10:48:45 -08:00
James Valleroy
092c7e70d1
upgrades: Allow grub-pc upgrade without reinstalling grub
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-28 10:48:38 -08:00
James Valleroy
d7c70b74d7
upgrades: Hold mumble-server during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor refactor to reuse list of packages with prompts]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-13 15:53:34 -08:00
James Valleroy
d794b575e1
upgrades: Fix sources list for dist upgrade from buster
- Check apt sources list regardless of whether we are upgrading to
stable or testing.

- Replace stable code name with new stable code name.

- When testing, also replace "stable" with code name to be tested.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-28 21:29:14 -08:00
James Valleroy
3455e72cbf
upgrades: Print steps in dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:27 -08:00
James Valleroy
6c801f117f
upgrades: Use apt_hold contextmanager
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:23 -08:00
James Valleroy
5750373ea0
upgrades: Hold freedombox package during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:18 -08:00
James Valleroy
890ab87589
upgrades: Fix flag name in info message
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:31:45 -08:00
James Valleroy
49c21cabc8
upgrades: Add a setting to enable dist upgrade
- Setting is stored in kvstore.

- Enable for new installs, and once when upgrading from version
  without flag.

- Split action subcommands for activating backports and for performing
  dist upgrade.

Tests:
- Dist upgrade is enabled by default in stable container.
- Dist upgrade is disabled by default in testing container.
- Enable dist upgrade. Dist upgrade is attempted periodically.
- Disable dist upgrade. Dist upgrade is no longer attempted.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: kvstore need to be locally imported anymore, import as usual]
[sunil: Minor changes to comments for PEP compliance]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 20:41:58 -07:00
James Valleroy
0e9e80f435
upgrades: Check free space before dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:12:37 -07:00
James Valleroy
bacfc4bcee
upgrades: Set a flag so interrupted dist-upgrade can be continued
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:12:34 -07:00
James Valleroy
0308d783e3
upgrades: Detect and upgrade to next stable release
Process can be tested by upgrading to testing:
$ sudo ./actions/upgrades --develop --test-upgrade

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: cosmetic: isort fixes]
[sunil: Restore BACKPORTS_REQUESTED_KEY that was accidentally removed]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:11:19 -07:00
James Valleroy
036f917efb
upgrades: Extend function to check for normal dist availability
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:09:47 -07:00
Joseph Nuthalapati
690cece2eb
matrix-synapse: Upgrade to 1.19
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-29 11:04:09 -04:00
Sunil Mohan Adapa
0c8ed13882
matrixsynapse: Allow upgrade to version 1.17
Tests:

- Run with new code. Setup is run for upgrades modules.
/etc/apt/preferences.d/51-freedombox-apps.pref contains the changes for
python3-attr.

- On stable. apt policy python3-attr shows version 19.3.0-3~bpo10+1 is the
preferred version.

- On stable, installing matrix-synapse leads to installation of python3-attr
=19.3.0-3~bpo10+1.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-07-19 07:42:52 -04:00
James Valleroy
f8eb6c8cc6
upgrades: Refactor use of lsb_release
Tests:
- On unstable and testing:
  - Ran functional tests for upgrades.
  - Ran actions "upgrades setup" and "upgrades setup-repositories".
- On testing:
  - In develop mode, activated backports.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:32 -07:00
James Valleroy
a60341f0d8
upgrades: Split apt preferences into 2 files
- 50freedombox4.pref handles freedombox package from current
  backports. Whenever upgrading to a new Debian release, backports can
  be activated for the new release, and will always include the
  freedombox package at the start.

- 51freedombox-apps.pref handles apps, and each entry is particular to
  a Debian release. For example, after bullseye release, entries from
  bullseye-backports can be added, and entries from buster-backports
  can be removed.

Tests:
- In testing container, run setup in development mode. Apt preferences
  files have the expected content.
- In stable vagrant box, install deb with these changes. Apt
  preferences files have the expected content.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:29 -07:00
James Valleroy
6ac3d85bae
upgrades: Enable backports for testing only in development mode
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:23 -07:00
James Valleroy
2176ab97ad
upgrades: Rewrite apt prefs file when activating backports
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:19 -07:00
James Valleroy
ca91120fdc
upgrades: Check that backports is for current release
- If backports is for older release, then it can be activated again to
  upgrade to latest release. (Plan is to make this automatic, but
  leave the manual option as a fallback.)

- Security notice still shown if older backports are enabled.

Tests:

- On Buster system, change distribution in
  /etc/apt/sources.list.d/freedombox2.list to
  stretch-backports. Updates page shows button to activate backports
  again. Activate and check the source list to confirm that it has
  buster-backports again.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:16 -07:00
James Valleroy
6b0744c1c7
upgrades: Use only sources file to determine if backports enabled
Tests:
- Build deb and install in buster image. Manually remove backports
  sources file. Security page does not show backports notice. Updates
  page shows button to activate backports.
- Activate backports from updates page. Success message is shown and
  button to activate backports is removed. Security page shows
  backports notice.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:13 -07:00
James Valleroy
d042c3c60c
upgrades: Use codename to pin freedombox from backports
Skip writing apt preferences when running sid.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:22 -07:00
James Valleroy
92d39c0b6c
upgrades: Use current release codename when enabling backports
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:15 -07:00
James Valleroy
d758a917e3
upgrades: Parameterize backports dist name
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:06 -07:00
James Valleroy
ba023c3ef8
upgrades: Skip enabling backports on testing and unstable
Uses lsb-release which is a dependency of unattended-upgrades.

Closes: #1844.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor change to the printed message]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-28 21:57:10 -07:00
James Valleroy
0f54fab067
apt: Run dpkg --configure -a before other actions
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-25 11:54:55 -07:00