76 Commits

Author SHA1 Message Date
James Valleroy
db993ecb16
upgrades: Disable searx during dist-upgrade
Avoid high CPU usage due to uwsgi continuously restarting before searx
config is upgraded.

Helps #1644.

Tests: Ran dist-upgrade for each case:

- Without searx installed -> no log messages related to searx.

- With searx installed but not enabled -> searx config is upgraded.

- With searx installed and enabled -> searx is disabled, later config
  is upgraded, and searx is re-enabled. Searx page can be accessed at
  the end.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-02-24 11:34:25 +02:00
James Valleroy
4db96b17e5
upgrades: Disable apt snapshots during dist upgrade
Take a snapshot before dist upgrade, if supported.

Tests:
- Ran dist upgrade test with apt snapshots enabled. Snapshot is taken
  at beginning. Apt snapshots are enabled at end.
- Ran dist upgrade test with apt snapshots disabled. Snapshot is taken
  at beginning. Apt snapshots are disabled at end.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 17:08:38 -05:00
James Valleroy
72716366c1
upgrades: Add 10 minute delay before apt update
Ensure that plinth is fully started so that it can detect the apt
update, and perform force_upgrades.

Test: Perform dist upgrade. At the end, all packages have been
upgraded.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 17:08:10 -05:00
James Valleroy
f83c763ab7
upgrades: Only check free space bytes before dist upgrade
Remove the percentage check, which was incorrectly checking used
percent, not free percent.

Closes: #2046.

Tests:

- Fill up disk space so that 4.5G is free. Dist upgrade does not start
  due to not enough free space.

- Fill up disk space so that 5.5G is free. Dist upgrade can be
  started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-10 07:54:52 -05:00
James Valleroy
975dfca7ad
roundcube: Allow upgrade to 1.4.*
Hold roundcube during dist upgrade.

Closes: #2035.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-04 17:41:31 -08:00
Joseph Nuthalapati
15f6fb1f61
matrix-synapse: python3-psycopg2 from backports
matrix-synapse 1.26 fails to start on FreedomBox stable if it doesn't
find the right version of python3-psycopg2. Version 2.8 of this library
is currently not available in Debian stable and has to be installed from
the buster-backports repository.

Tested the upgrade on a FreedomBox stable virtual machine.

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-02-04 09:51:20 -08:00
James Valleroy
3ada482b90
upgrades: Move start-dist-upgrade result string to app
Separate checking for dist upgrade from other setup. We don't want to
start a dist upgrade during the module's setup.

Tests: Checked logs for the following:
- Up to date.
- Updates not enabled.
- Dist upgrade started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:34:25 +02:00
James Valleroy
d747f77f80
upgrades: Get result of start-dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:31:39 +02:00
James Valleroy
906816f7cb
upgrades: Return reason when checking for dist upgrade
With default logging settings, only print the result.

Tests: Checked output for following cases:
- Dist upgrade flag exists.
- System is Debian testing.
- Latest stable release without --test.
- Automatic updates not enabled.
- Not enough free space.
- Dist upgrade started.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-31 22:31:15 +02:00
James Valleroy
37a38549bf
upgrades: Requires at least 5 GB free space for dist upgrade
With all apps installed, the used space increases by 4.9 GB.

Helps: #1644.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-25 14:35:30 +02:00
James Valleroy
566a1bd243
upgrades: Hold tt-rss during dist upgrade, if available
Performs this hold separately from the others, and ignore errors only
for tt-rss.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:44:03 -08:00
James Valleroy
a9914128f3
upgrades: Use full path to searx action script
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:59 -08:00
James Valleroy
8a3603cd5f
upgrades: Restart FreedomBox service at end of dist-upgrade
This was removed in an earlier change, so restore it.

Avoids a transient "IndexError: list index out of range" error caused
by django when visiting a page.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:56 -08:00
James Valleroy
2ae60802a9
upgrades: Write dist-upgrade service file in /run
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:53 -08:00
James Valleroy
2e95077d74
upgrades: Check before starting dist upgrade process
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:50 -08:00
James Valleroy
a66520f3ff
upgrades: Don't allow needrestart to restart freedombox-dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:46 -08:00
James Valleroy
0241b82479
upgrades: Install python3-systemd for unattended-upgrades
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:43 -08:00
James Valleroy
8934c22ef3
upgrades: Add service for dist upgrade
Only restart plinth if needed.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:40 -08:00
James Valleroy
66b0238146
upgrades: Ensure freedombox package is upgraded during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-01-10 20:43:31 -08:00
James Valleroy
30a0e69548
upgrades: Update searx search engines during dist upgrade
Flush stdout after printing debug info.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-28 10:48:45 -08:00
James Valleroy
092c7e70d1
upgrades: Allow grub-pc upgrade without reinstalling grub
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-28 10:48:38 -08:00
James Valleroy
d7c70b74d7
upgrades: Hold mumble-server during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor refactor to reuse list of packages with prompts]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-12-13 15:53:34 -08:00
James Valleroy
d794b575e1
upgrades: Fix sources list for dist upgrade from buster
- Check apt sources list regardless of whether we are upgrading to
stable or testing.

- Replace stable code name with new stable code name.

- When testing, also replace "stable" with code name to be tested.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-28 21:29:14 -08:00
James Valleroy
3455e72cbf
upgrades: Print steps in dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:27 -08:00
James Valleroy
6c801f117f
upgrades: Use apt_hold contextmanager
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:23 -08:00
James Valleroy
5750373ea0
upgrades: Hold freedombox package during dist upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:32:18 -08:00
James Valleroy
890ab87589
upgrades: Fix flag name in info message
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-11-25 21:31:45 -08:00
James Valleroy
49c21cabc8
upgrades: Add a setting to enable dist upgrade
- Setting is stored in kvstore.

- Enable for new installs, and once when upgrading from version
  without flag.

- Split action subcommands for activating backports and for performing
  dist upgrade.

Tests:
- Dist upgrade is enabled by default in stable container.
- Dist upgrade is disabled by default in testing container.
- Enable dist upgrade. Dist upgrade is attempted periodically.
- Disable dist upgrade. Dist upgrade is no longer attempted.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: kvstore need to be locally imported anymore, import as usual]
[sunil: Minor changes to comments for PEP compliance]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 20:41:58 -07:00
James Valleroy
0e9e80f435
upgrades: Check free space before dist-upgrade
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:12:37 -07:00
James Valleroy
bacfc4bcee
upgrades: Set a flag so interrupted dist-upgrade can be continued
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:12:34 -07:00
James Valleroy
0308d783e3
upgrades: Detect and upgrade to next stable release
Process can be tested by upgrading to testing:
$ sudo ./actions/upgrades --develop --test-upgrade

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: cosmetic: isort fixes]
[sunil: Restore BACKPORTS_REQUESTED_KEY that was accidentally removed]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:11:19 -07:00
James Valleroy
036f917efb
upgrades: Extend function to check for normal dist availability
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-05 15:09:47 -07:00
Joseph Nuthalapati
690cece2eb
matrix-synapse: Upgrade to 1.19
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-29 11:04:09 -04:00
Sunil Mohan Adapa
0c8ed13882
matrixsynapse: Allow upgrade to version 1.17
Tests:

- Run with new code. Setup is run for upgrades modules.
/etc/apt/preferences.d/51-freedombox-apps.pref contains the changes for
python3-attr.

- On stable. apt policy python3-attr shows version 19.3.0-3~bpo10+1 is the
preferred version.

- On stable, installing matrix-synapse leads to installation of python3-attr
=19.3.0-3~bpo10+1.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-07-19 07:42:52 -04:00
James Valleroy
f8eb6c8cc6
upgrades: Refactor use of lsb_release
Tests:
- On unstable and testing:
  - Ran functional tests for upgrades.
  - Ran actions "upgrades setup" and "upgrades setup-repositories".
- On testing:
  - In develop mode, activated backports.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:32 -07:00
James Valleroy
a60341f0d8
upgrades: Split apt preferences into 2 files
- 50freedombox4.pref handles freedombox package from current
  backports. Whenever upgrading to a new Debian release, backports can
  be activated for the new release, and will always include the
  freedombox package at the start.

- 51freedombox-apps.pref handles apps, and each entry is particular to
  a Debian release. For example, after bullseye release, entries from
  bullseye-backports can be added, and entries from buster-backports
  can be removed.

Tests:
- In testing container, run setup in development mode. Apt preferences
  files have the expected content.
- In stable vagrant box, install deb with these changes. Apt
  preferences files have the expected content.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:29 -07:00
James Valleroy
6ac3d85bae
upgrades: Enable backports for testing only in development mode
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:23 -07:00
James Valleroy
2176ab97ad
upgrades: Rewrite apt prefs file when activating backports
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:19 -07:00
James Valleroy
ca91120fdc
upgrades: Check that backports is for current release
- If backports is for older release, then it can be activated again to
  upgrade to latest release. (Plan is to make this automatic, but
  leave the manual option as a fallback.)

- Security notice still shown if older backports are enabled.

Tests:

- On Buster system, change distribution in
  /etc/apt/sources.list.d/freedombox2.list to
  stretch-backports. Updates page shows button to activate backports
  again. Activate and check the source list to confirm that it has
  buster-backports again.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:16 -07:00
James Valleroy
6b0744c1c7
upgrades: Use only sources file to determine if backports enabled
Tests:
- Build deb and install in buster image. Manually remove backports
  sources file. Security page does not show backports notice. Updates
  page shows button to activate backports.
- Activate backports from updates page. Success message is shown and
  button to activate backports is removed. Security page shows
  backports notice.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:44:13 -07:00
James Valleroy
d042c3c60c
upgrades: Use codename to pin freedombox from backports
Skip writing apt preferences when running sid.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:22 -07:00
James Valleroy
92d39c0b6c
upgrades: Use current release codename when enabling backports
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:15 -07:00
James Valleroy
d758a917e3
upgrades: Parameterize backports dist name
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-07-13 14:42:06 -07:00
James Valleroy
ba023c3ef8
upgrades: Skip enabling backports on testing and unstable
Uses lsb-release which is a dependency of unattended-upgrades.

Closes: #1844.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Minor change to the printed message]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-28 21:57:10 -07:00
James Valleroy
0f54fab067
apt: Run dpkg --configure -a before other actions
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-25 11:54:55 -07:00
James Valleroy
5424e1e23f
apt: Run apt-get -f install before other commands
Run `apt-get --fix-broken install` before installing package or manual
update. This will attempt to correct broken dependencies.

Tests:
- Install a package without its dependencies using `dpkg -i`.
- Both app install and manual update successfully recover from this
  situation.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-25 11:54:45 -07:00
James Valleroy
34a28e35c9
upgrades: Append unattended-upgrades-dpkg.log for more detail
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-23 20:14:30 -07:00
James Valleroy
2d1ced84cd
upgrades: Use a custom service for manual update
- Do not enable/start service during package install/upgrade
- Configure needrestart to skip restarting service

Closes: #1638.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Provide proper regex string in needrestart configuration with qr()]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-15 13:33:31 -07:00
James Valleroy
848845c432
upgrades: Don't enable backports on Debian derivatives
- Avoid introducing dependency on dpkg-vendor.

Tested:

- Install a base-files package from Ubuntu. Change
  /etc/dpkg/origins/default to point to it. Running the
  setup-repositories action does not create the backports list in apt
  sources.

Closes: #1654.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-06-03 21:39:06 -07:00
James Valleroy
6d1c637c1d
upgrades: Allow installation of python3-twisted from backports
- matrix-synapse >= 1.12 requires python3-twisted >= 18.9.0-8~.

- python3-twisted requires matching version of python3-twisted-bin.

- Increment upgrades module version.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-04-19 11:23:59 -07:00