- If an app declares dependency on package that is also a dependency for
freedombox, then during the app's uninstall, the dependency is attempt to be
removed and will fail (because freedombox package will be held state).
- Add freedombox (and thus its dependencies) to the list packages that should be
removed from list of packages to be removed during app uninstall.
- In test case, update list of packages attempted removal as the 'freedombox'
package is installed only in some environments.
Tests:
- Uninstall janus works. Log messages show that libjs-bootstrap5 and
node-popper2 are in the list of packages originally set to removed during app
uninstall but are later filtered out.
- Run pytest with 'freedombox' package installed and ensure all tests pass.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Ensure that packages that are not installable to negative priority are not
shown as available.
Tests:
- Set priority of an available package to less than 0. This package will be
shown as not-available in the app install page.
- Normal apps are shown as available and can be installed as usual.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- When backports repository or unstable repository freshly added by the updates
app. We will like apps to become available due to newly available Debian
packages. For this to happen 'apt update' must be called before checking if an
app is available.
Tests:
- Freshly apply the patches for upgrades app. Setup is re-run and unstable
sources file is introduced. Immediately visit the Matrix app and notice that is
shown as available and can be installed immediately.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Install matrix-synapse app by adding 'unstable' in apt sources.list. Then
remove 'unstable' from apt sources.list. Then matrix-synapse package will no
longer be found in the apt's cache.
- Try to uninstall the package. Without patch, the process errors out. With
patch, uninstall completes successfully.
- While matrix-synapse app is installed and apt cache does not contain
matrix-synapse package, install and uninstall bepasty app. Without patch,
uninstall fails. With patch, uninstall succeeds.
- Install and uninstall minetest app. 3d armor mod package is successfully
installed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Fixes: #1358
- Refresh the apt cache if required packages for an app are not found and if the
cache is more than 1 hour old (or non-existent).
- If required packages are found, don't refresh the package cache even if the
cache is outdated. This is because the check operation could lead to many
minutes of waiting before app can be installed.
Tests:
- Remove /var/lib/apt/lists/* and /var/cache/apt/pkgcache.bin. Visit an app
setup page. apt cache is updated and it take a while to check that the app is
available. App is shown as available. If page is refreshed, this time, the cache
is not updated.
- Set the modification of /var/cache/apt/pkgcache.bin file to more than 2 hours
ago with 'touch -d "2 hours ago" /var/cache/apt/pkgcache.bin'. Then refreshing
the page will not refresh the cache.
- Repeat test with an app that is not available such as Janus. Again apt cache
is refreshed. App is shown as not available. On refresh, the cache is not
updated.
- Set the modification of /var/cache/apt/pkgcache.bin file to more than 2 hours
ago with 'touch -d "2 hours ago" /var/cache/apt/pkgcache.bin'. Then refreshing
the page will not refresh the cache.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Remove redundant if condition in setup.html template
- Use JavaScript fetch() API instead of XMLHTTPRequest class
- Update a comment in test_package.py
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
- Using AJAX request instead of loading the initial page slowly.
Tests:
- Unit tests passes.
- Deluge app is not available in bookworm and is available in Trixie.
- When app is available, no message is shown. Install button is enabled.
- When app is not available a proper warning alert message is shown. Install
button is disabled.
- During check for the availability, the progress message is shown. Install
button is disabled.
- When Javascript is disabled on the page, no availability check is performed.
Install button is enabled.
- When an exception is raised in the is-available view, error message is shown.
Install button is enabled.
- When is-available view return HTML response, error message is shown. Install
button is enabled.
- When is-available view invalid JSON response, error message is shown. Install
button is enabled.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Closes: #1447
Find and rerun setup for apps after a dpkg operation is completed.
This is needed in a couple of situations:
1) Some Debian packages don't manage the database used by the package. When
these packages are updated, their database schema is left at an older version
and service might become unavailable. FreedomBox can perform the database schema
upgrade. However, FreedomBox needs to know when a package has been updated so
that database schema can be upgraded.
2) A package is installed but FreedomBox has not modified its configuration.
Newer version of package becomes available with a new configuration file. Since
the original configuration file has not changed at all, the new configuration
file overwrites the old one and unattended-upgrades deals with this case. Now,
say, the configuration file modifies some defaults that FreedomBox expects
things might break. In this case, FreedomBox can apply the require configuration
changes but it needs to notified as soon as the package has been updated.
When apt runs dpkg, after the operation is completed it triggers commands listed
under the configuration 'Dpkg::Post-Invoke'. This in turn calls this class via a
DBus notification. Here, we iterate through all the apps. If an app is currently
installed and interested in rerunning setup after dpkg operations, then its
setup is rerun. Interest is expressed using the 'rerun_setup_on_upgrade' flag on
the Package() component. If all packages of the app have not be upgraded since
the last check, we skip the operation.
Tests:
- When an app is installed from FreedomBox, the trigger is not run.
- When a package is installed from command line with apt, the trigger is run. It
does nothing.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2376.
Fixes: #2317.
When an app is removed, its packages are purged. However, there is another
installed app that needs these packages, we should keep those packages. We have
already implemented checking the packages against other apps' packages. However,
we are not checking if we are removing dependencies of other apps' packages.
This will still result in removal apps' packages.
To solve this problem, get list of packages of all the apps, then iterate over
their dependencies recursively and compile a comprehensive list of packages to
keep. Use this to reduce the set of packages to remove.
Tests:
- Without the patch, install bepasty and janus. Uninstall janus app and notice
that bepasty package is removed. With the patch, the problem is not observed.
- Printing the comprehensive list of packages to keep shows an extensive set
computed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- We have new way to show extra details with exceptions in HTML.
- This handling of error details in PackageException is mostly unused and
unnecessary complexity.
Tests:
- Introduce exception into package.install() and package.uninstall() methods.
Test that exceptions are being shown properly in Django error messages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
The primary reason for the existence of uninstall feature is to clean up the
existing working or non-working setup and re-install freshly. It is not to
remove the effects of the installed app since disable works well for that. It is
seldom to free up space since even on a microSD card, the space occupied by most
apps is negligible.
Tests:
- Unit tests pass
- Functional tests for all apps pass (except for known failures).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Warning that exim4 will be removed is incorrectly not shown during email app
installation. Fix that.
Tests:
- Unit tests pass.
- On a fresh testing container, trying to install email app shows the warning
message that exim will be removed. Installing the email app succeeds. exim is
removed during installation.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Check if package manager is busy works
- DONE: Power app shows status in app/restart/shutdown pages
- DONE: Upgrades app shows in app page and first boot wizard page
- DONE: When attempting force upgrade, busy state results in a back-off
- DONE: An app's packages can be installed/uninstalled successfully
- DONE: apt update is run before install
- DONE: If network is not available during package install, error message is shown
- DONE: Filtering packages with configuration file prompts works. Tested with
firewall 1.0.3 to 1.2.1.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Allows multiple apps to be queued up for installation. The operation for
installing the package will wait for the package manager to become available.
Wait for 24 hours before giving up.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Task of managing an operation's progress is now performed by the new Operation
class. Drop them from setup helper.
- Task of providing install() method is now moved to package module. Instead of
storing operation specific data in setup_helper like objects, store them in
thread specific storage that can retrieved anywhere during the operation without
holding references.
- Progress of an operation show as a progress bar is currently missing. This
will be regression until fixed later.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This was required in Python 2 but useless in Python 3.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Make terminology more consistent managed vs. possible, resolve vs. actual.
- Fix regression in security report caused by comparing package expressions with
package names.
- Fix regression in package upgrades caused by comparing package expressions
with package names.
- Update API method names to improve readability and prevent accidental
mismatching of package names and package expressions. Update variable names for
same reason during usage.
Tests:
- minetest install successfully in testing.
- Security report shows non-zero value in the current vulnerabilities column.
- When an unavailable package is added to list of packages in an app, the app
can't be installed.
- When PackageOr expressions is added to an essential package, running
--list-dependencies shows an expressions with '|' in it.
- Unit tests succeed.
- Find a package with conffile prompt and add that to list of a packages in an
app like bepasty and implement a stub force_upgrade() method in the app. Run
'apt update' and that triggers and analysis of packages with conf file prompts.
This should call force_upgrade() method in bepasty and with proper argument for
list of packages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When a package expression cannot be resolved (i.e. not installable),
add a diagnostic failure result with appropriate message.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- managed_packages() finds all possible packages that could be
installed. This is used for the check in the action script.
- resolve() finds actual packages to be installed. This is used in
setup, diagnose, and has_unavailable_packages.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- Package represents a package to potentially be installed.
- PackageOr allows an alternate package, in case the first one is not
available.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2148.
Tests:
- For an app with older version of package installed, run diagnostics. A warning
is shown. Latest version available is shown correctly in the message.
- For an app with latest version of package installed, run diagnostics. Test
shows as passed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is help in eliminating the module level package_conflicts declarations.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Include the new component reference in the components index]
[sunil: Minor changes to component doc string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Functions needed to spot and remove installed conflicting packages
before installation of apps.
- Remove all packages in a single operation as this way apt can search for
solutions to conflicts more easily.
- Use type hints rather than a lot of type checking. Type hints shall later be
enforced using offline checking (with mypy) or at runtime (with enforce, etc.).
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Run single remove operation on all packages]
[sunil: Use type hints instead of extensive type checking]
[sunil: Trim down the test case as it would only succeed after install]
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
package
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Help: #1938.
- package.py: ugettext_lazy applied to 'installing', 'downloading' and other
statuses.
- setup.html: trans function applied to "Check again" in template.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Mark remaining to status strings also]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Also add ability restore missing configuration files during reinstall.
- Reinstall is useful for restoring the original configuration files of the
package.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This includes list of packages for which conffile prompts will be shown. For
each package current version of the package, new version of the package and list
of configuration files that were modified.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
