Fixes: #2376.
Fixes: #2317.
When an app is removed, its packages are purged. However, there is another
installed app that needs these packages, we should keep those packages. We have
already implemented checking the packages against other apps' packages. However,
we are not checking if we are removing dependencies of other apps' packages.
This will still result in removal apps' packages.
To solve this problem, get list of packages of all the apps, then iterate over
their dependencies recursively and compile a comprehensive list of packages to
keep. Use this to reduce the set of packages to remove.
Tests:
- Without the patch, install bepasty and janus. Uninstall janus app and notice
that bepasty package is removed. With the patch, the problem is not observed.
- Printing the comprehensive list of packages to keep shows an extensive set
computed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- We have new way to show extra details with exceptions in HTML.
- This handling of error details in PackageException is mostly unused and
unnecessary complexity.
Tests:
- Introduce exception into package.install() and package.uninstall() methods.
Test that exceptions are being shown properly in Django error messages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Enable all apps, and run diagnostics. Diagnostic descriptions are formatted
as expected.
- Change the language to Spanish, and view the diagnostic results. Diagnostic
descriptions are translated as expected.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Ensure that each diagnostic test category can be identified by easy prefix
matching on the test ID.
- Give a different unique IDs each different kind of test. More specific tests
of a type get a different kind of ID.
- Make comparison of diagnostic test results in test cases more comprehensive.
- Simplify code that shows the number if issues identified.
- In many languages, there is complex logic to write plural forms. Plurals
can't be handled by assuming singular = 1 item and plural is > 1. Translation of
messages in Notification does not support plurals properly. Avoid this for now
by using sometimes incorrect plural form.
- For i18n we should avoid joining phrases/words. Words don't always maintain
order after translation.
- Notify about the total number of issues in diagnostics and not just the most
severe category. This is likely to draw more attention and avoid i18n
complexity.
- Dismiss the diagnostic notification if the latest run succeeded completely.
Tests:
- Unit tests pass.
- Diagnostics for following apps works: networks (drop-in config),
apache (daemon, listen address, internal firewall, external firewall),
tor (netcat), torproxy (internal only firewall, torproxy url, torproxy using
tor), privoxy (privoxy url, package available, package latest),
- Untested: Is release file available method in upgrades app.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Set unique check_id for each diagnostic check.
- Result is a string-based enumeration. The default value (NOT_DONE) can be
used for diagnostic checks that have not been completed yet.
- Result is StrEnum so that the return value of check_url can still be used
directly as a diagnostic result.
Closes: #2375
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
The primary reason for the existence of uninstall feature is to clean up the
existing working or non-working setup and re-install freshly. It is not to
remove the effects of the installed app since disable works well for that. It is
seldom to free up space since even on a microSD card, the space occupied by most
apps is negligible.
Tests:
- Unit tests pass
- Functional tests for all apps pass (except for known failures).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Warning that exim4 will be removed is incorrectly not shown during email app
installation. Fix that.
Tests:
- Unit tests pass.
- On a fresh testing container, trying to install email app shows the warning
message that exim will be removed. Installing the email app succeeds. exim is
removed during installation.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Check if package manager is busy works
- DONE: Power app shows status in app/restart/shutdown pages
- DONE: Upgrades app shows in app page and first boot wizard page
- DONE: When attempting force upgrade, busy state results in a back-off
- DONE: An app's packages can be installed/uninstalled successfully
- DONE: apt update is run before install
- DONE: If network is not available during package install, error message is shown
- DONE: Filtering packages with configuration file prompts works. Tested with
firewall 1.0.3 to 1.2.1.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Allows multiple apps to be queued up for installation. The operation for
installing the package will wait for the package manager to become available.
Wait for 24 hours before giving up.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Task of managing an operation's progress is now performed by the new Operation
class. Drop them from setup helper.
- Task of providing install() method is now moved to package module. Instead of
storing operation specific data in setup_helper like objects, store them in
thread specific storage that can retrieved anywhere during the operation without
holding references.
- Progress of an operation show as a progress bar is currently missing. This
will be regression until fixed later.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This was required in Python 2 but useless in Python 3.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Make terminology more consistent managed vs. possible, resolve vs. actual.
- Fix regression in security report caused by comparing package expressions with
package names.
- Fix regression in package upgrades caused by comparing package expressions
with package names.
- Update API method names to improve readability and prevent accidental
mismatching of package names and package expressions. Update variable names for
same reason during usage.
Tests:
- minetest install successfully in testing.
- Security report shows non-zero value in the current vulnerabilities column.
- When an unavailable package is added to list of packages in an app, the app
can't be installed.
- When PackageOr expressions is added to an essential package, running
--list-dependencies shows an expressions with '|' in it.
- Unit tests succeed.
- Find a package with conffile prompt and add that to list of a packages in an
app like bepasty and implement a stub force_upgrade() method in the app. Run
'apt update' and that triggers and analysis of packages with conf file prompts.
This should call force_upgrade() method in bepasty and with proper argument for
list of packages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When a package expression cannot be resolved (i.e. not installable),
add a diagnostic failure result with appropriate message.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- managed_packages() finds all possible packages that could be
installed. This is used for the check in the action script.
- resolve() finds actual packages to be installed. This is used in
setup, diagnose, and has_unavailable_packages.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- Package represents a package to potentially be installed.
- PackageOr allows an alternate package, in case the first one is not
available.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2148.
Tests:
- For an app with older version of package installed, run diagnostics. A warning
is shown. Latest version available is shown correctly in the message.
- For an app with latest version of package installed, run diagnostics. Test
shows as passed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is help in eliminating the module level package_conflicts declarations.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Include the new component reference in the components index]
[sunil: Minor changes to component doc string]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Functions needed to spot and remove installed conflicting packages
before installation of apps.
- Remove all packages in a single operation as this way apt can search for
solutions to conflicts more easily.
- Use type hints rather than a lot of type checking. Type hints shall later be
enforced using offline checking (with mypy) or at runtime (with enforce, etc.).
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Run single remove operation on all packages]
[sunil: Use type hints instead of extensive type checking]
[sunil: Trim down the test case as it would only succeed after install]
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
package
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Help: #1938.
- package.py: ugettext_lazy applied to 'installing', 'downloading' and other
statuses.
- setup.html: trans function applied to "Check again" in template.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Mark remaining to status strings also]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Also add ability restore missing configuration files during reinstall.
- Reinstall is useful for restoring the original configuration files of the
package.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This includes list of packages for which conffile prompts will be shown. For
each package current version of the package, new version of the package and list
of configuration files that were modified.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is optional and does not affect normal installations. However, when
performing configuration migration in FreedomBox (due to unattended-upgrades
refusing it), it is useful as a part of strategy to read configuration, force
install new configuration files and apply configuration again. This option can
be used on such cases.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This is a fallback solution to manually refresh package lists on AWS images
since they come with no apt package lists.
- This can also be occasionally useful for people running the testing
distribution where packages might be frequently added and removed.
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
async is a reserved keyword in Python 3.7. It can no longer be used as method
parameter. Change the name so that we are ready for Python 3.7.
See: https://www.python.org/dev/peps/pep-0492/#deprecation-plans
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
