- Now that we have a mechanism for properly collecting, transmitting, and display
the stdout and stderr. There is no reason not to collect all of the stdin and
stderr.
- Also, the stdin/stderr=subprocess.PIPE is redundant and prevents the output
from getting collected for debugging. So, remove it.
Tests:
- Ran functional tests on backups, calibre, ejabberd, email, gitweb, ikiwiki,
infinoted, kiwix, mediawiki, mumble, nextcloud,, openvpn, samba, wireguard,
zoph. 2-3 issues were found but did not seem like new errors.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
- systemd daemon-reload is performed during infinoted setup.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- When an exception is raised in subprocess.run(), for that call the stdout and
stderr are not being collected. Any previous successful calls are being
collected.
- This also fixes issues with adding an existing backup repository back after
removal. Capturing stderr is essential for raising the proper exceptions and
working correctly.
Tests:
- Remove an existing backup repository and add it back again. It fails with the
patches and succeeds with the patches.
- Remove an existing encrypted backup repository and add it back again with the
wrong password. A proper error message is shown 'Incorrect encryption
passphrase'.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- A local storage object must exist globally shared by all threads. Then
object.__dict__ is the thread specific storage. Absent this, when multiple
actions run in parallel, one will erase the thread local object of another.
Tests:
- When an error is raised in a privileged method, then the HTML error shown
contains stdout and stderr of the involved processes.
- Running functional tests on a lot of apps does not show this error anymore.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests:
- Upto 200 lines are shown in the logs. The description and unit name of the app
is correct.
- Apps without systemd units don't have 'View Logs' menu item.
- Nextcloud container logs are shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests:
- If there is a syntax error in communication with privileged server. 'stdout'
and 'stderr' keys are present in 'exception' dictionary of the reply.
- If there is a error in the privileged method in communication with privileged
server. 'stdout' and 'stderr' keys are present in 'exception' dictionary of the
reply. The values are filled with output of the command that have been run.
- If a privileged method uses action_utils.run, then raising an exception in the
method shows proper stdout and stderr in the UI HTML message.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- To be used to run specific command as another user.
Tests:
- Unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Tests:
- Functional tests for bepasty, nextcloud, torproxy app work.
- Adding/removing a domains works.
- After first setup, nscd service is masked.
- Re-run setup on miniflux app works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- We have a hook that triggers when 'apt update' is successfully run. This hook
handles the force upgrading mechanism. It's intended purpose is to handle
packages with configuration file prompts that unattended-upgrades does not
touch. 'apt update' is run on behalf of unattended-upgrades every day on a
schedule. This is the primary time the hook is intended to run. However, the
hook also run every time FreedomBox runs 'apt update' before installing an app.
Also no operations are performed, there is a race to see of apt is available for
the operation.
- Avoid these unnecessary runs by setting an environmental variable and by
checking it before running the trigger.
- There is one place where we want to genuinely run the trigger. That is after a
distribution upgrade. Handle this case.
Tests:
- When apt update is run on the command line, the hook is triggered.
- When installing an app, however, the hook is not triggered.
- During a dist-upgrade, the hook is triggered at the end.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This could be used when the daemon can't be running while configuration is
being updated.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- When freedombox is installed on a Debian system as a package, the folder
/var/lib/freedombox does not exist. When the service is run and systemd-resolved
is attempted to be installed, we try to hold the freedombox package. We also set
a flag to indicate this in /var/lib/freedombox. Since the folder does not exist,
it fails. Presumably this will also fail when installing any package. Fix this
by creating the parent directory.
Tests:
- On a fresh Debian system, install the freedombox package with these changes.
Wait for setup to run on names app. Install operation fails without this fix and
succeed with the fix (this only happens when names app is setup before backups
app as backups app creates /var/lib/freedombox directory).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
I introduced this code quality issue when handling a merge conflict in
711c19b511f969d0dce5c36221428e8caa0e7473.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reload a service if it supports reloading, otherwise restart. Do nothing if
service is not running.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Let the .container file be created and modified even when quadlets are not
available. This is harmless.
- When upgrading from bookworm to trixie, the fallback service file is removed
if setup is re-run.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
[jvalleroy: Add daemon-reload after creating service file]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Retrieve a database password from configuration if one is set instead of
generating new one.
- Create database after starting the container. This is okay as database
configuration is not set until maintenance:install operation is run.
- Minor change to setting administrator password during install.
Tests:
- Update profile in Nextcloud and re-run setup. The updated settings are still
available.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is required because when services are restarted, their Unix domain sockets
are removed and new ones are created. The container will still be using the old
sockets and will fail to connect to the service.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This prevents timeout of the service if the image pull is slow.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Configuring just php-fpm is easier compared to configuring Apache + mod_php.
There is no need to configure trusted proxies as the requests are made using the
FastCGI protocol.
- There is no need for a full web server as we already run Apache.
- Place nextcloud data in /var/lib/container so that non-PHP files can be served
directly without php-fpm involved. This location is more suitable for switching
to nextcloud based on a .deb file (if ever). This is done by configuring the
volume to serve a bind mounted directory of our choice.
- Update Apache configuration to proxy to php-fpm instead of another web server.
Include the changes needed for Apache configuration to serve non-php files
directly.
- Managed the volume using quadlet podman systemd generator.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This is not ideal and reduces security. However it simplifies quite a bit of
setup.
- Services on the host network are already exposed to the container (however,
they could easily be protected with firewall rules).
- Container has full access to external networks already. So this part does not
change.
- This setup would be at par with how other services run on FreedomBox right
now. We can think of generalized solution for all the apps later.
- FirewallLocalProtection for the single service the runs in the container works
as usual without change.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- See quadlet(5).
- Using 'podman generate systemd' is deprecated. Quadlets are recommended.
- When using the systemd generator, enable/disable is not possible. The
container is automatically started when system is booted or systemd is reloaded
after .container file changes.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
It is unlikely that other containers will need the same volume with path for
/var/www/html.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Reduce nesting necessary nesting.
- Add some type annotations.
- Simplify writing command output to a file by passing file handle to
subprocess.run().
- Create a path for volume to eliminate some duplication.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Used when a service is needed for an operation but we don't wish to keep it
running after the operation.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Helps: #2410.
- Ensure that diagnostics methods and parameters are type checked so that we can
catch any potential issues.
- Move plinth/modules/diagnostics/check.py to plinth/diagnostic_check.py to
avoid many circular dependencies created. This is due to
plinth.modules.diagnostics automatically imported when
plinth.modules.diagnostics.check is imported. Also app.py is already (type)
dependent on diagnostic_check due to diagnose() method. To make the Check
classes independent of diagnostic module is okay.
Tests:
- Run make check-type.
- Run full diagnostics with following apps installed: torproxy, tor.
- Test to netcat to 9051 in tor works.
- Test 'port available for internal/external networks' in firewall works.
- Test 'Package is latest' works.
- Test 'Access url with proxy' in privoxy works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
[jvalleroy: Also move tests for diagnostic_check]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- There hasn't been a need for this for a long time. non-systemd environments
haven't been worked on or tested for in a long time.
- Keep the is_systemd_running() method for future use.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- All tests in patch series have been done with this patch applied
- Install and uninstall of apps works
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This was required in Python 2 but useless in Python 3.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
`apt-mark hold PACKAGES` accepts a list of packages. But if one of the
package is missing from the apt repository, then it will fail to hold
any of the listed packages. So it is necessary to try to hold each
package by itself.
Test:
- Run dist-upgrade from bullseye to bookworm. mumble-server package is
currently missing from bookworm, but it should not cause an error in
dist-upgrade.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Avoid flake8 warnings.
- Makes the call more explicitly readable in case an exception is expected but
check=True is not passed by mistake.
Tests:
- Many tests are skipped since the changes are considered trivial.
check=False is already the default for subprocess.run() method.
- actions/package: Install an app when it is not installed.
- actions/upgrade: Run manual upgrades.
- actions/users: Change a user password. Login. Create/remove a user.
- actions/zoph: Restore a database.
- container: On a fresh repository, run ./container up,ssh,stop,destroy for a
testing container.
- plinth/action_utils.py: Enable/disable an app that has a running service.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Some actions that use this function are expected to output
JSON. Any output from apt-mark can interfere with this.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Guard against removing a hold while the package manager is busy.
Test:
1. Place hold on freedombox package.
2. Wait 3 minutes in development mode.
- Package is held.
3. Touch /var/lib/freedombox/package-held.
4. Wait 3 minutes in development mode.
- Package is not held.
- Flag is removed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
This is to recover from a situation where the package is held, and
then plinth process is interrupted so it cannot release the hold.
We check for and recover from this situation when installing new apps,
and when running dist upgrade. This provides another way to recover
from the problem, by running manual update.
Tests:
- (normal) No hold on freedombox package, and flag is not set. Run
manual update. Afterwards, there is no hold, and flag is not set.
- (admin preference) Place hold on freedombox package, but flag is not
set. Run manual update. Afterwards, there is still a hold, but flag is
not set.
- (recovery) Place hold on freedombox package, but flag is
set. Run manual update. Afterwards, there is no hold, and flag is not
set.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
In case the plinth process is interrupted, the "finally" block that is
meant to unhold the package may not be executed, and the package will
stay held. The flag is used to indicate this situation, so it can be
resolved the next time apt_hold_freedombox is used.
Tests:
- (normal) No hold on freedombox package, and flag is not set. Install
any app. Afterwards, there is no hold, and flag is not set.
- (admin preference) Place hold on freedombox package, but flag is not
set. Install any app. Afterwards, there is still a hold, but flag is
not set.
- (recovery) Place hold on freedombox package, but flag is
set. Install any app. Afterwards, there is no hold, and flag is not
set.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Performs this hold separately from the others, and ignore errors only
for tt-rss.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
