- Now that we have a mechanism for properly collecting, transmitting, and display
the stdout and stderr. There is no reason not to collect all of the stdin and
stderr.
- Also, the stdin/stderr=subprocess.PIPE is redundant and prevents the output
from getting collected for debugging. So, remove it.
Tests:
- Ran functional tests on backups, calibre, ejabberd, email, gitweb, ikiwiki,
infinoted, kiwix, mediawiki, mumble, nextcloud,, openvpn, samba, wireguard,
zoph. 2-3 issues were found but did not seem like new errors.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
During functional tests, it was noticed that getattr() failed at the following
line. The original intent of the code is to ensure that there are no failures
when 'stdout'/'stderr' attribute are not present or when they return None.
stdout = (getattr(err, 'stdout') or b'').decode()
Tests:
- Make the UI raise incorrect password error. Notice that the error is shown
properly.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests:
- Mounting/unmounting of remote SSH repositories works.
- Creating repo, creating/deleting/list archives work.
- If a privileged method raises an exception after outputting to stdout (using
action_utils.run) then stdout is shown in the HTML UI message.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Regression: downloading does not work with sudo based action anymore. However,
sudo based actions are to be removed in later patches.
Tests:
- Downloading tar backup archive works. Untar works. Downloading gives upto
10MiB/s speed.
- If API is not called with _raw_output=True, then special exception is raised.
- Downloading tar file from command line using nc also works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Many times, merging old and new data folders is not ideal and could lead to
unexpected outcomes. Perhaps removing all the backup folders and files before
restore is ideal. However, this patch tries to introduce that approach slowly on
an experimental basis.
Tests:
- Unit tests work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Fill up disk to have very little space left. Perform a large backup operation
by writing a large file to one of the directories to be backed up. Try to
perform the backup and notice that a nice error message is shown that there is
not enough disk space instead of a generic error message.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2218.
- When borg can't acquire a lock due to it being busy, any borg operation can
fail. Show a readable message instead of a generic error.
- Also handle errors for archive already existing and archive to be deleted not
existing.
Tests:
- Functional tests for backup app work.
- Creating archive works with proper message. Providing the name of existing
archive shows proper error.
- Deleting archive works with proper message. Open two tabs by clicking on the
delete archive button. Then delete with one and try to delete the it again with
the next one. Proper error message is shown.
- While downloading an archive, delete an archive. Proper error message that
borg is busy is shown.
- Upload archive works. A proper success message is shown.
- Restore backup from archive works. A proper success message is shown.
- Restore backup from file upload works. A proper success message is shown after
upload and after restoration.
- Adding local repository works. A proper success message is shown.
- Adding remote repository works. A proper success message is shown when SSH key
is verified and repository is added.
- Removing repository works. A proper success message is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Currently, in the repository class, if an operation fails, the error messages
from borg are interpreted and re-raised. Everywhere else, the errors are
interpreted. Fix this by wrapping privileged methods at the privileged module
level instead of a context manager at the place of calling.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Run affected privileged actions through UI and notice that secret strings are
not logged (except initializing init/info a new SSH repository).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
borg throws and warning "Warning: Attempting to access a previously unknown
unencryped repository!\nDo you want to continue?" and then fails while asking
for a manual confirmation. Avoid this by telling borg not to prompt (and confirm
yes).
This commit provides a fix for the issue described here:
https://discuss.freedombox.org/t/how-do-i-restore-freedombox/2022/10
[sunil: Drop the part about handling the error message]
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- mypy does not show any errors.
- Installing ejabberd app works. Privileged actions run fine.
- Unit tests work.
- No additional testing was done as type annotations don't have any effect at
runtime.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Functional tests works
- DONE: Initial setup works
- DONE: Borg repository is created at /var/lib/freedombox/borgbackup
- DONE: With regular and with encrypted repository
- DONE: Creating a repository works
- DONE: Getting information works. When adding a existing location, incorrect
password leads to error in the add form.
- DONE: Listing archives works
- DONE: Creating/restoring an archive works
- DONE: Backup manifest is created in /var/lib/plinth/backups-manifests/
- DONE: Including an app that dumps/restores its settings works
- DONE: Exporting an archive as tar works
- DONE: Exporting a large archive yields reasonable download speeds. 31
MB/s. 1GB file in about 30 seconds.
- DONE: Restoring from an uploaded archive works
- DONE: Listing the apps inside an archive works before restore
- DONE: Errors during operations are re-raises as simpler errors
- DONE: Get info
- DONE: List archives
- DONE: Delete archive (not handled)
- FAIL: Export tar
- DONE: Init repo
- DONE: Get archive apps (not handled)
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Mounting an SSH repository works
- If an known error is thrown during mounting, a simplified error is shown.
- Unmounting an SSH repository works
- If an known error is thrown during mounting, a simplified error is shown.
- Correct status of whether the repository is mounted is shown.
- If an known error is thrown during mounting, a simplified error is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
