Closes: #2218.
- When borg can't acquire a lock due to it being busy, any borg operation can
fail. Show a readable message instead of a generic error.
- Also handle errors for archive already existing and archive to be deleted not
existing.
Tests:
- Functional tests for backup app work.
- Creating archive works with proper message. Providing the name of existing
archive shows proper error.
- Deleting archive works with proper message. Open two tabs by clicking on the
delete archive button. Then delete with one and try to delete the it again with
the next one. Proper error message is shown.
- While downloading an archive, delete an archive. Proper error message that
borg is busy is shown.
- Upload archive works. A proper success message is shown.
- Restore backup from archive works. A proper success message is shown.
- Restore backup from file upload works. A proper success message is shown after
upload and after restoration.
- Adding local repository works. A proper success message is shown.
- Adding remote repository works. A proper success message is shown when SSH key
is verified and repository is added.
- Removing repository works. A proper success message is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #1603.
- This introduces seconds and timezone in the name of the archive. When seconds
are not used in the name, taking backups too quickly on after the other fails as
the names clash.
Tests:
- Taking a backup works and creates the name in the expected ISO time format.
- Changing the timezone and restarting service creates timestamps in that
timezone.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Sending GET requests to the two views by opening theirs URLs in the browser
show a blank page. Actions are not performed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In current stable and testing, verifying SSH remote hosts using RSA is not
working. After selecting the verified RSA fingerprint, paramiko fails to connect
- A change introduced in paramiko 2.9 lead to failures when connecting to hosts
that have a verified RSA host key[1][2][3]. To fix the issue,
disabled_algorithms must be used to drop some of the other algorithms supported
by the server to force paramiko behavior. A better solution to the problem was
introduced in paramiko 3.2. Both these solutions require careful update to the
code. Considering the utility paramiko provides, the regression annoyance,
effort required for this fix, and the security implications (it is an completely
independent SSH implementation), the library does not seem to be worth the
effort in our case.
- Switch to using sshpass command line utility instead of paramiko library. The
only reason to use paramiko seems that 'ssh' command by default does not allow
us to input password easily while paramiko does.
- Another place where paramiko is being used is to check if a host is already
verified in the known_hosts file. This has been trivially replaced with
'ssh-keygen -F'.
- Exit codes provided by sshpass can replace the specific exception raised by
paramiko.
Links:
1) https://www.paramiko.org/changelog.html
2) https://github.com/paramiko/paramiko/issues/2017
3) https://github.com/paramiko/paramiko/issues/1984
Tests:
- Add a remote backup repository with and without encryption.
- Add remote backup repository with all three types of algorithms.
- Add a remote repository again with wrong password. Authentication error is
properly shown.
- Add a remote backup repository and remove it. Host remains verified. Add a
repository again.
- Add a remote backup repository and remove it. Host remains verified. Change
the fingerprint the /var/lib/plinth/.ssh/known_hosts file. Add a repository
again. A proper error is shown that remote host could not be verified.
- Add a remote backup repository and remove it. Host remains verified. Stop SSH
server on the remote host. A generic error is shown that ssh command on remote
host failed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Take app_id in a URL fragment and fill that as the default selected app in
create backup form. This URL can be used in apps to create a backup link.
Tests:
- Visit /plinth/sys/backups/create/bepasty/. Only bepasty app will be selected.
- Visit /plinth/sys/backups/create/foo/. No apps are selected.
- Visit /plinth/sys/backups/create/. All apps are selected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- DONE: Functional tests works
- DONE: Initial setup works
- DONE: Borg repository is created at /var/lib/freedombox/borgbackup
- DONE: With regular and with encrypted repository
- DONE: Creating a repository works
- DONE: Getting information works. When adding a existing location, incorrect
password leads to error in the add form.
- DONE: Listing archives works
- DONE: Creating/restoring an archive works
- DONE: Backup manifest is created in /var/lib/plinth/backups-manifests/
- DONE: Including an app that dumps/restores its settings works
- DONE: Exporting an archive as tar works
- DONE: Exporting a large archive yields reasonable download speeds. 31
MB/s. 1GB file in about 30 seconds.
- DONE: Restoring from an uploaded archive works
- DONE: Listing the apps inside an archive works before restore
- DONE: Errors during operations are re-raises as simpler errors
- DONE: Get info
- DONE: List archives
- DONE: Delete archive (not handled)
- FAIL: Export tar
- DONE: Init repo
- DONE: Get archive apps (not handled)
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is recommended by PEP-0597: https://peps.python.org/pep-0597/
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests performed:
- Ensure that backups module is setup from version zero by deleting entry in DB.
Start FreedomBox. Observe the notification comes up.
- Dismiss button dismiss the notification.
- Go to button takes us to the backups module.
- Incrementing the version number of the backups app does not show the
notification again.
- Starting with old code, the notification show up when FreedomBox is run.
Incrementing the version number of the backups app after that does not show the
notification again.
- Enable a schedule and the notification should be dismissed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Simplify save() method such that a simple load(uuid).save() does not destroy
data. Currently, the verified parameter is lost.
- Drop unused store_credentials argument to save().
- Make verified a property of the SSH repository and instantiate with
it properly.
Tests performed:
- Adding a new remote repository works with SSH verification from unknown and
known hosts.
- Mounting and unmounting works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- In a container, fill up space. Start FreedomBox in develop mode wait 3 minutes
for storage warning to show.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When there are no additional disks available, don't show an empty drop down
for disks and throw an error when submitting. Instead make it clear that no
additional disks are available and that is why repository can't be created.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Simplify repository initialization using inheritance.
- Share the code for saving repository for disk and remote repositories. Error
messages are properly handled for disk repositories too.
- Move logic to create remote SSH directory to SSH repository class instead of
views.
- Create a new error for handling borg repository already existing while
initialization.
- Get information of repository after initializing so that password errors are
caught early.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
The constructor of the Repository object is being used for two distinct
purposes. One is to load the object from database and other to instantiate it
with parameter such that it can be saved to database. Separating the two usages
to different methods simplifies code and parameter passing for consumers.
Also turn some class specific constants from globals to class constants.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
It is being used to mean initialize the borg repository as well as creating an
instance of the repository class object. Use 'initialize' for former and
'get_instance' for latter.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Use repository class instances instead of custom ad-hoc data structures such as
repo_data.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Make store an internal implementation detail of the repository class and achieve
better abstraction.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Since the method is being called on Repository classes, _repository suffix is
redundant.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
get_repositories method will return repositories instead of dictionaries for
view content. This will make it usable in more situations.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Same module is being used to store/retrieve local disk repositories also.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Use sort_order property to decide which type of repositories should be listed
first.
- Remove getting repositories of a given type and retrieve all of them at the
same time.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Also make storage_type an abstract property so that derived classes are forced
to override it.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove incorrect instantiation of SSH repository for local disk repositories.
- Clarify the removal message to convey that backups are not removed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Replace subsubmenu with buttons
- Left align icons in table header (restore and delete)
- Replace the technical term "repository" with "location"
- Confirmation dialogs shouldn't have app name, description, manual etc.
- Confirmation dialogs don't need Cancel buttons
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
[sunil@medhas.org: Remove 'primary' for all but one button]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Remove Python code to handle the extra item in the submitted form
- The select-all checkbox is never submitted in the form since it is missing the
"name" attribute
- Replace selector ':checkbox' with the faster selector '[type=checkbox]'
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This is less tiresome for users and also improves the speed of functional tests.
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Ensure that the fingerprint accepted is the one verified by user. If they
fingerprints and public keys are retrieved separately, there is chance that what
was verified by the user is not what is added to the known hosts file.
- Avoid creating a temporary file when fetching keys
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
- There is no compelling reason to make the file configurable. Simplifies
configuration file if we make it relative to FreedomBox data directory.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
- Fix encrypted repositories getting created without encryption.
- Set verified=False by during save operation for safety.
- Handle common error scenarios and show proper messages. Such as authentication
failure.
- Use pathlib to simplify file handling code.
- Split nested code for readability and do better function splits.
- Expand ~ only if it is at the beginning of the path.
- Allow empty repository path as allowed by SSH.
- Don't internationalize log messages.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
- Add docstrings.
- Add blank lines for extra readability.
- Use <span> instead of <i> tag for icon in template.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Remove auto-mounting of repositories during instantiation entirely. It is better
to explicitly mount later.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
