52 Commits

Author SHA1 Message Date
Alice Kile
eb83e00011
fix formatting issues
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-12-07 13:08:35 -05:00
Sunil Mohan Adapa
9db1f186cd
backups: Show error when password is provided for unencrypted repo
- Without this error, users will believe that a repository is encrypted (as
  password is provided) but it is not (repository was created earlier without a
  password).

- Remove the need to send credentials, an instance property, while getting
  encryption data, an instance method.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:53:11 -04:00
Sunil Mohan Adapa
00da744a94
backups: Show lock icon for encrypted repositories
- Remove unused CSS.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:53:08 -04:00
Sunil Mohan Adapa
0465ce0efd
backups: Handle errors when adding disk repository
- Simplify repository initialization using inheritance.

- Share the code for saving repository for disk and remote repositories. Error
  messages are properly handled for disk repositories too.

- Move logic to create remote SSH directory to SSH repository class instead of
  views.

- Create a new error for handling borg repository already existing while
  initialization.

- Get information of repository after initializing so that password errors are
  caught early.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:53:03 -04:00
Sunil Mohan Adapa
c159c484ec
backups: Improve handling borg errors
- Use regular expression search instead of string search.

- Fix issue with incorrect password error not getting recognized properly.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:57 -04:00
Sunil Mohan Adapa
77a93d967f
backups: Minor simplification in running of action script
- No need to override run method in derived class if known_credentials is a
  class property exposed by base class.

- Move a method for cosmetic.

- Minor formatting.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:55 -04:00
Sunil Mohan Adapa
357e9a412a
backups: Minor cosmetic changes
- Add documentation strings.

- Styling blank lines.

- Yapf auto-formatting.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:52 -04:00
Sunil Mohan Adapa
7467fa4553
backups: Separate repository loading from instantiation
The constructor of the Repository object is being used for two distinct
purposes. One is to load the object from database and other to instantiate it
with parameter such that it can be saved to database. Separating the two usages
to different methods simplifies code and parameter passing for consumers.

Also turn some class specific constants from globals to class constants.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:49 -04:00
Sunil Mohan Adapa
5b01689a92
backups: Clarify two separate uses of name create_repository
It is being used to mean initialize the borg repository as well as creating an
instance of the repository class object. Use 'initialize' for former and
'get_instance' for latter.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:46 -04:00
Sunil Mohan Adapa
6c14e34875
backups: Implement hostname property on SSH repository
Use repository class instances instead of custom ad-hoc data structures such as
repo_data.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:44 -04:00
Sunil Mohan Adapa
143c9e494d
backups: Make mountpoint property private
It is not required by any consumers. Allows for better abstraction on a complex
class.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:39 -04:00
Sunil Mohan Adapa
d05bbab751
backups: Rename repo_path to borg_path for clarity
borg_path clearly signifies that it is to be used by borg and consumers of the
class will not be confused by it.

Also rename some repo_path variables in test cases.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:36 -04:00
Sunil Mohan Adapa
5865fbea26
backups: Minor change to disk repository name
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:33 -04:00
Sunil Mohan Adapa
68fed450a9
backups: Rename remove_repository method to remove
Since the method is being called on Repository classes, _repository suffix is
redundant.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:30 -04:00
Sunil Mohan Adapa
461741c33f
backups: Expose repository path as property
Make consumers depend on the repository classes instead of lower level store
API.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:27 -04:00
Sunil Mohan Adapa
e8b324eece
backups: Minor cosmetic fixes
- Remove some pylint warnings

- Add documentation strings.

- Yapf auto-formatting.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:22 -04:00
Sunil Mohan Adapa
0b0791d78f
backups: Introduce method for checking if a repository is usable
get_repositories method will return repositories instead of dictionaries for
view content. This will make it usable in more situations.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:20 -04:00
Sunil Mohan Adapa
0df07f5cf2
backups: Rename network_storage module to store
Same module is being used to store/retrieve local disk repositories also.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:17 -04:00
Sunil Mohan Adapa
5df34d1927
backups: Simplify listing repositories in index page
- Use sort_order property to decide which type of repositories should be listed
  first.

- Remove getting repositories of a given type and retrieve all of them at the
  same time.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:14 -04:00
Sunil Mohan Adapa
b5d7a910dd
backups: Simplify checking repository capabilities using flags
Also make storage_type an abstract property so that derived classes are forced
to override it.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:11 -04:00
Sunil Mohan Adapa
5136304212
backups: Fix removing local repository
- Remove incorrect instantiation of SSH repository for local disk repositories.

- Clarify the removal message to convey that backups are not removed.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:08 -04:00
Joseph Nuthalpati
de4a019063
backups: Save new backup location to plinth database
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:51:58 -04:00
Joseph Nuthalpati
7c7ad6d56a
backups: Refactor class hierarchy in repository.py
- Reduce repetition between various kinds of BorgBackup repositories

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil@medhas.org Undo blank line removals]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:51:54 -04:00
Nikolas Nyby
d2e987ef3c
Fix a handful of typos in docs and comments
Found with [codespell](https://github.com/codespell-project/codespell/)

Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-07-24 12:39:11 -07:00
Sunil Mohan Adapa
2c721659f4
backups: Un-mount SSH repositories before deleting them
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:45:28 +05:30
Sunil Mohan Adapa
6821d73025
backups: Minor cleanup
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:45:20 +05:30
Sunil Mohan Adapa
dd5d93637f
backups: Don't send passphrase on the command line
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:44:54 +05:30
Sunil Mohan Adapa
463c620c65
backups: Remove known_hosts file from config file
- There is no compelling reason to make the file configurable. Simplifies
  configuration file if we make it relative to FreedomBox data directory.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:44:34 +05:30
Sunil Mohan Adapa
76efccce37
backups: Fix and refactor adding a new remote repository
- Fix encrypted repositories getting created without encryption.

- Set verified=False by during save operation for safety.

- Handle common error scenarios and show proper messages. Such as authentication
  failure.

- Use pathlib to simplify file handling code.

- Split nested code for readability and do better function splits.

- Expand ~ only if it is at the beginning of the path.

- Allow empty repository path as allowed by SSH.

- Don't internationalize log messages.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:44:18 +05:30
Sunil Mohan Adapa
661a00198e
backups: Minor styling changes
- Add docstrings.

- Add blank lines for extra readability.

- Use <span> instead of <i> tag for icon in template.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:49 +05:30
Sunil Mohan Adapa
984b7dca88
backups: Cleanup auto-mounting SSH repositories
Remove auto-mounting of repositories during instantiation entirely. It is better
to explicitly mount later.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:40 +05:30
Joseph Nuthalapati
2c97e1e02e
backups: Read file path of known_hosts directly from plinth.config
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:02 +05:30
Joseph Nuthalapati
3a6dcbe7a7
Verify SSH hostkey before mounting
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:42:41 +05:30
Joseph Nuthalapati
0b43caf81d
Add SSH hostkey verification
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:42:25 +05:30
Sunil Mohan Adapa
c2f7bd6354
backups: Fix failing test case
When run on empty directory FileNotFoundError is raised. Include it in the list
of expected errors.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-18 21:51:29 -05:00
Sunil Mohan Adapa
655c2be214
backups: Improve performance of backup download
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:34 -05:00
Sunil Mohan Adapa
01492895c4
backups: Fix incomplete download archives
- Downloaded archives can't be fully extracted as tar.gz is incomplete at the
  end and corrupt. This is due to complete gzip streaming implementation that is
  does not flush the final bytes of gzip stream. Remove custom implementation
  and get gzipped stream directly from borg.

- Fix mimetype for .tar.gz to application/gzip.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:32 -05:00
Sunil Mohan Adapa
eb2b3bd86e
backups: Minor refactoring
- Mark some methods private.

- Fix some flake8 warnings.

- Remove unused exception.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:29 -05:00
Sunil Mohan Adapa
de22c79665
backups: Minor styling fixes
- Run yapf

- Fix flake8 errors/warnings.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:24 -05:00
Sunil Mohan Adapa
34e0394713
backups: Minor styling fixes
- Run yapf

- Address some flake8 errors/warnings.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:19 -05:00
Sunil Mohan Adapa
623bbc87e8
backups: Remove incorrectly set buffer size during download
- Don't set bufsize to 1 while streaming backup download. This is only effective
  with text streams with universal_newline flag set. An actual buffer size of 1
  is very inefficient and plain wrong. Leave the python default of
  io.DEFAULT_BUFFER_SIZE.

- Minor simplification to argument passing.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:15 -05:00
Michael Pimmer
bb95229a91
Backups: unittests for accessing repository with borg directly
- adapt action and write tests for accessing a borg repo directly
  via borg+ssh, without mounting it
- some docstring updates

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:51 -05:00
Michael Pimmer
b2f5d68ca6
Backups: allow creating archive in unmounted repository
- mount repository if necessary before creating an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:38 -05:00
Michael Pimmer
067fea7d17
Backups: functional tests update; restoring backup bugfix
- updated functional tests to work with changed UI
- fixed bug when restoring an uploaded file

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:31 -05:00
Michael Pimmer
3724dac9e6
Backups: Cleanup and improved error handling
- fixes issues as supposed by jvalleroy
- new repositories always get a UUID so they can immediately be fully
  used (mounted, queried etc) also before saving them
- remove test connection page -- errors are shown on form submission
- improved error handling when creating remote repositories

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:24 -05:00
Michael Pimmer
eab8991b54
Backups: support for encrypted repositories
- implement download, restore, delete archives of encrypted
  repositories
- change how BorgRepository and SshBorgRepository handle path
- when/before creating remote repositories, check whether the
  connection works
- updated tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:17 -05:00
Michael Pimmer
6651c0468e
Backups: remove archive name when creating an archive
- use datetime as archive name w/o asking user
- updated functional tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:10 -05:00
Michael Pimmer
c223250569
Backups: remove unittests to backups test directory
And add one bugfix to reset cfg.actions_dir after changing it

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:00 -05:00
Michael Pimmer
3c24c9d63b
Backups, remote repositories: change network_storage to dict
- change network_storage storage to dict format for easier handling
- added unittests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:53 -05:00
Michael Pimmer
15e26caa23
Backups, remote repositories: create/delete/restore of remote repos
- updated unittests
- implemented create/delete/restore remote archives

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:48 -05:00