88 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
039461b846
backups: yapf fixes
Reviewed-by: Michael Pimmer <info@fonfon.at>
2018-12-11 23:38:30 +01:00
Sunil Mohan Adapa
f4b0c4dc9f
backups: Don't fail tests when borg is not installed
When a user running tests is root and borg is not installed, several tests fail
due to missing borg. Skip these tests when borg is not installed.

Reviewed-by: Michael Pimmer <info@fonfon.at>
2018-12-11 23:37:38 +01:00
James Valleroy
6a3f158266
backups: Make validator errors translatable
Fix spelling error.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 20:47:08 -05:00
Michael Pimmer
493fdd572b
Backups: bump module version
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:58 -05:00
Michael Pimmer
bb95229a91
Backups: unittests for accessing repository with borg directly
- adapt action and write tests for accessing a borg repo directly
  via borg+ssh, without mounting it
- some docstring updates

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:51 -05:00
Michael Pimmer
a5ab22babf
Backups: notify that credentials of remote backups are stored
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:48 -05:00
Michael Pimmer
fc9e406bf8
Backups: allow using keyfile as credentials for sshfs mounts
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:44 -05:00
Michael Pimmer
b2f5d68ca6
Backups: allow creating archive in unmounted repository
- mount repository if necessary before creating an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:38 -05:00
Michael Pimmer
067fea7d17
Backups: functional tests update; restoring backup bugfix
- updated functional tests to work with changed UI
- fixed bug when restoring an uploaded file

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:31 -05:00
Michael Pimmer
3724dac9e6
Backups: Cleanup and improved error handling
- fixes issues as supposed by jvalleroy
- new repositories always get a UUID so they can immediately be fully
  used (mounted, queried etc) also before saving them
- remove test connection page -- errors are shown on form submission
- improved error handling when creating remote repositories

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:24 -05:00
Michael Pimmer
eab8991b54
Backups: support for encrypted repositories
- implement download, restore, delete archives of encrypted
  repositories
- change how BorgRepository and SshBorgRepository handle path
- when/before creating remote repositories, check whether the
  connection works
- updated tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:17 -05:00
Michael Pimmer
6651c0468e
Backups: remove archive name when creating an archive
- use datetime as archive name w/o asking user
- updated functional tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:10 -05:00
Michael Pimmer
c223250569
Backups: remove unittests to backups test directory
And add one bugfix to reset cfg.actions_dir after changing it

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:00 -05:00
Michael Pimmer
13e8ae1bb2
Backups, remote repository: adapt functional tests
- fix functional tests of backups
- rename export-and-download url to download

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:57 -05:00
Michael Pimmer
3c24c9d63b
Backups, remote repositories: change network_storage to dict
- change network_storage storage to dict format for easier handling
- added unittests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:53 -05:00
Michael Pimmer
15e26caa23
Backups, remote repositories: create/delete/restore of remote repos
- updated unittests
- implemented create/delete/restore remote archives

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:48 -05:00
Michael Pimmer
37aa2f9992
Backups, remote backups: fix unittests
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:45 -05:00
Michael Pimmer
cf6bbd6bba
Backups, remote repositories: use object-oriented repositories
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:42 -05:00
Michael Pimmer
27fbc982c7
Backups, remote repositories: re-use template for root location
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:38 -05:00
Michael Pimmer
e3817a1a31
Backups, remote repositories: integrate to backups index page
- integrate remote repositories into backups index page
- remove, mount and unmount repositories via UI

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:35 -05:00
Michael Pimmer
3a8b69fc82
Backups, remote repositories: start using sshfs
- actions to mount/umount via sshfs
- tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:31 -05:00
Michael Pimmer
4be1b0be76
Backups, remote repositories: uniform parameter handling
- introduce env_vars in backups script and access_params for more
  uniform handling of access parameters
- added tests for creating and deleting an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:27 -05:00
Michael Pimmer
0e2489ec23
Backups, remote repositories: implement init, info and some test
- added functionality to use remote repositories
- added some tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:22 -05:00
Michael Pimmer
4eeceaa695
Backups: allow testing the connection of ssh locations
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:17 -05:00
Michael Pimmer
b6d1237c0c
Backups: first UI sceleton for remote / encrypted backups
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:10 -05:00
Michael Pimmer
e2584be45d
Backups: Make Manifest a dict instead of a list
So it's possible to add more information like metadata etc.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-18 13:58:41 -05:00
Michael Pimmer
9cba16e0d0
Backups: minor adaption of upload file size warning
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:46 -05:00
Michael Pimmer
d4d5c15566
Backups: show free disk space on upload+restore page
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:36 -05:00
Michael Pimmer
8cc74bd203
Backups: minor cleanups
- some variable names, indentation and documentation changes
- removed unused backups action
- changed name of upload session variable to 'fbx-backups-upload-path'

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:26 -05:00
Michael Pimmer
2319b40d87
Backups: do not hardcode uploaded backup file path
And use a decorator instead of a middleware to delete backup files.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:18 -05:00
Michael Pimmer
1f9bb624e8
Backups: Stream archive downloads/exports
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:13 -05:00
Michael Pimmer
b92b66b7e9
Backups: clean up forms, names and templates
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:59 -05:00
Michael Pimmer
c8554bef9d
Backups: fix test
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:53 -05:00
Michael Pimmer
50e0bae67c
Backups: relative paths for borg extract in action script
'borg extract' expects relative paths as patterns for PATH(s)

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:46 -05:00
Michael Pimmer
35305d5e37
Backups: clean up exporting archives functionality
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:40 -05:00
Michael Pimmer
c770a7adfb
Backups: Restore directly from archive
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:29 -05:00
Michael Pimmer
51b0950ec4
Backups: uploading and import with temporarily stored file
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:24 -05:00
Michael Pimmer
ff673b0d73
Backups: export and download archives in one step
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:03 -05:00
Joseph Nuthalapati
c33b192f71
backups: Don't rely on disk labels during export/restore
Disk labels are unreliable. They may not exist. There are not unique and two
disks in the system may have the same label. Instead use the device path of the
disk/partition.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-22 19:26:22 -04:00
Joseph Nuthalapati
d5860ac179
backups: Fix issue with showing exports from disks without labels
When a disk does not have a label, its device path is used. This contains a '/'
which by default is not encoded by 'urlencode' filter in Django templates. This
leads to the expected URL not matching the regex in urls.py. Fix this by
ensuring that '/' is also encoded by 'urlencode'.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-22 19:26:17 -04:00
Joseph Nuthalapati
316d765629
backups: Implement app hooks
Each application will be able to implement backup_pre, backup_post, restore_pre
and restore_post hooks that get called before/after backup/restore
appropriately. This is to handle any edge cases that backup manifest mechanism
does not handle.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-21 19:18:50 -04:00
Joseph Nuthalapati
6cfd5774f0
backups: Enable backup/restore (no data currently)
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-21 19:18:11 -04:00
Michael Pimmer
d01c36a8f5
Backups: bugfix for downloading extracted archive files
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-15 15:10:02 -04:00
Joseph Nuthalapati
b18a80f0f2
backups: Implement disabling web configuration during backup
- Introduce a BackupApp class to store all information about application being
  backed up. This cleans up apps lists vs. manifest lists spread out in the
  code.

- Introduce ServiceHandler to abstract dealing with services and web
  configuration.

- Add enable and disable actions in apache action.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-08 18:28:54 -04:00
Joseph Nuthalapati
0be12affba
backups: Minor styling fixes
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-03 19:18:52 -04:00
Joseph Nuthalapati
8d7ede728e
backups: List apps that don't require backup too
- This serves two purposes. First is to assure user in the interface that backup
  of the module is not required. Second is to make sure that if an application
  is installed during backup it is also reinstalled during restore process (this
  need to be implemented).

- Allow backup test to run independently. Initialize the cfg module so that
  load_modules() works.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-03 19:18:50 -04:00
Joseph Nuthalapati
26764b7370
backups: Rename test_backup to test_api
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-03 19:18:48 -04:00
Joseph Nuthalapati
0ce97005b2
backups: Rename the backups API module
- Also other minor refactoring.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-03 19:18:43 -04:00
Joseph Nuthalapati
6b8b40cda6
backups: Relax schema for backup manifest data
Only check validity of keys if they are present. Allows writing simpler manifest
files without losing any information.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-03 19:18:37 -04:00
Michael Pimmer
d8fc2057d4
Backup module: added some unittests; minor doc updates
- added unittests
- introduced function backups.get_location_path()
- minor docstring / comment and naming updates

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-10-01 07:01:03 -04:00