103 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
655c2be214
backups: Improve performance of backup download
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:34 -05:00
Sunil Mohan Adapa
01492895c4
backups: Fix incomplete download archives
- Downloaded archives can't be fully extracted as tar.gz is incomplete at the
  end and corrupt. This is due to complete gzip streaming implementation that is
  does not flush the final bytes of gzip stream. Remove custom implementation
  and get gzipped stream directly from borg.

- Fix mimetype for .tar.gz to application/gzip.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:32 -05:00
Sunil Mohan Adapa
eb2b3bd86e
backups: Minor refactoring
- Mark some methods private.

- Fix some flake8 warnings.

- Remove unused exception.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:29 -05:00
Sunil Mohan Adapa
de22c79665
backups: Minor styling fixes
- Run yapf

- Fix flake8 errors/warnings.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:24 -05:00
Sunil Mohan Adapa
c22bec1cf4
backups: Remove dead code
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:21 -05:00
Sunil Mohan Adapa
34e0394713
backups: Minor styling fixes
- Run yapf

- Address some flake8 errors/warnings.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:19 -05:00
Sunil Mohan Adapa
623bbc87e8
backups: Remove incorrectly set buffer size during download
- Don't set bufsize to 1 while streaming backup download. This is only effective
  with text streams with universal_newline flag set. An actual buffer size of 1
  is very inefficient and plain wrong. Leave the python default of
  io.DEFAULT_BUFFER_SIZE.

- Minor simplification to argument passing.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-03 17:00:15 -05:00
Sunil Mohan Adapa
41d9419ddd
storage, backups: Minor styling and yapf fixes
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-01-27 11:10:42 -05:00
Sunil Mohan Adapa
9e316baa99
storage: Fix false error message visiting home page
When visiting homepage as a non-admin user, don't throw an error into the
console about not being able to get root partition information. This is a
regression from refactoring during backups change.

Fixes: #1468.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-01-27 11:10:36 -05:00
Sunil Mohan Adapa
983f4d13e8
backups: Fix showing not-installed apps in create backup page
- Also minor styling.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-25 17:59:55 -08:00
Sunil Mohan Adapa
f32239d32b
backups: Upgrade apps before restoring them
- All install/upgrade in all cases and not just for non-snapshot cases.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-25 17:11:15 -08:00
Joseph Nuthalapati
7d6284a829
backups: Add a loader to the restore button to indicate progress
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-25 17:02:30 -08:00
Joseph Nuthalapati
32b470bc7c
backups: Automatically install required apps before restore
Fixes #1460

Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-25 17:02:05 -08:00
Sunil Mohan Adapa
de5ba012e1
icons: Fixes for switching to fork-awesome
- Update documentation pointing to Bootstrap Glyphicons.

- Consistently use trash-o. It is also more clear at the sizes we are using.

- Use hourglass-o instead of hourglass.

- Use diaspora icon instead of generic icon for diaspora app.

- In cards, don't set 3x size. It is not used. Instead we are setting our custom
  size.

- Remove unused CSS rule for .sidebar .fa

- Align the FreedomBox logo to center with 'Home' text.

- Fix swapped icons for 'Client Apps' button.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-14 17:38:01 -08:00
Prachi Srivastava
5d68f6bf52
Replace glyphicons with forkawesome icons
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-14 17:37:58 -08:00
Sunil Mohan Adapa
039461b846
backups: yapf fixes
Reviewed-by: Michael Pimmer <info@fonfon.at>
2018-12-11 23:38:30 +01:00
Sunil Mohan Adapa
f4b0c4dc9f
backups: Don't fail tests when borg is not installed
When a user running tests is root and borg is not installed, several tests fail
due to missing borg. Skip these tests when borg is not installed.

Reviewed-by: Michael Pimmer <info@fonfon.at>
2018-12-11 23:37:38 +01:00
James Valleroy
6a3f158266
backups: Make validator errors translatable
Fix spelling error.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 20:47:08 -05:00
Michael Pimmer
493fdd572b
Backups: bump module version
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:58 -05:00
Michael Pimmer
bb95229a91
Backups: unittests for accessing repository with borg directly
- adapt action and write tests for accessing a borg repo directly
  via borg+ssh, without mounting it
- some docstring updates

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:51 -05:00
Michael Pimmer
a5ab22babf
Backups: notify that credentials of remote backups are stored
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:48 -05:00
Michael Pimmer
fc9e406bf8
Backups: allow using keyfile as credentials for sshfs mounts
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:44 -05:00
Michael Pimmer
b2f5d68ca6
Backups: allow creating archive in unmounted repository
- mount repository if necessary before creating an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:38 -05:00
Michael Pimmer
067fea7d17
Backups: functional tests update; restoring backup bugfix
- updated functional tests to work with changed UI
- fixed bug when restoring an uploaded file

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:31 -05:00
Michael Pimmer
3724dac9e6
Backups: Cleanup and improved error handling
- fixes issues as supposed by jvalleroy
- new repositories always get a UUID so they can immediately be fully
  used (mounted, queried etc) also before saving them
- remove test connection page -- errors are shown on form submission
- improved error handling when creating remote repositories

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:24 -05:00
Michael Pimmer
eab8991b54
Backups: support for encrypted repositories
- implement download, restore, delete archives of encrypted
  repositories
- change how BorgRepository and SshBorgRepository handle path
- when/before creating remote repositories, check whether the
  connection works
- updated tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:17 -05:00
Michael Pimmer
6651c0468e
Backups: remove archive name when creating an archive
- use datetime as archive name w/o asking user
- updated functional tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:10 -05:00
Michael Pimmer
c223250569
Backups: remove unittests to backups test directory
And add one bugfix to reset cfg.actions_dir after changing it

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:00 -05:00
Michael Pimmer
13e8ae1bb2
Backups, remote repository: adapt functional tests
- fix functional tests of backups
- rename export-and-download url to download

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:57 -05:00
Michael Pimmer
3c24c9d63b
Backups, remote repositories: change network_storage to dict
- change network_storage storage to dict format for easier handling
- added unittests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:53 -05:00
Michael Pimmer
15e26caa23
Backups, remote repositories: create/delete/restore of remote repos
- updated unittests
- implemented create/delete/restore remote archives

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:48 -05:00
Michael Pimmer
37aa2f9992
Backups, remote backups: fix unittests
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:45 -05:00
Michael Pimmer
cf6bbd6bba
Backups, remote repositories: use object-oriented repositories
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:42 -05:00
Michael Pimmer
27fbc982c7
Backups, remote repositories: re-use template for root location
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:38 -05:00
Michael Pimmer
e3817a1a31
Backups, remote repositories: integrate to backups index page
- integrate remote repositories into backups index page
- remove, mount and unmount repositories via UI

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:35 -05:00
Michael Pimmer
3a8b69fc82
Backups, remote repositories: start using sshfs
- actions to mount/umount via sshfs
- tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:31 -05:00
Michael Pimmer
4be1b0be76
Backups, remote repositories: uniform parameter handling
- introduce env_vars in backups script and access_params for more
  uniform handling of access parameters
- added tests for creating and deleting an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:27 -05:00
Michael Pimmer
0e2489ec23
Backups, remote repositories: implement init, info and some test
- added functionality to use remote repositories
- added some tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:22 -05:00
Michael Pimmer
4eeceaa695
Backups: allow testing the connection of ssh locations
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:17 -05:00
Michael Pimmer
b6d1237c0c
Backups: first UI sceleton for remote / encrypted backups
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:10 -05:00
Michael Pimmer
e2584be45d
Backups: Make Manifest a dict instead of a list
So it's possible to add more information like metadata etc.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-18 13:58:41 -05:00
Michael Pimmer
9cba16e0d0
Backups: minor adaption of upload file size warning
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:46 -05:00
Michael Pimmer
d4d5c15566
Backups: show free disk space on upload+restore page
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:36 -05:00
Michael Pimmer
8cc74bd203
Backups: minor cleanups
- some variable names, indentation and documentation changes
- removed unused backups action
- changed name of upload session variable to 'fbx-backups-upload-path'

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:26 -05:00
Michael Pimmer
2319b40d87
Backups: do not hardcode uploaded backup file path
And use a decorator instead of a middleware to delete backup files.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:18 -05:00
Michael Pimmer
1f9bb624e8
Backups: Stream archive downloads/exports
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 09:00:13 -05:00
Michael Pimmer
b92b66b7e9
Backups: clean up forms, names and templates
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:59 -05:00
Michael Pimmer
c8554bef9d
Backups: fix test
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:53 -05:00
Michael Pimmer
50e0bae67c
Backups: relative paths for borg extract in action script
'borg extract' expects relative paths as patterns for PATH(s)

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:46 -05:00
Michael Pimmer
35305d5e37
Backups: clean up exporting archives functionality
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-11-17 08:59:40 -05:00