83 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
9a4905e832
backups: Use privileged decorator for backup actions
Tests:

- DONE: Functional tests works
- DONE: Initial setup works
  - DONE: Borg repository is created at /var/lib/freedombox/borgbackup
- DONE: With regular and with encrypted repository
  - DONE: Creating a repository works
  - DONE: Getting information works. When adding a existing location, incorrect
    password leads to error in the add form.
  - DONE: Listing archives works
  - DONE: Creating/restoring an archive works
    - DONE: Backup manifest is created in /var/lib/plinth/backups-manifests/
    - DONE: Including an app that dumps/restores its settings works
  - DONE: Exporting an archive as tar works
    - DONE: Exporting a large archive yields reasonable download speeds. 31
      MB/s. 1GB file in about 30 seconds.
  - DONE: Restoring from an uploaded archive works
  - DONE: Listing the apps inside an archive works before restore
- DONE: Errors during operations are re-raises as simpler errors
  - DONE: Get info
  - DONE: List archives
  - DONE: Delete archive (not handled)
  - FAIL: Export tar
  - DONE: Init repo
  - DONE: Get archive apps (not handled)

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2022-10-08 18:53:57 -04:00
Sunil Mohan Adapa
900c0d30b9
*: Drop module level app property
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2022-08-15 10:36:29 -04:00
Sunil Mohan Adapa
75f6abac1e
*: Make setup method part of App class for all apps
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.

- This brings us closer to not needing to implement setup() method for some of
the typical apps.

- Remove default value None for old_version parameter.

  - A valid integer value is always passed to this call.

  - The value of None is undefined.

  - Simplifies the App API slightly.

- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.

  - Simplifies the setup methods significantly. Eliminates a class of
  bugs (some of them seen earlier).

  - The UI can show a simple 'installing...' or progress spinner instead of
  individual stages.

  - There are currently many inconsistencies where many operations are not
  wrapped in helper.call() calls.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2022-08-15 10:36:16 -04:00
Sunil Mohan Adapa
3c7bc4a192
*: pylint: Explicitly specify encoding when open a file
This is recommended by PEP-0597: https://peps.python.org/pep-0597/

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2022-07-04 19:45:57 -04:00
Sunil Mohan Adapa
019a785cc3
*: Drop module level depends declaration
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-04 16:39:49 -05:00
Sunil Mohan Adapa
a3d4d99b33
*: Drop use of module level version
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-04 16:38:08 -05:00
Sunil Mohan Adapa
cf36a9d385
*: Drop use of module level is_essential flag
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-04 16:37:59 -05:00
Sunil Mohan Adapa
4bf57c5707
*: Drop use of managed_packages and rely on Packages component
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-12-04 16:34:52 -05:00
Sunil Mohan Adapa
143bff3984
backups: Split app initialization
Tests:

- After startup, every 3 minutes, backup schedule check is performed if a
schedule is set.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-11-15 10:12:19 -05:00
Joseph Nuthalapati
cd4bd17173
components: Introduce new component - Packages
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2021-11-08 16:03:42 -08:00
Sunil Mohan Adapa
74214c18ae
*: Use Django gettext functions instead of ugettext
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.

- ugettext is already a simple alias of gettext. So, no regressions are
expected.

Tests:

- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.

- Using Django 3.2 there are no warnings related to removal of ugettext
functions.

- Ran regular unit tests.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-09-20 16:50:16 -04:00
Sunil Mohan Adapa
a022eed0eb
backups: Show notification on error during scheduled backups
- Keep a count of number of consecutive failures. Reset the count on a
successful backup.

- Keep the count for each repository.

Tests performed:

- Insert an exception into method for running a scheduled backup.

- When the FreedomBox is run in develop mode, it attempts to take backups every
3 minutes.

- It fails and shows the notification. Text is good. The error count is 1 and
error text is same as exception text. Go to backups button redirects to backup
page.

- In another 3 minutes another attempt is made and count increments this time.

- If the notification is dismissed, after another attempt is made, notification
comes back with incremented count.

- When the exception is removed and FreedomBox is restarted, schedule backup
succeeds and notification is dismissed automatically.

- When backup is deleted and exception is reintroduced, the notification comes
back with count reset to 1.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-01-21 20:59:48 -05:00
Sunil Mohan Adapa
d4e9e7a965
backups: Add a notification to suggest users to enable schedules
Tests performed:

- Ensure that backups module is setup from version zero by deleting entry in DB.
Start FreedomBox. Observe the notification comes up.

- Dismiss button dismiss the notification.

- Go to button takes us to the backups module.

- Incrementing the version number of the backups app does not show the
notification again.

- Starting with old code, the notification show up when FreedomBox is run.
Incrementing the version number of the backups app after that does not show the
notification again.

- Enable a schedule and the notification should be dismissed.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-01-21 20:56:05 -05:00
Sunil Mohan Adapa
b33362cb7a
backups: Trigger schedules every hour
- This will lead to backups only once a day or so.

- In case of errors, backups are tried every hour.

- Cleanups are also triggered but cleanups happen only after a successful
backup.

Tests performed:

- In development mode, a new backup is taken after 3 minutes of enabling a
repository's schedule.

- Error in one repository does not effect the backup of other repositories.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-01-21 20:53:42 -05:00
Sunil Mohan Adapa
01e00cdde4
backups: Allow comments to be added to archives during backup
Tests performed:

- Schedules are able to store and retrieve comments properly. Information about
schedule backups stored in comments is extracted properly.

- Unit tests run.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2021-01-21 20:33:47 -05:00
Sunil Mohan Adapa
fb1898befc
backups: Use the backup component in all apps
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2021-01-04 13:47:38 +02:00
James Valleroy
f88ced9706
app: Add donation links in dropdown menu
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Undo a minor isort refactor in searx]
[sunil: Change the menu label to 'Donate']
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-11 19:49:22 -07:00
Joseph Nuthalapati
d92ca09e19
framework: Remove module init() functions
Fixes #1906

Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-03 18:06:48 -04:00
Sunil Mohan Adapa
dc75a52d67
backups: Make app available by default
We have already progress quite far along in supporting backups and restore. This
is a critical functionality of FreedomBox.

Fixes: #1459.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-07-19 18:51:57 -04:00
Sunil Mohan Adapa
9368504da5
*.py: Use SPDX license identifier
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2020-02-19 14:38:55 +02:00
Sunil Mohan Adapa
b576a77c3d
app: Refactor all apps to use the Info component
- Remove the need to pass all the individual information elements to the AppView
  separately. This eliminates many issues with elements that were mistakenly not
  sent to AppView. Also reduces a lot of code duplication.

- Create App classes for power and sso for consistency.

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-02-16 18:22:01 -05:00
Alice Kile
eb83e00011
fix formatting issues
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-12-07 13:08:35 -05:00
Sunil Mohan Adapa
357e9a412a
backups: Minor cosmetic changes
- Add documentation strings.

- Styling blank lines.

- Yapf auto-formatting.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:52 -04:00
Sunil Mohan Adapa
7467fa4553
backups: Separate repository loading from instantiation
The constructor of the Repository object is being used for two distinct
purposes. One is to load the object from database and other to instantiate it
with parameter such that it can be saved to database. Separating the two usages
to different methods simplifies code and parameter passing for consumers.

Also turn some class specific constants from globals to class constants.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:52:49 -04:00
Joseph Nuthalpati
8a93b5b90c
backups: Allow adding backup repositories on multiple disks
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil@medhas.org Fix showing form choices, undo blank line removals]
[sunil@medhas.org Fix typo in tooltip for add repo button]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-09-07 12:51:40 -04:00
Nikolas Nyby
f81b1751ce
Fix typos in module init docs
Intialize -> Initialize

Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-07-24 14:55:22 -07:00
Sunil Mohan Adapa
dd5d93637f
backups: Don't send passphrase on the command line
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:44:54 +05:30
Sunil Mohan Adapa
463c620c65
backups: Remove known_hosts file from config file
- There is no compelling reason to make the file configurable. Simplifies
  configuration file if we make it relative to FreedomBox data directory.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:44:34 +05:30
Sunil Mohan Adapa
661a00198e
backups: Minor styling changes
- Add docstrings.

- Add blank lines for extra readability.

- Use <span> instead of <i> tag for icon in template.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:49 +05:30
Sunil Mohan Adapa
9c8674baa3
backup: Allow SSH directory paths with : in them
- From SSH manual, a path may contain : if it is presented as absolute path.
  Update regular expression for splitting accordingly.

- Allow Null paths similar to SSH.

- Perform a full regular expression match when splitting path.

- Simplify regular expression.

- Update tests.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:28 +05:30
Joseph Nuthalapati
bd1874d774
backups: Add regex validation for ssh_repository field
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:08 +05:30
Joseph Nuthalapati
2c97e1e02e
backups: Read file path of known_hosts directly from plinth.config
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:43:02 +05:30
Joseph Nuthalapati
3a6dcbe7a7
Verify SSH hostkey before mounting
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:42:41 +05:30
Joseph Nuthalapati
67404a1c2d
backups: Make paramiko a dependency of freedombox package
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:41:28 +05:30
Joseph Nuthalapati
e618e08fe7
backups: Improve UX of adding ssh remote
- Create empty directory if not exists
- Check if the directory is empty
  - if not empty, check if it's an existing backup repository
  - else throw an error

Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
2019-07-03 12:40:35 +05:30
Sunil Mohan Adapa
e4351b6b97
Introduce daemon component to handle systemd units
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-06-13 20:18:02 -04:00
Sunil Mohan Adapa
46f162d093
app: Add unique ID to each app class
Also maintain a global list of apps

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-06-13 20:17:59 -04:00
Sunil Mohan Adapa
b96d901071
Introduce component architecture and menu component
- Introduce base class for all apps that will contain components. With
  unittests.

- Introduce base classes for components. With unittests.

- Turn Menu class into an app component.

  - Further cleanup Menu class.

  - Update tests.

  - Maintain a global list of menu items and look them up easily. Generalize
    such that subsubmenus can later be merged into Menu class.

  - Cleanup scope of main menu initialization.

  - Use None instead of empty strings for various values. Ensure that
    printing short_description does not show 'None' in output.

  - Use enable/disable instead of promote/demote.

- Use menu component in all apps.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-06-07 11:48:04 -07:00
Sunil Mohan Adapa
ff8d85a88f
backups: Add link to manual page
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-02-07 19:12:04 -05:00
Prachi Srivastava
5d68f6bf52
Replace glyphicons with forkawesome icons
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2019-01-14 17:37:58 -08:00
Michael Pimmer
493fdd572b
Backups: bump module version
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:58 -05:00
Michael Pimmer
067fea7d17
Backups: functional tests update; restoring backup bugfix
- updated functional tests to work with changed UI
- fixed bug when restoring an uploaded file

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:31 -05:00
Michael Pimmer
eab8991b54
Backups: support for encrypted repositories
- implement download, restore, delete archives of encrypted
  repositories
- change how BorgRepository and SshBorgRepository handle path
- when/before creating remote repositories, check whether the
  connection works
- updated tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:28:17 -05:00
Michael Pimmer
15e26caa23
Backups, remote repositories: create/delete/restore of remote repos
- updated unittests
- implemented create/delete/restore remote archives

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:48 -05:00
Michael Pimmer
cf6bbd6bba
Backups, remote repositories: use object-oriented repositories
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:42 -05:00
Michael Pimmer
27fbc982c7
Backups, remote repositories: re-use template for root location
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:38 -05:00
Michael Pimmer
e3817a1a31
Backups, remote repositories: integrate to backups index page
- integrate remote repositories into backups index page
- remove, mount and unmount repositories via UI

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:35 -05:00
Michael Pimmer
3a8b69fc82
Backups, remote repositories: start using sshfs
- actions to mount/umount via sshfs
- tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:31 -05:00
Michael Pimmer
4be1b0be76
Backups, remote repositories: uniform parameter handling
- introduce env_vars in backups script and access_params for more
  uniform handling of access parameters
- added tests for creating and deleting an archive

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:27 -05:00
Michael Pimmer
0e2489ec23
Backups, remote repositories: implement init, info and some test
- added functionality to use remote repositories
- added some tests

Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2018-12-13 18:27:22 -05:00