- Add typing information for init methods Info, Shortcut, and Menu to easily
identify problems.
- Update docstrings for these components.
- Updates test cases to deal with tags instead of short description.
- Update custom shortcuts code to read tags and ignore short description.
- Update API to send tags instead of custom shortcuts.
- OpenVPN special treatment of info.description in Shortcut
Tests:
- All unit tests pass and type checking succeeds.
- All apps show icons with tags in apps and system section.
- In help section cards don't show tags.
- In front page, enabled apps show shortcuts with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
On systems where the grub-pc package is not available (e.g. ARM),
dpkg-query will have an exit status of 1. Handle the error that is
raised in this case.
Tests:
- Added unit tests for storage._diagnose_grub_configured.
- Tested on Raspberry Pi 4.
Closes: #2441
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
If the GRUB install device has not been selected, then grub package
configuration will fail during upgrades.
Tests:
- Install freedombox package with this change in a VM. Check that the
diagnostic is passed.
- Re-install the grub-pc package with DEBIAN_FRONTEND set to
noninteractive, so that it fails to be configured. Check that the
diagnostic is failed.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Change the partition to test to '/mnt'. Mount a loopback filesystem on /mnt.
'dd if=/dev/zero of=/test-file bs=1M count=100; mkfs.ext4 /test-file; mount -o
loop /test-file /mnt'. Turn it to read-only with 'mount -o remount,ro /mnt'.
Wait about 3 minutes for the notification to show up.
- The notification shows icon, title and message as expected. The button power
app appears and works as expected.
- When the filesystem is mount rw again, the notification goes away in 3
minutes. 'mount -o remount,rw /mnt'.
[sunil: Let glib.schedule decide time when debugging]
[sunil: Perform exact matching in partition mount options]
[sunil: Simplify notification message. Minor grammar change]
[sunil: Minor refactoring for styling]
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Print the interval time in schedule() method and verify that the times are as
expected in develop mode and production mode.
- Notification shows up for RAM usage if the check hardcoded to True.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Originally there was a separate module for udiskie, which later got
merged into storage module. Since storage is an essential module,
skip_recommends has no effect. (Recommends are never installed for
essential modules.)
Closes: #2203.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- SKIPPED: Functional tests work
- DONE: Initial setup works
- DONE: Root partition is expanded when space is available
- DONE: When there is free space for root partition it shows up in the interface
- DONE: Expand partition from user interface works
- DONE: Getting storage usage information works
- DONE: Disks and free space shown in app page
- DONE: Showing share mounts in samba works
- DONE: Backups add repository form shows disk choices
- DONE: Samba shows proper list of mounted shares and unavailable shares
- DONE: Directory validator works
- DONE: In deluge and transmission
- DONE: Auto-mounting a device works
- DONE: Ejecting a mounted disk from UI works
- DONE: Error are graciously handled
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- In a container, fill up space. Start FreedomBox in develop mode wait 3 minutes
for storage warning to show.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When running inside a container, it is not possible to retrieve information
about the disk that is mounted on '/'. Ignore errors in such cases.
Tests performed:
- Inside a container, start the service using freedombox-devel. Every 3 minutes
a check for disk space will be done in developer mode. If UDisks is being used
as source of information, the free space check will fail with an error logged.
- With the changes, the error is not shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Rename get_disks() to get_mounts() and use it in for backups and samba shares.
Create a new get_disks() similar to get_mounts() but use df information only for
showing free space. This inverts the importance of 'df' and UDisks. Use UDisks
as primary source of information for showing list of disks and then use df to
fill in the free space information.
- Retrieve all the mount points of a device and return them as part of
get_disks() in an extra 'mount_points' property.
- For storage listing, this fixes showing up of /.snapshots as separate disk and
showing of vboxsf, network mounts etc. Only shows mounts that are related to
block devices.
- Update various uses of get_disks() within storage module to use
'mounts_points' instead of 'mount_point' to be accurate in cases where there are
multiple mounts for a given device. Use get_mounts() where appropriate instead.
- Display all the mount points against a devices in multiple lines.
- Also show devices that are not currently mounted.
Tests performed:
- Filling up a disk shows a disk space warning properly. Warning contains the
free disk space correctly.
- Calling get_root_device(get_disks()) return the correct root device.
- In Deluge, the download directory contains a list of all samba current shares.
If a disk with samba share is unmouted, it does not show up in the list.
- In the Samba app page, all disks are shown properly. Root disk is shown as
'disk'. All other mount points such as .snapshots and /vagrant also show up.
- In the Samba app page, unavailable shares list shows up when a disk with a
share is unmounted.
- Upload a backup, warning on the form shows available disk space properly.
- When adding a backup location. The list includes all mount points. Duplicated
mount points are not shown. Root disk is not shown in the list. When all the
disks are used up for backup location, a warning that no additional disks are
available is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
In ed09028fcd2c850c3b87b65de66187b214190150, when eject was made to run as
superuser inside storage action, parsing of the error messages was not handled
properly. Fix it to show simple error messages about why the eject was not
successful.
Tests performed:
- In a terminal, switch to the directory where a disk is mounted to keep the
mount point busy. Attempt to eject the disk. A large stack trace is shown
without the patch and a clean error message is shown with it.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Bring us closer to avoiding the use of two different methods to access UDisks
DBus API: UDisks2 client library and direct DBus access with GDBus.
Perform formatting of the bytes outside of udisks2 module to avoid depending on
Django.
Tests performed:
- Visit the storage page. Disks are listed properly.
- Sizes are formatted to be human readable.
- Filesystem type is show properly: ext4, btrfs
- Labels for disks are shown as set by tune2fs etc.
- Device paths are shown properly.
- Mount point is shown properly.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove freedombox-udiskie.service file. Don't run udiskie anymore. Use our own
implementation of auto-mounting.
- Schedule disk failure checking to 3 seconds after application initialization.
Also perform auto-mounting at that time.
- Listen to new filesystems added and auto-mount them.
- Listen to disk failing attribute and report to user via a notification.
- Add rules to polkit-1 to allow plinth user to mount drives.
- Add simple abstractions over DBusProxy objects make accessing properties
simpler.
- Replicate udiskie's approach to mounting disks.
- Mount as root user for now using command line instead of DBus API. This is to
keep compatibility with older code that mounted under /media/root with relaxed
permissions.
Udiskie analysis:
- On device added, media added, perform auto_add
- On device changed and is addable and old state is not addable or removeable
- Automount condition:
- Matches configuration
- Not ignored
- is_filesystem and not mounted -> mount
- crypto device -> try unlock -> if success, mount
- is partition table
- Get all non-ignored devices, if partition then mount
- Mount condition:
- Is not ignored
- Is filesystem
- Find device with path
- Get options from configuration
- Is ntfs and executable ntfs-3g is not available
- Call mount
- No support for udisks1
- Built-in rules
- {'symlinks': '/dev/mapper/docker-*', 'ignore': True}
- {'symlinks': '/dev/disk/by-id/dm-name-docker-*', 'ignore': True}
- {'is_loop': True, 'is_ignored': False, 'loop_file': '/*', 'ignore': False}
- {'is_block': False, 'ignore': True}
- {'is_external': False, 'is_toplevel': True, 'ignore': True}
- {'is_ignored': True, 'ignore': True}
Tests performed:
- Create a CDROM in VM, inject media. Disk should get mounted.
- Create a temp file. mkfs.ext4 it at top level. losetup it. It should not get
auto mounted as it is a top level internal device.
- Create a temp file. Create two partitions and format the partitions. kpartx
-a on it. Both the file systems should get mounted.
- Create a temp file. luksformat it. Create a filesystem. luksopen the file.
It should get auto mounted.
- Checking for disk space repeatedly happens every 3 minutes.
- Drives are checked for healthy status only once, 3 seconds after FreedomBox is started.
- FreedomBox is able to mount disks while running as 'plinth' user with
policykit-1 version 0.105-26.
- FreedomBox is able to mount disks while running as 'plinth' user with
policykit-1 version 0.116-2 from experimental.
- Temporarily flip the is_failing condition in report_failing_drive. When
FreedomBox is restarted, notification about drives failing show up. When the
condition is reverted to normal, the notification is withdrawn.
- Build new Debian package and upgrade system with 20.8 installed. Two files
should be removed:
/var/lib/systemd/deb-systemd-helper-enabled/freedombox-udiskie.service.dsh-also
/etc/systemd/system/multi-user.target.wants/freedombox-udiskie.service .
systemctl status freedombox-udiskie.service should report no such unit.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Introduce new API to mark an app that it can't be disabled.
- Mark jsxc, storage, config, upgrade and firewall apps as can't be disabled.
- Fixed functional tests
- Replaced AppForm with forms.Form in all modules' forms.py.
- Remove app.template.js.
- Remove unused styles.
- Remove app status checks in form_valid of Deluge, Diaspora, Matrix, Ejabberd,
MediaWiki, Storage, Transmission, Quassel
- Purge unused is_enabled context variables (Ikiwiki)
- ejabberd: Minor cleanup in template
- jsxc: Cleanup unneeded overrides
- tahoe: Cleanup unnecessary overrides
Tests performed:
- For all apps affected, test enable/disable button works and submitting
configuration form works: with changes updates message and without changes
'settings unchanged' message.
- avahi
- bind
- cockpit
- SKIP: coquelicot
- datetime
- deluge
- SKIP: diaspora
- ejabberd
- gitweb
- i2p
- infinoted
- ikiwiki
- matrixsynapse
- mediawiki
- minetest
- minidlna
- mldonkey
- mumble
- pagekite
- privoxy
- quassel
- radicale
- roundcube
- SKIP: samba
- searx
- SKIP: shaarli
- shadowsocks
- ssh
- tahoe
- transmission
- FAIL: tt-rss (not installable)
- wireguard
- Deluge test that configuration changes when app is disabled work
- Quassel test that setting the domain works when app is diabled
- Transmission test that setting the domain works when app is diabled
- Ikiwiki create form works properly
- Enable/disable button appears as expected when enabled and when disabled
- Enable/disable button works without Javascript
- Functional tests work for affected apps, Tor and OpenVPN
- AppForm is removed from developer documentation
- Forms reference
- Customizing tutorial
- Test all apps using directory select form
- Transmission
- Deluge
- Visit each template that overrides block configuration and ensure that it is
loaded properly and the display is as expected.
- All apps that use AppView that are not tested above should not have an
enable/disable button. That is JSXC, update, config, firewall, storage, users.
Signed-off-by: Alice Kile <buoyantair@protonmail.com>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Get disks info from df command and add info from udisks, if available.
Before, it was other way round.
- Add a functional test that asserts root disk is available
- Remove unused key file_system_type from disks info
Closes#1765
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- samba app doesn't need anymore to change mount permissions
Fixes#1692 (in a different way)
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
On Raspberry Pi 3B+ image, it was observed that resizing partition fails during
initial setup. Due to this, Apache, SSH and Plinth become unavailable. Since
resizing the partition is not a critical operation to perform to bring up
FreedomBox, it is safe it ignore the error. In these cases, the user will have
to the storage app and retry manually after bring prompted by low disk space
message.
Tests:
- Introduce deliberate error in expand partition operation. Modify code to
always detect that partition can be expanded. Remove storage module from
plinth_module table in plinth.sqlite3. Run plinth. Notice that storage setup
is run but expanding partition fails. Even after expanding partition fails,
storage module is fully setup. Plinth proceeds with starting web server and
storage module is found in the plinth_module table of plinth.sqlite3.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Regression was introduced in a2d07fef0b9e2ae54f98fb5209b9cb0c5fd6f456 when
backup data import from manifest was inadvertently removed.
Closes: #1636.
Test:
Create a backup with this patch and 'storage' as one of the selected apps for
backup. Try to restore the backup without the patch. Observe that it fails.
Restore the backup with the patch. Observe that it succeeds.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Introduce base class for all apps that will contain components. With
unittests.
- Introduce base classes for components. With unittests.
- Turn Menu class into an app component.
- Further cleanup Menu class.
- Update tests.
- Maintain a global list of menu items and look them up easily. Generalize
such that subsubmenus can later be merged into Menu class.
- Cleanup scope of main menu initialization.
- Use None instead of empty strings for various values. Ensure that
printing short_description does not show 'None' in output.
- Use enable/disable instead of promote/demote.
- Use menu component in all apps.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Fetch disk information for all disks using udisks
- Call df as superuser so that all disks are listed (udisks doesn't need sudo)
- Improved implementation to check if device is removable
Signed-off-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Without the patch, every time the storage page is visited an error is logged.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Other types could be 'disk' (where there is no partition table) and 'loop' when
the disk is a loop back device. As long as they are mounted on a important
folders, don't let user unmount them.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
