- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
There are configuration items that users of syncthing can accidentally
set, resulting in the UI becoming unavailable. Previously, if such a
thing happened, the FreedomBox admin would have to ssh into the server and
edit the xml manually or reinstall/restore the app. With this patch, it
is enough to re-run the setup to make the UI accesible again.
- Remove http basic authentication from the settings (gui/user and
gui/password. This is unnecessary as FreedomBox already provides
authentication.
- Make sure these options are properly set:
- 'Use HTTPS for GUI' is off
- 'GUI Listen Address' is 127.0.0.1:8384
- GUI is enabled
There is a forum discussion that inspired these changes:
https://discuss.freedombox.org/t/solved-cant-access-syncthing-administration-panel/2137
Tests done:
- Fresh install: after the setup, confirm the UI is working properly and
the xml is configured with the expected values.
- Make the changes to break the Syncthing UI, then re-run setup. Confirm
the UI is back online.
Signed-off-by: Benedek Nagy <contact@nbenedek.me>
[sunil: Initialize conf_changed variable to prevent failures accessing it]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add typing information for init methods Info, Shortcut, and Menu to easily
identify problems.
- Update docstrings for these components.
- Updates test cases to deal with tags instead of short description.
- Update custom shortcuts code to read tags and ignore short description.
- Update API to send tags instead of custom shortcuts.
- OpenVPN special treatment of info.description in Shortcut
Tests:
- All unit tests pass and type checking succeeds.
- All apps show icons with tags in apps and system section.
- In help section cards don't show tags.
- In front page, enabled apps show shortcuts with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Fixes#2460
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Update the style to provide spacing after tags lines]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Don't use title casing, instead use simple capitalization.
- Add some tags.
- Drop outdated tags like 'VoIP', 'IM' while emphasizing 'Audio chat', 'Video
chat', 'Encrypted messaging' instead.
- Try to clarify server vs. web client with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add tags to Info component of apps. Use only English tags for all operations.
Localized tags are used for presentation to the user only. Add tags to all the
apps. Conventions (English):
1. Tags describing use cases should be in kebab case.
2. Protocols in tag names should be in their canonical format.
3. Tags needn't be 100% technically correct. This can get in the way of
comparing apps using a tag. Words that describe use cases that users can
easily understand should be preferred over being pedantic.
4. Tags should be short, ideally not more than 2 words. Avoid conjunctions
like "and", "or" in tags.
5. Avoid redundant words like "server", or "web-clients". Most apps on
FreedomBox are either servers or web clients.
6. Keep your nouns singular in tags.
- Use query
params to filter the Apps page by tags. When all tags are removed, redirect to /apps.
- Add UI elements to add and remove tag filters in the Apps page. Make the UI
similar to GitLab issue tags. Since there are 40 apps, there will be at least 40
tags. Selecting a tag from a dropdown will be difficult on mobile devices. A
fuzzy search is useful to find tags to add to the filter. Allow user to find the
best match for the search term and highlight it visually. The user can then
press Enter to select the highlighted tag. Make tag search case-insensitive.
Make the dropdown menu scrollable with a fixed size. User input is debounced by
300 ms during search.
- tests: Add missing mock in test_module_loader.py
- Add functional tests
[sunil]
- 'list' can be used instead of 'List' for typing in recent Python versions.
- Reserve tripe-quoted strings for docstrings.
- Undo some changes in module initialization, use module_name for logging
errors.
- isort and yapf changes.
- Encode parameters before adding them to the URL.
Tests:
- Tested the functionality of filtering by tag with one tag and two tags.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Syncthing from Debian testing uses new config directory if the
legacy configuration folder doesn't exist.
Tests performed in stable and testing containers:
- All syncthing tests pass when running twice.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Config files are all symlinks in /etc/
- Web interface works
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This improves modularity. Each app brings its own icons in its directory
instead of a centralized directory.
Tests:
- Install an app and notice that the installation notification has the icon.
- Visit an app's page in Apps and System (cockpit) section and ensure that the
app's icon is being shown.
- Visit the Apps and System section and notice that apps' icons are being shown.
- Visit the home page and notice that shortcuts' icons are being shown.
- Visit the URL /plinth/api/0/shortcuts and notice that the URLs for icons are
accessible and lead to proper icon files.
- Build developer documentation and notice that Tutorial -> View and Reference >
Menu pages show the expected updates.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Remove config file when app is uninstalled
Tests:
1. Install app and add a share
2. Reinstall app and confirm that the share doesn't exist
The order of the functional tests should be modified
since 'test_add_remove_folder' fails
Signed-off-by: nbenedek <contact@nbenedek.me>
[sunil: Update docstrings, make uninstall fail-safe]
[sunil: Remove the entire configuration directory]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- When app is freshly installed, nft rules are inserted.
- Trying to connect to local daemon from fbx user fails.
- Functional tests pass.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Functional tests work (failing already)
- DONE: Showing front page shortcuts according to user groups works
- DONE: Only user who is party of syncthing group is shown syncthing
- DONE: Admin users are always shown all the apps
- DONE: Syncthing:
- Not tested: When upgrading from version 2 or below, renaming group works
- DONE: Syncthing is added to freedombox-share group
- DONE: Initial setup of users app works
- DONE: freedombox-share group is created
- DONE: Retriving last admin user works
- DONE: Last admin is not allowed to delete account
- DONE: Creating a new user works
- DONE: Password is set properly (user can login with 'su - user' after)
- DONE: Incorrect confirmation password leads to error
- DONE: Adding the user to groups works (edit page shows correct list of groups)
- DONE: Editing a user works
- DONE: User is renamed properly
- DONE: Removing user from groups works
- DONE: Adding user to new groups works
- DONE: Providing incorrect auth password results in error message
- DONE: Enabling/disabling account work (confirm with 'su - user'). See #2277.
- DONE: Updating user password works
- DONE: New password is set (confirm with 'su - user')
- DONE: Providing incorrect auth password results in error message
- DONE: Initial user account creation works
- DONE: User account can be used (confirm with 'su - user')
- DONE: User is added to admin group
- DONE: Exception while getting SSH keys results in showing empty field
- DONE: Removing a user works
- DONE: Command provided in a message in users_firstboot.html works for
deleting users.
- DONE: If an admin users exists when running first wizard, list of admin users
is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Functional tests succeed (noticed intermittent failure)
- Initial setup succeeds
- User/group are created. /var/lib/syncthing is created with proper user/group
ownership.
- In configuration file, authentication notification is disabled
- Syncthing web interface is accessible
- Authentication related notification is not shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
This is now the preferred location in Debian. See:
https://lintian.debian.org/tags/systemd-service-in-odd-locationhttps://bugs.debian.org/992465https://bugs.debian.org/987989d70caa69c6https://lists.debian.org/debian-devel/2021/08/msg00275.html
Tests:
- Lintian no longer shows errors:
E: freedombox: systemd-service-in-odd-location lib/.../calibre-server-freedombox.service
- Comparing the old .deb and newly generated .deb with these changes. All the
systemd files show that they are moved from /lib to /usr/lib/systemd.
- After upgrading the deb from older version to a version these changes,
services installed by the package are available (tested after restart with
wordpress and claibre). Services tweaked by the package have the changed
configuration reflected as shown by systemctl show
{service-name}.service (tested after restart with quassel).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Add nslcd.service as a dependency to the services that depend on users
and groups defined in LDAP. deluged, mldonkey-server, syncthing@syncthing
and transmission-daemon services depend on freedombox-share LDAP group.
Closes#2061
Tests done with apps deluge, mldonkey, syncthing and transmission,
in both debian stable and testing dev containers, after applying changes:
- After installing an app and after reboot, the daemon user is a member
of the freedombox-share group.
- Checked with the `systemctl show` command that nslcd.service is added to
After=... dependencies.
- All the functional tests pass (in Debian stable, closed manually
the syncthing usage reporting form - #2059).
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Hide the warning 'The Syncthing admin interface is configured to allow
remote access without a password. ... '. This warning is unnecessary because
we are using authentication through Apache.
- Hide the usage reporting notification by declining reporting, if the user
has not made a choice yet.
- Call add_user_to_share_group() in setup() before starting syncthing to
avoid another syncthing daemon restart.
- Add a functional test for hidden notification messages.
- Functional tests: Improve the method to check if the javascript
loading process is complete.
Closes#1581
Tests performed:
- The syncthing app is installed successfully on Debian testing
- All syncthing tests pass on Debian stable and testing
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
When installing the syncthing app, create a LDAP group named "syncthing-access"
instead of "syncthing", because the app creates the "syncthing" system group
to run the syncthing daemon. Duplicate group names can cause some ambiguity as
described in #2008.
- Rename the existing "syncthing" LDAP and Django group to "syncthing-access".
- Update existing web shares to be accessible with new group name
- Functional tests: Add tests to check access to the syncthing site with a user
in the syncthing-access group and no group.
- Functional tests: Scroll to the edit button before clicking. Fixes some test
failures on a smaller browser window.
Fixes#2008
Tests performed on Debian stable and testing:
- Check that the existing "syncthing" group is renamed after upgrade:
1) Without patch applied, install syncthing, create a user in group "syncthing".
2) Apply patch, update Apache2 config file /etc/apache2/conf-available/syncthing-plinth.conf,
reload Apache2, restart plinth.
3) Check that the created user is now in the "syncthing-access" group and can
access /syncthing site.
- Check that the app upgrade succeeds when there are no users in the syncthing group.
- Create a web share accessible by the 'syncthing' group. Check that after the upgrade,
the share is accessible to a member of syncthing-access group.
- All the syncthing app tests pass.
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Each app does not have to call the validator by itself.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Undo a minor isort refactor in searx]
[sunil: Change the menu label to 'Donate']
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Move non-reusable app specific step definitions and helper methods into
<app>/tests/test_functional.py.
- Merge reusable helper methods into plinth.tests.functional
- Merge reusable step definitions into plinth.tests.functional.step_definitions
- avahi, datetime, ikiwiki: Reuse common methods to avoid repetition. Avoid
mapping from app nicknames to actual app names.
- deluge, transmission: Make a copy of sample.torrent for each app to avoid
clogging common place.
- Implement functional.visit() to simplify a lot of browser.visit() calls.
- Ensure that name of the mark on functional tests for an app is same as name of
the app. This will help with predicting the mark when running tests for a
particular app.
Tests performed:
- Run all functional tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Add pytest hooks to ignore all functional tests if pytest_bdd is not
installed.
- Update pytest hooks to skip tests in file named 'test_functional.py' if
--include-functional argument is not provided.
- Move functional_tests/install.py into plinth/tests/functional and update
reference in Vagrantfile.
- Move scenario files into individual app folders. Rename them after the app
they are testing. Merge TODO items listed in todo.org into corresponding feature
files.
- Add test_functional.py in each app to build tests from the features file using
pytest_bdd.
- Move all step_definitions, support and data into plinth/tests/functional/.
Include all step_definitions from conftest.py. Update to relative imports
instead of absolute imports.
Tests performed:
- Run py.test-3 --collect-only shows all functional tests and lists 574 tests.
No errors show that name of feature files are correct. The number says that all
functional test features are included.
- Remove pytest_bdd (or modify the import name) and run py.test-3 --collect-only
skips collecting all functional tests and shows only 300+ tests.
- Run functional tests for a few apps with py.test-3 --include-functional -m
app. For storage, deluge.
- Run unit tests with py.test-3. Functional tests are listed by skipped.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Closes: #1808
Test performed:
- Install samba in development VM and create open share (owned from root:freedombox-share).
- install syncthing and create syncronization folder in the above openshare.
- Sync files from desktop peer to development VM one on the above folder.
Signed-off-by: Nektarios Katakis <iam@nektarioskatakis.xyz>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Handle groups needed by an app.
- Handle reserved usernames for an app.
- Updated documentation
- Updated unit tests
Tests performed:
- Reserved usernames: ez-ipupd, ejabberd, Debian-minetest, mldonkey,
monkeysphere, mumble-server, privoxy, quasselcore, radicale, debian-tor,
debian-transmission
- Reserved usernames checks should work in the following forms:
- Create user
- Update user
- First boot user creation
- Full list of available groups should appear in following cases:
- Create user form
- Update user form
- Full list of groups should get created in Django DB during:
- Update user form display
- First boot form save
- When updating the last admin user, the 'admin' group choice is checked
and disabled.
- Following groups show up (sorted by group name):
- bit-torrent: Download files using BitTorrent applications
- git-access: Read-write access to Git repositories
- i2p: Manage I2P application
- wiki: View and edit wiki applications
- minidlna: Media streaming server
- ed2k: Download files using eDonkey applications
- freedombox-share: Access to the private shares
- web-search: Search the web
- syncthing: Administer Syncthing application
- feed-reader: Read and subscribe to news feeds
- admin: Access to all services and system settings
- Directory validation form checks for write permissions for following apps:
- deluge with debian-deluged user
- transmission with debian-transmission user
- Sharing app should show all the groups in add/edit share forms
- The following apps should get added to share group during setup:
debian-transmission
debian-deluged
- Unit tests pass
- Functional tests for users and groups pass
- Test that an app (example syncthing) provides the necessary
permissions to users in that group (but not in admin group).
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Fix i18n of group descriptions]
[sunil: Update developer documentation]
[sunil: Separate out cosmetic changes]
[sunil: Fix component ID for mumble]
[sunil: sharing: Remove unneeded dependency on users app]
[sunil: Implement better API for getting groups in component]
[sunil: Fix incorrect regression change ttrss app]
[sunil: Make iterating over gourps more readable]
[sunil: Improve tests, drop single use fixtures]
[sunil: Simplify test_view.py fixture]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
If at least one related service/daemon is not running, show the alert if app
is enabled, otherwise set alert as hidden (for functional tests).
Closes#1752
Signed-off-by: Veiko Aasa <veiko17@disroot.org>
[sunil: Minor code simplification in app template, status section]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Reuse the app_id already available to the view.
- Implement automatically detecting if an app has implemented diagnostics.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- move header section to it's own file so that it can be imported across
the app (app.html, simple_app.html, setup.html)
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
