- First of the list is the most important one and may be used as "primary"
domain in apps.
- Change the return type of DomainName.list() from set to list so that order can
be preserved. Update all users of the API accordingly. Add type hints to all the
methods using this API to catch any errors.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Add typing information for init methods Info, Shortcut, and Menu to easily
identify problems.
- Update docstrings for these components.
- Updates test cases to deal with tags instead of short description.
- Update custom shortcuts code to read tags and ignore short description.
- Update API to send tags instead of custom shortcuts.
- OpenVPN special treatment of info.description in Shortcut
Tests:
- All unit tests pass and type checking succeeds.
- All apps show icons with tags in apps and system section.
- In help section cards don't show tags.
- In front page, enabled apps show shortcuts with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
Fixes#2460
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Update the style to provide spacing after tags lines]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Don't use title casing, instead use simple capitalization.
- Add some tags.
- Drop outdated tags like 'VoIP', 'IM' while emphasizing 'Audio chat', 'Video
chat', 'Encrypted messaging' instead.
- Try to clarify server vs. web client with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Add tags to Info component of apps. Use only English tags for all operations.
Localized tags are used for presentation to the user only. Add tags to all the
apps. Conventions (English):
1. Tags describing use cases should be in kebab case.
2. Protocols in tag names should be in their canonical format.
3. Tags needn't be 100% technically correct. This can get in the way of
comparing apps using a tag. Words that describe use cases that users can
easily understand should be preferred over being pedantic.
4. Tags should be short, ideally not more than 2 words. Avoid conjunctions
like "and", "or" in tags.
5. Avoid redundant words like "server", or "web-clients". Most apps on
FreedomBox are either servers or web clients.
6. Keep your nouns singular in tags.
- Use query
params to filter the Apps page by tags. When all tags are removed, redirect to /apps.
- Add UI elements to add and remove tag filters in the Apps page. Make the UI
similar to GitLab issue tags. Since there are 40 apps, there will be at least 40
tags. Selecting a tag from a dropdown will be difficult on mobile devices. A
fuzzy search is useful to find tags to add to the filter. Allow user to find the
best match for the search term and highlight it visually. The user can then
press Enter to select the highlighted tag. Make tag search case-insensitive.
Make the dropdown menu scrollable with a fixed size. User input is debounced by
300 ms during search.
- tests: Add missing mock in test_module_loader.py
- Add functional tests
[sunil]
- 'list' can be used instead of 'List' for typing in recent Python versions.
- Reserve tripe-quoted strings for docstrings.
- Undo some changes in module initialization, use module_name for logging
errors.
- isort and yapf changes.
- Encode parameters before adding them to the URL.
Tests:
- Tested the functionality of filtering by tag with one tag and two tags.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2362.
Passing ?transport= parameter in STUN URIs is invalid. It always uses UDP.
Chrome and perhaps Firefox has recently started enforcing the correct syntax
leading to failures using the Coturn server URIs we set in Janus. This also
likely effects matrix-syanpse and ejabberd clients.
Links:
1) https://www.rfc-editor.org/rfc/rfc7064#section-3.1
2) https://bugs.chromium.org/p/chromium/issues/detail?id=1385735
Tests:
- Install Coturn. Observe that STUN URIs shown don't contain the 'transport'
parameter.
- Install Janus and launch the meeting room. Notice that the STUN URIs in the
room page don't have 'transport' parameter.
- Install ejabberd and notice that the auto-configured STUN URIs don't have
'transport' parameter.
- Install matrix-synapse and notice that the auto-configured STUN URIs don't
have 'transport' parameter.
- Install ejabberd and matrix-synapse. Ensure that STUN URIs manually. They are
not allowed to 'transport' parameter for the STUN URIs but must have transport
parameter for TURN URIs.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This improves modularity. Each app brings its own icons in its directory
instead of a centralized directory.
Tests:
- Install an app and notice that the installation notification has the icon.
- Visit an app's page in Apps and System (cockpit) section and ensure that the
app's icon is being shown.
- Visit the Apps and System section and notice that apps' icons are being shown.
- Visit the home page and notice that shortcuts' icons are being shown.
- Visit the URL /plinth/api/0/shortcuts and notice that the URLs for icons are
accessible and lead to proper icon files.
- Build developer documentation and notice that Tutorial -> View and Reference >
Menu pages show the expected updates.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
* Verify from the UI that after reinstalling the app a new secret is generated
* Functional tests passed
Signed-off-by: nbenedek <contact@nbenedek.me>
[sunil: Update docstrings, make uninstall fail-safe]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Closes: #2328.
Coturn depends on sqlite3 package. As sqlite3 is listed as a package in
roundcube, it is removed when roundcube is uninstalled. This results in the
removal of coturn as well. Prevent this by listing sqlite3 package explicitly in
coturn app.
Tests:
- Without the patch, install coturn and roundcube. When roundcube is
uninstalled, coturn package is no longer installed.
- Apply the patch, install coturn and roundcube. When roundcube is uninstalled,
coturn and sqlite3 packages are still installed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Initial setup of coturn succeeds
- Configuration file is created and required configuration is set.
- Coturn is restarted
- Coturn configuration is shown on app page.
- Changing the domain succeeds and coturn configuration reflects the new domain.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Clicking on the link in the coturn app description takes us to ejabberd app
page.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Add basic video room based on demo.
- Set port range to use for RTP.
- coturn: Add component for time-limited TURN configuration.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
[sunil: Don't error out when coturn is not installed/configured]
[sunil: Prepend data- to custom attribute in HTML]
[sunil: Convert SVG with embedded bitmap to vector graphics]
[sunil: Hide Javascript license information in footer]
[sunil: Minor changes to comments for styling]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Fixes: #2069.
Without a listening port, coturn will try to enumerate the non-local IP
addresses and try to listen on them. If coturn is started before network is
fully setup, it finds no usable IP addresses and fails. Furthermore, if IPs are
added to the system, it does not automatically listen on them.
A better approach as advised by systemd NetworkTarget documentation is to listen
on a wildcard address. This does not require network to be online and works well
for IP addresses being added/removed from the system. coturn is itself unable to
make changes to its default listening behavior for backward compatibility.
Tests:
- Freshly install coturn. Observe that listening-ip is properly set in the
configuration file. coturn is listening on 3478, 3479, 5349, 5350. coturn is
listening on ::1 and * addresses instead of individual IP addresses.
- Install coturn without the patch. Apply the patch and restart FreedomBox.
coturn setup will run. listening-ips get added to the configuration file. The
static-auth-secret is not changed from earlier. coturn will be restarted. coturn
is listening on 3478, 3479, 5349, 5350. coturn is listening on ::1 and *
addresses instead of individual IP addresses.
- Install coturn without the patch. Disable coturn. Apply the patch and restart
FreedomBox. coturn setup will run. coturn will not be enabled. coturn will be
running after setup.
- Functional tests pass.
- All ports able to connect using netcat (nc command) with IPv4 (-4 option) and
IPv6 (-6 option).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
This is now the preferred location in Debian. See:
https://lintian.debian.org/tags/systemd-service-in-odd-locationhttps://bugs.debian.org/992465https://bugs.debian.org/987989d70caa69c6https://lists.debian.org/debian-devel/2021/08/msg00275.html
Tests:
- Lintian no longer shows errors:
E: freedombox: systemd-service-in-odd-location lib/.../calibre-server-freedombox.service
- Comparing the old .deb and newly generated .deb with these changes. All the
systemd files show that they are moved from /lib to /usr/lib/systemd.
- After upgrading the deb from older version to a version these changes,
services installed by the package are available (tested after restart with
wordpress and claibre). Services tweaked by the package have the changed
configuration reflected as shown by systemctl show
{service-name}.service (tested after restart with quassel).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- Re-use the same validator in Matrix Synapse.
- Avoid importing plinth classes in actions files.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This implementation is very similar to that of Matrix Synapse with a lot
of code duplicated.
One major difference is that ejabberd doesn't have a conf.d/ directory.
So, the managed configuration and overridden configuration cannot be
cleanly separated.
Whether the configuration is managed or not is determined by the
presence of a file under `/etc/ejabberd`. Managed coturn configuration
isn't stored in ejabberd, since only one set of configuration can be
stored at a time. If the admin chooses to use the managed configuration,
the current coturn configuration is fetched and used to configure
ejabberd.
Fixes#1978
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Sunil:
- Join the string in the template instead of the view.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Sunil:
- Add TurnConfiguration to reference documentation. Add more details in
docstrings.
- Rename the component to TurnConsumer since 'Component' in the name is
redundant and unconventional. Also, hopefully, the component will retain the
API over multiple TURN servers.
- Log when notifying other components about configuration change.
- Use TurnConfiguration class more widely.
- Refactor for simplicity.
- Additional tests.
- Move URI generation code into TurnConfiguration.
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Without these changes, with Coturn not-installed, change the domain name.
Notice that certificate events for Coturn fails due to missing domain
information.
- With these changes, with Coturn not-installed, change the domain name. Notice
that certificate events for Coturn don't result in any actions.
- With these changes, when Coturn is installed, certificate is properly setup
for a domain.
- With these changes, with Coturn installed, change the domain name. Notice that
certificate events for Coturn succeed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Without these changes, with Coturn not-installed, change the domain name.
Notice that certificate events for Coturn fails due to missing domain
information.
- With these changes, with Coturn not-installed, change the domain name. Notice
that certificate events for Coturn don't result in any actions.
- With these changes, with Coturn installed, change the domain name. Notice that
certificate events for Coturn succeed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Start showing port ranges properly.
- Fixes issue with Coturn TURN relay ports not being shown.
Closes: #1851.
Tests:
- Visit each of affected apps and see the port forwarding information. The
information is same as before.
- HTTP and HTTPS ports are not shown.
- Coturn app shows additional port ranges for TURN relay ports.
- Shadowsocks app does not show port forwarding information as it is internal
only.
- Visit one of the apps not effected by the patch. There is no section related
to port forwarding.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- Add pytest hooks to ignore all functional tests if pytest_bdd is not
installed.
- Update pytest hooks to skip tests in file named 'test_functional.py' if
--include-functional argument is not provided.
- Move functional_tests/install.py into plinth/tests/functional and update
reference in Vagrantfile.
- Move scenario files into individual app folders. Rename them after the app
they are testing. Merge TODO items listed in todo.org into corresponding feature
files.
- Add test_functional.py in each app to build tests from the features file using
pytest_bdd.
- Move all step_definitions, support and data into plinth/tests/functional/.
Include all step_definitions from conftest.py. Update to relative imports
instead of absolute imports.
Tests performed:
- Run py.test-3 --collect-only shows all functional tests and lists 574 tests.
No errors show that name of feature files are correct. The number says that all
functional test features are included.
- Remove pytest_bdd (or modify the import name) and run py.test-3 --collect-only
skips collecting all functional tests and shows only 300+ tests.
- Run functional tests for a few apps with py.test-3 --include-functional -m
app. For storage, deluge.
- Run unit tests with py.test-3. Functional tests are listed by skipped.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Shows URLs and shared secret that communication servers like matrix-synapse
should be configured to. Later we will implement auto-configuring those servers.
- Allow selecting domain for the sake of TLS/DTLS certificate installation.
- Simplify systemd service file options. Drop log file and pid file support as
they are not needed with systemd. Add security options.
- Set custom configuration file by overriding systemd service file options so
that we don't have a problem with conffile prompts.
- Implement functional tests (and automatic diagnostics).
- Custom icon selected from the Noun project as Coturn project does not have
one.
- Backup/restore configuration file and certificates.
- Document some questions regarding configuration options.
Tests performed:
- App is not listed in the app page if 'advanced' flag is disabled.
- App name, icon and short description shows up correctly in apps page.
- App name, icon, short description, description, manual link, enable/disable
button and diagnostics link show up currently in app page.
- Verify that configuration used by coturn server is the FreedomBox
configuration by checking the cert path in the log output.
- PID file is not created in /var/run/turnserver/. It goes into /dev/null
according to the log output.
- No log file is created other than what is collected by systemd from command
line.
- systemctl show coturn.service shows all the intended restrictions such as
NoNewPrivileges, Protect* options.
- Run functional tests.
- Ensure that backup of configuration file works by taking backup, changing the
secret and restoring. During backup and restore coturn should be stopped and
started as per logs.
- Build Debian package. No warnings about the copyright file.
- Enabling the app enables the service and runs it.
- Disabling the app disables the service and stop it.
- All diagnostics tests pass.
- Diagnostic tests show firewall port coturn-freedombox for internal and
external networks, service coturn, and each listening port for udp4, udp6, tcp4
and tcp6.
- Information in the firewall page shows up properly. Enabling the app opens
firewall ports, and disabling it closes them.
- When the app is installed, if a cert domain is available, it will be used.
When multiple domains are available, one of them is picked.
- Status shows 4 URLs with the currently selected domain and secret key.
- Changing domain to another domain succeeds and reflects in the status
information.
- When no domain is configured. Installing the app succeeds. No domain is shown
in the list of domains.
- When domain is changed, the certificates files in /etc/coturn/certs are
overwritten.
- Certificates have the ownership turnserver:turnserver. Public key is cert.pem
has 644 permissions. Private is pkey.pem has 600 permissions. /etc/coturn/certs
is owned by root:root.
- Let's encrypt certificates are setup immediately after install.
- Port forwarding information shows all ports except for relay ports.
- Trying to create a user with username 'turnserver' throws an error. This
happens even when coturn is not installed yet.
- After installing coturn, the configuration file /etc/coturn/freedombox.conf is
created with ownership root:turnserver and permissions 640. The directory
/etc/coturn is created with ownership root:root and permissions 755.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
[jvalleroy: Fix copied form_valid comment]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
