Closes: #2161.
- Sections are ordered by importance on which administrator must act after
setting up the system.
- Consistent order across all the languages.
- Update the styling for the section hearers.
- For system section, make them compact.
- Make them look like a header text (with underline) rather than a
divider (like in a menu).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is so that the methods will be checked by mypy. This should help identify
any incorrect initialization of components.
- Remove unused self.repos in GitwebApp.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Print the interval time in schedule() method and verify that the times are as
expected in develop mode and production mode.
- Notification shows up for RAM usage if the check hardcoded to True.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Try to mark class variables in component classes.
- Leave typing hints generic, such as 'list' and 'dict' where content is usually
not filled, too complex, or context is unimportant.
- backups: Handle failure for tarfile extraction so that methods are not called
on potentially None valued variables.
- backups: Prevent potentially passing a keyword argument twice.
- dynamicdns: Deal properly with outcome of urlparsing.
- ejabberd: Deal with failed regex match
- email: Fix a mypy compliant when iterating a filtered list.
- tor: Don't reuse variables for different typed values.
- tor: Don't reuse variables for different typed values.
- operation: Return None explicitly.
- operation: Ensure that keyword argument is not repeated.
Tests:
- Where only typing hints were modified and no syntax error came up, additional
testing was not done.
- `mypy --ignore-missing-imports .` run successfully.
- Generate developer documentation.
- Service runs without errors upon start up.
- backups: Listing and restoring specific apps from a backup works.
- backups: Mounting a remote backup repository works.
- NOT TESTED: dynamicdns: Migrating from old style configuration works.
- ejabberd: Verify that setting coturn configuration works.
- email: Test that showing configuration from postfix works.
- tor: Orport value is properly shown.
- transmission: Configuration values are properly set.
- users: Running unit tests as root works.
- operation: Operation status messages are show properly during app install.
- ./setup.py install runs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- This will leave /etc/{plinth,freedombox} empty by default making service more
robust to run across various environments and situations. See systemd's
explanation for more details.
- Use Debian maintainer scripts remove all the existing files in
/etc/plinth/modules-enabled.
- Read from /usr/share/freedombox/modules-enabled then from
/etc/plinth/modules-enabled and finally from /etc/freedombox/modules-enabled.
Later read ones override previously read files. Any file pointing to /dev/null
will mean the module must be ignored.
Tests:
- Clean up /etc/plinth, /etc/freedombox and
/usr/share/freedombox/modules-enabled. Run service and notice that files are
getting loaded from development folder using a debug message.
- Run setup.py and notice that files get installed in
/usr/share/freedombox/modules-enabled/ and in the next run they get loaded from
there.
- Create a override file in /etc/plinth/modules-enabled/transmission and notice
that overriden file gets priority over the one in
/usr/share/freedombox/modules-enabled.
- Link the file /etc/plinth/modules-enabled/transmission to /dev/null and notice
that is not loaded.
- Create another file in /etc/freedombox/modules-enabled/transmission and notice
that it overrides the previous two files.
- All affected modules are loaded.
- Build a new Debian package and ensure that upgrading 23.8 to new version
removes are all configuration files.
- Build developer documentation and test that Tutorial -> Full Code and Tutorial
-> Skeleton sections have been updated with references to
-.../modules-enabled/... paths.
- Install quassel and notice that certificates were copied to /var/lib/quassel
directory. Change domain to another domain and notice that certificates were
copied again to that directory.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- test_uninstall is skipped for dynamicdns.
- test_uninstall is passed for bepasty.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Functional tests work.
- Initial setup works.
- Setting the setup version to 1 and running the service upgrades to version 2.
During this, export_config() and clean() work successfully.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
module.app property usage is greatly reduced because setup() and force_upgrade()
method are now part of App class instead of at the module level. Remove the
remaining minor cases of usage and drop the property altogether.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Primary purpose is to complete the App API and allow for multiple apps to be
present in a module without a single clashing setup() method. Secondary
objective is to get rid of SetupHelper instance simple use App instance instead.
- This brings us closer to not needing to implement setup() method for some of
the typical apps.
- Remove default value None for old_version parameter.
- A valid integer value is always passed to this call.
- The value of None is undefined.
- Simplifies the App API slightly.
- Drop setting 'pre', 'post' values to indicate the stage of setup for the App.
- Simplifies the setup methods significantly. Eliminates a class of
bugs (some of them seen earlier).
- The UI can show a simple 'installing...' or progress spinner instead of
individual stages.
- There are currently many inconsistencies where many operations are not
wrapped in helper.call() calls.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
When old configuration is not set and is exported with new code into newer
format, the result is a domain added with domain name 'null'. This causes issues
with UI not showing configuration and with null domain being added into
configurations of various daemons.
Tests:
- To reproduce the issue, switch to a revision with old dynamicdns code. Then
switch to a latest version without the fix. A 'null' domain is added to
configuration.
- To reproduce the issue, switch to a revision with old dynamicdns code. Then
switch to a latest version with the fix. A 'null' domain is not added to
configuration.
- With null domain in the configuration. Start FreedomBox with the fix. The null
domain should be removed and null domain should not be announced to other
daemons.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Drop all the bash code.
- Run the update URL code with same logic as before. Don't need to use action
code for it.
- Completely new way to handle configuration: using key/value store. Import old
configuration once and delete it.
- Use a glib scheduler instead of creating a cron job.
- Store and show status from key/value store.
- Handle multiple domains when getting/setting configuration and status. The UI
still shows a single configuration form. To be improved later.
- Catch and report all errors during the update process.
- Drop all NAT detection code.
- Drop selfhost.bz. German only, no free account, no proper TLS on domain, no
easy to test. Existing accounts will continue to work with "other" as the
service type.
- For gnudip update code, add a timeout of 10 seconds, set a buffer size of two
powers and fix handling error messages from server.
Tests:
- GnuDIP:
- Upon submission of the form, the IP is updated if app is enabled. IP is not
updated if app is disabled.
- Every 5 minutes, check is made again and IP is updated.
- If IP lookup URL is available, update calls are not made if the DNS is
already up-to-date.
- If IP lookup URL is not available, update calls are made unconditionally
every 5 minutes.
- For each of noip.com, freedns.afraid.org and other service:
- Upon submission of the form, the IP is updated if app is enabled. IP is not
updated if app is disabled.
- Every 5 minutes, check is made again and IP is updated.
- If IP lookup URL is available, update calls are not made if the DNS is
already up-to-date.
- If IP lookup URL is not available, update calls are made unconditionally
every 5 minutes.
- Form validation:
- Domain field is always mandatory.
- When type is selected as gnudip, the fields server, username, and password
are mandatory.
- When type is selected other than gnudip, the field update URL is mandatory.
The rest are optional.
- When the update URL contains a field contains <User>, username is mandatory.
For <Pass>, password is mandatory. For <Ip>, ip_lookup_url is mandatory.
- When use HTTP basic auth is checked, the fields username and password are
mandatory.
- Password is optional only if a previous password exists. If configuration is
deleted from kvstore, password is mandatory.
- Configuration import:
Install dynamicdns without the patch. Add configuration with each of the
service types. For GnuDIP service type, set two configurations with one with
and without IP lookup URL. Update to code with the patch. Setup should run.
- All fields in the configuration should be imported properly.
- If the previous configuration is disabled, app should be disabled after
import. Enabled otherwise.
- Updating the IP address should work immediately after import.
- Enable/Disable: when enabled, IP URL should be enabled every 5 minutes.
When disabled, updates should not happen.
- Status:
- When status is removed from the DB, it should show that no status is
available yet.
- When the form is updated or update happens via the timer, the status is
shown. It should show success for a proper update. Proper external IP address
should be shown.
- Set the server to localhost and submit. Status should show 'Server refused
connection' message. IP address should be '-'.
- Set the server to an unknown domain. Status should show 'Could not find
server' message. IP address should be '-'.
- Set the server to a known domain. Status should show 'Connection timed out'
message. IP address should be '-'.
- Last update time should keep increasing as time passes.
- Backup/restore:
- Functional tests.
- Javascript:
- When GnuDIP is selected as the type, the fields server, username, password,
domain, show password, and IP lookup URL should be shown while other fields
should be hidden. Same on page load with GnuDIP as pre-selected type.
- When GnuDIP is not selected as the type, the fields update URL, accept all
SSL certificates, use basic HTTP auth, domain name, username, password, show
password, IP lookup URL and use IPv6 fields should be shown and rest of the
fields should be hidden. Same on page load with non-GnuDIP as pre-selected
type.
- When show password is checked, password should be shown and when it is
unchecked, password is masked.
- When other service types are selected, the update URL values changes to the
respective service's URL.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Add Python implementation of GnuDIP client.
Tests:
- In testing container, configure Dynamic DNS with a (previously
offlined) freedombox.rocks account. FreedomBox interface shows that
the address has been updated. GnuDIP server also shows the correct
IP address.
- Running "gnudip update" and "dynamicdns update" actions produce the
expected results.
- For zoph, drop dependency on php7.4 as it will cause issues for future
versions of php. The dependency was a hack and not needed for Bullseye and
higher.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
By December 15th, all freedomboxes should no longer be pointing to the old
server. Hence, we need to have these changes done much before the migration date
for the changes to propagate.
The new website ddns.freedombox.org does not allow registrations yet. However,
the new website has a link to announcement. There is now an advice for new users
seeking to create accounts during the migration period.
Tests:
- In Dynamic DNS app, in the about page, the link is to the new server and it
works.
- The example link in the configuration form, IP URL field description is to new
server and it works.
- Functional tests for the Dynamic DNS app work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When configured, after startup, Dynamic DNS domain shows up in names module.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: isort all files]
[sunil: Remove component in datetime component as managed_packages is empty]
[sunil: Minor refactor in minidlna for consistency]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- In Django 2.2 django.conf.urls.url() is an alias to django.urls.re_path().
- In Django 4.0, url() function will be removed. On Django 3.2, it throws a
warning that this function will be removed in future.
Tests:
- Run unit tests with Django 3.2 and Django 2.2.
- With Django 3.2 there are no warnings when running unit tests and when running
FreedomBox Service.
- Visit a few affected apps with both Django versions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- ugettext functions will be removed in Django 4.0. Each use emits a warning
when running with Django 3.2. Since we have warnings enabled in developer mode,
we see quite a few messages because of this.
- ugettext is already a simple alias of gettext. So, no regressions are
expected.
Tests:
- Accessing an affected app in UI with Django 3.2 and Django 2.2 works fine.
- Using Django 3.2 there are no warnings related to removal of ugettext
functions.
- Ran regular unit tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This was used before to ensure the domain name was ASCII. However, str
does not convert to ASCII in Python 3.
Note that in config module, which sets the system domain name, the
domain is already restricted to alphanumerics, hyphen, and period.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Domain name is not case sensitive, but Let's Encrypt certificate paths
use lower-case domain name.
Add an extra 1 second delay to tests that configure DynamicDNS domain.
Test: DynamicDNS functional tests are passing.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
After a domain name change, Let's Encrypt will restart the webserver
and could cause a connection failure.
Test: DynamicDNS functional tests are passing.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Helps: #1938.
backups/forms.py:
- ChoiceField labeled to allow translation.
- Translation applied to hard coded literals.
config/forms.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
diagnostics_results.html:
Apply translation to results. Use gettext_noop to mark for translation.
dynamicdns/forms.py:
Apply translation to choice literals.
i2p/views.py:
Lazy translation applied to literals that were translated but still
displayed in english to non-english users.
names.html:
Apply translation to table headers.
performance/__init__.py:
Apply translation to description literals.
radicale/forms.py:
ChoiceField labeled to allow translation.
users/forms.py:
CharField labeled to allow translation.
QA:
- Literals visually verified.
- No errors in py.test-3.
- Yapf applied (only) to changed files.
- No remarks by flake8 to changed file.
Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Separate out the translations]
[sunil: Fix i18n for diagnostics]
[sunil: dynamicdns: Also do i18n for string GnuDIP]
[sunil: searx: Revert an incorrect removal of import]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
- Move non-reusable app specific step definitions and helper methods into
<app>/tests/test_functional.py.
- Merge reusable helper methods into plinth.tests.functional
- Merge reusable step definitions into plinth.tests.functional.step_definitions
- avahi, datetime, ikiwiki: Reuse common methods to avoid repetition. Avoid
mapping from app nicknames to actual app names.
- deluge, transmission: Make a copy of sample.torrent for each app to avoid
clogging common place.
- Implement functional.visit() to simplify a lot of browser.visit() calls.
- Ensure that name of the mark on functional tests for an app is same as name of
the app. This will help with predicting the mark when running tests for a
particular app.
Tests performed:
- Run all functional tests.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Add pytest hooks to ignore all functional tests if pytest_bdd is not
installed.
- Update pytest hooks to skip tests in file named 'test_functional.py' if
--include-functional argument is not provided.
- Move functional_tests/install.py into plinth/tests/functional and update
reference in Vagrantfile.
- Move scenario files into individual app folders. Rename them after the app
they are testing. Merge TODO items listed in todo.org into corresponding feature
files.
- Add test_functional.py in each app to build tests from the features file using
pytest_bdd.
- Move all step_definitions, support and data into plinth/tests/functional/.
Include all step_definitions from conftest.py. Update to relative imports
instead of absolute imports.
Tests performed:
- Run py.test-3 --collect-only shows all functional tests and lists 574 tests.
No errors show that name of feature files are correct. The number says that all
functional test features are included.
- Remove pytest_bdd (or modify the import name) and run py.test-3 --collect-only
skips collecting all functional tests and shows only 300+ tests.
- Run functional tests for a few apps with py.test-3 --include-functional -m
app. For storage, deluge.
- Run unit tests with py.test-3. Functional tests are listed by skipped.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Handle groups needed by an app.
- Handle reserved usernames for an app.
- Updated documentation
- Updated unit tests
Tests performed:
- Reserved usernames: ez-ipupd, ejabberd, Debian-minetest, mldonkey,
monkeysphere, mumble-server, privoxy, quasselcore, radicale, debian-tor,
debian-transmission
- Reserved usernames checks should work in the following forms:
- Create user
- Update user
- First boot user creation
- Full list of available groups should appear in following cases:
- Create user form
- Update user form
- Full list of groups should get created in Django DB during:
- Update user form display
- First boot form save
- When updating the last admin user, the 'admin' group choice is checked
and disabled.
- Following groups show up (sorted by group name):
- bit-torrent: Download files using BitTorrent applications
- git-access: Read-write access to Git repositories
- i2p: Manage I2P application
- wiki: View and edit wiki applications
- minidlna: Media streaming server
- ed2k: Download files using eDonkey applications
- freedombox-share: Access to the private shares
- web-search: Search the web
- syncthing: Administer Syncthing application
- feed-reader: Read and subscribe to news feeds
- admin: Access to all services and system settings
- Directory validation form checks for write permissions for following apps:
- deluge with debian-deluged user
- transmission with debian-transmission user
- Sharing app should show all the groups in add/edit share forms
- The following apps should get added to share group during setup:
debian-transmission
debian-deluged
- Unit tests pass
- Functional tests for users and groups pass
- Test that an app (example syncthing) provides the necessary
permissions to users in that group (but not in admin group).
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil: Fix i18n of group descriptions]
[sunil: Update developer documentation]
[sunil: Separate out cosmetic changes]
[sunil: Fix component ID for mumble]
[sunil: sharing: Remove unneeded dependency on users app]
[sunil: Implement better API for getting groups in component]
[sunil: Fix incorrect regression change ttrss app]
[sunil: Make iterating over gourps more readable]
[sunil: Improve tests, drop single use fixtures]
[sunil: Simplify test_view.py fixture]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tested-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Remove the need to pass all the individual information elements to the AppView
separately. This eliminates many issues with elements that were mistakenly not
sent to AppView. Also reduces a lot of code duplication.
- Create App classes for power and sso for consistency.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
This introduces flake8 and fixes a bunch of flake8 errors.
flake8 is run with: ./venv/bin/flake8 plinth
if you're using a python3 venv.
We can eventually further integrate this with gitlab ci.
https://salsa.debian.org/freedombox-team/plinth/issues/58
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Due a missing dependency on 'names' app, 'dynamicdns' app could be initialized
before 'names' app. This will lead to domain_added signal by dynamicdns being
ignored. Dynamic DNS domain name will not be make it into the list of global
domains.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@thoughtworks.com>
