- Since we are going to be an OpenID Provider, we need to fix the URLs that
other apps will be configured with for authentication. So change now from
/plinth to /freedombox. If done later, it will be harder since all the
configuration files for all dependent apps will need to be updated.
Tests:
- App availability checking works. Request goes to /freedombox URL
- Favicon is served properly and through /favicon.ico URL
- Redirection happens from / to /freedombox directly
- UI is available on /freedombox and on /plinth
- Manual page show /freedombox as the URL in two places
- Static files are successfully served from /freedombox URLs. URLs inside page
start with /freedombox
- backup, bepasty, calibre, config, dynamicdns, ejabberd, featherwiki, gitweb,
ikiwiki, kiwix, miniflux, names, openvpn, shadowsocks, shadowsocksserver,
sharing, shapshot, tiddlywiki, users, wireguard, jsxc, matrixsynapse, first
wizard, storage, samba, tags functional tests work. Backup/restore test for
matrixsynapse fails due to an unrelated bug (server not restarted after
restore).
- Setting the home page works:
- Having /plinth in the home page configuration works. Shows selection
correctly.
- Setting to app works. Shows selection correctly.
- Setting to user home page (sets /freedombox). Shows selection correctly.
- Setting to apache default works. Shows selection correctly.
- Changing back to FreedomBox service works. Shows selection correctly.
- Unit tests work
- Configuration page shows /freedombox in description but not /plinth
- Diagnostics show /freedombox in tests
- Roundcube URL link in email app has /freedombox
- email loads the page /.well-known/autoconfig/mail/config-v1.1.xml correctly
- email app shows /freedombox/apps/roundcube for /roundcube if roundcube is not
installed.
- networks: router configuration page shows URL starting with /freedombox.
- snapshot: Shows URL starting with /freedombox on the app page
- js licenses page uses /freedombox prefix for JSXC.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- When /var/lib/plinth/.ssh is deleted and add remote repository form is
visited, the directory along with SSH key files are created. The .pub file has
the expected comment freedombox@configured_domain.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Provide incorrect passphrase when adding a remote repository. Should show a
message that passphrase is incorrect and redirect back to add remote repository
form.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- As a safe guard although it should exist because of a prior verification of
SSH key.
- Minor refactor to make the method flatter.
Tests:
- Remove /var/lib/plinth/.ssh and visit add remote repository form. The public
key is displayed in the form. The files in /var/lib/plinth/.ssh are created with
expected permissions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Integrate SSH error handling into borg error handling.
- Move logic to migrate SSH keys into lower levels (Repository class) so that it
can performed at more instances such as when initializing repository. It also
provides better abstraction keeping the view logic simpler.
- Drop ability to mount repository using password. This is important next step
for mounting using systemd unit files.
- Use exceptions to eliminate return value checking.
- Create a special exception for exceptions raised during SSH operations. Raise
this at lower levels and handle these using the common error handler.
Tests:
- Adding a remote repository with key and password authentication works with and
without encryption. Adding works with SSH host key pre-verified works too.
- Trying to add a remote repository with incorrect passpharse fails with the
simplified error message. Redirect happens to add remote repository page. Error
message with SSH host key pre-verified works too. Repository is removed.
- Trying to provide wrong SSH password fails with a simplified error message.
Redirect happens to add remote repository page. Repository is removed.
- Mounting a repository after unmounting it works.
- Mounting a repository with SSH password in its configuration works. Migration
is performed and SSH password is replaced with SSH key file path.
- A schedule for a repository with SSH password runs successfully. An archive is
created. Migration is performed and SSH password is replaced with SSH key file
path.
- SSH identity files are created with plinth:plinth ownership. Private key file
is created with 0o600 permissions and public key file is created with 0o644
permissions.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- Adding a remote repository with key and password authentication works with and
without encryption.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- This makes the functional test focus more on the repository that is added
rather than all remote repositories.
Tests:
- Functional tests for backups app works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Use bootstrapform templates where possible to reduce code.
- Fix incorrect tag for radio button for password authentication.
- Drop borders and instead:
- Style each group as a section. This includes encryption section and
repository path section.
- Utilize the simplicity as passwords fields are not displayed.
- Retain <label> tag so that clicking on it works and it also easier for
accessibility tools.
Tests:
- The option for password authentication works are expected. The fields for
encryption passphrase, ssh password work as before.
- Clicking on form labels focuses the form element.
- Adding a repository with key and password authentication works with and
without encryption.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
- Unlike the case of network forms, for example, there is nothing the user could
infer from a disabled form element. If they see a disabled DNS field, they would
understand that it is an editable value but has been disabled due to other
option values. It is important to allow users to discover this. However, in case
of password fields, they are not needed to be shown to the user unless the
appropriate option is selected.
Tests:
- In the add remote repository form, selecting the authentication type radio
options shows and hides the password field. Selecting the value for encryption
type shows and hides the encryption password field.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- In remote repository add form, selecting radio options for authentication
types does not throw an error. The password field is shown/hidden as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Tests:
- On main branch, add a remote repository with SSH password. Unmount
the remote location.
- Switch to branch with this change. Mount the remote location. Logs
show that it is migrated from password to key authentication. Plinth
database no longer contains password for this remote.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- Use javascript to disable or enable password fields.
- If SSH key auth is selected, then try the connection.
- If SSH password auth is selected, then copy the key.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
- After copying the SSH client public key to the remote host, replace
the SSH password credential with keyfile.
- Also use SSH key when checking that remote directory exists.
Tests:
- Add remote backup location "tester@localhost:~backups". Test various
operations like create backup, download backup, unmount and
mount. Confirm that SSH password is no longer present in plinth
sqlite database.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- In development VM, add a remote backup location of "tester@localhost:~/backups".
Verify the SSH host key. plinth@freedombox key is listed in
/home/tester/.ssh/authorized_keys.
- Remove the remote backup location, and delete /home/tester/.ssh/authorized_keys.
Add the same remote backup location again. plinth@freedombox key is again
listed in /home/tester/.ssh/authorized_keys.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Click on Add Remote Backup Location. Logs show that SSH client key is
generated. The private key is readable only by plinth user.
- Go back, and click on Add Remote Backup Location again. Logs show that SSH
client key already exists.
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Fixes: #2554
- Update permissions on the backups-data directory so that files are only
accessible by root users.
- Ensure that the directory is created by the 'backups' app and not by each of
the apps that take the backup.
Tests:
- Run functional tests for miniflux, dynamicdns, wordpress, zoph, and nextlcoud.
There was an unrelated functional test case failure in nextcloud.
- On a fresh installation, apply patch. Service is restarted. The directory is
created with proper permissions and ownership.
- On a fresh installation, without the patch. Backup the dynamicdns app. The
directory is created with incorrect permissions. Apply the patch. Service is
restarted. Proper permissions are set on the directory.
- On a setup with incorrect permissions, re-run backups app's setup. The
permissions are updated correctly.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Now that we have a mechanism for properly collecting, transmitting, and display
the stdout and stderr. There is no reason not to collect all of the stdin and
stderr.
- Also, the stdin/stderr=subprocess.PIPE is redundant and prevents the output
from getting collected for debugging. So, remove it.
Tests:
- Ran functional tests on backups, calibre, ejabberd, email, gitweb, ikiwiki,
infinoted, kiwix, mediawiki, mumble, nextcloud,, openvpn, samba, wireguard,
zoph. 2-3 issues were found but did not seem like new errors.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
- This is to capture stdout and stderr and transmit that from privileged daemon
back to the service to be displayed in HTML.
Tests:
- Unit tests and code checks pass.
- Some of the modified actions work as expected.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
During functional tests, it was noticed that getattr() failed at the following
line. The original intent of the code is to ensure that there are no failures
when 'stdout'/'stderr' attribute are not present or when they return None.
stdout = (getattr(err, 'stdout') or b'').decode()
Tests:
- Make the UI raise incorrect password error. Notice that the error is shown
properly.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Veiko Aasa <veiko17@disroot.org>
Tests:
- Mounting/unmounting of remote SSH repositories works.
- Creating repo, creating/deleting/list archives work.
- If a privileged method raises an exception after outputting to stdout (using
action_utils.run) then stdout is shown in the HTML UI message.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Regression: downloading does not work with sudo based action anymore. However,
sudo based actions are to be removed in later patches.
Tests:
- Downloading tar backup archive works. Untar works. Downloading gives upto
10MiB/s speed.
- If API is not called with _raw_output=True, then special exception is raised.
- Downloading tar file from command line using nc also works.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Fallback to sudo based privileged implementation. Privileged daemon tests are
still to be implemented.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- The most pleasant font on any system is the default system font.
- It is the most optimized and styled font for the system considering screen
type and screen sizes.
- Used by all the system apps. Websites can become consistent with system apps
by using system fonts. GNOME, KDE, Ubuntu, Android, Chrome OS, iOS, and MacOS,
all have their own system fonts.
- Changed by the user using OS settings if they don't like it.
- Many popular sites have started using system fonts.
- No extra fonts have to be loaded making page loading jerk free and much
faster. On the first FreedomBox UI page load, the largest item is the font.
- We won't have carry the binary woff files in FreedomBox source tree anymore.
Also eliminates a bunch of lintian warnings.
- Lato font was used because it is prescribed by the FreedomBox identity manual.
Lato can still be used in other places such as marketing materials.
Tests:
- System font is used in the UI. When system font is changed in Gnome settings
and browser is restarted, the new font is shown in the UI.
- Check that the overall layout of the app grids is not effected by the font
size change.
- Check that all the tables in the UI are not affected by the font change.
- Backups repository listing shows each backup archive in one line.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- Many times, merging old and new data folders is not ideal and could lead to
unexpected outcomes. Perhaps removing all the backup folders and files before
restore is ideal. However, this patch tries to introduce that approach slowly on
an experimental basis.
Tests:
- Unit tests work.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- For scheduled backups and automatic backups generated when uninstalling apps,
make the generated archive name similar to automatic name given no name is
specified when creating an archive.
Tests:
- Set schedule on a local repository, uninstall an application, and create a
backup archive without a name. The timestamp in generated backup archives in all
three cases is similar in format.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2354.
- Process spawned for raw IO should be collected using wait(). Closing the
specially created pipe early causes an exception in the child process when it
tries to write to stderr the JSON result of executed method.
Tests:
- Monitor the output of 'journalctl -f'. Download and archive. Without the
patch, an error is printed as described in the bug. With the but no such error
is printed. Downloaded file is the same in both cases.
- Writing a log message in cleanup_func shows that the process has been waited
for and the FDs have been closed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Fill up disk to have very little space left. Perform a large backup operation
by writing a large file to one of the directories to be backed up. Try to
perform the backup and notice that a nice error message is shown that there is
not enough disk space instead of a generic error message.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Add typing information for init methods Info, Shortcut, and Menu to easily
identify problems.
- Update docstrings for these components.
- Updates test cases to deal with tags instead of short description.
- Update custom shortcuts code to read tags and ignore short description.
- Update API to send tags instead of custom shortcuts.
- OpenVPN special treatment of info.description in Shortcut
Tests:
- All unit tests pass and type checking succeeds.
- All apps show icons with tags in apps and system section.
- In help section cards don't show tags.
- In front page, enabled apps show shortcuts with tags.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Joseph Nuthalapati <njoseph@riseup.net>
- So that on slow machines, the tests don't fail.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2218.
- When borg can't acquire a lock due to it being busy, any borg operation can
fail. Show a readable message instead of a generic error.
- Also handle errors for archive already existing and archive to be deleted not
existing.
Tests:
- Functional tests for backup app work.
- Creating archive works with proper message. Providing the name of existing
archive shows proper error.
- Deleting archive works with proper message. Open two tabs by clicking on the
delete archive button. Then delete with one and try to delete the it again with
the next one. Proper error message is shown.
- While downloading an archive, delete an archive. Proper error message that
borg is busy is shown.
- Upload archive works. A proper success message is shown.
- Restore backup from archive works. A proper success message is shown.
- Restore backup from file upload works. A proper success message is shown after
upload and after restoration.
- Adding local repository works. A proper success message is shown.
- Adding remote repository works. A proper success message is shown when SSH key
is verified and repository is added.
- Removing repository works. A proper success message is shown.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #1603.
- This introduces seconds and timezone in the name of the archive. When seconds
are not used in the name, taking backups too quickly on after the other fails as
the names clash.
Tests:
- Taking a backup works and creates the name in the expected ISO time format.
- Changing the timezone and restarting service creates timestamps in that
timezone.
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- When archives are listed using list_archives() method, return datetime objects
instead of pre-formatted strings. datetime objects can be compared easily and
shown in a more human readable format.
Tests:
- Unit tests pass.
- Backups with a future date are ignored when considering recent backup
times (when they logged to console).
- Most recent scheduled backup times are retrieved correctly (when they logged
to console).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Tests:
- Sending GET requests to the two views by opening theirs URLs in the browser
show a blank page. Actions are not performed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- Currently, in the repository class, if an operation fails, the error messages
from borg are interpreted and re-raised. Everywhere else, the errors are
interpreted. Fix this by wrapping privileged methods at the privileged module
level instead of a context manager at the place of calling.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
Closes: #2476.
- The collapse click logic is triggered when clicking on the schedule button (or
the other buttons). In case of other buttons they do activate eventually but
the schedule button does not activate.
- Solve this properly by making sure that collapse logic is not triggered when
the buttons are clicked.
Tests:
- On stable and testing containers, ensure that collapse works. Collapse/expand
icon works. The three buttons work. And mobile view looks good.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- In current stable and testing, verifying SSH remote hosts using RSA is not
working. After selecting the verified RSA fingerprint, paramiko fails to connect
- A change introduced in paramiko 2.9 lead to failures when connecting to hosts
that have a verified RSA host key[1][2][3]. To fix the issue,
disabled_algorithms must be used to drop some of the other algorithms supported
by the server to force paramiko behavior. A better solution to the problem was
introduced in paramiko 3.2. Both these solutions require careful update to the
code. Considering the utility paramiko provides, the regression annoyance,
effort required for this fix, and the security implications (it is an completely
independent SSH implementation), the library does not seem to be worth the
effort in our case.
- Switch to using sshpass command line utility instead of paramiko library. The
only reason to use paramiko seems that 'ssh' command by default does not allow
us to input password easily while paramiko does.
- Another place where paramiko is being used is to check if a host is already
verified in the known_hosts file. This has been trivially replaced with
'ssh-keygen -F'.
- Exit codes provided by sshpass can replace the specific exception raised by
paramiko.
Links:
1) https://www.paramiko.org/changelog.html
2) https://github.com/paramiko/paramiko/issues/2017
3) https://github.com/paramiko/paramiko/issues/1984
Tests:
- Add a remote backup repository with and without encryption.
- Add remote backup repository with all three types of algorithms.
- Add a remote repository again with wrong password. Authentication error is
properly shown.
- Add a remote backup repository and remove it. Host remains verified. Add a
repository again.
- Add a remote backup repository and remove it. Host remains verified. Change
the fingerprint the /var/lib/plinth/.ssh/known_hosts file. Add a repository
again. A proper error is shown that remote host could not be verified.
- Add a remote backup repository and remove it. Host remains verified. Stop SSH
server on the remote host. A generic error is shown that ssh command on remote
host failed.
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
- The output of ssh-keyscan scan contain comments that start with '#'. When
these are present, they are incorrectly assumed to be valid keys. Ignore these
lines.
Tests:
- Output of ssh-keyscan with latest OpenSSH in Debian testing contains comments.
Trying to verify the identity of the remove host without the patch fails when
adding remote backup repository. When patch is applied, it works (except RSA
key).
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
