3 Commits

Author	SHA1	Message	Date
Sunil Mohan Adapa	91c4d6742e	cfg: In develop mode, use /var/lib for DB and sessions ... - Only effects develop mode. - To primarily avoid writing to the source code directory. Multiple containers or VMs using the source folder won't fight with the database file (the overlay file system plan is not working out well for containers #1873). - In the earlier days, we used to allow running from source code directory without even doing ./setup.py install. Currently it is not possible anyway. We pretty much install freedombox package before running from source directory. - If the build process itself learns not to write to source directory, then containers/VMs won't have to write to source directory at all. Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org> Reviewed-by: James Valleroy <jvalleroy@mailbox.org>	2020-06-28 21:01:53 -04:00
Sunil Mohan Adapa	7f7f4cfb52	container: Remove sqlite3 file early enough ... During --list-dependencies if an old sqlite3 file is present with gitweb enabled, then a2enconf -c gitweb-freedombox-auth get executed. In this case, setting up apache2 module fails because authpubtkt tokens are not yet generated but they are being referred to in the configuration files. Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org> Reviewed-by: Veiko Aasa <veiko17@disroot.org>	2020-06-25 01:21:39 +03:00
Sunil Mohan Adapa	2e5a1857f7	container: Add script to manage systemd-nspawn containers for dev. ... Inspired by: https://salsa.debian.org/vexch/plinth/snippets/391 This script creates a simple container using systemd-nspawn for developing FreedomBox. It has many advantages over running a VM using Vagrant. RAM is allocated to processes in the container as needed without any fixed limit. Also RAM does not have to be statically allocated so it is typically much lighter than running an VM. There is no hardware emulation when running a container with same architecture, so processes run as fast as they would on the host machine. Environment: The script will run only run on hosts having systemd-nspawn and network-manager installed, typical GNU/Linux distributions. It has been primarily developed and tested on Debian Buster but should work on most modern GNU/Linux distributions. Disk image: systemd-nspawn accepts not only a directory for starting a container but also a disk image. This disk image is loop-back mounted and container is started from that mounted directory. The partition to use is determined by looking at the boot flag in the partition table. This happens to work well with all existing FreedomBox images. In future, we may be able to run different architectures in this manner. After downloading, the disk image is expanded along with the partition and file system inside so that development can be done without running into disk space issues. Expanding the disk does not immediately consume disk space because it will be a sparse file. As data is written to the disk, it will occupy more and more space but the upper limit is the size to which disk has been expanded. Downloading images: Images are downloaded from FreedomBox download server using fixed URLs for each distribution. Signature is verified for the download images. The fingerprint of the allowed signing key is hard-coded in this script. Downloaded images are kept even after destroying the extracted raw image along with container. This allows for quickly resetting the container without downloading again. Booting: systemd-nspawn is run in 'boot' mode. This means that init process (happens to be systemd) is started inside the container. It then spawns all the other necessary daemons including openssh-server, firewalld and network-manager. A login terminal can be opened using 'machinectl login' because container is running systemd. SSH into the container is possible because network is up, configured by network-manager, and openssh server is running. Shared folder: Using systemd-nspawn, the project directory is mounted as /freedombox inside the container. The project directory is determined as directory in which this script resides. The project folder from the container point of view will be read-only. Container should be able to write various files such as build files, FreedomBox sqlite3 database and session files into the /freedombox folder. To enable writing, an additional read-write folder is overlayed onto /freedombox folder in the container. This directory can't be created under the project folder and is created instead in $XDG_DATA_HOME/freedombox-container/overlay/$DISTRIBUTION. If XDG_DATA_HOME is not set, it is assumed to be $HOME/.local/shared/. Whenever data is written into /freedombox directory inside the container, this directory on the host receives the changes. See documentation for Overlay filesystem for further details. When container is destroyed, this overlay folder is destroyed to ensure clean state after bringing up the container again. Users: PrivateUsers configuration flag for systemd-nspawn is currently off. This means that each user's UID on the host is also the same UID in the container as along as there is an entry in the container's password database. In future, we may explore using private users inside the container. 'fbx' is the development user and its UID is changed during setup phase to 10000 hoping it would not match anything on the host system. 'fbx' user has full sudo access inside the container without needing a password. Password for this user is not set by default, but can be set if needed. If there is no access to the container in any way, one can run 'sudo machinectl shell' and then run 'passwd fbx' to set the password for the 'fbx' user. 'plinth' user's UID in the container is also changed and set to the UID of whichever user owns the project directory. This allows the files to written by 'plinth' container user in the project directory because UID of the owner of the directory is same as the 'plinth' user's UID in container. Network: A private network is created inside the container using systemd-nspawn feature. Network interfaces from the host are not available inside the container. A new network interface called 'host0' is configured inside the container which is automatically configured by network-manager. On the host a new network interface is created. This script creates configuration for a 'shared' network using network-manager. When bringing up the container, this network connection is also brought up. A DHCP server and a DNS server are started network-manager on the host side so that DHCP and DNS client functions work inside the container. Traffic from the container is also masqueraded so that Internet connectivity inside the container works if the host has one. If necessary, the network interface on host side can be differently configured. For example, it can be bridged with another interface to expose the container on a network that the host machine participates in. The network IP address inside the container can be queried using machinectl. This script queries that IP address and presents the address in its console messages. All ports in the container can be reached from the host using this IP address as long as the firewall inside the container allows it. There is no need to perform port forwarding or mapping. SSH: It is assumed that openssh-server is installed inside the container. SSH server keys in the container are created if missing. Client side keys are created in .container/ssh directory and the public key is installed in the authorized keys file of the 'fbx' user. The 'ssh' sub-command to this script is simply a convenience mechanism for quick launch of ssh with the right IP address, user name and identity file. Role of machinectl: Most of the work is done by systemd-nspawn. machinectl is useful for running systemd-nspawn in the background and querying its current state. It also helps with providing the IP address of the container. machinectl is made to recognize the container by creating a link in /var/lib/machines/ to the image file. systemd-nspawn options are added by creating a temporary file in /run/systemd/nspawn. All machinectl commands should work. Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org> Reviewed-by: Veiko Aasa <veiko17@disroot.org>	2020-06-04 18:20:03 +03:00