integrate exmachina configuration management layer

- add exmachina code and test code
- modify plinth.py to listen for shared secret on stdin at start
  (if appropriate flag is set) and try to connect to exmachina daemon
- use exmachina to read and set /etc/hostname as a demo
- update plinth init.d script to start exmachina and share keys
- update docs with new deps and run instructions
This commit is contained in:
bnewbold 2012-07-12 17:21:54 -04:00
parent 337560b0d1
commit f1e764f2e5
11 changed files with 544 additions and 24 deletions

23
INSTALL
View File

@ -2,10 +2,10 @@
## Installing Plinth
Install the python-cheetah package and pandoc:
apt-get install python-cheetah pandoc
Install the python-cheetah package, pandoc, python-augeas, and
bjsonrpc:
apt-get install python-cheetah pandoc python-augeas python-bjsonrpc
Install the python-simplejson
@ -13,9 +13,16 @@ apt-get install python-simplejson
Unzip and untar the source into a directory. Change to the directory
containing the program. Do `make` and then run `./plinth.py` and
point your web browser at `localhost:8000`. The default username is
"admin" and the default password is "secret".
containing the program. Run:
$ make
$ ./plinth.py
and point your web browser at `localhost:8000`. The default username is "admin"
and the default password is "secret". To actually edit the configuration of
your local/dev machine, also run:
$ sudo ./exmachina/exmachina.py -v &
## Dependencies
@ -27,6 +34,10 @@ point your web browser at `localhost:8000`. The default username is
* *GNU Make* is used to build the templates and such.
* bjsonrpc - used for configuration management layer
* python-augeas and augeas - used for configuration management
The documentation has some dependencies too.
* *Markdown* is used to format and style docs.

12
NOTES
View File

@ -2,6 +2,18 @@
%
% February 2012
# Edits by bnewbold
## 2012-07-12 "exmachina" configuration management layer
- this new code is very ugly and in the "just make it work" style
- add exmachina code and test code
- modify plinth.py to listen for shared secret on stdin at start
(if appropriate flag is set) and try to connect to exmachina daemon
- use exmachina to read and set /etc/hostname as a demo
- update plinth init.d script to start exmachina and share keys
- update docs with new deps and run instructions
# Edits by seandiggity
## 2012-02-27 new theme based upon bootstrap

3
README
View File

@ -50,4 +50,5 @@ interface will overwrite those changes at first opportunity. This
interface is not a tool for super admins facing complex scenarios. It
is for home users to do a wide variety of basic tasks.
See comments in exmachina/exmachina.py for more details about the configuration
management process seperation scheme.

0
exmachina/__init__.py Normal file
View File

353
exmachina/exmachina.py Executable file
View File

@ -0,0 +1,353 @@
#!/usr/bin/env python
"""
Author: bnewbold <bnewbold@robocracy.org>
Date: July 2012
License: GPLv3 (see http://www.gnu.org/licenses/gpl-3.0.html)
(two helper functions copied from web, as cited below)
Package Requirements: python-augeas, bjsonrpc
This file implements both ends (privilaged daemon and unprivilaged python
client library) of a crude system configuration message bus, intended for use
(initially) with the Plinth web interface to the FreedomBox operating system.
The goal is to provide partially-untrusted processes (such as the web interface
running as the www-data user) access to core system configuration files
(through the Augeas library) and daemon control (through the init.d scripts).
The daemon process (started in the same startup script as Plinth) runs as root
and accepts JSON-RPC method calls through a unix domain socket
(/tmp/exmachina.sock by default). Because file access control may not be
sufficiently flexible for access control, a somewhat-elaborate secret key
mechanism can be used to control access to the RPC mechanism.
The (optional) shared secret-key mechanism requires clients to first call the
"authenticate" RPC method before any other methods. The secret key is passed to
the server process through stdin at startup (command line arguments could be
snooped by unprivilaged processes), and would presumably be passed on to the
client in the same way. The init_test.sh script demonstrates this mechanism.
"""
import os
import sys
import optparse
import logging
import socket
import subprocess
import stat
import time
import base64
import bjsonrpc
import bjsonrpc.handlers
import bjsonrpc.server
import augeas
log = logging.getLogger(__name__)
def execute_service(servicename, action, timeout=10):
"""This function mostly ripped from StackOverflow:
http://stackoverflow.com/questions/1556348/python-run-a-process-with-timeout-and-capture-stdout-stderr-and-exit-status
"""
# ensure service name isn't tricky trick
script = "/etc/init.d/" + os.path.split(servicename)[1]
if not os.path.exists(script):
return "ERROR: so such service"
command_list = [script, action]
log.info("executing: %s" % command_list)
proc = subprocess.Popen(command_list,
bufsize=0,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
poll_seconds = .250
deadline = time.time() + timeout
while time.time() < deadline and proc.poll() == None:
time.sleep(poll_seconds)
if proc.poll() == None:
if float(sys.version[:3]) >= 2.6:
proc.terminate()
raise Exception("execution timed out (>%d seconds): %s" %
(timeout, command_list))
stdout, stderr = proc.communicate()
return stdout, stderr, proc.returncode
class ExMachinaHandler(bjsonrpc.handlers.BaseHandler):
# authentication state variable. If not None, still need to authenticate;
# if None then authentication not require or was already successful for
# this instantiation of the Handler. This class variable gets optionally
# overridden on a per-process basis
secret_key = None
def _setup(self):
self.augeas = augeas.Augeas()
def authenticate(self, secret_key):
if not self.secret_key:
log.warn("Unecessary authentication attempt")
return
if not secret_key.strip() == self.secret_key.strip():
# fail hard
log.error("Authentication failed!")
sys.exit()
self.secret_key = None
# ------------- Augeas API Passthrough -----------------
def augeas_save(self):
if not self.secret_key:
log.info("augeas: saving config")
return self.augeas.save()
def augeas_set(self, path, value):
if not self.secret_key:
log.info("augeas: set %s=%s" % (path, value))
return self.augeas.set(path.encode('utf-8'),
value.encode('utf-8'))
def augeas_setm(self, base, sub, value):
if not self.secret_key:
log.info("augeas: setm %s %s = %s" % (base, sub, value))
return self.augeas.setm(base.encode('utf-8'),
sub.encode('utf-8'),
value.encode('utf-8'))
def augeas_get(self, path):
if not self.secret_key:
# reduce verbosity
log.debug("augeas: get %s" % path)
return self.augeas.get(path.encode('utf-8'))
def augeas_match(self, path):
if not self.secret_key:
# reduce verbosity
log.debug("augeas: match %s" % path)
return self.augeas.match("%s" % path.encode('utf-8'))
def augeas_insert(self, path, label, before=True):
if not self.secret_key:
log.info("augeas: insert %s=%s" % (path, value))
return self.augeas.insert(path.encode('utf-8'),
label.encode('utf-8'),
before=before)
def augeas_move(self, src, dst):
if not self.secret_key:
log.info("augeas: move %s -> %s" % (src, dst))
return self.augeas.move(src.encode('utf-8'), dst.encode('utf-8'))
def augeas_remove(self, path):
if not self.secret_key:
log.info("augeas: remove %s" % path)
return self.augeas.remove(path.encode('utf-8'))
# ------------- init.d Service Control -----------------
def initd_status(self, servicename):
if not self.secret_key:
return execute_service(servicename, "status")
def initd_start(self, servicename):
if not self.secret_key:
return execute_service(servicename, "start")
def initd_stop(self, servicename):
if not self.secret_key:
return execute_service(servicename, "stop")
def initd_restart(self, servicename):
if not self.secret_key:
return execute_service(servicename, "restart")
class EmptyClass():
# Used by ExMachinaClient below
pass
class ExMachinaClient():
"""Simple client wrapper library to expose augeas and init.d methods.
In brief, use augeas.get/set/insert to modify system configuration files
under the /files/etc/* namespace. augeas.match with a wildcard can be used
to find variables to edit.
After making any changes, use augeas.save to commit to disk, then
initd.restart to restart the appropriate system daemons. In many cases,
this would be the 'networking' meta-daemon.
See test_exmachina.py for some simple examples; see the augeas docs for
more in depth guidance.
"""
def __init__(self,
socket_path="/tmp/exmachina.sock",
secret_key=None):
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
self.sock.connect(socket_path)
self.conn = bjsonrpc.connection.Connection(self.sock)
if secret_key:
self.conn.call.authenticate(secret_key)
self.augeas = EmptyClass()
self.initd = EmptyClass()
self.augeas.save = self.conn.call.augeas_save
self.augeas.set = self.conn.call.augeas_set
self.augeas.setm = self.conn.call.augeas_setm
self.augeas.get = self.conn.call.augeas_get
self.augeas.match = self.conn.call.augeas_match
self.augeas.insert = self.conn.call.augeas_insert
self.augeas.move = self.conn.call.augeas_move
self.augeas.remove = self.conn.call.augeas_remove
self.initd.status = self.conn.call.initd_status
self.initd.start = self.conn.call.initd_start
self.initd.stop = self.conn.call.initd_stop
self.initd.restart = self.conn.call.initd_restart
def close(self):
self.sock.close()
def run_server(socket_path, secret_key=None):
if not 0 == os.geteuid():
log.warn("Expected to be running as root!")
if os.path.exists(socket_path):
os.unlink(socket_path)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.bind(socket_path)
sock.listen(1)
# TODO: www-data group permissions only?
os.chmod(socket_path, 0666)
if secret_key:
ExMachinaHandler.secret_key = secret_key
serv = bjsonrpc.server.Server(sock, handler_factory=ExMachinaHandler)
serv.serve()
def daemonize (stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
"""
From: http://www.noah.org/wiki/Daemonize_Python
This forks the current process into a daemon. The stdin, stdout, and
stderr arguments are file names that will be opened and be used to replace
the standard file descriptors in sys.stdin, sys.stdout, and sys.stderr.
These arguments are optional and default to /dev/null. Note that stderr is
opened unbuffered, so if it shares a file with stdout then interleaved
output may not appear in the order that you expect. """
# Do first fork.
try:
pid = os.fork()
if pid > 0:
sys.exit(0) # Exit first parent.
except OSError, e:
sys.stderr.write ("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror) )
sys.exit(1)
# Decouple from parent environment.
os.chdir("/")
os.umask(0)
os.setsid()
# Do second fork.
try:
pid = os.fork()
if pid > 0:
sys.exit(0) # Exit second parent.
except OSError, e:
sys.stderr.write ("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror) )
sys.exit(1)
# Now I am a daemon!
# Redirect standard file descriptors.
si = open(stdin, 'r')
so = open(stdout, 'a+')
se = open(stderr, 'a+', 0)
os.dup2(si.fileno(), sys.stdin.fileno())
os.dup2(so.fileno(), sys.stdout.fileno())
os.dup2(se.fileno(), sys.stderr.fileno())
return pid
# =============================================================================
# Command line handling
def main():
global log
parser = optparse.OptionParser(usage=
"usage: %prog [options]\n"
"%prog --help for more info."
)
parser.add_option("-v", "--verbose",
default=False,
help="Show more debugging statements",
action="store_true")
parser.add_option("-q", "--quiet",
default=False,
help="Show fewer informational statements",
action="store_true")
parser.add_option("-k", "--key",
default=False,
help="Wait for Secret Access Key on stdin before starting",
action="store_true")
parser.add_option("--random-key",
default=False,
help="Just dump a random base64 key and exit",
action="store_true")
parser.add_option("-s", "--socket-path",
default="/tmp/exmachina.sock",
help="UNIX Domain socket file path to listen on",
metavar="FILE")
parser.add_option("--pidfile",
default=None,
help="Daemonize and write pid to this file",
metavar="FILE")
(options, args) = parser.parse_args()
if len(args) != 0:
parser.error("Incorrect number of arguments")
if options.random_key:
sys.stdout.write(base64.urlsafe_b64encode(os.urandom(128)))
sys.exit(0)
log = logging.getLogger()
hdlr = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
hdlr.setFormatter(formatter)
log.addHandler(hdlr)
if options.verbose:
log.setLevel(logging.DEBUG)
elif options.quiet:
log.setLevel(logging.ERROR)
else:
log.setLevel(logging.INFO)
secret_key = None
if options.key:
log.debug("Waiting for secret key on stdin...")
secret_key = sys.stdin.readline().strip()
log.debug("Got it!")
if options.pidfile:
with open(options.pidfile, 'w') as pfile:
# ensure file is available/writable
pass
os.unlink(options.pidfile)
daemonize()
pid = os.getpid()
with open(options.pidfile, 'w') as pfile:
pfile.write("%s" % pid)
log.info("Daemonized, pid is %s" % pid)
run_server(secret_key=secret_key, socket_path=options.socket_path)
if __name__ == '__main__':
main()

13
exmachina/init_test.sh Executable file
View File

@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Test init.d-style initialization; run this script as root (or sudo it)
export key=`./exmachina.py --random-key`
echo $key | ./exmachina.py -vk --pidfile /tmp/exmachina_test.pid
sleep 1
echo $key | sudo -u www-data -g www-data ./test_exmachina.py -k
kill `cat /tmp/exmachina_test.pid` && rm /tmp/exmachina_test.pid
sleep 1
jobs

71
exmachina/test_exmachina.py Executable file
View File

@ -0,0 +1,71 @@
#!/usr/bin/env python
"""
This file tests the "client side" of the exmachina layer.
To use with secret keys, do the following in seperate terminals:
$ echo "<key>" | ./exmachina.py -vk
$ echo "<key>" | ./test.py -k
To use without, do the following in seperate terminals:
$ echo "<key>" | ./exmachina.py -vk
$ echo "<key>" | ./test.py -k
Use the init_test.sh script to test shared key passing and privilage seperation
at the same time:
$ sudo ./init_test.sh
"""
import sys
import optparse
import logging
import socket
import bjsonrpc
import bjsonrpc.connection
import augeas
from exmachina import ExMachinaClient
# =============================================================================
# Command line handling
def main():
socket_path="/tmp/exmachina.sock"
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect(socket_path)
secret_key = None
if sys.argv[-1] == "-k":
print "waiting for key on stdin..."
secret_key = sys.stdin.readline()
print "sent!"
print "========= Testing JSON-RPC connection"
c = bjsonrpc.connection.Connection(sock)
if secret_key:
c.call.authenticate(secret_key)
print "/*: %s" % c.call.augeas_match("/*")
print "/augeas/*: %s" % c.call.augeas_match("/augeas/*")
print "/etc/* files:"
for name in c.call.augeas_match("/files/etc/*"):
print "\t%s" % name
print c.call.initd_status("bluetooth")
print "hostname: %s" % c.call.augeas_get("/files/etc/hostname/*")
print "localhost: %s" % c.call.augeas_get("/files/etc/hosts/1/canonical")
sock.close()
print "========= Testing user client library"
client = ExMachinaClient(secret_key=secret_key)
print client.augeas.match("/files/etc/*")
#print client.initd.restart("bluetooth")
print client.initd.status("greentooth")
print "(expect Error on the above line)"
print client.initd.status("bluetooth")
client.close()
if __name__ == '__main__':
main()

2
fabfile.py vendored
View File

@ -138,7 +138,7 @@ def apache():
@task
def deps():
"Basic plinth dependencies"
sudo('apt-get install --no-install-recommends -y python make python-cheetah pandoc python-simplejson python-pyme')
sudo('apt-get install --no-install-recommends -y python make python-cheetah pandoc python-simplejson python-pyme python-augeas python-bjsonrpc')
@task
def update():

View File

@ -41,20 +41,18 @@ def valid_hostname(name):
def set_hostname(hostname):
"Sets machine hostname to hostname"
cfg.log.info("Writing '%s' to /etc/hostname" % hostname)
unslurp("/etc/hostname", hostname+"\n")
cfg.log.info("Writing '%s' to /etc/hostname with exmachina" % hostname)
try:
retcode = subprocess.call("/etc/init.d/hostname.sh start", shell=True)
if retcode < 0:
cfg.log.error("Hostname restart terminated by signal: return code is %s" % retcode)
else:
cfg.log.debug("Hostname restart returned %s" % retcode)
cfg.exmachina.augeas.set("/files/etc/hostname/*", hostname)
cfg.exmachina.augeas.save()
# don't persist/cache change unless it was saved successfuly
sys_store = filedict_con(cfg.store_file, 'sys')
sys_store['hostname'] = hostname
cfg.exmachina.initd.restart("hostname.sh") # is hostname.sh debian-only?
except OSError, e:
raise cherrypy.HTTPError(500, "Hostname restart failed: %s" % e)
sys_store = filedict_con(cfg.store_file, 'sys')
sys_store['hostname'] = hostname
class general(FormPlugin, PagePlugin):
url = ["/sys/config"]
order = 30
@ -72,8 +70,11 @@ class general(FormPlugin, PagePlugin):
def main(self, message='', **kwargs):
sys_store = filedict_con(cfg.store_file, 'sys')
hostname = cfg.exmachina.augeas.get("/files/etc/hostname/*")
# this layer of persisting configuration in sys_store could/should be
# removed -BLN
defaults = {'time_zone': "slurp('/etc/timezone').rstrip()",
'hostname': "gethostname()",
'hostname': "hostname",
}
for k,c in defaults.items():
if not k in kwargs:
@ -81,6 +82,8 @@ class general(FormPlugin, PagePlugin):
kwargs[k] = sys_store[k]
except KeyError:
exec("if not '%(k)s' in kwargs: sys_store['%(k)s'] = kwargs['%(k)s'] = %(c)s" % {'k':k, 'c':c})
# over-ride the sys_store cached value
kwargs['hostname'] = hostname
## Get the list of supported timezones and the index in that list of the current one
module_file = __file__
@ -120,7 +123,8 @@ class general(FormPlugin, PagePlugin):
old_val = sys_store['hostname']
try:
set_hostname(hostname)
except:
except Exception, e:
cfg.log.error(e)
cfg.log.info("Trying to restore old hostname value.")
set_hostname(old_val)
raise

View File

@ -17,6 +17,9 @@ from util import *
from logger import Logger
#from modules.auth import AuthController, require, member_of, name_is
from exmachina.exmachina import ExMachinaClient
import socket
__version__ = "0.2.14"
__author__ = "James Vasile"
__copyright__ = "Copyright 2011, James Vasile"
@ -71,9 +74,17 @@ def parse_arguments():
parser = argparse.ArgumentParser(description='Plinth web interface for the FreedomBox.')
parser.add_argument('--pidfile', default="",
help='specify a file in which the server may write its pid')
parser.add_argument('--listen-exmachina-key', default=False, action='store_true',
help='listen for JSON-RPC shared secret key on stdin at startup')
args=parser.parse_args()
if args.pidfile:
cfg.pidfile = args.pidfile
if args.listen_exmachina_key:
# this is where we optionally try to read in a shared secret key to
# authenticate connections to exmachina
cfg.exmachina_secret_key = sys.stdin.readline().strip()
else:
cfg.exmachina_secret_key = None
def setup():
parse_arguments()
@ -85,6 +96,13 @@ def setup():
except AttributeError:
pass
try:
cfg.exmachina = ExMachinaClient(
secret_key=cfg.exmachina_secret_key or None)
except socket.error:
cfg.exmachina = None
print "couldn't connect to exmachina daemon, but continuing anyways..."
os.chdir(cfg.file_root)
cherrypy.config.update({'error_page.404': error_page_404})
cherrypy.config.update({'error_page.500': error_page_500})

View File

@ -12,23 +12,53 @@
# This file is /etc/init.d/plinth
DAEMON=/usr/local/bin/plinth.py
EXMACHINA_DAEMON=/usr/local/bin/exmachina.py
PID_FILE=/var/run/plinth.pid
EXMACHINA_PID_FILE=/var/run/exmachina.pid
PLINTH_USER=www-data
PLINTH_GROUP=www-data
test -x $DAEMON || exit 0
test -x $EXMACHINA_DAEMON || exit 0
set -e
. /lib/lsb/init-functions
start_plinth (){
if [ -f $PID_FILE ]; then
echo Already running with a pid of `cat $PID_FILE`.
echo Already running with a pid of `cat $PID_FILE`.
else
$DAEMON --pidfile=$PID_FILE
if [ -f $EXMACHINA_PID_FILE ]; then
echo exmachina was already running with a pid of `cat $EXMACHINA_PID_FILE`.
kill -15 `cat $EXMACHINA_PID_FILE`
rm -rf $EXMACHINA_PID_FILE
fi
SHAREDKEY=`$EXMACHINA_DAEMON --random-key`
touch $PID_FILE
chown $PLINTH_USER:$PLINTH_GROUP $PID_FILE
echo $SHAREDKEY | $EXMACHINA_DAEMON --pidfile=$EXMACHINA_PID_FILE || rm $PID_FILE
sleep 0.5
echo $SHAREDKEY | sudo -u $PLINTH_USER -g $PLINTH_GROUP $DAEMON --pidfile=$PID_FILE
fi
}
stop_plinth () {
if [ -f $PID_FILE ]; then
kill -15 `cat $PID_FILE`
kill -15 `cat $PID_FILE` || true
rm -rf $PID_FILE
echo "killed plinth"
else
echo "No pid file at $PID_FILE suggests plinth is not running."
fi
if [ -f $EXMACHINA_PID_FILE ]; then
kill -15 `cat $EXMACHINA_PID_FILE` || true
rm -rf $EXMACHINA_PID_FILE
echo "killed exmachina"
else
echo "No pid file at $EXMACHINA_PID_FILE suggests exmachina is not running."
fi
}
test -x $DAEMON || exit 0
@ -45,5 +75,12 @@ case "$1" in
$0 stop
$0 start
;;
status)
status_of_proc -p $PID_FILE "$DAEMON" plinth && exit 0 || exit $?
;;
*)
echo "Usage: $NAME {start|stop|restart|status}" >&2
exit 1
;;
esac