Multi-system administration with Func

Managing multiple computer systems can involve a lot of repetitive tasks: connecting to each, performing some update, status check, or configuration tweak, and then moving on to the next machine. These kinds of things can be scripted of course, but scripts of that nature typically need to be adjusted frequently as machines come and go or the tasks change. The Fedora Unified Network Controller (Func) is a tool that will help simplify system administration, but there is more to it than that—it is a framework for doing two-way secure communication, from the command line, scripts, or applications.

Func is written in Python, providing an API for scripts written in that language, but it can also be used from the command line. Each client machine—minion in Func-speak—runs the funcd daemon which contacts the master server or overlord. From the overlord machine, commands can then be issued to individual minions or to subsets of them. Some of the power of Func can be seen in simple commands like:

    func "*" call service restart httpd

Similar kinds of tasks—but with more control—can be handled through the Python API. A somewhat contrived example from the Func website gives a sense of what can be done:

    import func.overlord.client as fc

    results = fc.Client("*").service.status("httpd")
    for (host, returns) in results.iteritems():
       if returns == 0:
	   fc.Client(host).reboot.reboot()

Managing keys can be a hassle when using ssh as an administrative tool, so Func uses another tool, Certmaster, to assist with keys. Certmaster provides a set of utilities and a Python API for managing SSL certificates. Clients generate certificate signing requests (CSRs), which contain their public key, that are sent to the Certmaster on the overlord. Administrators can either sign them from the command line or enable auto-signing. The minion then retrieves the signed certificate so that the overlord and minion communicate over an encrypted channel after that.

Func is not meant to replace ssh, instead it is intended to provide multi-system and scripting capabilities which are not the strengths of ssh. Like ssh, though, Func is meant to be easy to deploy—eventually ubiquitous, at least for Fedora—simple to use as well as easy to extend. It also has a pluggable architecture that allows Python modules to be integrated easily into Func, expanding the abilities of the minions. The documentation shows how to use the func-create-module command to generate template code which allows the administrator to ignore the Func requirements and concentrate on the task at hand.

There is nothing particularly Fedora-specific about Func, that's just where it was born. There are some efforts underway to add it for other distributions. Most of the work would be in creating distribution-specific analogs for things like restarting services and querying hardware configurations.

Red Hat has been releasing a steady stream of system administration tools over the last year or so. The Emerging Technology (ET) group has developed quite an ecosystem of tools to support installations with large numbers of servers that are frequently installed and upgraded. One might think they have a large infrastructure of such servers.

One of those tools that is frequently discussed in conjunction with Func is Cobbler. It is meant to simplify the configuration of a server to handle network installation and booting for a large server farm. From the web page:

Cobbler and the other tools coming out of the ET group are not just targeted at physical machines, but also virtualized environments. By using Cobbler, the puppet configuration manager, and the oVirt virtual machine manager, thousands of systems of various kinds can be managed in a centralized fashion. As would be expected, all of the code is available as free software.

These tools are quite interesting for system administrators, particularly those who use Fedora and have lots of systems to maintain. Even for small home networks, though, Func at least could come in handy. For overworked administrators—no matter the size of their domain—better tools are always welcome.

Note: you can avoid this step in the future by logging into your LWN account.