Bcfg2 Documentation

1. Bcfg2 Architecture
2. Installing Bcfg2
   1. Standard Group Names
   2. Example Bundles
3. Bcfg2 Specifications
4. Probes
   1. TCheetah Plugin
5. DynamicGroups
6. Client Tool Drivers
   1. Using Actions
7. Deploying Bcfg2
   1. Simple Deployments
   2. A Near-Literal Deployment
   3. An Abstract Deployment
   4. Bcfg2 Server Administration
   5. An example application of bcfg2
8. Developing for Bcfg2
9. BCFG2 Reports
   1. Dynamic Reports
      1. Dynamic Reports Installation
      2. Dynamic Reports Usage
   2. Static Reports
   3. Demo Repository
   4. Troubleshooting
   5. Manual Pages

Bcfg2 Architecture

Bcfg2 Architecture

Bcfg2 is based on a client-server architecture. The client is responsible for interpreting (but not processing) the configuration served by the server. This configuration is literal, so no local process is required. After completion of the configuration process, the client uploads a set of statistics to the server. This section will describe the goals and then the architecture motivated by it.

Goals

• Model configurations using declarative semantics.
  Declarative semantics maximize the utility of configuration management tools; they provide the most flexibility for the tool to determine the right course of action in any given situation. This means that users can focus on the task of describing the desired configuration, while leaving the task of transitioning clients states to the tool.
• Configuration descriptions should be comprehensive.
  This means that configurations served to the client should be sufficient to reproduce all desired functionality. This assumption allows the use of heuristics to detect extra configuration, aiding in reliable, comprehensive configuration definitions.
• Provide a flexible approach to user interactions.
  Most configuration management systems take a rigid approach to user interactions; that is, either the client system is always correct, or the central system is. This means that users are forced into an overly proscribed model where the system asserts where correct data is. Configuration data modification is frequently undertaken on both the configuration server and clients. Hence, the existence of a single canonical data location can easily pose a problem during normal tool use. Bcfg2 takes a different approach.

The default assumption is that data on the server is correct, however, the client has option to run in another mode where local changes are catalogued for server-side integration. If the Bcfg2 client is run in dry run mode, it can help to reconcile differences between current client state and the configuration described on the server. The Bcfg2 client also searches for extra configuration; that is, configuration that is not specified by the configuration description. When extra configuration is found, either configuration has been removed from the configuration description on the server, or manual configuration has occurred on the client. Options related to two-way verification and removal are useful for configuration reconciliation when interactive access is used.

• Plugins and administrative applications.
• Incremental operations.

The Bcfg2 Client

The Bcfg2 client performs all client configuration or reconfiguration operations. It renders a declarative configuration specification, provided by the Bcfg2 server, into a set of configuration operations which will, if executed, attempt to change the client's state into that described by the configuration specification. Conceptually, the Bcfg2 client serves to isolate the Bcfg2 server and specification from the imperative operations required to implement configuration changes.

This isolation allows declarative specifications to be manipulated symbolically on the server, without needing to understand the properties of the underlying system tools. In this way, the Bcfg2 client acts as a sort of expert system that "knows" how to implement declarative configuration changes.

The operation of the Bcfg2 client is intended to be as simple as possible. The normal configuration process consists of four main steps:

• Probe Execution
  During the probe execution stage, the client connects to the server and downloads a series of probes to execute. These probes reveal local facts to the Bcfg2 server. For example, a probe could discover the type of video card in a system. The Bcfg2 client returns this data to the server, where it can influence the client configuration generation process.
• Configuration Download and Inventory
  The Bcfg2 client now downloads a configuration specification from the Bcfg2 server. The configuration describes the complete target state of the machine. That is, all aspects of client configuration should be represented in this specification. For example, all software packages and services should be represented in the configuration specification. The client now performs a local system inventory. This process consists of verifying each entry present in the configuration specification. After this check is completed, heuristic checks for configuration not included in the configuration specification. We refer to this inventory process as 2-way validation, as first we verify that the client contains all configuration that is included in the specification, then we check if the client has any extra configuration that isn't present. This provides a fairly rigorous notion of client configuration congruence. Once the 2-way verification process has been performed, the client has built a list of all configuration entries that are out of spec. This list has two parts: specified configuration that is incorrect (or missing) and unspecified configuration that should be removed.
• Configuration Update
  The client now attempts to update its configuration to match the specification. Depending on options, changes may not (or only partially) be performed. First, if extra configuration correction is enabled, extra configuration can be removed. Then the remaining changes are processed. The Bcfg2 client loops while progress is made in the correction of these incorrect configuration entries. This loop results in the client being able to accomplish all it will be able to during one execution. Once all entries are fixed, or no progress is being made, the loop terminates.Once all configuration changes that can be performed have been, bundle dependencies are handled. Bundle groupings result in two different behaviors. Contained entries are assumed to be inter-dependant. To address this, the client re-verifies each entry in any bundle containing an updates configuration entry. Also, services contained in modified bundles are restarted.
• Statistics Upload
  Once the reconfiguration process has concluded, the client reports information back to the server about the actions it performed during the reconfiguration process. Statistics function as a detailed return code from the client. The server stores statistics information. Information included in this statistics update includes (but is not limited to):
  • Overall client status (clean/dirty)
  • List of modified configuration entries
  • List of uncorrectable configuration entries

Architecture Abstraction

The Bcfg2 client internally supports the administrative tools available on different architectures. For example, rpm and apt-get are both supported, allowing operation of Debian, Redhat, SUSE, and Mandriva systems. The client toolset is specified in the configuration specification. The client merely includes a series of libraries which describe how to interact with the system tools on a particular platform.

Three of the libraries exist. There is a base set of functions, which contain definitions describing how to perform POSIX operations. Support for configuration files, directories, and symlinks are included here. Two other libraries subclass this one, providing support for Debian and rpm-based systems.

The Debian toolset includes support for apt-get and update-rc.d. These tools provide the ability to install and remove packages, and to install and remove services.

The Redhat toolset includes support for rpm and chkconfig. Any other platform that uses these tools can also use this toolset. Hence, all of the other familiar rpm-based distributions can use this toolset without issue.

Other platforms can easily use the POSIX toolset, ignoring support for packages or services. Alternatively, adding support for new toolsets isn't difficult. Each toolset consists of about 125 lines of python code.

The Bcfg2 Server

The Bcfg2 server is responsible for taking a network description and turning it into a series of configuration specifications for particular clients. It also manages probed data and tracks statistics for clients.

The Bcfg2 server takes information from two sources when generating client configuration specifications. The first is a pool of metadata that describes clients as members of an aspect-based classing system. That is, clients are defined in terms of aspects of their behavior. The other is a file system repository that contains mappings from metadata to literal configuration. These are combined to form the literal configuration specifications for clients.

The Configuration Specification Construction Process

As we described in the previous section, the client connects to the server to request a configuration specification. The server uses the client's metadata and the file system repository to build a specification that is tailored for the client. This process consists of the following steps:

• Metadata Lookup
  The server uses the client's IP address to initiate the metadata lookup. This initial metadata consists of a (profile, image) tuple. If the client already has metadata registered, then it is used. If not, then default values are used and stored for future use. This metadata tuple is expanded using some profile and class definitions also included in the metadata. The end result of this process is metadata consisting of hostname, profile, image, a list of classes, a list of attributes and a list of bundles.
• Abstract Configuration Construction
  Once the server has the client metadata, it is used to create an abstract configuration. An abstract configuration contains all of the configuration elements that will exist in the final specification without any specifics. All entries will be typed (ie the tagname will be one of Package, ConfigurationFile, Service, !Symlink, or Directory) and will include a name. These configuration entries are grouped into bundles, which document installation time interdependencies.
• Configuration Binding
  The abstract configuration determines the structure of the client configuration, however, it doesn't contain literal configuration information. After the abstract configuration is created, each configuration entry must be bound to a client-specific value. The Bcfg2 server uses plugins to provide these client-specific bindings. The Bcfg2 server core contains a dispatch table that describes which plugins can handle requests of a particular type. The responsible plugin is located for each entry. It is called, passing in the configuration entry and the client's metadata. The behavior of plugins is explicitly undefined, so as to allow maximum flexibility. The behaviours of the stock plugins are documented elsewhere in this manual. Once this binding process is completed, the server has a literal, client-specific configuration specification. This specification is complete and comprehensive; the client doesn't need to process it at all in order to use it. It also represents the totality of the configuration specified for the client.

The Literal Configuration Specification

Literal configuration specifications are served to clients by the Bcfg2 server. This is a differentiating factor for Bcfg2; all other major configuration management systems use a non-literal configuration specification. That is, the clients receive a symbolic configuration that they process to implement target states. We took the literal approach for a few reasons:

• A small list of configuration element types can be defined, each of which can have a set of defined semantics. This allows the server to have a well-formed model of client-side operations. Without a static lexicon with defined semantics, this isn't possible. This allows the server, for example, to record the update of a package as a coherent event.
• Literal configurations do not require client-side processing. Removing client-side processing reduces the critical footprint of the tool. That is, the Bcfg2 client (and the tools it calls) need to be functional, but the rest of the system can be in any state. Yet, the client will receive a correct configuration.
• Having static, defined element semantics also requires that all operations be defined and implemented in advance. The implementation can maximize reliability and robustness. In more ad-hoc setups, these operations aren't necessarily safely implemented.

The Structure of Specifications

Configuration specifications contain some number of clauses. Two types of clauses exist. Bundles are groups of inter-dependent configuration entities. The purpose of bundles is to encode installation-time dependencies such that all new configuration is properly activated during reconfiguration operations. T hat is, if a daemon configuration file is changed, its daemon should be restarted. Another example of bundle usage is the reconfiguration of a software package. If a package contains a default configuration file, but it gets overwritten by an environment-specific one, then that updated configuration file should survive package upgrade. The purpose of bundles is to describe services, or reconfigured software packages. Independent clauses contains groups of configuration entities that aren't related in any way. This provides a convenient mechanism that can be used for bulk installations of software.

Each of these clauses contains some number of configuration entities. Five types of configuration entities exist: ConfigurationFile, Package, SymLink, Directory, and Service. Each of these correspond to the obvious system item. Configuration specifications can get quite large; many systems have specifications that top one megabyte in size. An example of one is included in an appendix. These configurations can be written by hand, or generated by the server. The easiest way to start using Bcfg2 is to write small static configurations for clients. Once configurations get larger, this process gets unwieldy; at this point, using the server makes more sense.

Design Considerations

This section will discuss several aspects of the design of bcfg2, and the particular use cases that motivated them. Initially, this will consist of a discussion of the system metadata, and the intended usage model for package indices as well.

System Metadata

Bcfg2 system metadata describes the underlying patterns in system configurations. It describes commonalities and differences between these specifications in a rigorous way. The groups used by bcfg2's metadata are responsible for differentiating clients from one another, and building collections of allocatable configuration.

The Bcfg2 metadata system has been designed with several high-level goals in mind. Flexibility and precision are paramount concerns; no configuration should be undescribable using the constructs present in the bcfg2 repository. We have found (generally the hard way) that any assumptions about the inherent simplicity of configuration patterns tend to be wrong, so obscenely complex configurations must be representable, even if these requirements seem illogical during the implementation.

In particular, we wanted to streamline several operations that commonly occurred in our environment.

• Copying one node's profile to another node.

In many environments, many nodes are instances of a common configuration specification. They all have similar roles and software. In our environment, desktop machines were the best example of this. Other than strictly per-host configuration like SSH keys, all desktop machines use a common configuration specification. This trivializes the process of creating a new desktop machine.

• Creating a specialized version of a currently existing profile.

In environments with highly varied configurations, departmental infrastructure being a good example, "another machine like X but with extra software" is a common requirement. For this reason, it must be trivially possible to inherit most of a configuration specification from some more generic source, while being able to describe overriding aspects in a convenient fashion.

• 3. Compose several pre-existing configuration aspects to create a new profile.

The ability to compose configuration aspects allows the easy creation of new profiles based on a series of predefined set of configuration specification fragments. The end result is more agility in environments where change is the norm.

In order for a classing system to be comprehensive, it must be usable in complex ways. The Bcfg2 metadata system has constructs that map cleanly to first-order logic. This implies that any complex configuration pattern can be represented (at all) by the metadata, as first-order logic is provably comprehensive. (There is a discussion later in the document describing the metadata system in detail, and showing how it corresponds to first-order logic)

These use cases motivate several of the design decisions that we made:

• There must be a many to one correspondence between clients and groups. Membership in a given profile group must imbue a client with all of its configuration properties.

Package Management

The interface provided in the bcfg2 repository for package specification was designed with automation in mind. The goal was to support an append only interface to the repository, so that users do not need to continuously re-write already existing bits of specification.

Installing Bcfg2

QuickStart

The steps below should get you from just thinking about a configuration management system to an operational installation of Bcfg2. If you get stuck, be sure to check the mailing list or drop in to our IRC channel.

For distro-specific guides, choose one of the following:

• QuickStart/CentOS
• QuickStart/Ubuntu

Get & Install Bcfg2 Server

We recommend running the server on a Linux machine for ease of deployment due to the availability of packages for the dependencies.

First, you need to download and install Bcfg2. Our Download has both source and packages for common environments, while our Install page describes what to do once you have the packages in hand. To start you will need to install the server on one machine and the client on one or more machines. Yes, your server can also be a client (and should be by the time your environment is fully managed). Detailed installation instructions can be found on the Install page.

Set up Repository

The next step after installing the Bcfg packages is to configure the server. You can easily set up a personalized default configuration by running, on the server,

bcfg2-admin init

You will be presented with a series of questions that will build a Bcfg2 configuration file in /etc/bcfg2.conf, set up a skeleton repository (in /var/lib/bcfg2 by default), help you create ssl certificates, and do any other similar tasks needed to get you started.

Once this process is done, you can start the Bcfg2 server:

/etc/init.d/bcfg2-server start

You can try it out by running the Bcfg2 client on the same machine, acting like it is your first client. Note: The following command will tell the client to run in no-op mode, meaning it will only check the client against the repository and report any changes it sees. It won't make any changes (partially because you haven't populated the repository with any yet). However, nobody is perfect - you can make a typo, our software can have bugs, monkeys can break in and hit enter before you are done. Don't run this command on a production system if you don't know what it does and aren't prepared for the consequences. We don't know of anybody having problems with it before, but it is better to be safe than sorry. And now for the command:

bcfg2 -q -v -n

That can be translated as "bcfg2 quick verbose no-op". The output should be something similar to:

Loaded tool drivers:
 Chkconfig    POSIX        PostInstall  RPM         

Phase: initial
Correct entries:        0
Incorrect entries:      0
Total managed entries:  0
Unmanaged entries:      242


Phase: final
Correct entries:        0
Incorrect entries:      0
Total managed entries:  0
Unmanaged entries:      242

Perfect! We have started out with an empty configuration, and none of our configuration elements are correct. It doesn't get much cleaner than that. But what about those unmanaged entries? Those are the extra configuration elements (probably all packages at the moment) that still aren't managed. Your goal now is to migrate each of those plus any it can't see up to the "Correct entries" line.

Populate Repository

Finally, you need to populate your repository. Unfortunately, from here on out we can't write up a simple recipe for you to follow to get this done. It is very dependent on your local configuration, your configuration management goals, the politics surrounding your particular machines, and many other similar details. We can, however, give you guidance.

After the above steps, you should have a toplevel repository structure that looks like:

bcfg-server:~ # ls /var/lib/bcfg2
Bundler/  Cfg/  Metadata/  Pkgmgr/  Rules/  SSHbase/  Svcmgr/  etc/

The place to start is the Metadata directory, which contains two files: clients.xml and groups.xml. Your current clients.xml will look pretty close to:

<Clients version="3.0">
   <Client profile="basic" pingable="Y" pingtime="0" name="bcfg-server.example.com"/>
</Clients>

The clients.xml file is just a series of <Client /> tags, each of which describe one host you manage. Right now we only manage one host, the server machine we just created. This machine is bound to the basic profile, is pingable, has a pingtime of 0, and has the name bcfg-server.example.com. The two "ping" parameters don't matter to us at the moment, but the other two do. The name parameter is the fully qualified domain name of your host, and the profile parameter maps that host into the groups.xml file.

Our simple groups.xml file looks like:

<Groups version='3.0'>
   <Group profile='true' public='false' name='basic'>
      <Group name='suse'/>
   </Group>
   <Group name='ubuntu' toolset='debian'/>
   <Group name='debian' toolset='debian'/>
   <Group name='redhat' toolset='rh'/>
   <Group name='suse' toolset='rh'/>
   <Group name='mandrake' toolset='rh'/>
   <Group name='solaris' toolset='solaris'/>
</Groups>

There are two types of groups in Bcfg: profile groups (profile='true') and non-profile groups (profile='false'). Profile groups can act as top-level groups to which clients can bind, while non-profile groups only exist as members of other groups. In our simple starter case, we have a profile group named basic, and that is the group that our first client bound to. Our first client is a SuSE machine, so it contains the suse group. Of course, bcfg2-admin isn't smart enough to fill out the rest of your config, so the suse group further down is empty.

Let's say the first thing we want to set up on our machine is the message of the day. To do this, we simply need to create a Bundle and add that Bundle to an appropriate group. In this simple example, we start out by adding

<Bundle name='motd'/>

to the basic group.

Next, we create a motd.xml file in the Bundler directory:

<Bundle name='motd' version='2.0'>
  <ConfigFile name='/etc/motd' />
</Bundle>

Now when we run the client, we get slightly different output:

Loaded tool drivers:
 Chkconfig    POSIX        PostInstall  RPM         
Incomplete information for entry ConfigFile:/etc/motd; cannot verify

Phase: initial
Correct entries:        0
Incorrect entries:      1
Total managed entries:  1
Unmanaged entries:      242

In dryrun mode: suppressing entry installation for:
 ConfigFile:/etc/motd 

Phase: final
Correct entries:        0
Incorrect entries:      1
Total managed entries:  1
Unmanaged entries:      242

We now have an extra unmanaged entry, bringing our total number of managed entries up to one. To manage it we need to copy /etc/motd to /var/lib/bcfg2/Cfg/etc/motd/. Note the layout of that path: all plain-text config files live in the Cfg directory. The directory structure under that directory directly mimics your real filesystem layout, making it easy to find and add new files. The last directory is the name of the file itself, so in this case the fill path to the motd file would be /var/lib/bcfg2/Cfg/etc/motd/motd. Copy your real /etc/motd file to that location, run the client again, and you will find that we now have a correct entry:

Loaded tool drivers:
 Chkconfig    POSIX        PostInstall  RPM         

Phase: initial
Correct entries:        1
Incorrect entries:      0
Total managed entries:  1
Unmanaged entries:      242


Phase: final
Correct entries:        1
Incorrect entries:      0
Total managed entries:  1
Unmanaged entries:      242

Done! Now we just have 242 (or more) entries to take care of!

The Bundler is a relatively easy directory to populate. You can find many samples of Bundles in the BundleRepository?, many of which can be used without editing.

Next Steps

Several other utilities can help from this point on:

bcfg2-info is a utility that instantiates a copy of the bcfg2 server core (minus the networking code) for examination. From this, you can directly query:

• Client Metadata
• Which entries are provided by particular plugins
• Client Configurations

Run bcfg2-info, and type help and the prompt when it comes up.

bcfg2-admin can perform a variety of repository maintenance tasks. Run bcfg2-admin help for details.

Standard Group Names

Group Names

This pages describes a set of common group names. The goal is to build a small set of common group names that can be used across installations. With any luck, this will increase portability of repository contents. Use of these names is, and will continue to be entirely optional. None of these groups are profile groups. This will result in a highly functional example repository, hosted in svn

Group Name	Category	Semantic Content
linux	os	Clients running the Linux kernel
solaris	os	Clients running the Solaris kernel

The following image describes the base compatibility OS groups for bcfg2. This file is available from /trunk/repository/Metadata/bcfg2-os.xml.

Example Bundles

Bcfg2 Specifications

Writing Bcfg2 Specification

Bcfg2 specifications are logically divided in to three areas:

• Metadata
• Abstract
• Literal

The metadata portion of the configuration assigns a client to its profile group and to its non profle groups. The profile group is assigned in Metadata/clients.xml and the non profile group assignments are in Metadata/groups.xml.

The group memberships contained in the metadata are then used to constuct an abstract configuration for the client. An abstract configuration for a client identifies the configuration entities (packages, configuration files, service, etc) that a client requires, but it does not identify them explicitly. For instance an abstract configuration may identify that a client needs the bcfg2 package with

<Package name=bcfg2/>

but this does not explicitly identify that an RPM package version 0.9.2 should be loaded from http://rpm.repo.server/bcfg2-0.9.2-0.1.rpm. The abstract configuration is defined in the xml configuration files for the Base and Bundles plugins.

A combination of a clients metadata (group memberships) and abstract configuration is then used to generate the clients literal configuration. For instance the above abstract configuration entry may generate a literal configuration of

<Package name='bcfg2' version='0.9.2-0.1' type='yum'/>

A clients literal configuration is generated by a number of plugins that handle the different configuration entities.

Metadata

The metadata mechanism has two types of information, client metadata and group metadata. The client metadata describes which top level group a client is associated with.The group metadata describes groups in terms of what bundles and other groups they include. Each aspect grouping and clients' memberships are reflected in the Metadata/groups.xml and Metadata/clients.xml files, respectively.

Usage of Groups in Metadata

Clients are assigned membership of groups in the Metadata descriptions. Clients can be directly assigned to 'profile' or 'public' groups. Client membership of all other groups is by those groups being associated with the profile or public groups. This file can be indirectly modified from clients through use of the -p flag to bcfg2.

Clients are associated with profile groups in Metadata/clients.xml as shown below.

Metadata/clients.xml

The Metadata/clients.xml file contains the mappings of Profile Groups to clients. The file is just a series of <Client /> tags, each of which describe one host. A sample file is below:

<Clients version="3.0">
  <Client profile="backup-server" pingable="Y" pingtime="0" name="backup.example.com"/>
  <Client profile="console-server" pingable="Y" pingtime="0" name="con.example.com"/>              
  <Client profile="kerberos-master" pingable="Y" pingtime="0" name="kdc.example.com"/>
  <Client profile="mail-server" pingable="Y" pingtime="0" name="mail.example.com"/>
  <Client name='foo' address='10.0.0.1' pingable='N' pingtime='-1'>
      <Alias name='foo-mgmt' address='10.1.0.1'/>
  </Client>
</Clients>

Clients Tag

The Clients tag has the following possible attributes:

Name	Description	Values
version	Client schema version	String

Client Tag

Each entry in clients.xml must have the following properties:

Name	Description	Values
name	Host name of client. This needs do be the name (possibly a FQDN) returned by a reverse lookup on the connecting IP address.	String
profile	Profile group name to associate this client with	String

Additionally, the following properties can be specified:

Name	Description	Values
Alias	Alternative name and address for the client	XML Element
address	Establishes an extra ip address that resolves to this client	ip address
location	Requires requests to come from an IP address that matches the client record	fixed|floating
password	Establishes a per-node password that can be used instead of the global password	String
pingable	If the client is pingable (deprecated; for old reporting system)	(Y|N)
pingtime	Last time the client was pingable (deprecated; for old reporting system)	String
secure	Requires the use of the per-client password for this client	true|false
uuid	Establishes a per-node name that can be used to bypass dns-based client resolution	String

For detailed information on client authentication see Authentication

Metadata/groups.xml

The Metadata/groups.xml file contains Group and Profile definitions. Here's a simple Metadata/groups.xml file:

<Groups version='3.0'>
  <Group name='mail-server' profile='true'
                            public='false'
                            comment='Top level mail server group' >
    <Bundle name='mail-server'/>
    <Bundle name='mailman-server'/>
    <Group name='apache-server'/>
    <Group name='rhel-as-4-x86'/>
    <Group name='nfs-client'/>
    <Group name='server'/>
  </Group>
  <Group name='rhel-as-4-x86' toolset='rh'>
     <Group name='rhel'/>
  </Group>
  <Group name='apache-server'/>
  <Group name='nfs-client'/>
  <Group name='server'/>
  <Group name='rhel'/>
</Groups>

Nested/chained groups definitions are conjunctive (logical and). For instance, in the above example, a client associated with the Profile Group 'mail-server' is also a member of the apache-server, rhel-as-4-x86, nfs-client, server and rhel groups.

Groups describe clients in terms for abstract, disjoint aspects. Groups can be combined to form complex descriptions of clients that use configuration commonality and inheritance. Groups have several attributes, described below:

Metadata Groups Tag

The Groups tag has the following possible attributes:

Name	Description	Values
version	Group schema version	String
origin	URL of master version (for common repo)	String
revision	Master version control revision	String

Metadata Group Tag

The Group Tag has the following possible attributes:

Name	Description	Values
name	Name of the group	String
profile	If a client can be directly associated with this group	(True|False*)
public	If a client can freely associate itself with this group. For use with bcfg2 -p option on the client.	(True|False*)
toolset	Describes which client-side logic should be used to make configuration changes	(rh|debian|solaris|aix|auto|gentoo)
category	A group can only contain one instance of a group in any category. This provides the basis for representing groups which are conjugates of one another in a rigorous way. It also provides the basis for negation.	String
default	Set as the profile to use for clients that are not associated with a profile in clients.xml.	(True|False*)
comment	English text description of group	String

Groups can also contain other groups and bundles.

Use of XInclude

XInclude is a W3C specification for the inclusion of external XML documents into XML source files. Much like the use of #include in C, this allows complex definitions to be split into smaller, more manageable pieces. As of bcfg2-0.9.0pre1, the Metadata plugin supports the use of XInclude specifications to split the clients.xml and groups.xml files. This mechanism allows the following specification to produce useful results:

 <Groups version='3.0' xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="my-groups.xml" />
  <xi:include href="their-groups.xml" />
 </Groups>

Each of the included groups files has the same format. These files are properly validated by bcfg2-repo-validate. This mechanism is useful for composing group definitions from multiple sources, or setting different permissions in an svn repository.

Probes

The metadata plugin includes client-side probing functionality. This is fully documented here?.

Dynamic Groups

Dynamic groups are likewise complex, and are covered on their own page

Abstract Configuration (Structures)

A clients Abstract Configuration is the inventory of configuration entities that should be installed on a client. Two plugins provide the basis for the abstract configuration, the Bundler and Base.

The plugin Bundler builds descriptions of interrelated configuration entities. These are typically used for the representation of services, or other complex groups of entities.

The Base provides a laundry list of configuration entities that need to be installed on hosts. These entities are independent from one another, and can be installed individually without worrying about the impact on other entities.

Usage of Groups in Base and Bundles

Groups are used by the Base and Bundles plugins for selecting Configuration Entity Types for inclusion in a clients abstract configuration. They can be thought of as:

if client is a member of group1 then
    assign to abstract config

Nested groups are conjunctive (logical and).

if client is a member of group1 and group2 then
    assign to abstract config

Group membership maybe negated. See "Writing Bundles" for an example.

Configuration Entity Types

Entities in the abstract configuration (and correspondingly in the literal configuration) can have one of several types. In the abstract configuration, each of these entities only has a tag and the name attribute set.

The types of Configuration Entities that maybe assigned to the abstract configuration are as follows:

Tag	Description
Package	Software Package
ConfigFile	Configuration File
Service	Persistent system services and daemons
Directory	File system Directories
SymLink	Symbolic links
Permissions	The permissions (not contents) of a POSIX path
Action	Actions to be performed as part of a bundle installation

An example of each entity type is below.

<Package name='bcfg2'/>
<ConfigFile name='/etc/bcfg2.conf'/>
<Service name='ntpd'/>
<Directory name='/var/tmp'/>
<SymLink name='/dev/MAKEDEV'/>
<Permissions name='/etc/passwd'/>
<Action name='action_name'/>

Configuration Entity Name Attribute

With the exception of the Action Entity type, the value of the configuration entity name attribute is not arbitary. It must map to entities of the same type in the Literal Configuration, which in turn map to physical entities on the client.

For the most part the physical entity being mapped to is intuitive and are listed below.

Tag	Description
Package	name of the software package
ConfigFile	Full absolute path of the configuration file
Service	Name of the init script for the service which may not be that same as the name of the daemon started or the package name
Directory	Full absolute path of the directories
SymLink	Full absolute path of the symbolic linkss
Permissions	Full absolute path of the file or directory
Action	Arbitary name

Writing Bundles

Bundles consist of a set of configuration entities. These entities are grouped together due to a configuration-time interdependency. Basic services tend to be the simplest example of these. They normally consist of

• some software package(s)
• some configuration files
• an indication that some service should be activated

If any of these pieces are installed or updated, all should be rechecked and any associated services should be restarted.

All files in the Bundles/ subdirectory of the repository are processed. Each bundle must be defined in its own file and the filename must be the same as the bundle name with a .xml suffix.

# ls Bundler
Glide3.xml
LPRng.xml
Tivoli-backup.xml
Tivoli.xml
a2ps.xml
abiword.xml
account.xml
adsm-client.xml
amihappy.xml
apache-basic.xml
apache.xml
apache2-basic.xml
apt-proxy.xml
at.xml
atftp-server.xml
atftp.xml
....

Groups can be used inside of bundles to differentiate which entries particular clients will receive. This is useful for the case where entries are named differently across systems; for example, one linux distro may have a package called openssh while another uses the name ssh. Configuration entries nested inside of Group elements only apply to clients who are a member of those groups; multiply nested groups must all apply.

Also, groups may be negated; entries included in such groups will only apply to clients who are not a member of said group.

When packages in a bundle are verified by the client toolset, the ConfigFiles included in the same bundle are taken into consideration. That is, a package will not fail verification from a bcfg2 perspective if the package verification only failed because of configuration files that are defined in the same bundle.

The following is an annotated copy of a bundle:

<Bundle revision='$Revision: 2668 $' name='ssh' version='2.0'
        origin='https://svn.mcs.anl.gov/repos/bcfg/trunk/repository/Bundler/ssh.xml'>
  <ConfigFile name='/etc/ssh/ssh_host_dsa_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_rsa_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_dsa_key.pub'/>
  <ConfigFile name='/etc/ssh/ssh_host_rsa_key.pub'/>
  <ConfigFile name='/etc/ssh/ssh_host_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_key.pub'/>
  <ConfigFile name='/etc/ssh/sshd_config'/>
  <ConfigFile name='/etc/ssh/ssh_config'/>
  <ConfigFile name='/etc/ssh/ssh_known_hosts'/>
  <Group name='rpm'>
    <Package name='openssh'/>
    <Package name='openssh-askpass'/>
    <Service name='sshd'/>
    <Group name='fedora' >
       <Group name='fc4' negate='true'>
         <Package name='openssh-clients'/>
       </Group>
       <Package name='openssh-server'/>
    </Group>
  </Group>
  <Group name='deb'>
    <Package name='ssh'/>
    <Service name='ssh'/>
  </Group>
</Bundle>

In this bundle, most of the entries are common to all systems. Clients in group "deb" get one extra package and service, while clients in group "rpm" get two extra packages and an extra service. In addition, clients in group "fedora" and group "rpm" get one extra package entries, unless they are not in the fc4 group, in which case, they get an extra package. Notice that this file doesn't describe which versions of these entries that clients should get, only that they should get them. (Admittedly, this example is slightly contrived, but demonstrates how group entries can be used in bundles)

Group	Entry
all	/etc/ssh/ssh_host_dsa_key
all	/etc/ssh/ssh_host_rsa_key
all	/etc/ssh/ssh_host_dsa_key.pub
all	/etc/ssh/ssh_host_rsa_key.pub
all	/etc/ssh/ssh_host_key
all	/etc/ssh/ssh_host_key.pub
all	/etc/ssh/sshd_config
all	/etc/ssh/ssh_config
all	/etc/ssh/ssh_known_hosts
rpm	Package openssh
rpm	Package openssh-askpass
rpm	Service sshd
rpm and fedora	Package openssh-server
rpm and fedora and not fc4	Package openssh-clients
deb	Package ssh
deb	Service ssh

Bundle Tag

The Bundle Tag has the following possible attributes:

Name	Description	Values
name	The name of the bundle	String
version	Bundle schema version	String
origin	URL of master version (for common repo)	String
revision	Master version control revision	String

As mentioned above the Configuration Entity Tags may only have the name attribute in Bundle definitions.

Abstract Group Tag

In the Abstract Configuration plugins (Base and Bundle) the Group Tag may have the following attributes:

Name	Description	Values
name	Name of group.	String
negate	Negate the group association (is not a member of)	(True|False*)

An abstract group may contain any of the Configuration Entity types and other groups.

Using Base

The Base plugin provides a mechanism to add independent configuration entities to a client's abstract configuration. All files in the Base/ subdirectory of the repository are processed, and all entries that fall within the scope of the client metadata are included in its abstract configuration.

$ ls Base/
centos-4-x86.xml
fedora-core-4-x86.xml
rhel-as-4-x86.xml
rhel-es-4-x86.xml
rhel-ws-4-x86_64.xml
rhel-ws-4-x86.xml

<Base>
    <Group name='Centos4.4-Standard'>
        <Package name='audit'/>
        <Package name='audit-libs'/>
        <Package name='basesystem'/>
        <Package name='bash'/>
        <Package name='bcfg2'/>
        <Package name='beecrypt'/>
        ....
        <Package name='yum'/>
        <Package name='zlib'/>
        <Group name='x86_64'>
            <Package name='systemimager-x86_64initrd_template'/>
        </Group>
    </Group>
</Base>

The format of the Base files are similar to those used by the Bundler. The majority of the elements are usually Packages, but ConfigFiles, Directories, Services, Permissions, and SymLinks may all be defined. A partial example is below:

Base Tag

The Base Tag has no attributes

As mentioned above the Configuration Entity Tags contained in a Base definition may only have the name attribute in Base definitions.

Abstract Group Tag

In the Abstract Configuration plugins (Base and Bundle) the Group Tag may have the following attributes:

Name	Description	Values
name	Name of group.	String
negate	Negate the group association (is not a member of)	(True|False*)

An abstract group may contain any of the Configuration Entity types and other groups.

Literal Configuration (Generators)

A Generator is a bcfg2 piece of code that is run to generate the literal configuration for a host using a combination of the hosts metadata and abstract configuration.

A Generator can take care of a particular configuration element. Any time this element is requested by the client, the server dynamically generates it either by crunching data and creating new information or by reading a file off of disk and passes it down to the client for installation.

Usage of Groups in Generators

Similar to Abstract Configuration plugins, groups are used by generator plugins for selecting Configuration Entities for inclusion in a clients literal configuration. They can be thought of as:

if client is a member of group1 then
    assign to abstract config

Nested groups are conjunctive (logical and).

if client is a member of group1 and group2 then
    assign to abstract config

How the groups are configured is specific to the plugin, but here are two common methods:

• xml configuration file (Pkgmgr, Rules)
• file name encoding (Cfg, TCheetah, SSHBase)

Details are included on each plugin's page.

Generators

Each of the generators is covered on their own page.

Plugin	Description
Actions	Action entries are commands that are executed either before bundle installation, after bundle installation or both.
Cfg	The Cfg plugin provides a repository to describe configuration file contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines.
Deps	The deps plugin allows inter-entry dependencies (like package prerequisites) to be mapped independent of client roles.
Hostbase	Hostbase is an IP management system built on top of Bcfg2.
Pkgmgr	The Pkgmgr plugin define what packages are available for a host or image and where to find those packages.
Rules	The Rules plugin is responsible for defining the details about the Directory, Permissions, SymLink and Service elements.
SSHbase	SSHbase is a purpose build bcfg2 plugin for managing ssh host keys. It is responsible for making ssh keys persist beyond a client rebuild and building a proper ssh_known_hosts file, including a correct localhost record for the current system.
TCheetah	The TCheetah plugin allows you to use the cheetah templating system to create files, instead of the various diff-based methods offered by the Cfg plugin. It also allows you to include the results of probes executed on the client in the created files.
TGenshi	The TGenshi plugin uses the genshi toolkit for generating files and functions much like the TCheetah plugin

Probes

TCheetah Plugin

DynamicGroups

Dynamic Groups

As of bcfg2-0.9.0pre6, Bcfg2 supports the use of dynamic groups. These groups are not included in a client's profile group, but instead are derived from the results of probes executed on the client. These dynamic groups need not already exist in Metadata/groups.xml. If a dynamic group is defined in Metadata/groups.xml, clients that include this group will also get all included groups and bundles.

Setting up dynamic groups

In order to define a dynamic group, setup a probe that outputs the text based on system properties:

group:groupname

This output is processed by the bcfg2 server, and results in dynamic group membership in groupname for the client. See the Probes page for a more thorough description of probes.

Client Tool Drivers

Client Tool Drivers

Client tool drivers allow bcfg2 to execute configuration operations by interfacing with platform and distribution specific tools.

Tool drivers handle any reconfiguration or verification operation. So far we have tools that primarily deal with packaging systems and service management. The POSIX tool also handles file system and permissions/groups operations. To write your own tool driver, to handle a new packaging format, or new service architecture see WritingClientToolDrivers.

When the bcfg2 client is run, it attempts to instantiate each of these drivers. The succeeding list of drivers are printed as a debug message after this process has completed. Drivers can supercede one another, for example, the Yum driver conflicts (and unloads) the RPM driver. This behavior can be overridden by running the bcfg2 client with the -D flag. This flag takes a colon delimited list of drivers to use on the system.

Currently these are the tool drivers that are distributed with bcfg2:

Action (trunk/bcfg2/src/lib/Client/Tools/Action.py)

Pre and post-install tests and actions. This driver executes commands and supplies status information to the bcfg2 server via the statistics mechanism. It can also be used to prevent bundle installation when pre-conditions are not met. See the UsingActions? page for more details.

APT (trunk/bcfg2/src/lib/Client/Tools/APT.py)

Debian Packages. This tool driver is used to handle packages on dpkg based systems and employs the "apt" executable.

Blast (trunk/bcfg2/src/lib/Client/Tools/Blast.py)

Blastwave Packages. This tool driver is for blastwave packages on solaris

Chkconfig (trunk/bcfg2/src/lib/Client/Tools/Chkconfig.py)

To start and stop services on primarily Redhat based distros. NOTE: Start and stop are standard arguments, but the one for reload isn't consistent across services. You can specify which argument to use with the reload property in Service tags. Example: <Service name="ftp" reload="condrestart" status="on" type="chkconfig">

DebInit (trunk/bcfg2/src/lib/Client/Tools/DebInit.py)

Debian Service Support; exec's update-rc.d to configure services.

Encap (trunk/bcfg2/src/lib/Client/Tools/Encap.py)

Encap Packages.

FreeBSDInit (trunk/bcfg2/src/lib/Client/Tools/FreeBSDInit.py)

FreeBSD Service Support. Only bundle updates will work.

FreeBSDPackage (trunk/bcfg2/src/lib/Client/Tools/FreeBSDPackage.py)

FreeBSD Packages. Verifies packages and their version numbers but can't install packages.

launchd (trunk/bcfg2/src/lib/Client/Tools/launchd.py)

Mac OS X Services. To use this tool, you must maintain a standard launch daemon .plist file in /Library/LaunchDaemons/ (example ssh.plist) and setup a <Service name="com.openssh.sshd" type="launchd" status="on" /> entry in your config to load or unload the service. Note the name is the Label specified inside of the .plist file

Portage (trunk/bcfg2/src/lib/Client/Tools/Portage.py)

Gentoo Packages.

POSIX (trunk/bcfg2/src/lib/Client/Tools/POSIX.py)

Files and Permissions are handled by the POSIX driver. Usage well documented other places.

RcUpdate (trunk/bcfg2/src/lib/Client/Tools/RcUpdate.py)

Uses the rc-update executable to manage services on distributions such as Gentoo.

RPM (trunk/bcfg2/src/lib/Client/Tools/RPM.py)

Executes rpm to manage packages most often on redhat based systems. DEPRECATED: in favor of RPMng

RPMng (trunk/bcfg2/src/lib/Client/Tools/RPMng.py)

Next-generation RPM tool, will be default in upcoming release. Handles RPM sublties like epoch and prelinking and 64-bit platforms better than RPM client tool. RPMng/YUMng Documentation.

SMF (trunk/bcfg2/src/lib/Client/Tools/SMF.py)

Solaris Service Support.

SYSV (trunk/bcfg2/src/lib/Client/Tools/SYSV.py)

Handles System V Packaging format that is available on Solaris.

Yum (trunk/bcfg2/src/lib/Client/Tools/Yum.py)

Handles RPMs using the YUM package manager. DEPRECATED: in favor of YUMng

YUMng (trunk/bcfg2/src/lib/Client/Tools/YUMng.py)

Handles RPMs using the YUM package manager. Handles sublties better than the Yum client tool. RPMng/YUMng Documentation.

Using Actions

Deploying Bcfg2

Simple Deployments

A Near-Literal Deployment

An Abstract Deployment

Bcfg2 Server Administration

An example application of bcfg2

Developing for Bcfg2

Developing For Bcfg2

While the Bcfg2 server provides a good interface for representing general system configurations, its plugin interface offers the ability to implement configuration interfaces and representation tailored to problems encountered by a particular site. This chapter describes what plugins are good for, what they can do, and how to implement them.

Bcfg2 Plugins

Bcfg2 plugins are loadable python modules that the Bcfg2 server loads at initialization time. These plugins can contribute to the functions already offered by the Bcfg2 server or can extend its functionality. In general, plugins will provide some portion of the configuration for clients, with a data representation that is tuned for a set of common tasks. Much of the core functionality of Bcfg2 is implemented by several plugins, however, they are not special in any way; new plugins could easily supplant one or all of them.

The following table describes the various functions of bcfg2 plugins.

Name	Description
Probes	Plugins can issue commands to collect client-side state (like hardware inventory) to include in client configurations
ConfigurationEntry List	Plugins can construct a list of per-client configuration entry lists to include in client configurations.
ConfigurationEntry contents	Literal values for configuration entries.
XML-RPC functions	Plugins can export function calls that expose internal functions.

Writing Bcfg2 Plugins

Bcfg2 plugins are python classes that subclass from Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must be set in the new plugin. These values dictate how the new plugin will behave with respect to the above four functions. The following table describes all important member fields.

Name	Description	Format
name	The name of the plugin	string
version	The plugin version (generally tied to revctl keyword expansion).	string
author	The plugin author.	string
rmi	Set of functions to be exposed as XML-RPC functions	List of function names (strings)
Entries	Multidimentional dictionary of keys that point to the function used to bind literal contents for a given configuration entity.	Dictionary of ConfigurationEntityType, Name keys and function reference values
BuildStructures	Function that returns a list of the structures for a given client	Member function
GetProbes	Function that returns a list of probes that a given client should execute	Member function
ReceiveData	Function that accepts the probe results for a given client.	Member function

Example Plugin

import Bcfg2.Server.Plugin
class MyPlugin(Bcfg2.Server.Plugin.Plugin):
   '''An example plugin'''
   # All plugins need to subclass Bcfg2.Server.Plugin.Plugin
   __name__ = 'MyPlugin'
   __version__ = '1'
   __author__ = 'me@me.com'
   __rmi__ = ['myfunction']
   # myfunction is not available remotely as MyPlugin.myfunction
   
   def __init__(self, core, datastore):
       Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore)
       self.Entries = {'ConfigFile':{'/etc/foo.conf': self.buildFoo}}

   def myfunction(self):
       '''function for xmlrpc rmi call'''
       #do something
       return True

   def buildFoo(self, entry, metadata):
       '''Bind per-client information into entry based on metadata'''
       entry.attrib.update({'owner':'root', 'group':'root', 'perms':'644'})
       entry.text = '''contents of foo.conf'''

Example Connector

import Bcfg2.Server.Plugin

class Foo(Bcfg2.Server.Plugin.Plugin,
         Bcfg2.Server.Plugin.Connector):
    '''The Foo plugin is here to illustrate a barebones connector'''
    name = 'Foo'
    version = '$Revision: $'
    experimental = True

    def __init__(self, core, datastore):
        Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore)
	Bcfg2.Server.Plugin.Connector.__init__(self)
        self.store = XMLFileBacked(self.data, core.fam)

    def get_additional_data(self, metadata):

        mydata = {}
        for data in self.store.entries['foo.xml'].data.get("foo", []):
           
            mydata[data] = "bar"

        return  dict([('mydata', mydata)])

    def get_additional_groups(self, meta):
        return self.cgroups.get(meta.hostname, list())

BCFG2 Reports

Dynamic Reports

Dynamic Reports Installation

Overview

Installation of the new reporting system requires installation of a python module and configuration of the Apache webserver with a virtual host. Additionally, until fully integrated, periodically an "import script" must be executed via Cron or similar mechanism.

Versions 0.9.5pre1 and greater no longer need to be installed at the root url for a given host, and therefore no longer require their own virtual host.

Prerequisites

• sqlite3
• pysqlite2
• Django 0.96.x
• mod-python

Install

setup bcfg2.conf or bcfg2-web.conf

Be sure to include the specified fields included the example bcfg2.conf file. These can be specified in either /etc/bcfg2.conf, if it is readable by the webserver user, or /etc/bcfg2-web.conf. This changed in 0.9.5.

Install skeleton database

Inside the bcfg2-tarball/examples/ directory from the tarball you will find brpt.sqlite. Copy this file to <path-to-bcfg2-repository>/etc/brpt.sqlite

If you are not using sqlite (the default choice), please see the "Notes on Alternative Databases" section below.

At this point we can import statistics date in to the database from your clients.xml and statistics.xml files.

execute the following: python .../site-packages/Bcfg2/Server/Reports/importscript.py

(Versions 0.8.7 and prior must specify: python .../site-packages/Bcfg2/Server/Reports/importscript.py-c <repo>/Metadata/clients.xml -s <repo>/etc/statistics.xml )

It might be a good idea to write a 1-line shell script to execute this very long command; it is only an interim measure until the /etc/statistics.xml file is obsoleted.

At this point you shouldn't get any errors or tracebacks. Common problems include incorrect installation of the pysqlite2 module.

Next we configure Apache:

An example site config is included below for the vhost "reports.mcs.anl.gov":

<VirtualHost reports.mcs.anl.gov>
        ServerAdmin webmaster@mcs.anl.gov
        ServerName reports.mcs.anl.gov
        DocumentRoot /var/www/reports
        <Directory /var/www/reports>
                Order deny,allow
                Deny from all
                Allow from localhost #you may want to change this
                AllowOverride None
        </Directory>

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        ServerSignature Off

        # Stop TRACE/TRACK vulnerability
        <IfModule mod_rewrite.c>
                RewriteEngine on
                RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
                RewriteRule .* - [F]
        </IfModule>
<Location "/">
        SetHandler python-program
        PythonHandler django.core.handlers.modpython
        SetEnv DJANGO_SETTINGS_MODULE Bcfg2.Server.Reports.settings
        PythonDebug On
</Location>
<Location "/site_media/">
        SetHandler None
</Location>
</VirtualHost>

The first three lines of this configuration must be customized per site.

The bcfg2-tarball/reports/site_media/ directory needs to be copied to /var/www/reports/site_media/ It could live anywhere; as long as the contents are accessible on the virtual host at /site_media/.

At this point you should be able to point your web browser to the virtualhost you created and see the new reports.

One final step is to configure your system to import statistics data regularly. execute the following: python .../site-packages/Bcfg2/Server/Reports/importscript.py

(Versions 0.8.7 and prior must specify: python .../site-packages/Bcfg2/Server/Reports/importscript.py-c <repo>/Metadata/clients.xml -s <repo>/etc/statistics.xml )

on a regular basis, perhaps as often as every 5 or 10 minutes or as infrequent as daily, so your reports are always up to date.

Notes on Alternative Databases

If you choose to use a different database, you'll need to edit settings.py from inside of the brpt module. Instructions on this process can be found on the Django website. The provided sqlite database file is pre-populated with the appropriate tables, if you choose a different database engine, you will need to run "python manage.py syncdb" from the brpt module directory to configure it before importing any data. As of vers 0.9.5, these settings are moved to the more appropriate /etc/bcfg2.conf

If you encounter any problems or have suggestions for these instructions feedback is welcome and encouraged.

Dynamic Reports Usage

New Reporting System

Now available in Bcfg2 0.8.2

Installation

Installation Guide

Summary and Features

The new reporting system was implemented to address a number of deficiencies in the previous system. By storing statistics data in a relational database, we are now able to view and analyze more information about the state of the configuration, including information about previous configuration. Specific features in the new system include:

• The ability to look at past statistics information.

• More recent data concerning hosts. The import script may be run quite often, updating the database that contains all statistics information. In the future we anticipate development of a database based statistics module for the server that will allow statistics updates to happen immediately as configuration changes happen.

• Additional information display in reports. Primarily, reasons for configuration item verification failure are now accessible. This additional data is provided only by the most recent client.

• Instead of static pages, pages are generated on the fly, allowing users to drill down to find out about a specific host, rather than have one huge page with too much information.

Planned improvements include

- Accounts, customized displays for each admin. And privacy of config data.

- Config browsing capabilities; to look at your config in an interesting way. Fixing all the known bugs from below.

Unfortunately with all the improvements, there are a few less exciting elements about the new reporting system. The new reporting system moves away from static pages and towards a real web application, which causes mainly problems with dependancies and makes installation and more difficult. This should become less of a problem over time as Django is packaged and we develop a better installation process for a web app. This should become clear when reading the Installation section that follows.

Usage

You can use these new reports in tandem with the old system. Currently the new reporting system simply periodically runs an importer script via cron. This imports the XML statistics and clients files to the relational database, building historical information. In the future, a new statistics module in the server will allow direct writing to the database whenever a configuration interaction occurs, which will make the reports always up to date.

New reports script in 0.9.6pre2 and later

Command Line Use

bcfg2-reports

bcfg2-reports allows you to retrieve data from the database about clients, and the states of their current interactions. It also allows you to change the expired/unexpired states.

The utility runs as a standalone application. It does, however, use the models from /src/lib/Server/Reports/reports/models.py.

A number of different options can be used to change what bcfg2-reports displays:

• -a : shows all hosts, including expired hosts
• -b NAME : single-host mode - shows bad entries from the current interaction of NAME
• -c : shows only clean hosts
• -d : shows only dirty hosts
• -e NAME : single-host mode - shows extra entries from the current interaction of NAME
• -h : shows help and usage info about bcfg2-reports
• -s NAME : single-host mode - shows bad and extra entries from the current interaction of NAME
• -x NAME : toggles expired/unexpired state of NAME
• --badentry=KIND,NAME : shows only hosts whose current interaction has bad entries in of KIND kind and NAME name; if a single argument ARG1 is given, then KIND,NAME pairs will be read from a file of name ARG1
• --extraentry=KIND,NAME : shows only hosts whose current interaction has extra entries in of KIND kind and NAME name; if a single argument ARG1 is given, then KIND,NAME pairs will be read from a file of name ARG1
• --fields=ARG1,ARG2,... : only displays the fields ARG1,ARG2,... (name,time,state)
• --sort=ARG1,ARG2,... : sorts output on ARG1,ARG2,... (name,time,state)

Screenshots

Static Reports

The Bcfg2 Reporting System

Bcfg2's reporting system is its killer feature. It allows administrators to gain a broad understanding of the configuration state of their entire environment. It summarizes

• Configuration changes and when they were made
• Discrepancies between the specification and current client states
  • Clients can be grouped by misconfiguration type
• Configuration entries that are not specified
• Overall client summaries according to these types

There are two systems, the old system, which builds static reports based on a series of XSLT stylesheets and a new dynamic reporting system that uses django and a database backend.

• The Dynamic Reporting System (Installation Guide)
• The Static Reporting System

Demo Repository

Troubleshooting

Troubleshooting

From time to time, Bcfg2 produces results that the user finds surprising. This can happen either due to bugs or user error. This page describes several techniques to gain visibility into the bcfg2 client and server and understand what is going on.

Figure out if error is client or server side

• Cache a copy of the client using bcfg2 -c /tmp/config.xml
• Search for the entry of interest
• If it looks correct, then there is a client issue

This file contains all aspects of client configuration. It is structured as a series of bundles and base entries.
NOTE: Most often the entry is not correct and the issue lies in the specification.

Review server log messages

The bcfg2-server process logs to syslog facility LOG_DAEMON. The server produces a series of messages upon a variety of events and errors.

Check if all repository XML files conform to schemas

Bcfg2 comes with XML schemas describing all of the XML formats used in the server repository. A validation command bcfg2-repo-validate is included with the source distribution and all packages. Run it with the -v flag to see each file and the results if its validation.

If the bcfg2 server is not reflecting recent changes, try restarting the bcfg2-server process

If this fixes the problem, it is either a bug in the underlying file monitoring system (fam or gamin) or a bug in Bcfg2's file monitoring code. In either case, file a ticket in the tracking system. In the ticket, include:

• Filesystem monitoring system (fam or gamin)
• kernel version (if on linux)
• if any messages of the form "Processed N <fam/gamin> events in M seconds" appeared between the modification event and the client configuration generation request appeared in the server log
• Which plugin handled the file in the repostiory (Cfg, Rules, Pkgmgr, TCheetah, TGenshi, Metadata)
• If a touch of the file after the modification causes the problem to go away

bcfg2-info

Bcfg2 server operations can be simulated using the bcfg2-info command. The command is interactive, and has commands to allow several useful operations

• clients - Current client metadata (profile and group) settings
• groups - Current group metadata values
• mappings - Configuration entries provided by plugins
• buildfile <filename> <hostname> - Build a config file for a client
• showentries <client> <type> - Build the abstract configuration (list of entries) for a client
• build <hostname> <output-file> - Build the complete configuration for a client

Type help in bcfg2-info for more information.

FAQs

Why won't bcfg2-server start?

If your server doesn't seem to be starting and you see no error messages in your server logs, try running it in the foreground to see why.

Why am I getting a traceback?

If you get a traceback, please let us know. You can file a ticket, send the traceback to the Mailing List, or hop on IRC and let us know.

What is the most common cause of "The following entries are not handled by any tool"?

Often it corresponds to entries that aren't bound by the server (for which you'll get error messages on the server). You should try inspecting the logs on the server to see what may be the cause.

Error Messages

This page describes error messages produced by Bcfg2 and steps that can be taken to remedy them.

Error	Location	Meaning	Repair
Incomplete information for entry <EntryTag>:<EntryName>; cannot verify	Client	The described entry is not fully specified by the server, so no verification can be performed.	1
Incomplete information for entry <EntryTag>:<EntryName>; cannot install	Client	The described entry is not fully specified by the server, so no verification can be performed.	2
The following entries are not handled by any tool: <EntryTag>:<EntryName>	Client	The client cannot figure out how to handle this entry.	3
no server x509 fingerprint; no server verification performed!	Client	The client is unable to verify the server	4
Failed to bind entry: <EntryTag> <EntryName>	Server	The server was unable to find a suitable version of entry for client.	5
Failed to bind to socket	Server	The server was unable to bind to the tcp server socket.	6
Failed to load ssl key <path>	Server	The server was unable to read and process the ssl key.	7
Failed to read file <path>	Server	The server failed to read the specified file	8
Failed to parse file <path>	Server	The server failed to parse the specified XML file	9
Client metadata resolution error for <IP>	Server	The server cannot resolve the client hostname or the client is associated with a non-profile group.	10

---

1. 1. This entry is not being bound. Ensure that a version of this entry applies to this client.
2. 2. This entry is not being bound. Ensure that a version of this entry applies to this client.
3. 3. Add a type to the generator definition for this entry
4. 4. Run bcfg2-admin fingerprint on the server and add it to the client bcfg2.conf as mentioned here
5. 5. This entry is not being bound. Ensure that a version of this entry applies to this client.
6. 6. Ensure that another instance of the daemon (or any other process) is not listening on the same port.
7. 7. Ensure that the key is readable by the user running the daemon and that it is well-formed.
8. 8. Ensure that this file still exists; a frequent cause is the deletion of a temp file.
9. 9. Ensure that the file is properly formed XML.
10. 10. Fix hostname resolution for the client or ensure that the profile group is properly setup.

Manual Pages

Manual Pages

These are copies of the bcfg2 manual pages created with man2html. The most recent versions of these man pages are always in trunk/bcfg2/man.

• 1+bcfg2
• 5+bcfg2.conf
• 8+bcfg2-admin
• 8+bcfg2-build-reports
• 8+bcfg2-info
• 8+bcfg2-remote
• 8+bcfg2-repo-validate
• 8+bcfg2-server

These man pages are all from r5042, which is slightly after 0.9.6 - see also #636 for information on how these pages were created.