Writing Bcfg2 Specification

Bcfg2 specifications are logically divided in to three areas:

• Metadata
• Abstract
• Literal

The metadata portion of the configuration assigns a client to its profile group and to its non profle groups. The profile group is assigned in Metadata/clients.xml and the non profile group assignments are in Metadata/groups.xml.

The group memberships contained in the metadata are then used to constuct an abstract configuration for the client. An abstract configuration for a client identifies the configuration entities (packages, configuration files, service, etc) that a client requires, but it does not identify them explicitly. For instance an abstract configuration may identify that a client needs the bcfg2 package with

<Package name=bcfg2/>

but this does not explicitly identify that an RPM package version 0.9.2 should be loaded from http://rpm.repo.server/bcfg2-0.9.2-0.1.rpm. The abstract configuration is defined in the xml configuration files for the Base and Bundles plugins.

A combination of a clients metadata (group memberships) and abstract configuration is then used to generate the clients literal configuration. For instance the above abstract configuration entry may generate a literal configuration of

<Package name='bcfg2' version='0.9.2-0.1' type='yum'/>

A clients literal configuration is generated by a number of plugins that handle the different configuration entities.

Metadata

The metadata mechanism has two types of information, client metadata and group metadata. The client metadata describes which top level group a client is associated with.The group metadata describes groups in terms of what bundles and other groups they include. Each aspect grouping and clients' memberships are reflected in the Metadata/groups.xml and Metadata/clients.xml files, respectively.

Usage of Groups in Metadata

Clients are assigned membership of groups in the Metadata descriptions. Clients can be directly assigned to 'profile' or 'public' groups. Client membership of all other groups is by those groups being associated with the profile or public groups. This file can be indirectly modified from clients through use of the -p flag to bcfg2.

Clients are associated with profile groups in Metadata/clients.xml as shown below.

Metadata/clients.xml

The Metadata/clients.xml file contains the mappings of Profile Groups to clients. The file is just a series of <Client /> tags, each of which describe one host. A sample file is below:

<Clients version="3.0">
  <Client profile="backup-server" pingable="Y" pingtime="0" name="backup.example.com"/>
  <Client profile="console-server" pingable="Y" pingtime="0" name="con.example.com"/>              
  <Client profile="kerberos-master" pingable="Y" pingtime="0" name="kdc.example.com"/>
  <Client profile="mail-server" pingable="Y" pingtime="0" name="mail.example.com"/>
  <Client name='foo' address='10.0.0.1' pingable='N' pingtime='-1'>
      <Alias name='foo-mgmt' address='10.1.0.1'/>
  </Client>
</Clients>

Clients Tag

The Clients tag has the following possible attributes:

Name	Description	Values
version	Client schema version	String

Client Tag

Each entry in clients.xml must have the following properties:

Name	Description	Values
name	Host name of client. This needs do be the name (possibly a FQDN) returned by a reverse lookup on the connecting IP address.	String
profile	Profile group name to associate this client with	String

Additionally, the following properties can be specified:

Name	Description	Values
Alias	Alternative name and address for the client	XML Element
address	Establishes an extra ip address that resolves to this client	ip address
location	Requires requests to come from an IP address that matches the client record	fixed|floating
password	Establishes a per-node password that can be used instead of the global password	String
pingable	If the client is pingable (deprecated; for old reporting system)	(Y|N)
pingtime	Last time the client was pingable (deprecated; for old reporting system)	String
secure	Requires the use of the per-client password for this client	true|false
uuid	Establishes a per-node name that can be used to bypass dns-based client resolution	String

For detailed information on client authentication see Authentication

Metadata/groups.xml

The Metadata/groups.xml file contains Group and Profile definitions. Here's a simple Metadata/groups.xml file:

<Groups version='3.0'>
  <Group name='mail-server' profile='true'
                            public='false'
                            comment='Top level mail server group' >
    <Bundle name='mail-server'/>
    <Bundle name='mailman-server'/>
    <Group name='apache-server'/>
    <Group name='rhel-as-4-x86'/>
    <Group name='nfs-client'/>
    <Group name='server'/>
  </Group>
  <Group name='rhel-as-4-x86' toolset='rh'>
     <Group name='rhel'/>
  </Group>
  <Group name='apache-server'/>
  <Group name='nfs-client'/>
  <Group name='server'/>
  <Group name='rhel'/>
</Groups>

Nested/chained groups definitions are conjunctive (logical and). For instance, in the above example, a client associated with the Profile Group 'mail-server' is also a member of the apache-server, rhel-as-4-x86, nfs-client, server and rhel groups.

Groups describe clients in terms for abstract, disjoint aspects. Groups can be combined to form complex descriptions of clients that use configuration commonality and inheritance. Groups have several attributes, described below:

Metadata Groups Tag

The Groups tag has the following possible attributes:

Name	Description	Values
version	Group schema version	String
origin	URL of master version (for common repo)	String
revision	Master version control revision	String

Metadata Group Tag

The Group Tag has the following possible attributes:

Name	Description	Values
name	Name of the group	String
profile	If a client can be directly associated with this group	(True|False*)
public	If a client can freely associate itself with this group. For use with bcfg2 -p option on the client.	(True|False*)
toolset	Describes which client-side logic should be used to make configuration changes	(rh|debian|solaris|aix|auto|gentoo)
category	A group can only contain one instance of a group in any category. This provides the basis for representing groups which are conjugates of one another in a rigorous way. It also provides the basis for negation.	String
default	Set as the profile to use for clients that are not associated with a profile in clients.xml.	(True|False*)
comment	English text description of group	String

Groups can also contain other groups and bundles.

Use of XInclude

XInclude is a W3C specification for the inclusion of external XML documents into XML source files. Much like the use of #include in C, this allows complex definitions to be split into smaller, more manageable pieces. As of bcfg2-0.9.0pre1, the Metadata plugin supports the use of XInclude specifications to split the clients.xml and groups.xml files. This mechanism allows the following specification to produce useful results:

 <Groups version='3.0' xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="my-groups.xml" />
  <xi:include href="their-groups.xml" />
 </Groups>

Each of the included groups files has the same format. These files are properly validated by bcfg2-repo-validate. This mechanism is useful for composing group definitions from multiple sources, or setting different permissions in an svn repository.

Probes

The metadata plugin includes client-side probing functionality. This is fully documented here?.

Dynamic Groups

Dynamic groups are likewise complex, and are covered on their own page

Abstract Configuration (Structures)

A clients Abstract Configuration is the inventory of configuration entities that should be installed on a client. Two plugins provide the basis for the abstract configuration, the Bundler and Base.

The plugin Bundler builds descriptions of interrelated configuration entities. These are typically used for the representation of services, or other complex groups of entities.

The Base provides a laundry list of configuration entities that need to be installed on hosts. These entities are independent from one another, and can be installed individually without worrying about the impact on other entities.

Usage of Groups in Base and Bundles

Groups are used by the Base and Bundles plugins for selecting Configuration Entity Types for inclusion in a clients abstract configuration. They can be thought of as:

if client is a member of group1 then
    assign to abstract config

Nested groups are conjunctive (logical and).

if client is a member of group1 and group2 then
    assign to abstract config

Group membership maybe negated. See "Writing Bundles" for an example.

Configuration Entity Types

Entities in the abstract configuration (and correspondingly in the literal configuration) can have one of several types. In the abstract configuration, each of these entities only has a tag and the name attribute set.

The types of Configuration Entities that maybe assigned to the abstract configuration are as follows:

Tag	Description
Package	Software Package
ConfigFile	Configuration File
Service	Persistent system services and daemons
Directory	File system Directories
SymLink	Symbolic links
Permissions	The permissions (not contents) of a POSIX path
Action	Actions to be performed as part of a bundle installation

An example of each entity type is below.

<Package name='bcfg2'/>
<ConfigFile name='/etc/bcfg2.conf'/>
<Service name='ntpd'/>
<Directory name='/var/tmp'/>
<SymLink name='/dev/MAKEDEV'/>
<Permissions name='/etc/passwd'/>
<Action name='action_name'/>

Configuration Entity Name Attribute

With the exception of the Action Entity type, the value of the configuration entity name attribute is not arbitary. It must map to entities of the same type in the Literal Configuration, which in turn map to physical entities on the client.

For the most part the physical entity being mapped to is intuitive and are listed below.

Tag	Description
Package	name of the software package
ConfigFile	Full absolute path of the configuration file
Service	Name of the init script for the service which may not be that same as the name of the daemon started or the package name
Directory	Full absolute path of the directories
SymLink	Full absolute path of the symbolic linkss
Permissions	Full absolute path of the file or directory
Action	Arbitary name

Writing Bundles

Bundles consist of a set of configuration entities. These entities are grouped together due to a configuration-time interdependency. Basic services tend to be the simplest example of these. They normally consist of

• some software package(s)
• some configuration files
• an indication that some service should be activated

If any of these pieces are installed or updated, all should be rechecked and any associated services should be restarted.

All files in the Bundles/ subdirectory of the repository are processed. Each bundle must be defined in its own file and the filename must be the same as the bundle name with a .xml suffix.

# ls Bundler
Glide3.xml
LPRng.xml
Tivoli-backup.xml
Tivoli.xml
a2ps.xml
abiword.xml
account.xml
adsm-client.xml
amihappy.xml
apache-basic.xml
apache.xml
apache2-basic.xml
apt-proxy.xml
at.xml
atftp-server.xml
atftp.xml
....

Groups can be used inside of bundles to differentiate which entries particular clients will receive. This is useful for the case where entries are named differently across systems; for example, one linux distro may have a package called openssh while another uses the name ssh. Configuration entries nested inside of Group elements only apply to clients who are a member of those groups; multiply nested groups must all apply.

Also, groups may be negated; entries included in such groups will only apply to clients who are not a member of said group.

When packages in a bundle are verified by the client toolset, the ConfigFiles included in the same bundle are taken into consideration. That is, a package will not fail verification from a bcfg2 perspective if the package verification only failed because of configuration files that are defined in the same bundle.

The following is an annotated copy of a bundle:

<Bundle revision='$Revision: 2668 $' name='ssh' version='2.0'
        origin='https://svn.mcs.anl.gov/repos/bcfg/trunk/repository/Bundler/ssh.xml'>
  <ConfigFile name='/etc/ssh/ssh_host_dsa_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_rsa_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_dsa_key.pub'/>
  <ConfigFile name='/etc/ssh/ssh_host_rsa_key.pub'/>
  <ConfigFile name='/etc/ssh/ssh_host_key'/>
  <ConfigFile name='/etc/ssh/ssh_host_key.pub'/>
  <ConfigFile name='/etc/ssh/sshd_config'/>
  <ConfigFile name='/etc/ssh/ssh_config'/>
  <ConfigFile name='/etc/ssh/ssh_known_hosts'/>
  <Group name='rpm'>
    <Package name='openssh'/>
    <Package name='openssh-askpass'/>
    <Service name='sshd'/>
    <Group name='fedora' >
       <Group name='fc4' negate='true'>
         <Package name='openssh-clients'/>
       </Group>
       <Package name='openssh-server'/>
    </Group>
  </Group>
  <Group name='deb'>
    <Package name='ssh'/>
    <Service name='ssh'/>
  </Group>
</Bundle>

In this bundle, most of the entries are common to all systems. Clients in group "deb" get one extra package and service, while clients in group "rpm" get two extra packages and an extra service. In addition, clients in group "fedora" and group "rpm" get one extra package entries, unless they are not in the fc4 group, in which case, they get an extra package. Notice that this file doesn't describe which versions of these entries that clients should get, only that they should get them. (Admittedly, this example is slightly contrived, but demonstrates how group entries can be used in bundles)

Group	Entry
all	/etc/ssh/ssh_host_dsa_key
all	/etc/ssh/ssh_host_rsa_key
all	/etc/ssh/ssh_host_dsa_key.pub
all	/etc/ssh/ssh_host_rsa_key.pub
all	/etc/ssh/ssh_host_key
all	/etc/ssh/ssh_host_key.pub
all	/etc/ssh/sshd_config
all	/etc/ssh/ssh_config
all	/etc/ssh/ssh_known_hosts
rpm	Package openssh
rpm	Package openssh-askpass
rpm	Service sshd
rpm and fedora	Package openssh-server
rpm and fedora and not fc4	Package openssh-clients
deb	Package ssh
deb	Service ssh

Bundle Tag

The Bundle Tag has the following possible attributes:

Name	Description	Values
name	The name of the bundle	String
version	Bundle schema version	String
origin	URL of master version (for common repo)	String
revision	Master version control revision	String

As mentioned above the Configuration Entity Tags may only have the name attribute in Bundle definitions.

Abstract Group Tag

In the Abstract Configuration plugins (Base and Bundle) the Group Tag may have the following attributes:

Name	Description	Values
name	Name of group.	String
negate	Negate the group association (is not a member of)	(True|False*)

An abstract group may contain any of the Configuration Entity types and other groups.

Using Base

The Base plugin provides a mechanism to add independent configuration entities to a client's abstract configuration. All files in the Base/ subdirectory of the repository are processed, and all entries that fall within the scope of the client metadata are included in its abstract configuration.

$ ls Base/
centos-4-x86.xml
fedora-core-4-x86.xml
rhel-as-4-x86.xml
rhel-es-4-x86.xml
rhel-ws-4-x86_64.xml
rhel-ws-4-x86.xml

<Base>
    <Group name='Centos4.4-Standard'>
        <Package name='audit'/>
        <Package name='audit-libs'/>
        <Package name='basesystem'/>
        <Package name='bash'/>
        <Package name='bcfg2'/>
        <Package name='beecrypt'/>
        ....
        <Package name='yum'/>
        <Package name='zlib'/>
        <Group name='x86_64'>
            <Package name='systemimager-x86_64initrd_template'/>
        </Group>
    </Group>
</Base>

The format of the Base files are similar to those used by the Bundler. The majority of the elements are usually Packages, but ConfigFiles, Directories, Services, Permissions, and SymLinks may all be defined. A partial example is below:

Base Tag

The Base Tag has no attributes

As mentioned above the Configuration Entity Tags contained in a Base definition may only have the name attribute in Base definitions.

Abstract Group Tag

In the Abstract Configuration plugins (Base and Bundle) the Group Tag may have the following attributes:

Name	Description	Values
name	Name of group.	String
negate	Negate the group association (is not a member of)	(True|False*)

An abstract group may contain any of the Configuration Entity types and other groups.

Literal Configuration (Generators)

A Generator is a bcfg2 piece of code that is run to generate the literal configuration for a host using a combination of the hosts metadata and abstract configuration.

A Generator can take care of a particular configuration element. Any time this element is requested by the client, the server dynamically generates it either by crunching data and creating new information or by reading a file off of disk and passes it down to the client for installation.

Usage of Groups in Generators

Similar to Abstract Configuration plugins, groups are used by generator plugins for selecting Configuration Entities for inclusion in a clients literal configuration. They can be thought of as:

if client is a member of group1 then
    assign to abstract config

Nested groups are conjunctive (logical and).

if client is a member of group1 and group2 then
    assign to abstract config

How the groups are configured is specific to the plugin, but here are two common methods:

• xml configuration file (Pkgmgr, Rules)
• file name encoding (Cfg, TCheetah, SSHBase)

Details are included on each plugin's page.

Generators

Each of the generators is covered on their own page.

Plugin	Description
Actions	Action entries are commands that are executed either before bundle installation, after bundle installation or both.
Cfg	The Cfg plugin provides a repository to describe configuration file contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines.
Deps	The deps plugin allows inter-entry dependencies (like package prerequisites) to be mapped independent of client roles.
Hostbase	Hostbase is an IP management system built on top of Bcfg2.
Pkgmgr	The Pkgmgr plugin define what packages are available for a host or image and where to find those packages.
Rules	The Rules plugin is responsible for defining the details about the Directory, Permissions, SymLink and Service elements.
SSHbase	SSHbase is a purpose build bcfg2 plugin for managing ssh host keys. It is responsible for making ssh keys persist beyond a client rebuild and building a proper ssh_known_hosts file, including a correct localhost record for the current system.
TCheetah	The TCheetah plugin allows you to use the cheetah templating system to create files, instead of the various diff-based methods offered by the Cfg plugin. It also allows you to include the results of probes executed on the client in the created files.
TGenshi	The TGenshi plugin uses the genshi toolkit for generating files and functions much like the TCheetah plugin