= Overview =

[[TracNav]]

Configuration management is a necessary piece of maintaining many
machines and diverse architectures. The ultimate goal of
configuration management is to make machines as consistent, easily
manageable, and up to date as possible. This document describes a
typical installation of Bcfg2.

== Bcfg2 Server ==

Each deployment will have a central bcfg2 server.
This machine runs the Bcfg2 server daemon as well as keeps the Bcfg2 configuration
repository as well as serving package data when needed. The server uses XML-RPC
for its communications and this channel is encrypted using SSL. The
Bcfg2 server and client get the connection information from the `/etc/bcfg2.conf` 
configuration file. An example `/etc/bcfg2.conf` looks like:

{{{
[server]
repository = /var/lib/bcfg2
structures = Bundler,Base
generators = SSHbase,Cfg,Pkgmgr,Rules
# Uncomment to use the DBStats plugin (0.9.6pre2 and later)
plugins = DBStats

[statistics]
sendmailpath = /usr/lib/sendmail
database_engine = sqlite3
# 'postgresql', 'mysql', 'mysql_old', 'sqlite3' or 'ado_mssql'.
database_name =
# Or path to database file if using sqlite3.
#<repository>/etc/brpt.sqlite is default path if left empty
database_user =
# Not used with sqlite3.
database_password =
# Not used with sqlite3.
database_host =
# Not used with sqlite3.
database_port =
# Set to empty string for default. Not used with sqlite3.
web_debug = True

[communication]
protocol = xmlrpc/ssl
password = PASSWORD
key = /etc/bcfg2.key
# fingerprint of server (from bcfg2-admin fingerprint)
#fingerprint = [server fingerprint]

[components]
bcfg2 = https://server:6789
}}}

This file should be owned by user and group root and readable only by
user root, as it contains a shared secret password.

== Bcfg2 Abstractions ==

=== Bcfg2 Elements ===

Creating a specification for a host is a modular process. A
specification is built of several pieces and at the base are
configuration elements. The following are all valid configuration
elements:
 * Package : A package for the given system (RPM. Deb, ebuild, etc)
 * !ConfigFile : A config file
 * Service : A service to start or stop on boot (usually kept in `/etc/init.d/`)
 * Directory : A directory that should be present
 * !SymLink : A symbolic link that should be present

=== Bcfg2 Bundles ===

Bundles are the simplest reusable configuration blocks. A Bcfg2
Bundle is a collection of Elements that are tightly related. Bundles
are written in XML. An example Bundle might be one for syslog:

{{{
<Bundle name="syslog" version="2.0">
  <Group name="rhel">
    <ConfigFile name="/etc/syslog.conf"/>
    <Package name="sysklogd"/>
    <Service name="syslog"/>
  </Group>
</Bundle>
}}}

Here we can see that for the Group rhel, the syslog Bundle contains a
!ConfigFile element `/etc/syslog.conf`, a Package element sysklogd, and
a Service element syslog. This defines a relationship between all
these elements, so that if one of these changes, examination of all
the elements should be taken more closely. For example, say you
update the `/etc/syslog.conf` file. Once it's been updated the syslog
service should be restarted. Bcfg2 takes care of this automatically
for you.

Bundles should be fairly discrete and simple and their relationship
should be tight. We wouldn't want to put the sshd service in the
syslog Bundle even though sshd does use syslog. Restarting sshd when `/
etc/syslog.conf` changes or when a new sysklogd package comes out
doesn't make sense.

=== Bcfg2 Groups ===

Groups are the next abstraction up from Bundles. Groups can contain
Bundles, other Groups, or simply nothing at all. An example Group
might be for a mail-server:

{{{
  <Group profile='false' name='server'>
    <Bundle name='kerberos-client'/>
    <Group name='backup-client'/>
    <Group name='base'/>
    <Group name='ldap-client'/>
    <Group name='serial-console'/>
  </Group>
}}}

Here we see that the server Group is actually made up of a Bundle and
several other Groups. Groups are used throughout Bcfg2. They can be
used in Bundles to help distinguish which elements a host gets
depending on its group membership. They are also used in the Pkgmgr
plugin, the Svcmgr plugin, and the Cfg plugin.

=== Bcfg2 Profiles ===

The highest level of abstraction is that of Profiles. A Profile is a
special Group and is then mapped to a host. A host can have one and
only one profile mapped to it. Note that a Profile can also contain
other Profiles, but these Profiles aren't mapped to the host, they
are simple associated with the host as a Group. A mail-server Profile
might look like:

{{{
  <Group profile='true' public='false' name='mail-server'>
    <Bundle name='mail-server'/>
    <Bundle name='mailman-server'/>
    <Group name='apache-server'/>
    <Group name='linux-rhel-server-as-4-x86'/>
    <Group name='nfs-client'/>
    <Group name='server'/>
  </Group>
}}}

Here we can see that a mail-server is actually made up of a couple of
Bundles and a few Groups as well.

== Bcfg2 Plugins ==

Generators are a special case for Bcfg2. A Generator is a piece of
code that is run to generate Bcfg2 Elements automatically for a host.
To explain this the best way it's best to look at an example.
If you run SSH, one of the most tedious processes is keeping track of
SSH host keys. If you have to rebuild an existing machine you want to
keep its existing keys so that user's don't get warnings that the key
has changed. When you build a brand new machine, host keys are
automatically created for a machine. And you probably also want to
manage the global known_hosts file to contain all the host keys for
your machines. This can all be done by hand or even by copying each
host's key into the Cfg repository and hand updating the known_hosts
file, but it could easily be automated. Enter Generators.
With a Generator you can have it so the it takes care of the
management of the particular !ConfigFile elements such as `/etc/ssh/
known_hosts` and the host keys. What would happen is when a client
contacts the Bcfg2 server and requests those !ConfigFile elements
handled by the Generator, the Generator will take care of auto-
generation of new host keys for a host or giving a host the host key
that it had before. On top of that, if a new host key is created and
added to the repository, the Generator will automatically update the
known_hosts file with the new key and make it available for all the
hosts to install so that they automatically have the proper key for
the new host. And here's the kicker. Let's say a machine is
compromised. You have to rebuild the machine, but in this case you
don't want it to get the same host key and you have to remove the
host key out of all the other machine's known_hosts file. All you
have to do is remove the compromised key out of the repository and
rebuild the machine. When the machine runs Bcfg2 again, a new key
will be generated for the host and the known_hosts file will be
updated to have the new key and remove the old compromised key.
Pretty slick huh?

=== Bcfg2 Repository ===

The Bcfg2 repository is just a collection of directories and files
that direct the Bcfg2 server how to build configurations for hosts.
The majority of the repository is kept in `/var/db/bcfg` on ci-
www.uchicago.edu but the package repository is kept on ci-
nfs.uchicago.edu and NFS exported. The package repository is
automounted at `/autonfs/media`, for the original installation media,
and `/autonfs/updates`, for updated packages. Within the central Bcfg2
repository there are several directories that contain configuration
parameters for establishing a particular configuration for a host.

==== Metadata/ Directory ====

The `Metadata/` directory contains several configuration files that
dictate how to start generating a configuration for a host.

===== Metadata/groups.xml =====

The `groups.xml` contains Group and Profile definitions. Here's a very
basic `Metadata/groups.xml`:

{{{
<Groups version='3.0'>
  ...
  <Group profile='true' public='false' name='mail-server'>
    <Bundle name='mail-server'/>
    <Bundle name='mailman-server'/>
    <Group name='apache-server'/>
    <Group name='linux-rhel-server-as-4-x86'/>
    <Group name='nfs-client'/>
    <Group name='server'/>
  </Group>
  ...
</Groups>
}}}

===== Metadata/clients.xml =====

The `Metadata/clients.xml` file contains the mappings of Profiles to
clients. A sample from this file is:

{{{
<Clients version="3.0">
  <Client profile="backup-server" pingable="Y" pingtime="0" name="ci-backup.uchicago.edu"/>
  <Client profile="console-server" pingable="Y" pingtime="0"
name="ci-con.uchicago.edu"/>
  <Client profile="kerberos-master" pingable="Y" pingtime="0"
name="ci-kdc.uchicago.edu"/>
  <Client profile="mail-server" pingable="Y" pingtime="0" name="ci-
mail.uchicago.edu"/>
  ...
</Clients>
}}}

==== Bundler/ Directory ====

The `Bundler/` directory is where all the Bcfg2 bundle XML files are
kept. These are the files described above.

==== Pkgmgr/ Directory ====

The `Pkgmgr/` directory keeps the XML files that define what packages
are available for a host or image and where to find those packages.
Here's an example of the files:

{{{
$ ls Pkgmgr/
ci-backup.uchicago.edu.xml
linux-fedora-core-4-noarch-updates.xml
linux-fedora-core-4-x86-updates.xml
linux-fedora-core-4-x86.xml
linux-rhel-client-ws-4-noarch-udpates.xml
linux-rhel-client-ws-4-x86.xml
linux-rhel-client-ws-4-x86-updates.xml
linux-rhel-client-ws-4-x86_64.xml
linux-rhel-client-ws-4-x86_64-updates.xml
linux-rhel-server-as-4-noarch-updates.xml
linux-rhel-server-as-4-x86-updates.xml
linux-rhel-server-as-4-x86.xml
linux-rhel-server-es-4-noarch-updates.xml
linux-rhel-server-es-4-x86.xml
linux-rhel-server-es-4-x86-updates.xml
}}}

Here we can see a pattern of files. Looking at the names of these
files you can make a pretty good guess as to what they contain.
Let's take a look at what the contents of one of these files, linux-
rhel-server-as-4-x86.xml, might look like:

{{{
<PackageList uri='http://ci-www.uchicago.edu/media/rhel-server-as-4/x86/install/RedHat/RPMS' type='rpm' priority='0'>
  <Group name='linux-rhel-server-as-4-x86'>
    <Package file='4Suite-1.0-3.i386.rpm'/>
    <Package file='Canna-3.7p3-7.EL4.i386.rpm'/>
    <Package file='Canna-devel-3.7p3-7.EL4.i386.rpm'/>
    <Package file='Canna-libs-3.7p3-7.EL4.i386.rpm'/>
    <Package file='ElectricFence-2.2.2-19.i386.rpm'/>
    <Package file='FreeWnn-1.10pl020-5.i386.rpm'/>
    <Package file='FreeWnn-devel-1.10pl020-5.i386.rpm'/>
    <Package file='FreeWnn-libs-1.10pl020-5.i386.rpm'/>
    ...
  </Group>
</PackageList>
}}}

Here you can see that the data is encapsulated in a !PackageList which
describes the URI of the files described, the type of package, and a
priority of the files. The priority is used to decide which specific
file to use when there are multiple files that could be used for a
particular host. The highest priority file is the one that is used.
Using this system, you can have a file that contains all the Packages
from the original installation, linux-rhel-server-as-x86.xml in our
case, and then create a new file that contains updates that are made
available afterwards, linux-rhel-server-as-x86-updates.xml and linux-
rhel-server-as-noarch-updates.xml in our case. You simply make the
priority of the update Packages higher and they will be used instead
of the original installation Packages.
The names of the XML files have no special meaning to Bcfg2; they are
simply named so it's easy for the admin to know what the contents
hold. All Packages could be kept in a single file if you so desired.
Bcfg2 simply uses the Groups in the files and priorities to determine
how to assign Packages to a host. You may notice the file ci-
backup.uchicago.edu.xml. Its Packages have a higher priority than the
update Packages. This is because that particular host requires
special Packages that are older than the ones available in the updates.

==== Svcmgr/ Directory ====

The `Svcmgr` directory is responsible for defining the state of
Services. Like the `Pkgmgr` directory, the `Svcmgr` directory can contain
multiple files to allow the administrator to divide the definitions
of services up. Here's a simple example:

{{{
<Services priority='0'>
  <Service name="kudzu" status="off"/>
  <Group name="mail-server">
    <Service name="postfix" status="on"/>
  </Group>
  ...
</Services>
}}}

==== Base/ Directory ====

The Base directory is responsible for defining the base config for
all hosts of a given Group. The majority of the elements are
Packages, but you can have Services, !ConfigFiles, Directories, and
!SymLinks as well. Here's an example:

{{{
<Base >
  <Group name='linux-rhel-server-as-4-x86'>
    <ConfigFile name='/etc/group'/>
    <ConfigFile name='/etc/inittab'/>
    <ConfigFile name='/etc/passwd'/>
    <ConfigFile name='/etc/securetty'/>
    <ConfigFile name='/etc/shadow'/>
    <ConfigFile name='/etc/updatedb.conf'/>
    <ConfigFile name='/usr/bin/bu'/>
    <Package name='4Suite'/>
    ...
  </Group>
</Base>
}}}

Like the `Pkgmgr` directory and the `Svcmgr` directory, the Base
directory can contain multiple files and the appropriate file is
chosen based on the Group membership:

{{{
$ ls Base/
linux-fedora-core-4-x86.xml
linux-rhel-client-ws-4-x86_64.xml
linux-rhel-client-ws-4-x86.xml
linux-rhel-server-as-4-x86.xml
linux-rhel-server-es-4-x86.xml
}}}

==== Cfg/ Directory ====

The Cfg is where all the actual !ConfigFiles are kept. If you have a
!ConfigFile element listed in a Bundle you need to provide Bcfg with
the actual configuration file to give to the host. The directory
structure under the `Cfg/` directory defines where the configuration
file should be placed on the host machine relative to the /
directory. For example, if you want to manage `/etc/motd`  you would
create the directory `Cfg/etc/motd/` and then place the `/etc/motd` file
in that directory.

===== Specialized Configuration Files =====

Some hosts or Groups might need a different version of `/etc/motd` or
might just need to add or remove lines from it. You can create
specialized versions for a particular host by appending .H_ to the
configuration file where <hostname is the fully qualified hostname.
So if foo.example.com needed a special version your motd directory
might look like:

{{{
$ ls Cfg/etc/motd
motd
motd.H_foo.example.com
}}}

Now when you run Bcfg2 on foo.example.com it will get the specialized
version and all other hosts will get the generic version.
If you want specialized versions for a Group you would append .G##_
to the file where ## is a two digit number specifying priority and is
the name of the Group. Continuing our example if we wanted to have a
special motd for the mail-server Group we would have the following:

{{{
$ ls Cfg/etc/motd
motd
motd.G01_mail-server
motd.H_foo.example.com
}}}

Now if a host is a member of the mail-server Group it will get the
specialized Group version of motd. Now what happens if
foo.example.com is also a membere of the mail-server Group? Bcfg2
will use the highest priority, most specific file. So foo.example.com
will always get motd.H_foo.example.com. If you have specialized
versions for multiple groups, the priorities you assign come in to
play. Let's say we have the following:

{{{
$ ls Cfg/etc/motd
motd
motd.G01_mail-server
motd.G02_web-server
motd.H_foo.example.com
}}}

If you have a host, not foo.example.com, that is a member of both the
mail-server and the web-server Groups, it will get the web-server
version because it has a higher priority, 02 instead of 01.

=== Deltas ===

Bcfg2 has finer grained control over how to deliver configuration
files to a host. Let's say we have a Group named file-server. Members
of this group need the exact same `/etc/motd` as all other hosts except
they need one line added. We could copy motd to motd.G01_file-server,
add the one line to the Group specific version and be done with it,
but we're duplicating data in both files. What happens if we need to
update the motd? We'll need to remember to update both files then.
Here's where deltas come in. A delta is a small change to the base
file. There are two types of deltas: cats and diffs. The cat delta
simply adds or removes lines from the base file. The diff delta is
more powerful since it can take a unified diff and apply it to the
base configuration file to create the specialized file. Diff deltas
should be used very sparingly.

==== Cat Files ====

Continuing our example for cat files, we would first create a file
named `motd.G01_file-server.cat`. The .cat suffix designates that the
file is a diff. We would then edit that file and add the following
line:

{{{
+This is a file server
}}}

The + at the begining of the file tells Bcfg2 that the line should
be appended to end of the file. You can also start a line with - to
tell Bcfg2 to remove that exact line wherever it might be in the file.
How do we know what base file Bcfg2 will choose to use to apply a
delta? The same rules apply as before: Bcfg2 will choose the highest
priority, most specific file as the base and then apply deltas in the
order of most specific and then increasing in priority. What does
this mean in real life. Let's say our machine is a web server, mail
server, and file server and we have the following configuration files:

{{{
motd
motd.G01_web-server
motd.G01_mail-server.cat
motd.G02_file-server.cat
motd.H_foo.example.cat
}}}

If our machine isn't foo.example.com then here's what would happen:

Bcfg2 would choose `motd.G01_web-server` as the base file. It is the
most specific base file for this host.
Bcfg2 would apply the `motd.G01_mail-server.cat` delta to the
`motd.G01_web-server` base file. It is the least specific delta.
Bcfg2 would then apply the `motd.G02_file-server.cat` delta to the
result of the delta before it.
If our machine is foo.example.com then here's what would happen:

Bcfg2 would choose `motd.G01_web-server` as the base file. It is the
most specific base file for this host.
Bcfg2 would apply the `motd.H_foo.example.com.cat` delta to the
`motd.G01_web-server` base file.
The reason the other deltas aren't applied to foo.example.com is
because a .H_ delta is more specific than a .G##_ delta. Bcfg2
applies all the deltas at the most specific level.

== Bcfg2 Reports ==

Bcfg2 has the ability to generate reports to make it easier to see at
a glance the state of your environment. It can report via three
methods: HTML web page, RSS feed, email.

=== Report Configuration ===

Report generation requires the `etc/report-configuration.xml` and looks
something like:

{{{
<Reports>
  <Report name='Computation Institute' good='Y' modified='Y'>
    <Delivery mechanism='www' type='nodes-digest'>
      <Destination address='/var/db/bcfg/www/stats.html'/>
    </Delivery>
    <Delivery mechanism='rss' type='nodes-individual'>
      <Destination address='/var/db/bcfg/www/stats.rss'/>
    </Delivery>
    <Delivery mechanism='mail' type='nodes-digest'>
      <Destination address='somebody@ci.uchicago.edu'/>
    </Delivery>

    <Machine name='.*'/>
  </Report>
</Reports>
}}}

This is a good example showcasing several of the report generation
features. In this example we define a report names Computation
Institute which includes good as well as bad nodes and those nodes
that were modified in the last run. For this report there are three
delivery mechanisms: www, rss, mail. Thee www mechanism is the HTML
generation. The other two are pretty self explanatory. The last
parameter is which machines to include in this report. This report
includes all the machines that Bcfg2 knows about. here you can give a
regular expression to narrow down which machines are reported for.
Refer to the Bcfg2 documentation for more detail on the configuring
reports.

=== Report Scripts ===

Reports are generated by the `/usr/sbin/StatReports` script. 
!GenerateHostInfo
gathers the data about the given hosts and writes out `etc/hostinfo.xml`. 
!StatReports reads the `etc/hostinfo.xml` and generates
the reports based on `etc/report-configuration.xml`. !StatReports will 
automatically run !GenerateHostInfo if it has not run within 24 hours. 
!StatReports is usually run nightly from cron.

=== Report Interface ===

For the www and rss mechanisms, you need to define a way to get to
the reports generated. The CI reports are stored in `/var/db/bcfg/www`
and we need to define an Apache Alias to access it. `/etc/httpd/conf.d/
bcfg2.conf` defines the Apache configuration:

{{{
<IfModule mod_alias.c>
        Alias /bcfg2 /var/db/bcfg/www
</IfModule>

<Directory "/var/db/bcfg/www">
        Options Indexes FollowSymLinks
        AllowOverride None
        Order deny,allow
        Allow from all
</Directory>
}}}