= Overview = [[TracNav]] Configuration management is a necessary piece of maintaining many machines and diverse architectures. The ultimate goal of configuration management is to make machines as consistent, easily manageable, and up to date as possible. This document describes a typical installation of Bcfg2. == Bcfg2 Server == Each deployment will have a central bcfg2 server. This machine runs the Bcfg2 server daemon as well as keeps the Bcfg2 configuration repository as well as serving package data when needed. The server uses XML-RPC for its communications and this channel is encrypted using SSL. The Bcfg2 server and client get the connection information from the `/etc/bcfg2.conf` configuration file. An example `/etc/bcfg2.conf` looks like: {{{ [server] repository = /var/lib/bcfg2 structures = Bundler,Base generators = SSHbase,Cfg,Pkgmgr,Rules # Uncomment to use the DBStats plugin (0.9.6pre2 and later) plugins = DBStats [statistics] sendmailpath = /usr/lib/sendmail database_engine = sqlite3 # 'postgresql', 'mysql', 'mysql_old', 'sqlite3' or 'ado_mssql'. database_name = # Or path to database file if using sqlite3. #/etc/brpt.sqlite is default path if left empty database_user = # Not used with sqlite3. database_password = # Not used with sqlite3. database_host = # Not used with sqlite3. database_port = # Set to empty string for default. Not used with sqlite3. web_debug = True [communication] protocol = xmlrpc/ssl password = PASSWORD key = /etc/bcfg2.key # fingerprint of server (from bcfg2-admin fingerprint) #fingerprint = [server fingerprint] [components] bcfg2 = https://server:6789 }}} This file should be owned by user and group root and readable only by user root, as it contains a shared secret password. == Bcfg2 Abstractions == === Bcfg2 Elements === Creating a specification for a host is a modular process. A specification is built of several pieces and at the base are configuration elements. The following are all valid configuration elements: * Package : A package for the given system (RPM. Deb, ebuild, etc) * !ConfigFile : A config file * Service : A service to start or stop on boot (usually kept in `/etc/init.d/`) * Directory : A directory that should be present * !SymLink : A symbolic link that should be present === Bcfg2 Bundles === Bundles are the simplest reusable configuration blocks. A Bcfg2 Bundle is a collection of Elements that are tightly related. Bundles are written in XML. An example Bundle might be one for syslog: {{{ }}} Here we can see that for the Group rhel, the syslog Bundle contains a !ConfigFile element `/etc/syslog.conf`, a Package element sysklogd, and a Service element syslog. This defines a relationship between all these elements, so that if one of these changes, examination of all the elements should be taken more closely. For example, say you update the `/etc/syslog.conf` file. Once it's been updated the syslog service should be restarted. Bcfg2 takes care of this automatically for you. Bundles should be fairly discrete and simple and their relationship should be tight. We wouldn't want to put the sshd service in the syslog Bundle even though sshd does use syslog. Restarting sshd when `/ etc/syslog.conf` changes or when a new sysklogd package comes out doesn't make sense. === Bcfg2 Groups === Groups are the next abstraction up from Bundles. Groups can contain Bundles, other Groups, or simply nothing at all. An example Group might be for a mail-server: {{{ }}} Here we see that the server Group is actually made up of a Bundle and several other Groups. Groups are used throughout Bcfg2. They can be used in Bundles to help distinguish which elements a host gets depending on its group membership. They are also used in the Pkgmgr plugin, the Svcmgr plugin, and the Cfg plugin. === Bcfg2 Profiles === The highest level of abstraction is that of Profiles. A Profile is a special Group and is then mapped to a host. A host can have one and only one profile mapped to it. Note that a Profile can also contain other Profiles, but these Profiles aren't mapped to the host, they are simple associated with the host as a Group. A mail-server Profile might look like: {{{ }}} Here we can see that a mail-server is actually made up of a couple of Bundles and a few Groups as well. == Bcfg2 Plugins == Generators are a special case for Bcfg2. A Generator is a piece of code that is run to generate Bcfg2 Elements automatically for a host. To explain this the best way it's best to look at an example. If you run SSH, one of the most tedious processes is keeping track of SSH host keys. If you have to rebuild an existing machine you want to keep its existing keys so that user's don't get warnings that the key has changed. When you build a brand new machine, host keys are automatically created for a machine. And you probably also want to manage the global known_hosts file to contain all the host keys for your machines. This can all be done by hand or even by copying each host's key into the Cfg repository and hand updating the known_hosts file, but it could easily be automated. Enter Generators. With a Generator you can have it so the it takes care of the management of the particular !ConfigFile elements such as `/etc/ssh/ known_hosts` and the host keys. What would happen is when a client contacts the Bcfg2 server and requests those !ConfigFile elements handled by the Generator, the Generator will take care of auto- generation of new host keys for a host or giving a host the host key that it had before. On top of that, if a new host key is created and added to the repository, the Generator will automatically update the known_hosts file with the new key and make it available for all the hosts to install so that they automatically have the proper key for the new host. And here's the kicker. Let's say a machine is compromised. You have to rebuild the machine, but in this case you don't want it to get the same host key and you have to remove the host key out of all the other machine's known_hosts file. All you have to do is remove the compromised key out of the repository and rebuild the machine. When the machine runs Bcfg2 again, a new key will be generated for the host and the known_hosts file will be updated to have the new key and remove the old compromised key. Pretty slick huh? === Bcfg2 Repository === The Bcfg2 repository is just a collection of directories and files that direct the Bcfg2 server how to build configurations for hosts. The majority of the repository is kept in `/var/db/bcfg` on ci- www.uchicago.edu but the package repository is kept on ci- nfs.uchicago.edu and NFS exported. The package repository is automounted at `/autonfs/media`, for the original installation media, and `/autonfs/updates`, for updated packages. Within the central Bcfg2 repository there are several directories that contain configuration parameters for establishing a particular configuration for a host. ==== Metadata/ Directory ==== The `Metadata/` directory contains several configuration files that dictate how to start generating a configuration for a host. ===== Metadata/groups.xml ===== The `groups.xml` contains Group and Profile definitions. Here's a very basic `Metadata/groups.xml`: {{{ ... ... }}} ===== Metadata/clients.xml ===== The `Metadata/clients.xml` file contains the mappings of Profiles to clients. A sample from this file is: {{{ ... }}} ==== Bundler/ Directory ==== The `Bundler/` directory is where all the Bcfg2 bundle XML files are kept. These are the files described above. ==== Pkgmgr/ Directory ==== The `Pkgmgr/` directory keeps the XML files that define what packages are available for a host or image and where to find those packages. Here's an example of the files: {{{ $ ls Pkgmgr/ ci-backup.uchicago.edu.xml linux-fedora-core-4-noarch-updates.xml linux-fedora-core-4-x86-updates.xml linux-fedora-core-4-x86.xml linux-rhel-client-ws-4-noarch-udpates.xml linux-rhel-client-ws-4-x86.xml linux-rhel-client-ws-4-x86-updates.xml linux-rhel-client-ws-4-x86_64.xml linux-rhel-client-ws-4-x86_64-updates.xml linux-rhel-server-as-4-noarch-updates.xml linux-rhel-server-as-4-x86-updates.xml linux-rhel-server-as-4-x86.xml linux-rhel-server-es-4-noarch-updates.xml linux-rhel-server-es-4-x86.xml linux-rhel-server-es-4-x86-updates.xml }}} Here we can see a pattern of files. Looking at the names of these files you can make a pretty good guess as to what they contain. Let's take a look at what the contents of one of these files, linux- rhel-server-as-4-x86.xml, might look like: {{{ ... }}} Here you can see that the data is encapsulated in a !PackageList which describes the URI of the files described, the type of package, and a priority of the files. The priority is used to decide which specific file to use when there are multiple files that could be used for a particular host. The highest priority file is the one that is used. Using this system, you can have a file that contains all the Packages from the original installation, linux-rhel-server-as-x86.xml in our case, and then create a new file that contains updates that are made available afterwards, linux-rhel-server-as-x86-updates.xml and linux- rhel-server-as-noarch-updates.xml in our case. You simply make the priority of the update Packages higher and they will be used instead of the original installation Packages. The names of the XML files have no special meaning to Bcfg2; they are simply named so it's easy for the admin to know what the contents hold. All Packages could be kept in a single file if you so desired. Bcfg2 simply uses the Groups in the files and priorities to determine how to assign Packages to a host. You may notice the file ci- backup.uchicago.edu.xml. Its Packages have a higher priority than the update Packages. This is because that particular host requires special Packages that are older than the ones available in the updates. ==== Svcmgr/ Directory ==== The `Svcmgr` directory is responsible for defining the state of Services. Like the `Pkgmgr` directory, the `Svcmgr` directory can contain multiple files to allow the administrator to divide the definitions of services up. Here's a simple example: {{{ ... }}} ==== Base/ Directory ==== The Base directory is responsible for defining the base config for all hosts of a given Group. The majority of the elements are Packages, but you can have Services, !ConfigFiles, Directories, and !SymLinks as well. Here's an example: {{{ ... }}} Like the `Pkgmgr` directory and the `Svcmgr` directory, the Base directory can contain multiple files and the appropriate file is chosen based on the Group membership: {{{ $ ls Base/ linux-fedora-core-4-x86.xml linux-rhel-client-ws-4-x86_64.xml linux-rhel-client-ws-4-x86.xml linux-rhel-server-as-4-x86.xml linux-rhel-server-es-4-x86.xml }}} ==== Cfg/ Directory ==== The Cfg is where all the actual !ConfigFiles are kept. If you have a !ConfigFile element listed in a Bundle you need to provide Bcfg with the actual configuration file to give to the host. The directory structure under the `Cfg/` directory defines where the configuration file should be placed on the host machine relative to the / directory. For example, if you want to manage `/etc/motd` you would create the directory `Cfg/etc/motd/` and then place the `/etc/motd` file in that directory. ===== Specialized Configuration Files ===== Some hosts or Groups might need a different version of `/etc/motd` or might just need to add or remove lines from it. You can create specialized versions for a particular host by appending .H_ to the configuration file where }}} This is a good example showcasing several of the report generation features. In this example we define a report names Computation Institute which includes good as well as bad nodes and those nodes that were modified in the last run. For this report there are three delivery mechanisms: www, rss, mail. Thee www mechanism is the HTML generation. The other two are pretty self explanatory. The last parameter is which machines to include in this report. This report includes all the machines that Bcfg2 knows about. here you can give a regular expression to narrow down which machines are reported for. Refer to the Bcfg2 documentation for more detail on the configuring reports. === Report Scripts === Reports are generated by the `/usr/sbin/StatReports` script. !GenerateHostInfo gathers the data about the given hosts and writes out `etc/hostinfo.xml`. !StatReports reads the `etc/hostinfo.xml` and generates the reports based on `etc/report-configuration.xml`. !StatReports will automatically run !GenerateHostInfo if it has not run within 24 hours. !StatReports is usually run nightly from cron. === Report Interface === For the www and rss mechanisms, you need to define a way to get to the reports generated. The CI reports are stored in `/var/db/bcfg/www` and we need to define an Apache Alias to access it. `/etc/httpd/conf.d/ bcfg2.conf` defines the Apache configuration: {{{ Alias /bcfg2 /var/db/bcfg/www Options Indexes FollowSymLinks AllowOverride None Order deny,allow Allow from all }}}