wiki:TrackingDevelopmentTrunk

Version 36 (modified by solj, 14 years ago) (diff)

--

  1. Tracking Development Releases of Bcfg2
    1. Tested setups for 1.0 SSL rewrite
    2. User visible changes
      1. SSL Changes
      2. bcfg2.conf changes
      3. Snapshots
      4. New plugins
      5. Service Changes
    3. Release Information
      1. 1.0pre4
      2. 1.0pre3
        1. Server Core
        2. SSL
        3. Tool driver fixes
        4. Snapshots
        5. Other
        6. Bugfixes
      3. 1.0pre2
      4. 1.0pre1
  2. Tracking Development Releases of Bcfg2 (< 1.0)
    1. Fixed Issues
    2. Release Information
      1. 0.9.6rc1
      2. 0.9.6pre3
      3. 0.9.6pre2
      4. 0.9.6pre1
      5. 0.9.5pre5
      6. 0.9.5pre4
      7. 0.9.5pre3
      8. 0.9.5pre2
      9. 0.9.5pre1

Tracking Development Releases of Bcfg2

Currently, we are in the process of working on 1.0. The roadmap is located at source:trunk/bcfg2/doc/1.0-roadmap

Tested setups for 1.0 SSL rewrite

Server Version Client Version Works
[5215] ​0.9.5.7 No (Server failure: Protocol Error)
[5215] ​0.9.6 Yes (with old-style key/fingerprint setup)
[5215] ​1.0pre2 Yes (with old-style key/fingerprint setup)
[5215] [5215] Yes (with old-style key/fingerprint setup)

User visible changes

In bcfg2-1.0, the following minimal versions of python will be required:

python 2.4 - server
python 2.3 - client

SSL Changes

We have switched to the in-tree ssl module included with python 2.6. A backport exists for 2.3, 2.4, and 2.5. With this, M2Crypto is not needed, and tlslite is no longer included with bcfg2 sources. Information about building the ssl pre-req is here?

bcfg2.conf changes

The generators/structures lines of bcfg2.conf have been merged into a single plugins line. Bcfg2-admin init has been updated to generate a new-style bcfg2.conf. A minimal configuration may look something like this: 

[server]
repository = /var/lib/bcfg2
plugins = Base,Bundler,Cfg,Metadata,Pkgmgr,Rules,SSHbase

[communication]
protocol = xmlrpc/ssl
password = foobat
key = /etc/bcfg2.key
# same as 'key =' line unless using a full pki
certificate = /etc/bcfg2.key

[components]
bcfg2 = https://localhost:6789

svn = yes is also deprecated. You can get the same functionality by adding Svn to your plugins line (see Svn?).

Snapshots

Snapshots?

New plugins

Service Changes

  • New overall client service mode
    • default
      • perform all service manipulations
    • disabled
      • perform no service manipulations
    • build
      • attempt to stop all services started
      • deprecates/replaces -B
  • New per service mode
    • mode='manual'
      • do not start/stop/restart this service
    • mode='default'
      • perform appropriate service ops
    • mode='supervised'
      • default + ensure service is running when verification is performed
      • deprecates supervised='true'
    • mode='custom'
      • set non-default restart target in conjunction with target attr

Release Information

1.0pre4

  • Packages
    • fix handling of architectures for RawURL YumSources
    • add parse-caching support for YumSources
    • Improve YumSource performance dramatically
    • implement caching for APT
    • implement repomd support for yum sources
    • Fix cases where arch group mappings are unexpected
    • Fix traceback with missing config.xml (Reported by Jack Neely)
    • Make cache directory creation recursive
    • Add dictionary tracking YumSource architecture mappings for cache files
    • Implement RawURL support
  • Metadata
    • Implement query interface
    • improve resolution performance
    • bcfg2-admin: Migrate query to new Metadata query interface
  • SSL
    • Implement protocol selection in bcfg2.conf xmlrpc/tlsv1 can be used with DOE grid certs
    • Improve core error handling for key existence failures
  • Server
    • Improve core error handling
    • Fix Fam support
    • Make Pseudo fam work on systems without Gamin
    • Add sshbase alias and address support
    • SimpleXMLRPCServer patch for python 2.4 (Patch from Jack Neely)
  • Packaging
    • Debian/Ubuntu? packaging update
    • Debian: postrm fix from Paul Cannon
    • Made Debian packaging work with Python 2.6 and earlier
    • Gentoo packaging updates
  • Client
    • APT: Force cache reload to get data from source updates
    • Improve error message for attempted directory unlinks (Patch from Torsten Rehn) (Resolves Ticket #657)
    • APT: fix cache invalidation
    • Add back python 2.3 fixes for {RPM,YUM}ng
    • Fix yum api use for centos 4 compatibility (Resolves Ticket #670)
    • Implement -z (for Independent entries only, a la -b) (Resolves Ticket #616)
  • Misc
    • Implement profiling support in bcfg2-info (call profile <cmd> to use)
    • Snapshots: Add --date for detailed view for a particular date
    • Fixed bug in Init.py with setting up Bcfg2-admin for the first time.
    • Cfg: Fix hardwired filename ignores (Fixes Ticket #619)
    • Improve bundler error handling for .genshi files
    • Fall back to previous XMLRPCDispatcher calling convention (Resolves Ticket #669)
  • Doc updates

1.0pre3

  • Fix pseudofam for systems without Gamin

Server Core

  • Fix Pkgmgr virtual package target binding (Reported by TimL)
  • rework File Monitoring code/adapt to new server infrastructure
  • Fix updates for VCS plugins
  • New server performance interface
    • Provides scalable aggregate performance data for server operations
  • Report deprecated plugins, improve reporting for experimental plugins
  • Implement support for .genshi bundles in Bundler
  • Packages
    • Yum improvements and bugfixes
    • Support for multi-arch yum sources
    • Implement dependency resolver debugging
    • Improve error handling
    • patch from Tim Laszlo for redhat 4 repos
  • SGenshi: improve error handling
  • Schema update from TimL (related to the service schema change)

SSL

  • We now use the ssl module included with python2.6 (this has been backported to 2.3-2.5)
  • Certificate-based authentication is supported
  • Implementation is backward compatible to 0.9.6 clients
  • strict cert auth, cert or password, or bootstrap (password once, then cert only) are supported
  • Clients now authenticate servers by commonName (not fingerprint)
  • Use of certifications require a CA to be used
  • The server is now multithreaded

Tool driver fixes

  • APT.Remove: Split up package names properly
  • Chkconfig patch to properly disable services from Ti Leggett
  • Fix RcUpdate driver regressions
  • Initial IPS (Opensolaris) driver
  • Fix portage driver traceback (Resolves Ticket #649)
  • YUMng
    • Fix for RHEL5 (patch from Tim Lazlo)
    • Fix version=auto for epoch-sensitive packages
    • Fix multi-arch yum sources
    • Fix YUMng -r behavior
  • Update RcUpdate tool driver to catch all services
  • Remove deprecated RPM and Yum drivers

Snapshots

  • Add extra/bad entry reporting
  • Add revision to bcfg2-admin snapshots reports
  • Remove ad-hoc error handling in favor of normal bcfg2-admin mode handling
  • fix Statistics data location in importer
  • minor cosmetic updates

Other

  • Add bash completion for bcfg2-admin
  • Fix daemonize exit status
  • Fix builds with the redhat specific rpm packaging
  • lots of py 2to3 and pylint updates
  • Fix py2.4 portability (try/except/finally is 2.5+) (Reported by Lisa Giacchetti)
  • Include ignores for Pkgmgr updates (patch from zultron)
  • Update bcfg2 manpage for multiple bundles
  • bcfg2 client: remove agent support
  • BB plugin updates/bugfix

Bugfixes

  • Fix fam tracebacks for Ticket #650
  • Add support for probed groups in bcfg2-admin query (Resolves Ticket #647)
  • Display diff in interactive mode (for Ticket #526)
  • Fix fd leak caused by our use of the subprocess API
  • Fix reversed options (Reported by Kamil Kisiel)
  • Logging: Fix reconnect when using /dev/log
  • Handle import errors in the help path (Resolves Ticket #653)
  • Modify bcfg2-repo-validate to warn on xml duplicates (for Ticket #643)
  • Metadata: fix default group assertion
  • Fix exit in bcfg2-info
  • Specfile fixes

1.0pre2

  • Snapshots -- New reporting system backend
    • New data model
    • Importer plugin
    • basic command line interface
  • Improve priority conflict error message
  • Schema improvements
  • Client Tool Driver Updates
    • YUMng
      • version=auto/any support
      • switch to native yum calls (improved speed, decreased code cruddyness)
    • RPMng version=any support
    • launchd update for 10.5
    • Portage driver update
  • bcfg2-info
    • pretty print generated configs
    • add profiling support
  • Packages: implement yum support
  • Init script fixes
  • doc updates
  • Server-side checksum support (client side remains to be done)
  • Cfg: delta fixes
  • Bcfg2-admin viz bugfix
  • Solaris packaging updates
  • Annotate client configurations with bind failure information
  • Refine plugin interface to include partial metadata sources
  • Tickets

1.0pre1

  • Modular version control support for repositories (plus new Git support)
  • Plugin architecture revamp (described in trunk/bcfg2/doc/plugin-roles)
    • Switch probes and properties to connector interface
  • Add new Packages plugin
  • Add support for out of tree plugins
  • Add Genshi Bundler (will be replaced by genshi support in Bundler)
  • version=any/auto support for APT/SYSV/Blast client tool drivers
  • new logo
  • Python 3k fixes

Tracking Development Releases of Bcfg2 (< 1.0)

This page provides an overview of issues users should be aware of when running Bcfg2 prereleases.

Fixed Issues

Version Description Severity Fix Revision
0.9.5pre5
Cfg state machine error Serious [3784]
importscript performance problems Serious [3795]
apt-get update is run before updates to sources.list Workaround [3785]
create-debian-pkglist mishandles arch-specific packages
when packages are only available on one arch 		Workaround [3779]
YUMng traceback with incomplete Package entries
during entry reverification 		Cosmetic [3656]
0.9.5pre3
SSHbase traceback upon new entry addition Serious [3617]
Diffs not displayed in interactive client mode Serious [3618]
Failure on error path in POSIX driver Cosmetic [3622]

Release Information

0.9.6rc1

  • Bugfixes over 0.9.6pre3
  • Model support for django 1.0

0.9.6pre3

  • NagiosGen? plugin
  • Centralized per-entry installation decision support
  • Static file monitoring infrastructure
  • Reporting system SchemaEvolution support
  • Tons of bugfixes

0.9.6pre2

  • bcfg2-reports (command line client to the reporting system)
  • Editor plugin
  • Plugin interface for metadata and statistics backends
  • bcfg2-admin query (command line interface to metadata)
  • FreeBSD rc support
  • Pkgmgr optimizations (10X speedup on python2.4+)
  • Switch server startup operations to process filesystem updates before listening on network
  • Add direct logging support for bcfg2 and bcfg2-server

0.9.6pre1

  • Refactor Cfg/TCheetah/TGenshi/SGenshi/Metadata to use common client- and group-specific entries
    • client- and group-specific entries can now be used as
      • Probes
      • TGenshi templates
      • SGenshi templates
      • TCheetah templates
  • bcfg2-admin refactor
    • Mainly conversions
    • Implement non-interactive mode for bcfg2-admin pull
  • Simplify client-side proxy code
  • Implement Multi-fingerprint support
  • Rework option handling (everything is now flat, and all option parsing occurs in the outermost calling script)
  • A variety of trunk-specific bugfixes

0.9.5pre5

  • importscript performance improvements
  • Implement pruned directories
  • APT.py - move apt-get update call to be effective during the run
  • Cfg.py bugfix - deal properly with fam event trace we previously messed up (important for some cases)
  • create-debian-pkglist - deal properly with multiarch packages when packages aren't available for all architectures
  • rpmlisting.py patch from solj (to deal with athlon arch systems)
  • YUMng/RPMng bugfixes
  • Pkgmgr - deal with comments properly in multiarch situations

0.9.5pre4

  • infoxml support has been included in TCheetah and TGenshi (now group-based file permissions and ownership can be trivially specified for any ConfigFile plugin)
  • TCheetah error information is now presented upon templating failures.
  • Probedata is now written to disk, for use with bcfg2-info (This, in conjunction with improved TCheetah templating error display should remove probe-based TCheetah templates from the "black art" category)
  • altsrc consistency checking is implemented (ie an error is produced if a configuration containing the same entry with different altsrc tags is generated for a client)
  • RPMng: Improved debug output when required attributes are not present
  • Schema updates
  • Fix diff display for non-binary ConfigFiles in interactive mode
  • Fix sshbase bug
  • Fix POSIX driver error paths

0.9.5pre3

  • support for info.xml files (described here?)
  • support for package instance mappings. (described here?)
  • added output options to bcfg2-query suitable for use with pdsh
  • fixed binary file uploads and bcfg2-admin pull
  • removed spurious -q in RPM.py
  • Improve client error message when entries are incomplete (print missing fields)
  • child process management fix in server
  • dynamic reporting system tb fix

0.9.5pre2

  • changed django reports admin to dev e-mail address
  • Add secondary config file for webservers (so that webservers don't need access to bcfg2.conf)
  • updated init script for agent mode
  • Add support for altsrc bindings to TGenshi
  • Implementing selective forking server, which runs read-only requests in child processes. Should dramatically improve scalability
  • merge realname into path for altsrc templates in TCheetah
  • Switch default to RPMng/YUMng (RPM/Yum are still around just not enabled by default)
  • quiet down RPMng a little (patch from lueningh)
  • Implement bcfg2-remote -H -
  • Add a realname attribute in altsrc bound entries and add TCheetah support
  • Add default attribute to basic group in default config
  • Fix use of altsrc when using Package entries with file attributes
  • Improve error handling in bcfg2-admin (Step towards resolving Ticket #469)
  • Switch DebInit to use invoke-rc.d (Resolves Ticket #434)
  • Fix bcfg2-info build command
  • add altsrc to schema

0.9.5pre1

  • Reporting System Features:
    • Hosts may be "expired" to prevent them from showing up in reports. Data still exists for them, so if one views a calendar date before they were expired, they'll still show up.
    • requires action The config file for reports has changed locations from the inappropriate location inside the bcfg2 python module, to be part of bcfg2.conf. Please look at the included example bcfg2.conf to see the new format
    • requires action Reporting System may be installed at any URL, that is, it can be installed in a sub directory and not at the root of a virtual host. Django 0.96 is now required.
  • Bcfg2 Agent Mode:
    • Initial release of this functionality
    • Client may run in an idle mode waiting for reconfiguration command from the server
    • Use bcfg2-remote to initiate a client's reconfiguration operation
    • Uses fingerprint based authentication for https xml-rpc connection
    • Useful for instant reconfiguration of client without ssh'ing to host
    • still need to add daemonize code and init script support (next pre)
  • Binary diff uploads now work properly
  • Added the TGenshi plugin (contributed by Jeff Ollie)
  • Added the SGenshi plugin
    • is intended for use as a structure, like bundler or base.
    • only uses the genshi xml format
    • results in completely programmable bundles
    • this could eventually replace both bundler and base, as it is
    • more powerful
  • Added entry remapping support
    • with the addition of an extra altsrc attribute, entries can be remapped to a different source.
    • For example, if you have the same config file with different paths on different architectures, you can add altsrc to use the same plugin and repository on all architectures. ie: <ConfigFile name='/etc/inet/hosts' altsrc='/etc/hosts'/> will cause all linux and solaris systems to use the same plugin source for data on all clients.
    • This feature was motivated by the need manage a bunch of config files using the same template