bcfg2 incorrectly reports "All entries correct."

[naapuri]

Suppose that a config file belonging to a .deb package is changed, eg. /etc/screenrc. Running bcfg2 without -q notices this change and re-installs the 'screen' package. However, config files are not overwritten when re-installing packages (and they shouldn't be). So bcfg2 ends up re-installing 'screen' every time it is run, and still claiming "All entries correct." even though they are not.

The best fix I can think of for this problem (and other similar problems - I think these exist!) would be to re-run bcfg2 with -n whenever bcfg2 made choices and thought that everything was fixed. This should be done internally in bcfg2, and by default.

This will slow down the bcfg2 client, but only when there is something to fix.

[desai]

Here is a proposal to fix this. We could delete config files that don't conform (we would need to run 2 passes of debsums (for package files and config files)) and then could run apt-get with --force-confmiss. That would reinstall any missing config files. If there were any deliberately missing config files, they would need to be deliberately represented in bundles, but I think this would work.

[naapuri]

I found another case where bcfg2 incorrectly reports that eveything is ok.

If the package list contains a package 'foo' but not 'bar', on which 'foo' depends, bcfg2 gets confused.

How to reproduce (Debian etch, svn build 2534):

1. Check that packages 'ytnef' and 'libytnef1' are not installed. Package 'ytnef' depends on 'libytnef1'. No further dependencies.

2. Check that all that there are no unmanaged or incorrect entries.

# bcfg2 -v -q -r all
Reading package lists... Done
Building dependency tree... Done
Reading package lists... Done
Building dependency tree... Done
Loaded tool drivers:
 APT          DebInit      POSIX        PostInstall

Phase: initial
Correct entries:        164
Incorrect entries:      0
Total managed entries:  164
Unmanaged entries:      0

All entries correct.

Phase: final
Correct entries:        164
Incorrect entries:      0
Total managed entries:  164
Unmanaged entries:      0

All entries correct.

3. Add 'ytnef' to the bcfg2 package configuration for this host.

4. Run bcfg2:

# bcfg2 -v -q -r all
Reading package lists... Done
Building dependency tree... Done
Reading package lists... Done
Building dependency tree... Done
Loaded tool drivers:
 APT          DebInit      POSIX        PostInstall

Phase: initial
Correct entries:        164
Incorrect entries:      1
Total managed entries:  165
Unmanaged entries:      0

Trying single pass package install for pkgtype deb
Single Pass Succeded
Reading package lists... Done
Building dependency tree... Done
The Following Bundles have been modifed:



Phase: final
Correct entries:        165
Incorrect entries:      0
Total managed entries:  165
Unmanaged entries:      0

All entries correct.

5. Actually two packages were installed:

# dpkg -l *ytnef*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name                               Version                            Description
+++-==================================-==================================-====================================================================================
ii  libytnef0                          1.5-1                              improved decoder for application/ms-tnef attachments
ii  ytnef                              2.6-1                              improved decoder for application/ms-tnef attachments

Now there is an unmanaged entry (libytnef0) even though the latest bcfg2 run says there aren't any.

6. Now run bcfg2 again:

# bcfg2 -v -q -r all
Reading package lists... Done
Building dependency tree... Done
Reading package lists... Done
Building dependency tree... Done
Loaded tool drivers:
 APT          DebInit      POSIX        PostInstall

Phase: initial
Correct entries:        165
Incorrect entries:      0
Total managed entries:  165
Unmanaged entries:      1

Removing packages:
libytnef0
Reading package lists... Done
Building dependency tree... Done

Phase: final
Correct entries:        165
Incorrect entries:      0
Total managed entries:  165
Unmanaged entries:      0

All entries correct.

First it finds the unmanaged entry and tries to uninstall it. However, uninstalling the libytnef0 package also uninstalls 'ytnef', but bcfg2 doesn't notice this. The 'ytnef' package is now missing, but bcfg2 insists that everything is ok!

# dpkg -l *ytnef*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name                               Version                            Description
+++-==================================-==================================-====================================================================================
pn  libytnef0                          <none>                             (no description available)
pn  ytnef                              <none>                             (no description available)

---

In this example, dependencies are the problem. The server should be aware of the dependency graph. (Does the code for that exist yet?) But IMO, even with incorrect/missing dependency data the bcfg2 client should never claim that eveything is ok if it's clearly not.

[naapuri]

Another somewhat related instance of this issue.

I had a custom-made .deb package custom-jed-conf that installed config files under /etc/jed-root/.

Then I realized I'd better manage these config files via bcfg2's Cfg plugin. I imported the files there and removed the custom-jed-conf package from Base/*.xml. So the next bcfg2 client run should uninstall the package and install these config files via Cfg.

First run:

# bcfg2 -d -v -q -r all
{'profile': False, 'kevlar': False, 'interactive': False, 'dryrun': False, 'verbose': True, 'paranoid': False, 'setup': '/etc/bcfg2.conf', 'cache': False, 'remove': 'all', 'server': 'https://x.x.x.x:6789', 'quick': True, 'bundle': False, 'file': False, 'debug': True, 'build': False, 'password': 'xxx', 'help': False}
GetProbes completed successfully
GetConfig completed successfully
Reading package lists... Done
Building dependency tree... Done
Reading package lists... Done
Building dependency tree... Done
Loaded tool drivers:
 APT          DebInit      POSIX        PostInstall

Phase: initial
Correct entries:        496
Incorrect entries:      0
Total managed entries:  496
Unmanaged entries:      1

Removing packages:
custom-jed-conf
> apt-get remove -y --force-yes custom-jed-conf
< Reading package lists...
< Building dependency tree...
< The following packages will be REMOVED:
<   custom-jed-conf
< 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
< Need to get 0B of archives.
< After unpacking 2601kB disk space will be freed.
< (Reading database ... 11940 files and directories currently installed.)
< Removing custom-jed-conf ...
< dpkg - warning: while removing custom-jed-conf, directory `/etc/jed-root/lib' not empty so not removed.
< dpkg - warning: while removing custom-jed-conf, directory `/etc/jed-root' not empty so not removed.
Reading package lists... Done
Building dependency tree... Done

Phase: final
Correct entries:        496
Incorrect entries:      0
Total managed entries:  496
Unmanaged entries:      0

All entries correct.
RecvStats completed successfully

After this, the config files do not exist, but bcfg2 says everything is ok.

Second run installs the config files just fine:

# bcfg2 -d -v -q -r all
{'profile': False, 'kevlar': False, 'interactive': False, 'dryrun': False, 'verbose': True, 'paranoid': False, 'setup': '/etc/bcfg2.conf', 'cache': False, 'remove': 'all', 'server': 'https://x.x.x.x:6789', 'quick': True, 'bundle': False, 'file': False, 'debug': True, 'build': False, 'password': 'xxx', 'help': False}
GetProbes completed successfully
GetConfig completed successfully
Reading package lists... Done
Building dependency tree... Done
Reading package lists... Done
Building dependency tree... Done
Loaded tool drivers:
 APT          DebInit      POSIX        PostInstall
ConfigFile /etc/jed-root/lib/isearch.slc does not exist
(many more of these...)

Phase: initial
Correct entries:        179
Incorrect entries:      317
Total managed entries:  496
Unmanaged entries:      0

Installing ConfigFile /etc/jed-root/lib/docbook.sl
(many more of these...)
The Following Bundles have been modifed:



Phase: final
Correct entries:        496
Incorrect entries:      0
Total managed entries:  496
Unmanaged entries:      0

All entries correct.
RecvStats completed successfully

I still think that bcfg2 should dry-run after any changed to see that everything is fine. At least there should be an option. One of the worst things a configuration engine can do is falsely claim that everything is ok.

[desai]

I think that [bef7adca2083020a103eee4baf6fadc4c4537344] (SVN r2546) should resolve the pending bits of this.