Ticket #954 (closed defect: fixed)

Opened 12 years ago

Last modified 11 years ago

web_debug = true in bcfg2.conf causes memory over usage

Reported by: Owned by:
Priority: major Milestone: Bcfg2 1.2.1 Release (Bugfix)
Component: bcfg2-server Version: 1.0
Keywords: memory dbstats Cc:


When running tests using guppy. Adding this into bcfg2 code somewhere:

from guppy.heapy import Remote Remote.on()

Then from another window ran:

$ python -c "from guppy import hpy;hpy().monitor()" <Monitor> * Connection 1 opened * <Monitor> lc CID PID ARGV

1 12075 ['/usr/sbin/bcfg2-server', '-d']

<Monitor> sc 1 Remote connection 1. To return to Monitor, type <Ctrl-C> or .<RETURN> <Annex> int Remote interactive console. To return to Annex, type '-'.


At this point the memory counters are reset and I ran a bcfg2 client update on another server. Next taking a tip from a blog post about analysing django memory leaks[f5e4281313ff9004be488e420f90d5fe413bb5c1] (SVN r2) I run a method to extract information about the references hierarchy. This is as the post states magic!


Reference Pattern by <[dict of] class>.

0: _ --- [-] 1134 (Bcfg2.Server.Plugins.Metadata.ClientMetadata? |


1: a [-] 351 dict (no owner): 0xf083c64*2, 0xfcfc46c*2, 0xfe7dacc*2... 2: aa ---- [-] 2 list: 0x10066aac*4914, 0x1043a82c*8 3: a3 [-] 1 dict of

django.db.backends.mysql.base.DatabaseWrapper?: 0x8...

4: a4 ------ [-] 1 dict (no owner): 0x104992d4*3 5: a5 [R] 1 guppy.heapy.heapyc.RootStateType?: 0xf58f38acL 6: a3b ----- [-] 1 django.db.backends.mysql.base.DatabaseWrapper?: 0x8625194 7: a3ba [S] 12 dict of module: ..db..., ..importscript, ..models 8: a3bb ----- [-] 2 dict of


9: a3bba [-] 2

django.db.backends.mysql.creation.DatabaseCreation?: 0x8... <Type e.g. '_.more' for more.>

"Django keeps track of all queries for debugging purposes (connection.queries). This list is reset at the end of HTTP request. But in standalone mode, there are no requests. So you need to manually reset to queries list after each working cycle."

Django is only caching these queries if it's in debug mode. I have web_debug = true in my /etc/bcfg2.conf. I've set this to False and have reran some of the tests and the memory usage is much different.

Memory increase between client runs with web_debug:

False: 118856 bytes True: 1303532 bytes

So that's 1.2Mb increase per client update.


Change History

comment:1 Changed 12 years ago by laszlo

  • Owner changed from desai to laszlo
  • Status changed from new to assigned

comment:2 Changed 12 years ago by solj

  • Milestone changed from Bcfg2 1.2.0 Release to Bcfg2 1.2.1 Release (Bugfix)

comment:3 Changed 11 years ago by

  • Owner changed from laszlo to
  • Status changed from assigned to accepted

comment:4 Changed 11 years ago by

  • Status changed from accepted to closed
  • Resolution set to fixed

As I understand it, the memory leak caused by web_debug is pretty unavoidable; lots of Django things warn about memory leaks if DEBUG is true. Consequently, I have:

  • Disabled web_debug in the default bcfg2.conf created by 'bcfg2-admin init';
  • Disabled web_debug in the sample bcfg2.conf files referenced in the setup guides; and
  • Added a warning if web_debug is enabled.

WARNING! You need to establish a session before you can create or edit tickets. Otherwise the ticket will get treated as spam.


Add a comment

Modify Ticket

Change Properties
<Author field>
as closed
The resolution will be deleted. Next status will be 'reopened'

E-mail address and user name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.