Modify

Ticket #663 (closed defect: fixed)

Opened 12 years ago

Last modified 12 years ago

Running bcfg2s-server for a few days results in 'Too many open files' crash

Reported by: nhalpern@… Owned by: desai
Priority: major Milestone: Bcfg2 1.0.0 Release
Component: bcfg2-server Version: 1.0
Keywords: Cc:

Description

After upgrading from v0.9.6 to v1.0pre2, bcfg2-server crashes every few days with 'too many open files'.

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 8192
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8192
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
May 18 18:07:04 tauon bcfg2-server[7059]: Processed 1 gamin events in 0.000 seconds. 0 collapsed
May 18 18:07:04 tauon bcfg2-server[7059]: error in service loop Traceback (most recent call last):   File "/usr/sbin/bcfg2-server", line 243, in ?     BSERV.serve_forever()   File "/usr/lib/python2.4/site-packages/Bcfg2/Component.py", line 317, in serve_forever     self.handle_request()   File "/usr/lib64/python2.4/SocketServer.py", line 217, in handle_request     request, client_address = self.get_request()   File "/usr/sbin/bcfg2-server", line 102, in get_request     self.Core.Service()   File "/usr/lib/python2.4/site-packages/Bcfg2/Server/Core.py", line 222, in Service     self.revision = plugin.get_revision()   File "/usr/lib/python2.4/site-packages/Bcfg2/Server/Plugins/Svn.py", line 37, in get_revision     stdout=PIPE).stdout.readlines()   File "/usr/lib64/python2.4/subprocess.py", line 542, in __init__     errread, errwrite)   File "/usr/lib64/python2.4/subprocess.py", line 899, in _execute_child     errpipe_read, errpipe_write = os.pipe() OSError: [Errno 24] Too many open files
May 11 07:40:59 tauon bcfg2-server[0f07fb09115d71990bcf3844782ce4174f2158ff] (SVN r5584): error in service loop Traceback (most recent call last):   File "/usr/sbin/bcfg2-server", line 243, in ?     BSERV.serve_forever()   File "/usr/lib/python2.4/site-packages/Bcfg2/Component.py", line 317, in serve_forever     self.handle_request()   File "/usr/lib64/python2.4/SocketServer.py", line 217, in handle_request     request, client_address = self.get_request()   File "/usr/sbin/bcfg2-server", line 102, in get_request     self.Core.Service()   File "/usr/lib/python2.4/site-packages/Bcfg2/Server/Core.py", line 222, in Service     self.revision = plugin.get_revision()   File "/usr/lib/python2.4/site-packages/Bcfg2/Server/Plugins/Svn.py", line 37, in get_revision     stdout=PIPE).stdout.readlines()   File "/usr/lib64/python2.4/subprocess.py", line 542, in __init__     errread, errwrite)   File "/usr/lib64/python2.4/subprocess.py", line 899, in _execute_child     errpipe_read, errpipe_write = os.pipe() OSError: [Errno 24] Too many open files

Attachments

bcfg2-server-lsof.txt (27.8 KB) - added by nhalpern@… 12 years ago.
lsof output of a freshly started instance.

Change History

Changed 12 years ago by nhalpern@…

lsof output of a freshly started instance.

comment:1 Changed 12 years ago by anonymous

Same instance, 24 hours later: $ sudo /usr/sbin/lsof -n -p $(pgrep bcfg2-server) | wc -l 873 $ sudo /usr/sbin/lsof -n -p $(pgrep bcfg2-server) | grep pipe | wc -l 798

comment:2 Changed 12 years ago by desai

  • Status changed from new to closed
  • Resolution set to fixed

WARNING! You need to establish a session before you can create or edit tickets. Otherwise the ticket will get treated as spam.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.