Mailing List CGatePro@mail.stalker.com Message #92338
From: Robinson Maureira Castillo <rmaureira@solint.cl>
Subject: Re: Too many files open in this process
Date: Mon, 27 Aug 2007 18:07:08 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A more friendly way to do it

Create a file Startup.sh on your CGP base directory (usually
/var/CommuniGate on Linux), and put some of those parameters there.

# Begin Startup.sh
ulimit -n 65536
SUPPLPARAMS="--ClusterBackend  --DefaultStackSize 131072
- --AllowUnknownAccess --ForeignStatic --closeStuckSockets --useNonBlo
ckingSockets --CreateTempFilesDirectly 50"
# End Startup.sh

Startup.sh gets sourced everytime you start the CGP service from its
init script. This way you can update CGP without worrying about changes
on the init script.

The SUPPLPARAMS part, can be adjusted to match your environment, on our
deployments the --useNonBlockingSockets and --CreateTempFilesDirectly
switches had a nice impact on performance.

The other parameters can be put on /etc/sysctl.conf in the following format:

fs.file-max = 131072
kernel.threads-max = 131072
net.ipv4.ip_local_port_range=1024 65000
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_max = 262144
net.core.netdev_max_backlog = 4096

Then simply run sysctl -p to activate the changes, sysctl -p is run at
system boot, so there's no need to run it before starting CGP.

Best regards,

Wayde Nie wrote:
> On Mon, 27 Aug 2007 13:31:07 -0500
>  "Jason Hooper" <jhooper@tlcontact.com> wrote:
>> I just add :
>>
>> ulimit -n 16380
>> ulimit -u 2000
>> ulimit -c 2097151
>> umask 0
>>
>> to /etc/rc.d/init.d/CommuniGate
>>
>
> Yes, I think this is the gist of the OP's problem scaling up...
>
> After I sent my last message, I did a quick search thorough the CGatePro
> Help and found the following in the "/Guide/Scalability.html" page:
>
>> Following are some additional tuning options for Linux 2.6. For most
>> Linux distributions, these shell commands should be placed into a boot
>> script to be run at system startup. RedHat and a few other
>> distributions may also provide a facility to configure "sysctl" data
>> in the file /etc/sysctl.conf:
>>
> #!/bin/sh
> # Linux 2.6 tuning script
> # max open files
> echo  131072 > /proc/sys/fs/file-max
> # kernel threads
> echo 131072 > /proc/sys/kernel/threads-max
> # socket buffers
> echo 65536 > /proc/sys/net/core/wmem_default
> echo 1048576 > /proc/sys/net/core/wmem_max
> echo 65536 > /proc/sys/net/core/rmem_default
> echo 1048576 > /proc/sys/net/core/rmem_max
> # netdev backlog
> echo 4096 > /proc/sys/net/core/netdev_max_backlog
> # socket buckets
> echo 131072 > /proc/sys/net/ipv4/tcp_max_tw_buckets
> # port range
> echo '16384 65535' > /proc/sys/net/ipv4/ip_local_port_range
>
> Wayde Nie.
>
>> -----Original Message-----
>> From: CommuniGate Pro Discussions [mailto:CGatePro@mail.stalker.com]
>> On Behalf Of Wayde Nie
>> Sent: Monday, August 27, 2007 11:59 AM
>> To: CommuniGate Pro Discussions
>> Subject: Re: Too many files open in this process
>>
>> On Mon, 27 Aug 2007 18:33:54 +0300
>>  Wael Shahin <wael.shahin@gmail.com> wrote:
>>> 21:27:35.166 1 SMTPI-057956([192.168.1.195]) temp file cannot be
>>> created. Error Code=failed to create a temp file
>>> 21:27:44.984 0 SYSTEM server 5.1.11 stopped under Linux
>>> 21:28:35.658 0 SYSTEM server 5.1.11 started under Linux, open files
>>> limit=1024
>>> 21:28:35.658 0 SYSTEM process core dump limit=2047M
>>>
>>> I would appreciate any comments for the above
>>
>> What values does the following return?
>>
>>   ulimit -n
>>
>>   cat /proc/sys/fs/file-max
>>
>>   grep nofile /etc/security/limits.conf
>>
>> Looks like something's limiting the max number of open files to 1024
>> and to support the number of clients in your test, you need more than
>> that...
>>
>> --
>> Wayde Nie.
>>
>> #############################################################
>> This message is sent to you because you are subscribed to
>>  the mailing list <CGatePro@mail.stalker.com>.
>> To unsubscribe, E-mail to: <CGatePro-off@mail.stalker.com>
>> To switch to the DIGEST mode, E-mail to
>> <CGatePro-digest@mail.stalker.com>
>> To switch to the INDEX mode, E-mail to <CGatePro-index@mail.stalker.com>
>> Send administrative queries to  <CGatePro-request@mail.stalker.com>
>>
>>
>>
>>
>> #############################################################
>> This message is sent to you because you are subscribed to
>>  the mailing list <CGatePro@mail.stalker.com>.
>> To unsubscribe, E-mail to: <CGatePro-off@mail.stalker.com>
>> To switch to the DIGEST mode, E-mail to
>> <CGatePro-digest@mail.stalker.com>
>> To switch to the INDEX mode, E-mail to <CGatePro-index@mail.stalker.com>
>> Send administrative queries to  <CGatePro-request@mail.stalker.com>
>
> --
> Wayde Nie.
>
> #############################################################
> This message is sent to you because you are subscribed to
>  the mailing list <CGatePro@mail.stalker.com>.
> To unsubscribe, E-mail to: <CGatePro-off@mail.stalker.com>
> To switch to the DIGEST mode, E-mail to <CGatePro-digest@mail.stalker.com>
> To switch to the INDEX mode, E-mail to <CGatePro-index@mail.stalker.com>
> Send administrative queries to  <CGatePro-request@mail.stalker.com>

- --
Robinson Maureira Castillo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFG00sMu+2kmA0sEb4RAl0eAJ9GhtO1l+SelXlzqMqyC9OGKc/D5wCfXsTL
ArEJbVts8+0DTetYqpeNEn8=
=rbCi
-----END PGP SIGNATURE-----
Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster