Mailing List CGatePro@mail.stalker.com Message #92665
From: Christian Ellsworth Capo <k@ellsworth.cl>
Subject: Re: Filesystem layout and CGP performance
Date: Wed, 19 Sep 2007 22:33:05 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>
Jorge R. Macías L. wrote:

8<------------------------------------------------------------------------------
volume group formed with 5 FC 10Krpms 300 GB disks on RAID 5, the last one on a similar volume group but with 15Krpms disks.

Should I keep on using ext3?  Should I keep on using Redhat Enterprise 4? Do I upgrade to Entreprise 5?  Do I use any specific parameters for the kernel or for ext3 when I format the partitions in order to get a better performance?

Any suggestions?
Well Jorge... let me see if i can be of help

Filesystem:
Ext3 is a very stable filesystem, not the absolut fastest but in my "Point of View",but it is the best relation in performance/data integrity/support/tunning.

- EXT3 and many other filesystems beyond 90% of usage starts to have a performance hit. over 95% the IOwait levels of the OS are Skyhigh. Always keep at least a 20% free of the filesystem... it will help -trust me-.

-To make EXT3 faster you can play with a couple of mount parameters and some kernel flags.
in the mount options for each filesystem add: "noatime" this will ease at least a 20% of your current IOp/s (linux is not wasting IO in updating the last time the file was accessed, communigate uses the account.info file for that.. so is safe to do this trick.) still check the "man mount" info about noatime.

- The Kernel bootline you can use "elevator=deadline" in RHEL4 a kernel entry of my grub config is :

   title Red Hat Enterprise Linux AS (2.6.9-55.0.6.ELsmp)
           root (hd0,0)
           kernel /vmlinuz-2.6.9-55.0.6.ELsmp ro root=LABEL=/
   elevator=deadline
           initrd /initrd-2.6.9-55.0.6.ELsmp.img

   this will change the default kernel scheduller from CFQ to DEADLINE,
   making the system a lot more responsive in heavy load environments.


- Also check in the systems log about messages of not enough files... perhaps the OS needs a little extra tunning. of numbers of files open. (sysctl -a | grep fs.file-max) a heavy loaded systenm should have more that 131k  I use in the large cluster 256k. and tweak the networking too.
- Using recyclable temp files, helps in the communigate standpoint.
- If you have the filesystem space, lower the CGP threshold for indexing the mailboxes (it uses a little more space but saves IOp/s)

Hardware:
RAID5 is one of the slowest raid configurations, it doesn't sacrifice that much capacity but it's not fast. even in enterprise class storage systems.  the best performance would be a RAID10 (raid 1+0), this sacrifieces more capacity but you gain double the access speed, and better write times.

I would recommend in the new storage switching away from RAID5 to RAID10. depending also on the FC HBA maker/model/driver, check the tunning posible in command queues (this is a lot more advanced. so don't do nothing that you are not sure how it will affect the system).

my $0.02

I hope it'll help you hold until the hardware is working.

Christian Ellsworth

Thanks and best regards

Jorge

#############################################################
This message is sent to you because you are subscribed to
 the mailing list <CGatePro@mail.stalker.com>.
To unsubscribe, E-mail to: <CGatePro-off@mail.stalker.com>
To switch to the DIGEST mode, E-mail to <CGatePro-digest@mail.stalker.com>
To switch to the INDEX mode, E-mail to <CGatePro-index@mail.stalker.com>
Send administrative queries to  <CGatePro-request@mail.stalker.com>

Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster