[Zarafa Indexer] High CPU load

[Marcus]

Hello,
I faced a problem last week. 

I decided to do a stress test on my email server.

There goes the spec of the Zentyal 2.0 box:
Kernel and CPU    :            Linux 2.6.32-28-generic on x86_64
Processor information :     AMD Athlon(tm) 64 X2 Dual Core Processor 5400+, 2 cores
RAM:                                 2GB DDR2 800MHz
Hard Drives :                     RAID 10 w/ 4X1.0TB (SATA-II, WD Green)

Test
Total emails :                   35,000
Sending from :                 2 servers
Email type:                      text
Attachment :                    no


Send/Receive took about 10 minutes over a 8Mbps WiFi and 32Mbps CPL networks (real speed).

CPU and RAM on the Zarafa box was all fine until Zarafa's indexer kicked in >>  CPU jumped to a load of 16.0.  I had to put a stop to it after an hour of non sens CPU load so I restarted the indexer.  Everything was fine for something like 5 minutes then the load came back.

The indexer problem came from 2 obvious facts:
1) Much email came in simultaneously
2) Zarafa indexer was set to 5 minutes

The second observation is the most important one.  Not having the time to index everything within the default configuration, Zarafa was queuing indexing requests, bringing the load to the roof.

In order to fix the problem and avoid any problems related with a spam flood, I had to modify the configuration of the indexer.  This way, Zarafa indexer was able to finish the indexing process before the next one and keeping the CPU load to an acceptable level.

This is how I change the indexer from 5 minutes to 30 minutes:

In a terminal as root user

Code: (terminal) [Select]

# cd / && find . -name 'indexer.cf*' | xargs perl -pi -e 's/index_interval = 5/index_interval = 30/g' && zarafa-indexer restart
Explaining this command
It will look for 2 files on the server:
1) /etc/zarafa/indexer.cfg
2) /usr/share/ebox/stubs/zarafa/indexer.cfg.mas

Then, it will change the lines

Code: [Select]

index_interval = 5for

Code: [Select]

index_interval = 30
And at last, it will restart the Zarafa indexer service.


Note that you may have to repeat this after a Zentyal update (if zarafa module get an update)


Having all that said, it would be great to have a dropdown menu under:

Zentyal >> Groupware >> General configuration settings

to tweak the indexing frequency.


Best,

Marcus

[jsalamero]

Thanks for your suggestion, will consider introduce a conf key for this.