[SOLVED] Slow and stuck server without reason

[penguinjr]

Hello everybody!

I am new in Zentyal and I am getting a problem with it. Sometimes while I am just in the web panel or with ssh entering in a directory my server starts to be very slow.

If in that moment I try to CPU what is in the CPU running I can't see anything but if I am in front of the server desktop I can see in the panel plugin of LXDE the CPU usage widget totally green and using a high CPU.

Im suspecting the problem can be from the hard disk set up but I am using the same from the beginning of Zentyal 3.2 and before I hadn't this problem. Other thing I was wondering it was the antivirus, but the module is disabled.


I am running Zentyal 3.2.4 x86 over a HP DL 145 G3, inside of it I have two SATA disk Caviar Green with a RAID1 Software made with Zentyal installer.

This my zentyal.log: http://pastebin.com/DLabei0w
My list of packages:

Code: [Select]

ii  language-pack-zentyal-es             3.2.1                                   Zentyal translations for language Spanish
ii  zentyal-antivirus                    3.2                                     Zentyal - Antivirus
ii  zentyal-bwmonitor                    3.2                                     Zentyal - Bandwidth Monitor
ii  zentyal-ca                           3.2                                     Zentyal - Certification Authority
ii  zentyal-common                       3.2.1                                   Zentyal - Common Library
ii  zentyal-core                         3.2.4                                   Zentyal - Core
ii  zentyal-dhcp                         3.2.1                                   Zentyal - DHCP Service
ii  zentyal-dns                          3.2.4                                   Zentyal - DNS Service
ii  zentyal-ebackup                      3.2                                     Zentyal - Backup
ii  zentyal-firewall                     3.2                                     Zentyal - Firewall
ii  zentyal-ftp                          3.2.1                                   Zentyal - FTP
ii  zentyal-gateway                      3.2                                     Zentyal - Gateway Suite
ii  zentyal-infrastructure               3.2                                     Zentyal - Network Infrastructure Suite
ii  zentyal-ips                          3.2                                     Zentyal - Intrusion Prevention System
ii  zentyal-ipsec                        3.2                                     Zentyal - IPsec and L2TP/IPsec
ii  zentyal-jabber                       3.2                                     Zentyal - Jabber (Instant Messaging)
ii  zentyal-l7-protocols                 3.2                                     Zentyal - Layer-7 Filter
ii  zentyal-monitor                      3.2                                     Zentyal - Monitor
ii  zentyal-network                      3.2.3                                   Zentyal - Network Configuration
ii  zentyal-ntp                          3.2                                     Zentyal - NTP Service
ii  zentyal-objects                      3.2                                     Zentyal - Network Objects
ii  zentyal-openvpn                      3.2                                     Zentyal - VPN Service
ii  zentyal-remoteservices               3.2.1                                   Zentyal - Cloud Client
ii  zentyal-samba                        3.2.8                                   Zentyal - File Sharing and Domain Services
ii  zentyal-services                     3.2                                     Zentyal - Network Services
ii  zentyal-software                     3.2.1                                   Zentyal - Software Management
ii  zentyal-squid                        3.2                                     Zentyal - HTTP Proxy (Cache and Filter)
ii  zentyal-trafficshaping               3.2.1                                   Zentyal - Traffic Shaping
ii  zentyal-users                        3.2.2                                   Zentyal - Users and Computers
ii  zentyal-webserver                    3.2                                     Zentyal - Web Server


Thank you in advance! Because I was trying a lot of things and I am a bit lost with this.

[ctek]

Did you take a look at "top" from console ?
And how much ram do you have ?

Regards
Bogdan

[penguinjr]

Yes, I did, but I didn't see anything extrange.

But right know for example.

I tried to enter in the web panel (https://LOCAL_IP) I wrote my username / password and click enter and now is stucked! From ssh y try to see htop and it takes 1minute to execute it.

http://i.imgur.com/ILMt3aE.jpg

[robb]

How much memory does your server have? You notice a lot of swapping? I would recommend at least 2GB (more = better) for Zentyal 3.2

[penguinjr]

Hello Robb,

In the image I posted in the post before appear. I have 3GB of RAM and 7629MB of Swap.

http://i.imgur.com/ILMt3aE.jpg

The SWAP file is /dev/sdaN and /dev/sdbN, same amount in each disk and partition and It is outside of Software-RAID1.

[christian]

Indeed load average is > 6 (short).
Would you mind showing top instead of htop ?
How many CPU do you have there ?

[penguinjr]

I think is a AMD Opteron dual core, low end CPU, but at least is dual core.

...


More tests...

administrador@UPA-SERV-03F:~$ sudo hdparm -tT /dev/sda
/dev/sda:
 Timing cached reads:     2 MB in 22.62 seconds =  90.55 kB/sec
........ (and stucked! )

So the problem defitnely is in the disks. But I don't understand because It were working well.

[robb]

Here a 'load for dummies' explanation: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages
load of 6 is bad news...

[BrettonWoods]

Whilst you are having a look at top or htop have you noticed how much s4sync hits the system.

Anyone got a system with a lot of computers and users?

Would be interested in the results my procs ramp up to 60% on s4sync

Its also right about suricata but that just ticks away at about 5%

Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 4 cores
Real memory   1.60 GB used, 7.80 GB total

[penguinjr]

This is the list of the 5 higher cpu use proccess:

administrador@UPA-SERV-03F:~$ ps -e -o pcpu,state,args --sort pcpu | tail -n 5
 0.1 S /usr/bin/X :0 vt07 -nolisten tcp
 0.1 S /usr/sbin/named -u bind -4
 0.2 S [md1_raid1]
 0.6 S /usr/lib/firefox/firefox https://localhost
 8.1 S /usr/bin/perl /usr/share/zentyal-samba/s4sync


and its appear s4sync...

Actually I don't have too much users. (a few, for FTP usings...).


Can have the problem with s4sync any relation with this?

Code: [Select]

m line 204
        eval {...} called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
        ModPerl::RegistryCooker::run('ModPerl::Registry=HASH(0xbc8700a0)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 170
        ModPerl::RegistryCooker::default_handler('ModPerl::Registry=HASH(0xbc8700a0)') called at /usr/lib/perl5/ModPerl/Registry.pm line 31
        ModPerl::Registry::handler('ModPerl::Registry', 'Apache2::RequestRec=SCALAR(0xbc86fe5c)') called at -e line 0
        eval {...} called at -e line 0
2013/11/13 10:36:55 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (100 attempts)
2013/11/13 10:37:05 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (200 attempts)
2013/11/13 10:37:15 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (300 attempts)
2013/11/13 10:37:15 DEBUG> LDB.pm:201 EBox::LDB::safeConnect - FATAL: No es posible conectar con el servidor LDAP de samba: connect: Connection refused at /usr/share/perl5/EBox/LDB.pm line 201
        EBox::LDB::safeConnect('EBox::LDB=HASH(0xa7b73c0)') called at /usr/share/perl5/EBox/LDB.pm line 155
        EBox::LDB::connection('EBox::LDB=HASH(0xa7b73c0)') called at /usr/share/perl5/EBox/LDAPBase.pm line 95
        EBox::LDAPBase::search('EBox::LDB=HASH(0xa7b73c0)', 'HASH(0xa7bfb84)') called at /usr/share/perl5/EBox/LDB.pm line 272
        EBox::LDB::domainSID('EBox::LDB=HASH(0xa7b73c0)') called at /usr/share/zentyal-samba/s4sync line 71
2013/11/13 10:37:47 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (1 attempts)
2013/11/13 10:37:57 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (100 attempts)
2013/11/13 10:38:07 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (200 attempts)
2013/11/13 10:38:17 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (300 attempts)
2013/11/13 10:38:17 DEBUG> LDB.pm:201 EBox::LDB::safeConnect - FATAL: No es posible conectar con el servidor LDAP de samba: connect: Connection refused at /usr/share/perl5/EBox/LDB.pm line 201
        EBox::LDB::safeConnect('EBox::LDB=HASH(0xac683c0)') called at /usr/share/perl5/EBox/LDB.pm line 155
        EBox::LDB::connection('EBox::LDB=HASH(0xac683c0)') called at /usr/share/perl5/EBox/LDAPBase.pm line 95
        EBox::LDAPBase::search('EBox::LDB=HASH(0xac683c0)', 'HASH(0xac70b84)') called at /usr/share/perl5/EBox/LDB.pm line 272
        EBox::LDB::domainSID('EBox::LDB=HASH(0xac683c0)') called at /usr/share/zentyal-samba/s4sync line 71
2013/11/13 10:39:05 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (1 attempts)
2013/11/13 10:39:22 WARN> LDB.pm:197 EBox::LDB::safeConnect - Could not connect to Samba LDB: connect: Connection refused, retrying. (100 attempts)


[BrettonWoods]

That is your problem but what the answer is I am unsure.

I think you might be better doing a clean install.

As it doesn't look like the latest chmod bug

I have just been surprised at how much the ldap samba sync is putting on the system.

Yours is broken

[penguinjr]

This is my CPU, I thought it was a dual core  . Its a single core/thread CPU.

Opteron 248 - http://www.cpu-world.com/CPUs/K8/AMD-Opteron%20248%20-%20OSA248CEP5AU%20(OSA248AUBOX%20-%20OSA248AUWOF).html

BrettonWoods

¿¿Really?? Actually I can do it... but... If there is another solution. Re-install just partially (samba, and users groups)

[BrettonWoods]

Keeping trawling through your logs and see if it connects.

What you could do is disable the samba and users and see how things go.

I have never had much success purging and reinstalling compared to a clean install.

 

[penguinjr]

Yes. I am just testing disablen Users and Groups and everything is working well.

For example now, this is my hdparm and write speeds over the RAID1

administrador@UPA-SERV-03F:~$ sudo hdparm -tT /dev/sdb

/dev/sdb:
 Timing cached reads:   1876 MB in  2.00 seconds = 937.71 MB/sec
 Timing buffered disk reads: 344 MB in  3.01 seconds = 114.33 MB/sec
administrador@UPA-SERV-03F:~$ sudo hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   1826 MB in  2.00 seconds = 913.63 MB/sec
 Timing buffered disk reads: read(2097152) returned 270336 bytes
administrador@UPA-SERV-03F:~$ dd if=/dev/zero of=testing.file bs=8k count=512k
524288+0 records in
524288+0 records out
4294967296 bytes (4,3 GB) copied, 75,7688 s, 56,7 MB/s

[jbahillo]

Hi there:

I think the issue might come from I/O. It would be worthy to check percentage of wa processes in top, and if so try using some took like iotop to find which process is provoking all that IO

BR