[SOLVED] IP process is 100% CPU since last security update for Zentyal 4.1

[expertgeeks]

i've updated (long time ago) my servers to Ubuntu Vivid (15.04) LTS enablement stacks (actually kernel version:  3.19.0-33),
works without any issues like a charm!
https://wiki.ubuntu.com/Kernel/LTSEnablementStack

DESKTOP (if you use GUI (lxde) on server):

Code: [Select]

sudo apt-get install --install-recommends linux-generic-lts-vivid xserver-xorg-core-lts-vivid xserver-xorg-lts-vivid xserver-xorg-video-all-lts-vivid xserver-xorg-input-all-lts-vivid libwayland-egl1-mesa-lts-vivid
SERVER (if your server running headless (without GUI)):

Code: [Select]

sudo apt-get install --install-recommends linux-generic-lts-vivid

Have you upgraded to Zentyal 4.2 with vivid without problems ?

[julio]

i've updated (long time ago) my servers to Ubuntu Vivid (15.04) LTS enablement stacks (actually kernel version:  3.19.0-33),
works without any issues like a charm!
https://wiki.ubuntu.com/Kernel/LTSEnablementStack

DESKTOP (if you use GUI (lxde) on server):

Code: [Select]

sudo apt-get install --install-recommends linux-generic-lts-vivid xserver-xorg-core-lts-vivid xserver-xorg-lts-vivid xserver-xorg-video-all-lts-vivid xserver-xorg-input-all-lts-vivid libwayland-egl1-mesa-lts-vivid
SERVER (if your server running headless (without GUI)):

Code: [Select]

sudo apt-get install --install-recommends linux-generic-lts-vivid

Have you upgraded to Zentyal 4.2 with vivid without problems ?

yes, without problems...

[cpt.charisma]

Just a little more background & info:

This just got backported to 4.0, so I got to deal with it today.  In addition to IP, the Cut process was also at 100%.  It prevented DNS, DHCP, SAMBA, NTP and other important things from loading.

Here are Ubuntu bug reports on the issue:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1516052
https://bugs.launchpad.net/ubuntu/+source/linux-lts-utopic/+bug/1514785

It looks like there are several kernel versions affected, so this may come up again with future upgrades.

It stems from a bug in the kernel that makes the ip command output the first rule infinitely.  You can use this command to see if you're affected:
ip route ls

Broken Output:
0:   from all lookup local
0:   from all lookup local
0:   from all lookup local
0:   from all lookup local
0:   from all lookup local
0:   from all lookup local
0:   from all lookup local
<repeats indefinitely - ctrl+c to quit>

In Zentyal, this causes one of the network scripts to hang because it's waiting for that command to end.  This prevents loading of other services and resulted in my network being severely broken.

Besides the previously mentioned fix of rolling back the kernel, you can modify the script in question:
/usr/share/zentyal-network/flush-fwmarks

I just copied the file to flush-fwmarks.old, then edited the original commenting everything out.  I like this solution because I still get the benefit of the security fixes in the new kernel.  The down side is that the ip command is broken and firewall marks don't get flushed, but I don't really care about those   If you do care about those and have some shell scripting ability, it should be relatively easy to modify that script to notice that ip is broken and move on.

[lacostewin]

Thank you, expertgeeks. With linux-image-3.13.0-70-generic the same history. Downgrade to linux-image-3.13.0-68-generic is fix problem. Sorry for bad En. Your brother from Russia)))

[expertgeeks]

You're very welcome ! Thanks for the feedback about the 3.13.0-70 kernel.

[clockmoddersadm]

Problem is now completly solved with kernel 3.19.0-39-generic.

Upgraded today and issue is no more present.

[ctucc99]

We did this fix when we encountered it (after running the 4.2 update via the Dashboard)...  we followed your instructions, rolled back the Kernel and it fixed it  (Thank you very much for posting this).

My question now however is in my Dashboard, I am at 4.2.1 and I now have 2 component updates and 43 system updates queued up for installation.  Can I install these via the Dashboard or am I going to run into the same issue again?  (IE am I going to continually have to roll back the kernel each time I run Zentyal updates now?)

[BerT666]

seems that the bug is fixed and normal update is working again :-)

[jonathan38]

It's literally a process called "ip". If you run it manually, it starts looping indefinitely, causing the high CPU usage.

[pocc]

So does this mean I can reboot to a newer kernel now?  I've been stuck on -51 for awhile

[pocc]

So I bit the bullet and gave it a try today. 

In the process of trying to resolve this previously, I'd updated to the 15.04 hwe stack and the 3.19.x series kernel but was still having issues so I kept it on 3.16.0-51 as that seemed to be the last version that worked.

I'm pleased to say that both of my Zentyal boxes are now running 3.19.0-42 and things seem very snappily stable.