ebox memory footprint

[lucho115]

HI, i want to use ebox only like a powerfull gateway, i has been using ipcop + plugins for 5 years, now i want to build an server (P4 2.4mhz, 1gb ram, 3 nics), with ipcop stable (1.4.21) the footprint was of 50 or 60 MB with no charge, and 250 or 260 with full charge (200 pcs), but the first that i see when install ebox is that the fresh install without charge eat 500 mb , iam testing this ebox server so i dont know how memory eat with full charge (200 pcs) but i start thinking that all the ram will be eaten in seconds and the server will swap all the time.
Why the huge diference of memory footprint between ipcop and ebox?
Anybody have in production a similar server with a huge charge?
thanks and sorry about my english

[J. A. Calvo]

Hi lucho,

First of all, we should know how many ebox modules do you have installed, so if you can please paste here the output of the following command: dpkg -l | grep ebox

It would be also nice if you give us the output of the top command to see the state of the running processes and the memory.

Regards,

J. A. Calvo

[lucho115]

Hi lucho,

First of all, we should know how many ebox modules do you have installed, so if you can please paste here the output of the following command: dpkg -l | grep ebox

It would be also nice if you give us the output of the top command to see the state of the running processes and the memory.

Regards,

J. A. Calvo

ok
 

Code: [Select]

dpkg -l | grep ebox
ii  ebox                                  1.2.8-0ubuntu1~ppa1~hardy1                                      the eBox platform - Base framework
ii  ebox-ca                               1.2-0ubuntu1~ppa1~hardy1                                        eBox - Certificate Authority Manager for eBo
ii  ebox-dhcp                             1.2-0ubuntu1~ppa1~hardy1                                        eBox - DHCP server module
ii  ebox-dns                              1.2.2-0ubuntu1~ppa1~hardy1                                      eBox - DNS server
ii  ebox-firewall                         1.2.2-0ubuntu1~ppa1~hardy1                                      eBox - Firewall
ii  ebox-l7-protocols                     1.2-0ubuntu1~ppa1~hardy1                                        eBox - layer 7 protocols
ii  ebox-monitor                          1.2.2-0ubuntu1~ppa1~hardy1                                      eBox - Monitoring module
ii  ebox-network                          1.2.6-0ubuntu1~ppa1~hardy1                                      eBox - Network configuration module
ii  ebox-ntp                              1.2-0ubuntu1~ppa1~hardy1                                        eBox - NTP server
ii  ebox-objects                          1.2-0ubuntu1~ppa1~hardy1                                        eBox - Object management
ii  ebox-openvpn                          1.2.2-0ubuntu1~ppa1~hardy1                                      eBox - OpenVPN server module
ii  ebox-services                         1.2.1-0ubuntu1~ppa1~hardy1                                      eBox - Services management
ii  ebox-software                         1.2-0ubuntu1~ppa1~hardy2                                        eBox - Software management
ii  ebox-squid                            1.2.4-0ubuntu1~ppa1~hardy1                                      eBox - Proxy cache and content filter
ii  ebox-trafficshaping                   1.2-0ubuntu1~ppa1~hardy1                                        eBox - Traffic shaper for eBox
ii  ebox-usersandgroups                   1.2.2-0ubuntu1~ppa1~hardy1                                      eBox - User and Group management
ii  esofttool                             0.5.4                                                           ebox software tool
ii  libebox                               1.2.1-0ubuntu1~ppa1~hardy1                                      eBox common library for server and client
ii  samba-vscan                           0.3.6cbeta5ebox1-2                                              Samba virus scanning VFS module
and Top:

Code: [Select]

Tasks:  75 total,   3 running,  72 sleeping,   0 stopped,   0 zombie
Cpu(s):  5.7%us,  0.3%sy,  0.0%ni, 93.7%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1018976k total,   451088k used,   567888k free,    21480k buffers
Swap:  1686784k total,        0k used,  1686784k free,   191796k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 6264 ebox      20   0 55036  43m 5392 S  5.0  4.4   0:04.03 apache2
 4692 ebox      20   0  4240 2484 1796 S  0.3  0.2   0:02.20 gconfd-2
 6333 root      20   0 63852  11m 2160 S  0.3  1.1   0:00.53 collectd
    1 root      20   0  1808  772  548 S  0.0  0.1   0:01.36 init
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    4 root      15  -5     0    0    0 R  0.0  0.0   0:00.02 ksoftirqd/0
    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    6 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 events/0
    7 root      15  -5     0    0    0 S  0.0  0.0   0:00.01 khelper
   41 root      15  -5     0    0    0 S  0.0  0.0   0:00.03 kblockd/0
   44 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid
   45 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify
  118 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kseriod
  156 root      20   0     0    0    0 S  0.0  0.0   0:00.00 pdflush
  157 root      20   0     0    0    0 S  0.0  0.0   0:00.02 pdflush
  158 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kswapd0
  202 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
 1368 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0
 1374 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux
 1377 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd
 1384 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 khubd
 1391 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0
 1396 root      15  -5     0    0    0 S  0.0  0.0   0:00.01 scsi_eh_1
 2103 root      15  -5     0    0    0 S  0.0  0.0   0:00.01 scsi_eh_2
 2105 root      15  -5     0    0    0 S  0.0  0.0   0:00.01 scsi_eh_3
 2364 root      15  -5     0    0    0 S  0.0  0.0   0:00.18 kjournald

This info is with the system just booted.

[J. A. Calvo]

It is not eating 500mb, you have to substract the buffered and cached memory that in fact they are also available. So your real used memory is:

 451088-21480-191796=237812 (232Mb)

I don't know which software installs ipcop, but I think 232MB is not too much.

And looking at your top, apache is the main responsible of that memory consumption, so if you are going to use it as gateway I don't see any relation. I mean, maybe with 200 clients eBox will perform well if these clients are not using the ebox apache (they won't).

Hope this clarifies something.

Regards,

J. A. Calvo

[christian]

...And looking at your top, apache is the main responsible of that memory consumption...

Just curious: is there any pending investigation to change from Apache (that can be, as you state, quite memory/CPU consuming) to lighter HTTP engine (like Nginx, Lighttpd or Cherokee) ?

For sure Apache can be much more powerfull but is it required on this kind of server?

Christian

[J. A. Calvo]

Yes, investigate the replacement of the current http server with a ligther one is something that we have already talked about. But it's not a top priority right now in our roadmap.

Regards,

J. A. Calvo

[cheesyking]

It's also worth bearing in mind that the web server in ipcop is just used to run the admin pages.

In ebox it also has to run the groupware and any other web based system the network might need so many people would need to run apache in addition to whatever lightweight server the admin pages run on.

[javi]

If we didn't use apache we would save around 50MB.

We have thought about it, but with the current specs of computers we considered that for now we should work on other stuff first.

Thanks!

[lucho115]

It is not eating 500mb, you have to substract the buffered and cached memory that in fact they are also available. So your real used memory is:

 451088-21480-191796=237812 (232Mb)

I don't know which software installs ipcop, but I think 232MB is not too much.

And looking at your top, apache is the main responsible of that memory consumption, so if you are going to use it as gateway I don't see any relation. I mean, maybe with 200 clients eBox will perform well if these clients are not using the ebox apache (they won't).

Hope this clarifies something.

Regards,

J. A. Calvo

Iam back, ok i know that buffers and cache is not real mem used but also is true that the kernel never free that mem when the phisical memory is running out (i know by experience), so start swapping , and also that the ipcop take only 50-60 mb (incluided buffers and cache with kernel 2.4.35 in stable release) and 60-70 (incluided buffers and cache with kernel 2.6.27 in unstable release).
Also if we take out apache the mem used is still huge.
thanks
bye

[cheesyking]

Just a thought, the output of top that you posted doesn't seem to be ordered by memory use. You have to press "M" when top is running to do that (the default is to order by cpu use and lots of services site idle most of the time so you don't see them on that list)

here is top from one of my servers:

Code: [Select]

top - 22:28:16 up 50 days, 11:25,  2 users,  load average: 0.00, 0.00, 0.00
Tasks: 124 total,   2 running, 122 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1017960k total,   980308k used,    37652k free,   122600k buffers
Swap:  2980016k total,      908k used,  2979108k free,   316976k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 8482 clamav    20   0 86456  73m  980 S  0.0  7.4   2:48.57 clamd             
 8542 amavis    20   0 66852  56m 5816 S  0.0  5.7   0:05.67 amavisd-new       
30105 amavis    20   0 69156  56m 2660 S  0.0  5.6   0:01.72 amavisd-new       
30788 amavis    20   0 68464  55m 2560 S  0.0  5.6   0:00.79 amavisd-new       
 7465 ebox      20   0 51488  44m 4748 S  0.3  4.5 106:46.78 ebox-loggerd       
13850 ebox      20   0 53832  43m 5652 S  0.0  4.4   0:02.02 apache2           
 2032 www-data  20   0 46372  28m 5972 S  0.0  2.9   3:29.19 apache2           
 8507 root      20   0 31584  28m 2572 S  0.0  2.9   0:52.57 spamd             
 8567 root      20   0 31584  26m  640 S  0.0  2.7   0:00.27 spamd             
 8568 root      20   0 31584  26m  548 S  0.0  2.7   0:00.38 spamd             
 1620 www-data  20   0 42356  24m 6012 S  0.0  2.5   8:18.92 apache2           
14401 www-data  20   0 41752  23m 5624 S  0.0  2.4   1:27.62 apache2           
14429 www-data  20   0 40776  23m 5996 S  0.0  2.3   3:21.00 apache2           
 8512 ebox      20   0 26048  22m 3364 S  0.0  2.3   0:03.76 learnspamd         
 7695 postgres  20   0 40812  22m  20m S  0.0  2.2   3:38.85 postgres           
 4887 postgres  20   0 40144  21m  21m S  0.0  2.2   2:32.74 postgres           
27700 www-data  20   0 39244  20m 5536 S  0.0  2.1   0:26.95 apache2           

As you can see clamAV is the worst offender along with anti spam stuff, apache and postgres (but then I do use egroupware). Given it's been up for 50 days I don't think the memory usage is too bad, of course I'm not acting as a proxy for 200 clients though.

If you want a real memory hog have a look at a server2008 SBS install. It will boot with 4GB but you seem to need more like 8GB if you don't want it to keep crashing... and that's with 5 users 

[dragonslayr]

Memory usage on ebox is not problematic. A typical install of a bind9 server runs 200m, while most of my boxes running just a few services use 400m and my main incoming mail server regularly eats up 2 gig, while a kvm machine with 8 gigs is using 7gig running 7 servers..

And I don't think an ipcop box running any "added" services like filters etc runs on 50m even with just 1 client behind it. Sorry, I don't believe it. I'll look at a couple of ipcop boxes running tomarrow and see what I see...

[lucho115]

Just a thought, the output of top that you posted doesn't seem to be ordered by memory use. You have to press "M" when top is running to do that (the default is to order by cpu use and lots of services site idle most of the time so you don't see them on that list)

here is top from one of my servers:

Code: [Select]

top - 22:28:16 up 50 days, 11:25,  2 users,  load average: 0.00, 0.00, 0.00
Tasks: 124 total,   2 running, 122 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1017960k total,   980308k used,    37652k free,   122600k buffers
Swap:  2980016k total,      908k used,  2979108k free,   316976k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 8482 clamav    20   0 86456  73m  980 S  0.0  7.4   2:48.57 clamd             
 8542 amavis    20   0 66852  56m 5816 S  0.0  5.7   0:05.67 amavisd-new       
30105 amavis    20   0 69156  56m 2660 S  0.0  5.6   0:01.72 amavisd-new       
30788 amavis    20   0 68464  55m 2560 S  0.0  5.6   0:00.79 amavisd-new       
 7465 ebox      20   0 51488  44m 4748 S  0.3  4.5 106:46.78 ebox-loggerd       
13850 ebox      20   0 53832  43m 5652 S  0.0  4.4   0:02.02 apache2           
 2032 www-data  20   0 46372  28m 5972 S  0.0  2.9   3:29.19 apache2           
 8507 root      20   0 31584  28m 2572 S  0.0  2.9   0:52.57 spamd             
 8567 root      20   0 31584  26m  640 S  0.0  2.7   0:00.27 spamd             
 8568 root      20   0 31584  26m  548 S  0.0  2.7   0:00.38 spamd             
 1620 www-data  20   0 42356  24m 6012 S  0.0  2.5   8:18.92 apache2           
14401 www-data  20   0 41752  23m 5624 S  0.0  2.4   1:27.62 apache2           
14429 www-data  20   0 40776  23m 5996 S  0.0  2.3   3:21.00 apache2           
 8512 ebox      20   0 26048  22m 3364 S  0.0  2.3   0:03.76 learnspamd         
 7695 postgres  20   0 40812  22m  20m S  0.0  2.2   3:38.85 postgres           
 4887 postgres  20   0 40144  21m  21m S  0.0  2.2   2:32.74 postgres           
27700 www-data  20   0 39244  20m 5536 S  0.0  2.1   0:26.95 apache2           

As you can see clamAV is the worst offender along with anti spam stuff, apache and postgres (but then I do use egroupware). Given it's been up for 50 days I don't think the memory usage is too bad, of course I'm not acting as a proxy for 200 clients though.

If you want a real memory hog have a look at a server2008 SBS install. It will boot with 4GB but you seem to need more like 8GB if you don't want it to keep crashing... and that's with 5 users 

ok, but i dont use egroupware, mail, antispam, antivirus, database (postgres) so i didnt install that modules, but still the memory used is very high. I dont want to use window$, i can compare with other linux distros like ipcop that the only issue is that not support multiwan, and giving all same services that ebox use only 1/8 of the memory that ebox use.
I think there are 3 things that cause the mem problem:
1- the soft that ebox use to each service (ej. ebox for dns use bind and ipcop use dnsmasq).
2- Use the ubuntu kernel without tunning (if somebody used ubuntu anytime knows what about iam taking, the ubuntu kernel use a lot of the ram to buffers and cache, i dont why).
3- Dependencies, ex. i dont select postgres in advance packages selection but it installed anyway, and the same thing do with other packages.

thanks and sorry about may english.

[lucho115]

Memory usage on ebox is not problematic. A typical install of a bind9 server runs 200m, while most of my boxes running just a few services use 400m and my main incoming mail server regularly eats up 2 gig, while a kvm machine with 8 gigs is using 7gig running 7 servers..

And I don't think an ipcop box running any "added" services like filters etc runs on 50m even with just 1 client behind it. Sorry, I don't believe it. I'll look at a couple of ipcop boxes running tomarrow and see what I see...

Please doit, install ipcop (stable or unstable) and you will see the diference of mem usage. And yes ipcop can do (include filters) all that ebox do, the only issue is that not support multiwan.
thanks and sorry about my english.
bye

[isaac]

Hi Lucho!

The memory apache uses in eBox is used by the administration interface, once everything is set up and you don't use it it will be probably swapped out to disk if memory is needed. In addition you can go a step further and just stop ebox's apache when you are not using the web interface

Code: [Select]

sudo /etc/init.d/ebox apache stop
and of course start it if you want to do some administration

We still try to lower the memory footprint of eBox, but it's not such a big issue as you can see

Cheers!

[lucho115]

this is my top sort by mem usage

top - 13:30:50 up 18 min,  1 user,  load average: 0.06, 0.05, 0.08
Tasks:  73 total,   1 running,  71 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1018976k total,   433324k used,   585652k free,    45604k buffers
Swap:  1686784k total,        0k used,  1686784k free,   179180k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4536 clamav    20   0 79068  74m  396 S  0.0  7.5   0:00.00 clamd
 6033 ebox      20   0 47464  37m 4980 S  0.0  3.8   0:07.28 apache2
 5622 ebox      20   0 37852  31m 4268 S  0.0  3.1   0:02.04 ebox-loggerd
 5856 root      20   0 31172  22m 5520 S  0.0  2.3   0:02.14 apache2
 5951 root      20   0 28468  19m 4592 S  0.0  1.9   0:00.81 apache2
 6016 ebox-use  20   0 31304  18m  724 S  0.0  1.8   0:00.00 apache2
 5578 bind      20   0 34696 7416 2324 S  0.0  0.7   0:00.02 named
 4293 postgres  20   0 40148 4912 4296 S  0.0  0.5   0:00.44 postgres
 5773 proxy     20   0  7028 4604 1572 S  0.0  0.5   0:00.08 squid
 6088 root      20   0 12440 3928 2152 S  0.0  0.4   0:00.03 apache2
 4317 openldap  20   0 22220 3568 1596 S  0.0  0.4   0:00.01 slapd
 6266 pepe      20   0  5556 3000 1444 S  0.0  0.3   0:00.14 bash
 5771 postgres  20   0 40952 2980 1804 S  0.0  0.3   0:00.00 postgres
 6263 root      20   0  8056 2532 2072 S  0.0  0.2   0:00.03 sshd
 6090 www-data  20   0 12440 2456  672 S  0.0  0.2   0:00.00 apache2
 6093 www-data  20   0 12440 2456  672 S  0.0  0.2   0:00.00 apache2
 6095 www-data  20   0 12440 2456  672 S  0.0  0.2   0:00.00 apache2
 6097 www-data  20   0 12440 2456  672 S  0.0  0.2   0:00.00 apache2
 6098 www-data  20   0 12440 2456  672 S  0.0  0.2   0:00.00 apache2
 4647 ebox      20   0  4228 2440 1792 S  0.0  0.2   0:02.50 gconfd-2
 4193 klog      20   0  3028 1920  424 S  0.0  0.2   0:00.11 klogd
 6265 pepe      20   0  8056 1556 1076 S  0.0  0.2   0:00.05 sshd
 4297 postgres  20   0 40272 1400  676 S  0.0  0.1   0:00.02 postgres
 5686 root      20   0  4024 1396 1132 S  0.0  0.1   0:00.01 ntpd
 4295 postgres  20   0 40148 1352  724 S  0.0  0.1   0:00.03 postgres
 4296 postgres  20   0 40148 1172  544 S  0.0  0.1   0:00.02 postgres
 4298 postgres  20   0 11468 1148  464 S  0.0  0.1   0:00.01 postgres

why clamav is installed if i dont select antivirus? and if i want remove clamav with apt its say that will removo too squid danguardian and others things that i want.
if anybody can help me? thanks
bye