http proxy and content filtering slows down our internet connection

[kiaboy]

Hi,

I hope someone out there can help me figuring out what is wrong with my configuration. When content filtering is on i got 10 or more seconds delay before a website load, internet is very slow. Aside from that my port forwarding is intermittent. Below are the details about my setup.

Hardware            Q9550 quad core
RAM                   4Gb
Core version     2.2.7
System load     0.08, 0.18, 0.25
HTTP proxy          ON
Transparent proxy ON

[christian]

With such "huge" configuration, it should not be that slower.
However, by design:
- adding proxy is always slower than "no proxy" (except when you access pages from cache)
- transparent proxy is even slower
- filtering is one more step to slowness, and depending on filtering rules, it can be really noticeable.

This said, did you try, in order to investigate:
- proxy without content filtering?
- direct access to internet from Zentyal
- measure HTTP response time (depending on your client OS, Firebug, HTTPFox, Fiddler...)

[kiaboy]

@Christian

In addition on my setup, i have 70 concurrent users and yes i did some sort of isolation, and here are the result.

No filtering = faster internet speed
No caching = faster internet speed
With or without proxy speed is not noticeable

but from what i understand proxy and caching should should bumped up the speed of the internet. And no proxy no content filtering. All i want from this setup is to filter any web request coming from office.
 

[kiaboy]

@Christian

All i want to implement is to block unwanted website that may distract the productivity on a employee in the office ad also to open those blocked sites during their break periods.

[christian]

No filtering = faster internet speed

is difference huge ?

No caching = faster internet speed

How did you achieve this? Zentyal GUI doesn't allow to implement proxy without cache, as far as I know

With or without proxy speed is not noticeable

I can't correlate this with above sentence unless you tweaked Zentyal config to enable proxy without cache or did you set "cache size =0" and noticed that this impacts internet access speed?

but from what i understand proxy and caching should should bumped up the speed of the internet.

not 100% true. Not wrong neither  proxy by itself will not provide faster internet access. Without proxy, your client asks for a page directly to web server. With proxy, requested is received by proxy which performs same request again to web server, receives page and sends it back to client. How can this be faster    However, if you implement cache:
- when you want to access same page again, proxy cache might be more efficient than your local (browser) cache, because of size
- more important, when someone else accesses same page, benefit of proxy cache is obvious 

 

And no proxy no content filtering. All i want from this setup is to filter any web request coming from office.

This is THE point: if you want to filter, you have to implement proxy.
If I understand well what you explained, internet access is only slower when you enable filtering. What kind of rule did you enable? default configuration only or specific filter profile?

[kiaboy]

@Christian

default with uploaded blacklist.

[christian]

So, instead of jumping from "no filter" to "customized filter", give a try with the intermediate step that is to use real "default configuration". Still you can play with threshold 
This will give you an idea of what is really impacting performance.
Is it because of filtering feature or because of your customized blacklist. I'm pretty sure that without blacklist, you will have standard performance.
This said, you may need this blacklist too. There is nothing magic here: either this list is reasonably short and impact in term of performance is minimum or list is huge and impact can be huge too.

[jsalamero]

Have a look at the performance tuning options in /etc/zentyal/squid.conf.

[kiaboy]

Have a look at the performance tuning options in /etc/zentyal/squid.conf.

filter profile is set to default. without the filter and the blacklist i uploaded internet speed improves like 50%. turning off transparent proxy improves the internet more.

[kiaboy]

@Christian

default with uploaded blacklist.

filter profile is set to default. without the filter and the blacklist i uploaded internet speed improves like 50%. turning off transparent proxy improves the internet more.

[christian]

  this confirms what I told you at the very beginning of this thread:
- transparent proxy has performance impact
- filtering rules complexity impacts performance too.

However, except if you use performance analysis tool or if something is really wrong in your network, you should not notice impact of transparent proxy that much on performance: this is only one extra step redirecting requests to proxy at default gateway level. 

[jsalamero]

If your proxy goes that slow, hardware is your bottleneck, check RAM, CPU and disk IO in your server.

[christian]

Hardware            Q9550 quad core
RAM                   4Gb

I doubt such quad core can be the bottleneck unless there is major issue with memory that would result in some (serious) swapping. However, this is something you have to look at.