Author Topic: Zentyal's proxy in the real world  (Read 22166 times)

Sam Graf

  • Guest
Zentyal's proxy in the real world
« on: May 16, 2012, 05:07:43 pm »
Zentyal provides two basic types of proxy implementation: transparent, and non-transparent--where the proxy configuration details have to be declared explicitly on client devices. Which implementation to choose, and why?

At least some portion of the Zentyal user base is not going to use the proxy at all. This topic is for admins who are not going to not know the answer to the basic "which proxy implementation?" question beyond the "it works!" test. They will not immediately know what the choice really means to their work-flow, to their ability to conform the technology to company policy, and so on.

For some of us, Zentyal's transparent proxy implementation is the correct "just works" choice. It involves minimal fuss for all devices on the network, including mobile and visitor devices, while providing robust Web content filtering and virus screening.

For some of us, that level of service is great as a starting point, but eventually we would like to accomplish other things--tracking individual users as they browse the Web (to determine who is violating an acceptable use policy, for instance), or implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service. How do I get from where I am and what I know to where I would like to be, based on a vague idea of what I want?

The problem in answering that question is that Zentyal's explicit proxy is non-trivial to use if one lacks knowledge of how proxies work in general and how Zentyal's implementation works in particular. For purposes of this topic, I'm going to assume a simple admin--he knows what he wants to accomplish in general terms (say, block objectionable Web content, track policy violations) but does not understand the underlying technologies well, if at all. Knowledgeable admins may use Zentyal's GUI-managed services as a convenient starting point and then routinely hand configure the rest. Other admins may have Web content filtering for the very first time only because Zentyal makes it remarkably easy to integrate that service into an easy-to-manage network infrastructure, all in a single product, and that's all they know about it. And they love it because they can do something they might not otherwise be able to do! But some day something new arises...

If an exploration-minded admin sets up an explicit proxy (she can get that far since the process it relatively straightforward) and feels very good about that first step because everything is working great--until a staff member tries to access their hosting cPanel, and can't get in...now what? Since there is no "allow cPanel" proxy option, she's stumped. Does she give up on her experiment and go back to the transparent proxy? Is that the best choice for her to make?

Hopefully this topic can amass the tips and tricks necessary to exploit Zentyal's proxy to whatever purpose a simple admin might want, including the fundamental tips on making the choice in the first place--what's gained an lost in each case.

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #1 on: May 16, 2012, 05:23:05 pm »
Excellent starting point +++

If you don't mind, I would just add some explicit  ;D  statement for those not reading between the lines:
- access to cPanel example in Sam's description is because out-of-the-box, Zentyal proxy implementation may not support HTTPS access to port different from 443.
- notice that with transparent proxy, there is no such problem because HTTPS access is not done via proxy but directly via firewall.

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #2 on: May 16, 2012, 05:53:50 pm »
Thank you for the explicit clarifications. ;D

But consider my poor admin's problem...she has no idea why cPanel access is suddenly blocked. She can't read between the lines because she has no idea what's going on. I was hoping we could take a minute to feel her predicament. :P

I used that example deliberately, out of my own experience, in keeping with my intent to see this stay true to the real world. We had people suddenly unable to access cPanel-based webmail. It wasn't really suddenly, it was just that a period of time had elapsed between the start of the explicit proxy experiment and the attempt to access webmail (on a machine without a proper e-mail client). So there was no immediate connection between the two in anybody's mind. "Try again tomorrow" didn't work, so then the mental machinery had to crank it up a notch, to connect dots that didn't have a very obvious relationship--other HTTPS connections work, after all, and we're explicitly ;D including the port in connection attempt ...

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #3 on: May 16, 2012, 06:34:14 pm »
Much clearer.
For these Zentyal users, transparent proxy is the best choice.  8)
As you wrote, it works out-of-the-box. Let them use it like this until they have additional needs, maybe later nor never  ;)

Again, people looking only for cache feature and potentially basic filtering on HTTP only can be satisfied with transparent proxy. Not need to look further, especially if it has to be looked from the end-user standpoint.
As previously discussed in another topic, some Zentyal "users" made this choice because of its simplicity and their willingness not look at the technical details. If trigger is this one, then transparent proxy is their obvious best choice.
For them, it doesn't matter if client has to resolve names or if there is not HTTPS filtering.

One detail, having in mind such users: if only few users are behind Zentyal box using transparent proxy with no specific "proxy feature", then they should rather not use proxy at all, they will get better performance:
- cache efficiency ratio, with only few users, is below '1"
- because of the way it works, transparent proxy is slightly slower than explicit proxy, thus slower than not proxy at all.

This said, the point is "how to address all the other cases" if any  :D

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #4 on: May 16, 2012, 07:12:28 pm »
One detail, having in mind such users: if only few users are behind Zentyal box using transparent proxy with no specific "proxy feature", then they should rather not use proxy at all, they will get better performance:
- cache efficiency ratio, with only few users, is below '1"
- because of the way it works, transparent proxy is slightly slower than explicit proxy, thus slower than not proxy at all.
Very good point.

This said, the point is "how to address all the other cases" if any  :D
Let me suggest that for my simple admin, content filtering and virus screening likely are going to be an attraction. He may have downloaded Zentyal for that alone (my first interest in eBox was its VPN possibilities; I didn't care about the rest at first) and then discovered other wonderful tools in it. They may not care about caching or about authentication, at first. So transparent proxy seems the natural choice.

Given that interest, let's take it to the problem solving level. We had a local elementary school principal access child pornography using school equipment on the school network. The access was discovered during a routine audit, and he got dismissed for violating the school district's acceptable use policy. Some simple, just-getting-by non-profit admin works for an organization where all staff and volunteers have to pass a background check, so broadly speaking this kind of behavior is an issue, and he wonders about what happened at the school very seriously for the first time--I'm trying to block access, just like the school does, but would I know if somebody had gotten access anyway, just like happened at the school?

So he comes here, describes what he's after and why, and then asks: "Is there a way for me to know about Web access policy violations on an individual level? I don't see user names and site access connected together in the logs. Is the transparent proxy coupled with Zentyal subscription services my best option? Will that tell me what I need to know? Or will I need to use a non-transparent proxy, since I see something about user authentication there? Does anybody know exactly how the two (proxy and subscription services) work together?"

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #5 on: May 16, 2012, 11:59:04 pm »
I don't know anything about subscription service and even don't understand why this jumps in our debate  :-[
I've read your post already 3 times and still don't understand. Sorry. Do you mind elaborating a bit ???

Escorpiom

  • Zen Hero
  • *****
  • Posts: 897
  • Karma: +25/-1
    • View Profile
Re: Zentyal's proxy in the real world
« Reply #6 on: May 17, 2012, 03:27:03 am »
Subscription service allows for extensive logging/reports, perhaps that is what Sam meant?

To elaborate a bit more, I came from a Windows box with Squid installed. For me, caching was and is the main reason to use Squid.
Internet speed is rather limited at my end, so making efficient use of our Internet connection is most important.
The network grew over time and I decided to change to a all-in-one Linux based router solution.   
Zentyal is all that and it provides me with the web proxy. I have dedicated 10GB of disk space to the cache and it saves a heap of bandwidth every day.

Apart from the proxy cache, other goodies are the logs. Not so much per user based, but more like general stats of what type of content is being used on my network.
Then came the adzapper, I consider it a must-have. The content blocker also made it on my list.     

One of the things that do not really interest me (at the moment) is authentication. We don't need to track policy violations either. Perhaps at a later point in time.
So far for my real world scenario, I hope it adds to this topic.

Cheers.   
Marcus' Rule:
Blanks & capitals = avoid it and you'll avoid problems...

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #7 on: May 17, 2012, 03:33:33 am »
Subscription service allows for extensive logging/reports, perhaps that is what Sam meant?
Correct. For really real world examples of this type of question already discussed here, see:
http://forum.zentyal.org/index.php/topic,6180.0.html
http://forum.zentyal.org/index.php/topic,7628.msg30343.html#msg30343

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #8 on: May 17, 2012, 08:33:50 am »
Clearer, although link shown by Sam doesn't explain anything, at least to me, for what concerns subscription.
Let me explain: there is a link from sixstone to Zentyal website but this doesn't explain the detail of subscription neither how it works (and such link is not supposed to explain this level of detail, I agree).

However I understand now that debate is about getting detailed report about web usage (I focus on web usage as we discuss HTTP proxy, I'm sure subscription provides much more).
What I don't understand is why this debate about "internet usage analysis" while we discuss about transparent vs. explicit proxy. Is there any impact I do not perceive?

@Escorpiom: you explain benefit of use of proxy on your network. This is pretty clear and exactly what Sam meant with "real world". I understand also better what "subscription" can provide although this is a bit strange to me: you have limited internet bandwidth and decide to buy services consuming part of your bandwidth  :o I'm a bit confused. Well, I know Zentyal doesn't provide anything to really look at log content. Reasons why we discussed stuff like Awstats in one of the links Sam showed. But here again, as this log analysis stuff any impact on proxy design choice? I don't understand.
Last but not least, if internet browsing performance is your real concern (which I can easily understand), then we are entering in another dimension because achieving top perf with Squid requires tuning that is much more trick than transparent vs. explicit and very far from end-user approach. It starts with high performance disks dedicated to proxy cache, tuning of cache content cleaning, memory cache... well, another world, far from our simple debate  8)

Escorpiom

  • Zen Hero
  • *****
  • Posts: 897
  • Karma: +25/-1
    • View Profile
Re: Zentyal's proxy in the real world
« Reply #9 on: May 17, 2012, 11:00:52 am »
@Escorpiom: you explain benefit of use of proxy on your network. This is pretty clear and exactly what Sam meant with "real world". I understand also better what "subscription" can provide although this is a bit strange to me: you have limited internet bandwidth and decide to buy services consuming part of your bandwidth  :o I'm a bit confused.

I'm confused too. What do you mean by buying services that consume part of my bandwidth? If this is about a subscription service, I only have the free subscription to peek at the Zentyal Cloud, but most of the time it's offline. I'm not buying anything that I'm aware of.
I do understand that we can take the whole proxy thing to the next level but neither my wallet nor my knowledge is sufficient to do this. I do have a 10K raptor disc for the cache  :)

However, perhaps you're trying to get another point across: No proxy is faster than transparent or explicit proxy? If so, then I obviously don't agree.

Cheers.
Marcus' Rule:
Blanks & capitals = avoid it and you'll avoid problems...

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #10 on: May 17, 2012, 03:17:30 pm »
Escorpiom,

If you are convinced or better, have measured that proxy improve internet browsing speed, this is very good.
I can explain why this result is seldom achieved, however I would like to start with another comment:
HTTP proxy is used for 2 main reasons:
- security
- performance

Security is almost self-explanatory:
- profiling, authorization, blacklist, ad removal etc...

Performance is more questionable because:
1 - your browser brings its own local cache
   - HTTPS is not cached
   - more and more ignorant web developer are wrongly using "pragma no-cache"  >:(

2 - accessing web page directly will bring the page to your browser, directly (kind of) where it will stored in local cache(*).
  - doing the same via proxy, you will send the request to proxy that will act as web client itself, retrieve page and forward it to you while storing it in proxy cache(*)
 - doing the same with transparent proxy is like with explicit proxy except that client is not aware of proxy in the middle, thus sends request that is intercepted then redirected by proxy, so, slightly slower, even if you don't perceive it without measuring tool.

3 - proxy cache mechanism however as some benefit (do not think I'm trying to demonstrate that proxy is always slower  ;)): when one given page has already been accessed by someone else in your organization, then you will get it from proxy. Because of this, proxy efficiency directly depends on the number of users, reason why I said: with few users, proxy is likely slower than no proxy.

If you are not convinced, I suggest you give a closer look, using tool like HTTP analyser.

On top of that, tuning proxy is not an obvious task, much more complex than debating about transparent vs. explicit proxy.
Large cache size, even on fast disk, is not enough.
- some other parameters have more impact
- too large cache size will make it slower because of the amount of file it will store.

Escorpiom

  • Zen Hero
  • *****
  • Posts: 897
  • Karma: +25/-1
    • View Profile
Re: Zentyal's proxy in the real world
« Reply #11 on: May 18, 2012, 03:18:42 am »
True to some extend.
Not only HTTPS isn't cached, a lot of webpages (as you also pointed out) won't be cached either.
If we take a look at exactly what gets cached it becomes clear that proxy cache is surely not the holy grail.
I knew that from the beginning.
Furthermore, webpages as a whole seldom get cached. But certain elements on those pages might get cached.

But I'll give you an example of real world proxy cache:
It's that "Microsoft patch Tuesday". Around 40 computers on my net will get updated, these patches for both Windows and Office will sum anywhere from 10MB to 100MB in size.
Once Zentyal server has the patches in it's cache (depending on max. filesize given in the config file), other computers on the network can retrieve the updates directly from the cache and thus it's faster without the need to download over and over again.
You can calculate the savings in bandwidth. It may be necessary to adjust some settings in the Squid config file, but it is not hard to do.

We currently use a 3.5Mbit down, 850Kbit up Adsl2 connection. I know that some of you may be spoiled with Internet connections ranging from 10Mbits until 50Mbits or even 100Mbits.
The use of a proxy cache is perhaps less interesting having those Internet speeds available.

Cheers. 
Marcus' Rule:
Blanks & capitals = avoid it and you'll avoid problems...

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #12 on: May 18, 2012, 08:54:37 am »
Very good point  ::thumbs up::
In large companies, admins take care of this by handling OS update differently (because problem is exactly the same, whatever your bandwidth, if you have 1000, 2000 or 5000 PC performing Windows update)
Furthermore, with 40 PCs, you certainly benefit from proxy cache, no doubt about this.
My comment what in fact for SMBs and SOHO, maybe not clear enough.

Back to the (interesting) initial debate, launched by Sam:
we were comparing real world experiences because deploying explicit proxy was supposed to require too much technical understanding and manual actions not handled by Zentyal GUI.
As you are tuning your own Squid config, you are obviously very far from this  ;D

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #13 on: May 18, 2012, 05:40:54 pm »
What I don't understand is why this debate about "internet usage analysis" while we discuss about transparent vs. explicit proxy. Is there any impact I do not perceive?
Because I'm guessing that to take full advantage of the subscription service in the real world, I am going to have to run an explicit proxy. So for a simple admin, we will move from the frying pan and into the fire, all the time thinking we are solving the problem of detailed user tracking.

Now my simple admin is paying for a solution and he will find his system is in a place he wasn't expecting. Of course, I could have cut right to an explicit proxy, but you are already tempted to say he doesn't need it. So, I took the long way to make a point--that at least some Zentyal admins will wander into explicit proxy territory not because they decided up front that it is a better way to go, but because their evolving needs led them there even if they didn't know what they were getting into.

MY real point is that we need to do two things: lay out the advantages and disadvantages of Zentyal's proxy implementation, and we also need to understand that it's not necessarily that straightforward for admins who are just trying to solve a real world problem.

I'm not far outside my own real experience in everything I have written so far, including the road taken to even trying an explicit proxy in the first place (what a bad experience for a dummy!)

Marcus

  • Forum Moderator
  • Zen Samurai
  • *****
  • Posts: 395
  • Karma: +12/-0
    • View Profile
    • Professional IT Service
Re: Zentyal's proxy in the real world
« Reply #14 on: May 19, 2012, 04:17:43 pm »
Hello,

Quote
Since there is no "allow cPan*l" proxy option[...]
This is configured in the firewall and not the proxy.

(My 2 pennies) This issue could be fixed by adding the list of official/unofficial ports list to the default configuration (or at least the most commonly used services).

Regarding the proxy administration;
The same idea could be applied in the default domain filtering rules.


Quote
[...]eventually we would like to accomplish other things--tracking individual users as they browse the Web (to determine who is violating an acceptable use policy, for instance)
(My 2 pennies) This is some other problem that need to be solved. I'm currently using a third party software to get daily/weekly reports on usage.


Quote
It may be necessary to adjust some settings in the Squid config file, but it is not hard to do.
(My 2 pennies) That should be done with the Zentyal GUI.


Quote
[...] implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service
(My 2 pennies) No easy way of making a custom page for the offending users is also something that I do find harsh. Most of the users don't know what Zentyal is and they don't need to know anything about it.  Some companies prefer to let the user know that the requested domain is off limits with the company logo along with its policies.   


Quote
I do have a 10K raptor disc for the cache
RAM would give you more bang for your bucks.


Best,

Marcus