Poll

How is HTTP proxy set and used in your gateway?

Transparent proxy + firewall rules for HTTPS
18 (75%)
Explicit (non transparent) proxy
4 (16.7%)
proxy.pac
0 (0%)
WPAD DHCP
1 (4.2%)
WPAD DNS
1 (4.2%)
other
0 (0%)

Total Members Voted: 21

Author Topic: How are you using HTTP proxy?  (Read 7104 times)

stuartiannaylor

  • Guest
Re: How are you using HTTP proxy?
« Reply #15 on: September 13, 2011, 12:25:11 pm »
I am glad you mentioned that Christian. I have done my usual and thought whats the point of filtering https if it can't be done. As you rightly point out domain filtering can be. Thats a major battle done in one and my assumptions had overlooked this.

If you are involved in public, education or community systems there are a hell of a lot of sites and proxies that use the https loophole to bypass filtering.

Looks like when 2.2 arrives  :) I will be going back to my wpad and pac file config with the zentyal DNS module.

I have been having a google for https content filtering / squid / dansguardian and there would seem to be nothing open source.
I get your piggy in the middle https scheme with products such as http://www.komodia.com/products/komodias-ssl-decoderdigestor/ or http://www.cymphonix.com/EXNetworkComposer.html exist as commercial closed offerings.

If you have the time a how-to based on application and target IE "http & https domain filtering with http content filtering with browser auto-config in one". Would be an excellent addition as the documentation is fragmented at the moment.
We have the Zentyal technical offering, seperate browser auto-config and the fact the transparent proxy alllows many https loophole sites so forcing https through dans is required.

You have changed my mind on my installs and it now seems strange that the majority of us just use the transparent mode (from the poll)

I would love to get the authentication mechanisms going without plain-text authentication login-boxes. As I want to employ tiered filter policies for group hierarchies and want to log user activity to DB level.
https://addons.mozilla.org/en-US/firefox/addon/integrated-auth-for-firefox/ makes the settings a little easier in firefox.
But as I see it the Zentyal methods will have to wait for SSO and kerberos.

Thanks
Stuart

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #16 on: September 13, 2011, 02:22:51 pm »
1 - Figures in the pool: it makes sense. Transparent proxy = less administration at client level although a bit more at proxy and firewall. Furthermore, Microsoft way of working: as this is transparent and works (more or less), you don't need to understand how it works  :P

2 - Browser Single Sign On: better to hear a bit more about Zentyal plan in term of Kerberos implementation. A lot of debate to come here  :)

3 - look at this if you plan to "intercept" SSL flow:
http://wiki.squid-cache.org/Features/SslBump
and this too http://wiki.squid-cache.org/Features/DynamicSslCert
Hoops, I should refrain myself to add more entropy here and there  :-[ but this aspect of infrastructure is so funny  :P

vshaulsk

  • Zen Samurai
  • ****
  • Posts: 477
  • Karma: +9/-1
    • View Profile
Re: How are you using HTTP proxy?
« Reply #17 on: September 13, 2011, 03:09:37 pm »
Ok I have a couple of questions:

I have created a virtual host wpad.home.lan (bound to 100.100.100.100)  When I go to the DNS section I see my home.lan and when I click on it ... I see wpad under hosts with the IP above.

Does this mean for every single domain wifi.guest, wifi.lan, DMZ etc..... I will have to create a virtual host?

Second:
I have never created a virtual host before so I am not sure where zentyal creates these directories.

Also when I look at the example of the wpad.dat example.... I am having trouble figuring out which parts I would need to change and could I use the same code for every domain (wifi.guest, wifi.lan, DMZ).  I don't want my DMZ subnet (192.168.0.0) to use proxy and just go direct to the net.  (I take in the code where it says zentyal.yourdomain.com I would change that to something like server1.home.lan)

Third:
On the file browsers .... firefox... will I have to change the clients configuration to auto detect proxy settings???  I take it the default configuration does not work??

Thank you !!!!!

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #18 on: September 13, 2011, 07:18:35 pm »
Ok I have a couple of questions:
This just shows that I need to imrpove my documentation.

Quote
I have created a virtual host wpad.home.lan (bound to 100.100.100.100)  When I go to the DNS section I see my home.lan and when I click on it ... I see wpad under hosts with the IP above.

Does this mean for every single domain wifi.guest, wifi.lan, DMZ etc..... I will have to create a virtual host?

If you use DNS mechanism, then wpad.home.lan will be used only by hosts for which fqdn is something.*.home.lan but not, for instance by something.wifi.lan

This means that you have to create one virtualhost per domain.

Quote
Second:
I have never created a virtual host before so I am not sure where zentyal creates these directories.

it's up to you in the config file (/etc/apacheé/sites-available/ebox-wpad.home.lan  look at Document Root section)

Quote
Also when I look at the example of the wpad.dat example.... I am having trouble figuring out which parts I would need to change and could I use the same code for every domain (wifi.guest, wifi.lan, DMZ).  I don't want my DMZ subnet (192.168.0.0) to use proxy and just go direct to the net.  (I take in the code where it says zentyal.yourdomain.com I would change that to something like server1.home.lan)

Well... proxy.pac describes what you intend to reach, not who or what tries to reach it. Thus this may need specific virtual host for DMZ based clients but it could be tricky or over-complex. In such case, you can choose DHCP mechanism that will describe, per DHCP range, which wpad to use. DHCP will be tried before DNS, so even if yuou have DNS that is reach by multiple DHCP ranges, DHCP will go first.

Quote
Third:
On the file browsers .... firefox... will I have to change the clients configuration to auto detect proxy settings???  I take it the default configuration does not work??

on IE, as far as I know, default is auto-discover. On firefox, I don't remember but I don't think it is set like this so, yes, you have to change it.

I'll try to improve my documentation later.

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #19 on: September 15, 2011, 11:52:59 am »
FYI, I've updated this "Howto" http://trac.zentyal.org/wiki/Documentation/Community/HowTo/SelectRightHTTPproxyDesign  in order to reflect new features thanks to Zentyal 2.2
DNS SRV and TXT records can now be used to describe wpad location.
I will test it soon.

I also tried to make some sentences clearer, thanks to your remarks.

stuartiannaylor

  • Guest
Re: How are you using HTTP proxy?
« Reply #20 on: September 15, 2011, 12:07:38 pm »
Looks excellent to me. Many Thanks.

Christian this brings me to another thread.
You have submitted an excellent document into the community. You have added community product that is freely available.
Zentyal might want to use your document to provide your solution into the Enterprise GUI in the proxy section.
To me this is the perfect place to fork a commercial Enterprise product.
There should be no difference in the community version and enterprise product in terms of features.
The only difference is that the enterprise product should be polished so that hacking isn't required and works from the box. The community is supported by the community and yes we get the same product but have to provide some hacks (your excellent documentation) as opposed  to selecting certain options in the Admin GUI.

http://forum.zentyal.org/index.php/topic,8023.msg32734.html#msg32734
« Last Edit: September 15, 2011, 12:26:12 pm by stuartiannaylor »

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #21 on: September 15, 2011, 02:34:32 pm »
Stuart,

I'm not sure to get your point. Perhaps my English is not at the right level yet ;)
However debate you launch is an interesting one, except that it may lead us to very long exchanges  ::)

To me, different products may exist, one open and potentially supposed to be adapted and one polished and closed with SLA. This is one model. Another model, and I believe Zentyal is on this last one, is to have only one single product, same for community and commercial targets, difference being additional services for customers willing to pay for it.

If Zentyal benefits from inputs, add-on, documentation or whatever idea community brings in, this is fine with me. We are in a kind of win-win model: community benefits from free product, helps to improve it and this may increase number of customers asking for commercial services on top of the free, shared, backbone.

If Zentyal, as a company, wants to add, in HTTP proxy section, features to handle WPAD, I've no problem at all with this. I haven't invented anything here  ;) and my major contribution, if any, is to try to convince people that "transparent everything" is, most of the time, the wrong way of dealing with reduction of administration cost and burden.
So I'm more on the religious side, if I can said so  :-*


vshaulsk

  • Zen Samurai
  • ****
  • Posts: 477
  • Karma: +9/-1
    • View Profile
Re: How are you using HTTP proxy?
« Reply #22 on: September 15, 2011, 03:31:43 pm »
I have a different question along the proxy line?

 I have a couple of VLans running.... for most of them the WPAD approach would be awesome.  However I have one VLan (wifi.guest) which as you could have guessed only used for guests coming over.  With the new captive portal module I am planning on leaving the wireless AP for that Vlan wide open and have all traffic redirect to the captive portal screen.  This way when people come over I can just give them generic username and password (guest1 password1) to log in.  Most of those guest either don't have their browser setup to automatically find proxy settings or are using mobile phones which might not have that option.  I don't want them to have any issues accessing the internet and I don't want to change any settings on their end. 

My question is it possible to have transparent proxy setup for that one VLAN, but use the WPAD method for all other VLans???????


christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #23 on: September 15, 2011, 03:41:57 pm »
:-)  your question could have been simpler  :P
I'm not sure not to reach my own limit of competence on this specific topic...  :-[

Well, this could be achieved, for what I understand, with some specific rules at firewall level to redirect ports on this interface assuming users using captive portal are connect on dedicated subnet.

The idea is to define here same rules as the ones used for transparent proxy: when request is received by firewall, forwarding rules redirect it to proxy. BTW, this is the reason why HTTPS can no be filtered by proxy, just because of this redirection at FW level.

So you could enable such rules for your dedicated VLAN only and I believe it would do the trick. Still I'm not 100% sure and never tried.  Let us know in case you succeed.

Hey Zentyal gurus! any feedback or input?  :D

More specifically, it requires:
- Not to bind Squid on this VLAN interface
- not to let these devices to use DNS "well know alias" DNS mechanism
- to nevertheless create wpad.dat with "DIRECT" statement in case devices on this subnet pick-up proxy.pac

hummm, it definitely deserves to drill down  :o
« Last Edit: September 15, 2011, 03:46:12 pm by christian »

vshaulsk

  • Zen Samurai
  • ****
  • Posts: 477
  • Karma: +9/-1
    • View Profile
Re: How are you using HTTP proxy?
« Reply #24 on: September 15, 2011, 04:26:58 pm »
Trying to figure out what specific rules is whats going to be an issue.

Currently I can't isolate any subnets from one another using the firewall rules when transparent proxy is turned on.

Example.
I have setup a firewall rule between object wifi.guest (100.100.100.0) and object home.lan (200.200.200.0) to deny any connections.  When I try to ping or tracert everything works correctly and clients on which belong to these two objects subnests don't connect.
However when I try to access a control dashboard through HTTP it connects through the transparent proxy. Under logs nothing is showing up in the firewall, but under proxy logs (zentyal dashboard) it shows the connection.
I have tried several firewall rules to somehow block the transparent proxy from picking up these connections, but it does not work. 

I can't find a service that controls how transparent proxy picks up traffic and until I do I am not sure if I will be able to control my VLAN scenario.  The proxy should work with the rules set by the firewall, but it does not seem to be doing this. 

One other thing bases on your documentation for wpad  the clients on wifi.guest will never pick it up because they will be searching the wpad.wifi.guest domain which does not exist.  Since they will not be able to find this wpad.wifi.guest domain there should be no need for a rule in wpad.dat file for subnet (wifi.guest) = direct.  Am I thinking correct or am I still not understanding how DNS and WPAD work???

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #25 on: September 15, 2011, 04:46:31 pm »
You are definitely thinking correctly... or at least I'm thinking the same and have not the feeling I'm wrong here  ;D

Joke aside, if your domains are different, then you're perfectly correct about catching or not wpad DNS record.

Regarding use of transparent proxy: in transparent proxy mode, HTTP port is redirected to Squid (localhost 3129). At Kernel level or FW? I think it's FW but involves kernel feature. hummmm  ???

If you set up Squid in non transparent mode and tune FW redirection port to catch only source from 100.100.100.0, then it works:
from 100.100.100.0 you will have transparent proxy behaviour
from 200.200.200.0, you will never have transparent proxy and will access Zentyal web pages directly.

does it make sense to you or am I wrong?

What I need to refine tune is capability to tune these catching rules.

Your question is triggering another one that is capability to have Squid service binding on all - or not - interfaces. It requires some hook here to have Squid binding only on the home.lan subnet (http_port 200.200.200.x 3128).

This could be useful for some advanced configuration but then I'm afraid it will not ease use from SMBs... it's difficult to always try to find the right balance between flexibility and easiness  :-[
« Last Edit: September 15, 2011, 04:52:46 pm by christian »

vshaulsk

  • Zen Samurai
  • ****
  • Posts: 477
  • Karma: +9/-1
    • View Profile
Re: How are you using HTTP proxy?
« Reply #26 on: September 15, 2011, 08:27:18 pm »
Thank you for the information !!

I agree that this setup is probably more complicated than most SMB require. 

I will play with it and see if I can get it working.... I think once I figure out what firewall rules work with controlling how traffic gets directed to the transparent proxy than I will be able to set this scenario up on a basic level.... just to get me by for now.


Also I will try to search the internet about to how to hook squid to a particular Vlan just like you can do different DHCP servers/ VLAn.... I would like to setup different squid modes/VLan.

Perhaps if I find a way to set it up like I want (transparent with filter for my wifi.guest) and (authenticate + filter for my home.lan) so that squid binds to particular 802.1Q Vlan trunks.... I will take a look at how to make it possible to set something like this up in dashboard level.  IT will make things more complex, but I think it could be useful on an infrastructure level.

Sam Graf

  • Guest
Re: How are you using HTTP proxy?
« Reply #27 on: September 17, 2011, 02:30:43 am »
Since Christian dislikes the transparent proxy as not being the technically right way to do things, I thought I'd add some possible ammunition to the anti-transparent-proxy argument. It appears that trying to run at least some zero client hardware behind a Zentyal transparent proxy interferes with establishing at least a Citrix HDX connection.

I don't actually understand why, though. First of all, HDX connections get routed to HTTPS, as I understand it (maybe the problem occurs right there?). Second, there is no place to configure a proxy in the zero client I'm using. Yet simply disabling Zentyal's transparent proxy option seems to make a difference.

I need to experiment more, but I've lost a couple of hours and a handful of hair trying to figure out where my HDX connection was going wrong, and it's somewhere at the proxy. It may be an artifact of using a transparent proxy.

christian

  • Guest
Re: How are you using HTTP proxy?
« Reply #28 on: September 17, 2011, 09:48:22 am »
Sam, I do not "dislike" transparent proxy.  ;D ;D ;D

Trust me even if all what I write shows the opposite  ;)
What I really don't like with transparent proxy is not the technical implementation but the fact that people go for this not understanding what are the pros & cons.
In some cases, I really do understand that transparent proxy could be the best implementation (some few cases only, do not think I've totally changed my mind  :D)

And my is not with proxy only but with "transparent everything" approach, surprisingly especially with such solution targeting SMBs: this kind of design targets people or organization not necessarily understanding all the technical stuff and the goal is of course not to change this. I mean that because they don't understand, solution has to make it easy and reliable and the transparent approach which looks the easiest is in fact the most complex because of the side effects requiring tricks at FW level for HTTPS plus some other unexpected stuff.

Of course, this reflects my own personal view only  8)
I can see that large majority is using transparent mode: Zentyal documentation promotes this. Therefore my documentation trying to show "something else"...  :-*

stuartiannaylor

  • Guest
Re: How are you using HTTP proxy?
« Reply #29 on: September 17, 2011, 01:17:42 pm »
The transparent proxy is good for an "out of the box" solution.
There is a huge ammount of https holes and services out there that allow the proxy to be bypassed.

So like Christian if I was going to install and configure the proxy I wouldn't use the standard transparent settings.

Its great though that we have the choice and documentation to support this.

Stuart