Author Topic: Zentyal's proxy in the real world  (Read 22165 times)

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #15 on: May 20, 2012, 06:55:52 pm »
Quote
Since there is no "allow cPan*l" proxy option[...]
This is configured in the firewall and not the proxy.
Hi Marcus. I don't think so, if my recollection is any good. In the case of an explicit proxy, I think there is hand configuration of the proxy involved, unless I'm mistaken.

Quote
[...] implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service
(My 2 pennies) No easy way of making a custom page for the offending users is also something that I do find harsh. Most of the users don't know what Zentyal is and they don't need to know anything about it.  Some companies prefer to let the user know that the requested domain is off limits with the company logo along with its policies.   
I hadn't thought about that and I think you make an excellent point.

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #16 on: May 20, 2012, 07:09:47 pm »
I want to move forward on the idea that the explicit proxy, all by itself, isn't the goal of an administrator, I don't think. It's explicit proxy and something the explicit proxy makes possible--authentication, network objects, and so on.

So I want to start a list of things that happen in various scenarios, things that may take a simple admin by surprise. I'm going to start with explicit proxy plus authentication. The behavior of certain things changes:
  • Automatic updates of various things may fail unless configured to authenticate (when possible)
  • Mobil and guest devices become more difficult to manage, in the case of guest devices, especiaily
  • Standard HTTPS port assignments that are not 443 may fail
  • Re-authenitcation may be required over the course of the work day, causing some end user ill will :)
It would be nice, I think, to begin to develop the "workarounds" required to solve these "problems" (and any others people want to add) in detail, for those admins who might give up on using explicit proxy without them.

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #17 on: May 20, 2012, 08:47:23 pm »
Sam,

I'm not very comfortable with your approach  :-\
Of course, administrators should not select design because of the beauty of it but because of features or requirements. Given design is only what you have to implement so that requirements are covered.
Based on this, why do you force yourself to go in a direction that may not fit your requirements and that is also not aligned with what you feel to be the "standard design", i.e. transparent proxy  ::)

Explicit proxy is not mandatory  ;)

Each design has pros and cons and none is perfect, reason why I would suggest to start with requirement rather than design.

To elaborate a bit:
- Explicit proxy doesn't mean authentication (while the opposite is not true)
- Profiling means explicit proxy
- HTTPS filtering means explicit proxy
- Explicit proxy doesn't mean manual device configuration

If I try to rephrase it, first step can be to enable explicit proxy without authentication if you feel that prompting users for authentication is too much for them and also if authentication is not required.

There is no perfect workaround neither: one example is authentication
One can not ask for authentication because it brings some features at proxy and at the same time ask for SSO so that authentication occurs only once. Such approach will just show that security at workstation level is now more critical and not for proxy only.

Transparent proxy
Features:
HTTP filtering
No HTTPS filtering (control, if needed, has to be done at FW level)
No profiling (meaning no different proxy behaviour based on user) not user tracking

Pros
Easy deployment

Cons
Potential side effects because client is not aware of proxy and thinks to communicate directly with server
Difficult to analyse proxy log
Clients on LAN must resolve internet names
No cache for intranet servers

Explicit proxy
Features:
HTTP and HTTPS filtering
Profiling if authentication is enabled

Pros
Proxy log reflects internet usage
WPAD permits to control access to Intranet web server: direct or through proxy (useful for cache in case of server over WAN)

Cons
Requires to define proxy on each device if not "auto discovery service" is deployed
May require multiple settings per device for programs not using OS or browser settings
Non standard HTTPS ports have to be explicitly defined

To be completed...


Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #18 on: May 20, 2012, 10:06:03 pm »
I'm a little confused :-[.

I'm not trying to force anything. I feel like perhaps you assume all Zentyal admins can think of Zentyal's proxy in the abstract. For instance, until you brought it up the first time, I had never heard of WPAD. How can simple people like me be as intentional as to understand the pros and cons of this and that, as abstractions? Keep in mind that I installed eBox itself not because I clearly understood its pros and cons relative to other offerings or because I was convinced of the technical superiority of Linux, but because eBox had a VPN service that even I could use to solve a company problem. Not very scientific, but that's all you get when it comes to the real world admins I am hoping to help with this topic. The alternative is to tell people like me (which some have in fact suggested :) ) that people who aren't "real" admins shouldn't be managing SMB IT in the first place. I won't get into my response here. ;D

I bring up authentication not to force anything--well, the boss said, we want this, so I was tasked with making it happen. Authentication was the very first step to get the boss what she wanted. She hears things and reads things, even if it's just in the news, enough to be able to ask questions:
Q. If we had a user violating our AUP, would you know about it?
A. I would know that someone was violating our AUP, but not whom.
Q. Why not?
A. Because I'm not even close to being as smart as christian. :D
Q. I'll remember that at review time. In the meantime, what is the point of having an AUP when you can't even let me know if a particular person is violating it? Since it's better to get rid of you than the AUP, please don't leave me with only one option.
A. Yes ma'am.

OK, I'm exaggerating a little, but that's not far from how it really went.

So I deny I'm trying to force myself into anything. <rant>I "prefer" (because it is simply not true that I actually prefer a transparent proxy) a transparent proxy (when it comes to Zentyal) because the alternative isn't nearly so simple as just implementing WPAD. It's potentially half a dozen things I have to do by hand, and maintain by hand, just to do what the boss asked. And then there's the other things that I'm expected to manage by hand to do what we want ... the DHCP server for example. So at the end of the day, it's really Zentyal that pushed me in the direction of a transparent proxy, because the total cost of ownership of an explicit proxy keeps me from doing my other work.</rant>

That said, I don't mind at all following you as you take the approach to this that you think best :) . I am certain to learn something :) .

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #19 on: May 20, 2012, 11:47:22 pm »
Sam,

For sure at the very beginning, for someone not understanding all the technical stuff but having only needs and boss with requests, asking to have theoretical approach to balance and discuss pros & cons has little meaning. I fully share.
However, we are discussing this since months, which give us time to:
- understand better technical stuff
- measure pros & cons of different choices
- refine requirements if needed
- at the end, make the right decision hopefully not to far from the ideal target, or at least understanding why the "perfect" world doesn't exists  ;D

Goal is not to tell you that you have - or not - to manage SMB. You are in this position right now and have this responsibility. Goal is to exchange and share our view, share our knowledge and achieve the best result. You don't know everything, neither I do but working together, we should achieve better result because there are some aspects you know and some other I know. But this means that we have to move from our respective positions.
Rephrasing this, I mean that at some point, one has to learn some technical stuff in order to move ahead. If we keep debate at the end-user level, the is very little we can do  :-[

Back to this technical stuff, I don't understand why you stick on this position, thinking that explicit proxy will need half a dozen of manually managed configuration options.

If I try to summarize what I currently have in mind:

1 - Once explicit proxy is enabled, auto-discovery is highly suitable to avoid managing clients manually. This can be done 100% using Zentyal GUI if option you select is DNS, now that SRV and TXT records are available in Zentyal interface.
However, pushing this via DHCP is very suitable too but not available in Zentyal GUI.
2 - WPAD server can be managed via Zentyal GUI but proxy.pac file (wpad.dat) has to me manually managed.
3 - in case of use of non standard HTTPS ports, squid conf has to be manually tweaked.

What else?
- You may have some (very few) devices not implementing auto-discovery.
- You may have (here again very few) programs not using OS or browser settings to determine whenever proxy has to be used or not
- if you need profiling or identification, then authentication is required. Difficult here to have something (authentication) and the opposite (not to be bothered by authentication) at the same time  ::) and as I wrote, once SSO (thanks to Keberos) will be there, we will discuss at length about security on workstations  ;D ;D ;D

but this is what you have to put in the balance to decide whenever explicit proxy is better than the few drawback. Once you have this in your hands, no one can decide for you because you are, at the end, the one operating and managing.
This is the way I perceive it  8)

Then we may hope that Zentyal team, in a next version, will improve their platform and include these few interfaces so that everything can be done using Zentyal GUI. do not take it wrongly: it will never prevent to understand a bit of technique in order to make the right choice.



Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #20 on: May 21, 2012, 03:00:08 am »
We have been discussing this, true but I have miscommunicated. This isn't supposed to be about me, but about other admins who want to master what Zentyal offers. And it's about what Zentyal offers today, as is. It's about tips and tricks, with enough theory to explain the how and why, but with the emphasis on accomplishing real world tasks--because advanced planning presupposes the knowledge to plan. At bottom, it is about reducing any technical barriers to Zentyal adoption, if not eliminate them altogether.

Instead, it could be about generic drop-in scripts, detailed how-tos, and so on. It isn't about me and shouldn't be about me because my organization is committed to a different course. That's also something we have discussed before.

My goal was to point out that people like me need help not just understanding to make choices, but also enough handholding to make it work, at least the first time. Your earlier efforts at detailing the WPAD process are something of what I had in mind.

Keep in mind that people will have to role out their theory in production. I don't know of a single small business that can mirror production environments in a test setup. So very likely they will need to not only know what's ahead of them, but also just what's in front of them at the moment. They will learn more from their experience then from anything else, almost certainly. I say that as a person who is an educator by training, not a technician. Of course, I'm not doing too good at educating this community to think like a Linux noob and to see how Linux noobs can use Zentyal, with a little better help, so take what I say with a grain of salt. The proxy is a perfect case where more help is desperately needed, IMHO.

So, carry on as you see it. Keep in mind that so far, from how I see it, you are long on reassurances, but short on tips and tricks :) . As I said at the start, it might be pure ignorance that makes someone think they face a mountain when really it's not that bad. So your reassurances are encouraging. But I would still have no idea how to do what you're describing, personally speaking. So I would have hope if this were about me, but I would still be waiting for more information, the tips and tricks part to make the magic happen.

Carry on. :D

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #21 on: May 21, 2012, 07:27:28 am »
Keep in mind that so far, from how I see it, you are long on reassurances, but short on tips and tricks :)

 ;D ;D ;D ;D ;D
I can accept this, even understand but I can't really help because, although my doc is not perfect, I thought I already explained almost everything for someone wiling to deploy.
So in order to improve this, we need someone else to highlight what is missing, what is not clear and what is wrong so that we can update this HowTo.

However, as I tried to explain multiple times, such decision (deploying explicit proxy rather than transparent one) cannot be made without some investment on the technical side. do not expect me to produce a cookbook (e.g. like current Zentyal documentation) with screen-shot showing "click here and there" only because this is, to me, worst than nothing  :-X You have the howto but you don't understand why and as soon as you face the first either problem or unavoidable side effect, you don't understand what happens, neither why. Best case, you need a lot of support from this forum  :D Worst case, you revert back and decide that explicit proxy doesn't work  >:(

Zentyal's goal (that is to provide interface to allow people with no IT admin technical background but in charge it to achieve something easily) is a very tricky one. It reminds me Windows compared to Linux some years ago when you had thousands of Windows admins having deployed Windows based platforms without single understanding of how it works. And most of the time, it doesn't really work  ;D or at least poorly with security holes and poor quality. Is it because of Windows? No, of course. More than 20 years ago, Microsoft had the Orange book certification with Windows NT !
This is because of the approach that makes it happening too easily thank to GUI and "click and run" hiding the difficulty behind  :o

I'm not promoting the command line with complex grep/pipe/awk  ;D I'm fighting against the "cookbook only" documentation even if I do understand that one can not produce documentation re-explaining everything from scratch. To me there is something in the middle explaining the "how" and "why" so that beginner admin can make choices because he understand or can decide to learn a bit more of technique before diving.

So, what is missing with current HowTo preventing you to understand?  8)

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #22 on: May 21, 2012, 02:35:13 pm »
Some quick comments:

However, as I tried to explain multiple times, such decision (deploying explicit proxy rather than transparent one) cannot be made without some investment on the technical side. do not expect me to produce a cookbook (e.g. like current Zentyal documentation) with screen-shot showing "click here and there" only because this is, to me, worst than nothing  :-X You have the howto but you don't understand why and as soon as you face the first either problem or unavoidable side effect, you don't understand what happens, neither why. Best case, you need a lot of support from this forum  :D Worst case, you revert back and decide that explicit proxy doesn't work  >:(
Agreed, 100% in principle even if we are not yet agreeing on the how-to of a how-to. :D

I'm fighting against the "cookbook only" documentation even if I do understand that one can not produce documentation re-explaining everything from scratch. To me there is something in the middle explaining the "how" and "why" so that beginner admin can make choices because he understand or can decide to learn a bit more of technique before diving.
Again agreed, 100% in principle if nothing else. What I'm fighting is the tendency, as I perceive it, to push for the "buck up and learn Linux" type of documentation. You see it frequently here, in one form or another. "Issue sudo blah blah blah and see what happens." How is that any better than "point and click" stuff if I have no idea what sudo blah blah blah actually does? And since Zentyal itself keeps the command line out of my daily experience, how can anybody here expect it to become second nature to me? :o

I often can't decide which is more obtuse--the command line, or Perl. I generally end up thinking Perl is. ;D

The underlying problem is two-fold, I think. Those who understand the technology well often don't have patience to teach beginners. It takes too much time and concentration to conduct "special education classes for Linux dummies," especially in the context of a support forum.

Additionally, FOSS advocates aren't always very objective about their passion. I think Linus Torvalds said it best in a clip I heard recently (and I'm paraphrasing): "I would hope that people don't choose open source software because it is somehow the morally right thing to do. I would hope they choose open source software because it's better software." Some open source software is still very much a work in progress and isn't truly better yet. But if you "buck up and learn Linux," you can compensate. ::)

In any case, it is axiomatic in education that to teach, you must take a person from the known to the unknown. It's true that I must be willing to learn. It is also just as true that the teacher must be willing to teach. So in general, whatever we write as a how-to must start firmly within the known, and only then move to the unknown, to the new. I think a good general rule is to start with the Zentyal GUI and what it teaches (think of Zentyal as a teacher, because for good or ill, that's what it is), since that's reliable common ground. if the Zentyal GUI doesn't teach it, or teaches it poorly, then assume the student will not have a good grasp of the concepts. The whole proxy discussion is a textbook example, since the Zentyal GUI doesn't teach this concept particularly well, IMHO. Looked at this way, something like "Once explicit proxy is enabled, auto-discovery is highly suitable to avoid managing clients manually. This can be done 100% using Zentyal GUI if option you select is DNS, now that SRV and TXT records are available in Zentyal interface." can be perfectly clear and yet still fly right over the head of a student, since we may not have connected enough of the dots between the known and the unknown.

If we are unwilling to follow this known-to-the-unknown axiom as we write how-tos, then it is really unjust to criticize Zentyal users who "revert back and decide that explicit proxy doesn't work" as bad students. Only if they refuse to grapple with the technical side are they bad students. If we don't give them enough finger holds and toe holds as we drive them up what is to them a very steep mountain, they will naturally fall through no fault of their own.
« Last Edit: May 21, 2012, 02:43:30 pm by Sam Graf »

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #23 on: May 21, 2012, 02:59:08 pm »
1 - I do not criticize or, better expressed I hope, I try  not to do and if it looks like I'm criticizing, this is only because of my broken English  ;)
2 - I don't know Linux CLI and do not push anyone to use CLI first before GUI. I'm perhaps one of those using most often "man" command  :D
3 - However when I use something like mail, proxy or VPN service, what I do need is to understand how it works, in term of concept and protocol. So what I'm pushing for is not to learn Linux but to learn concepts. Trust me  ;)  e.g. knowing pretty well LDAP, I've been teaching and correcting quite often Windows administrators because I understand some aspects of AD as this is yet another LDAP server  8)
4 - to me, Zentyal GUI will never teach anything. Not that I like text. I do prefer clear drawing but can't figure out how to learn from GUI. Well, you may make some guess but hardly more than this. Hopefully, if at the end is works, you will shift from guess and assumptions to noun.
5 - So we are now in a crazy  loop: if I don't know what you know or don't know, I don't know where to start and definitely refuse to describe everything from scratch, although I'm not sure it will consume more time than debate we currently have  ;D ;D  But at least this debate is a funny one while writing doc is, for me, painful  :-[
6 - I'll try to introduce some screen-shots in my howto in case it helps. Please tell me which ones you would like to see here.

Few more posts and we will together win the price for the longer thread with the fewer posts  ;)

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #24 on: May 21, 2012, 03:30:32 pm »
4 - to me, Zentyal GUI will never teach anything.
All experience teaches. Zentyal is the common experience here. We are in serious trouble if we cannot use this to our advantage, as teachers.

5 - So we are now in a crazy  loop: if I don't know what you know or don't know, I don't know where to start
See above. Start with what we know. There is a reason why there are very few good technical writers in the world, because it is actually quite hard to think this way. Forget the physics for a minute; study Stephen Hawking's books for how they teach.

In this case (my example from above), it's not automatically clear what the proxy service and DNS service have in common, but to a certain point I can trust my teacher long enough to gain some desperately needed experience. So resort to the cookbook style documentation just long enough for me to know how to get the first steps behind me. Write the cookbook in such a way that I can't get any further than you want me to get, but at least get me to something new, some new system behavior that gives me the satisfaction of accomplishing something. The mountain in front of me automatically gets smaller. I can see that what you told me to do actually works, though I may not yet know exactly why. If you always put comprehension before experience, you will teach less. See Socrates. But, once I have had the experience of doing this and that and can see that it works and have had some reason for joy, then set me the task of understanding what I just did. It makes no difference if the tool was the command line or the GUI, the educational principle is still the same. I suspect even you didn't know all there is to know about mail servers before you had your first one up and running.

my broken English
Your English is very good. If we had to rely on my French, we could not talk at all. :-[

Few more posts and we will together win the price for the longer thread with the fewer posts  ;)
If we ever get to the original purpose, we can split this. In the meantime, what is the prize? I love winning prizes!!
« Last Edit: May 21, 2012, 04:19:41 pm by Sam Graf »

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #25 on: May 21, 2012, 04:59:51 pm »
<evenmoreofftopicthantherestofmyposts>So I had a phone conference in the past hour with a Web host. I want to redirect a domain they control to a domain I control through a different host--an acquisition transition. We are going to let the domain name they control expire, but we have a contract with them until fall; I am going to honor the contract but not transfer a domain I don't want. The W3C recommend issuing HTTP 301 headers so that search engines and bookmarks are properly updated. The host's server administrator says to me that I have to do that. I say, well, no, I need your server to issue the permanent change of address as things stand right now. What am I missing? I'm on speaker-phone which isn't the best situation anyway, but all I get is mumble mumble A records mumble mumble. And then, but I can do the 301, it's just not how I would do it.

This is a teachable moment. We have in common a specific task--redirecting one domain to another on a permanent basis the right way. If there is a better way to do it than my request for a 301 header from his server, and I ask for that information, I am teachable. But my teacher would not teach me. And I am the paying customer. >:( </evenmoreofftopicthantherestofmyposts>

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #26 on: May 21, 2012, 05:03:23 pm »
As explained in the HowTo, reason why DNS is involved in proxy design is because of this draft. I'm not aware it has ever been promoted from draft to RFC but this one is used everywhere.

Once you know it, this is just a matter of describing the right SRV and TXT or wpad.yourdomain entry (well known alias method).

Quote
All experience teaches. Zentyal is the common experience here. We are in serious trouble if we cannot use this to our advantage, as teachers.
So I am  :-[ because I can't tell you what you will learn once I will have shown screen-shot showing that you have to click here and there  :(

Sam, we are pretty much in line but you play a kind of biased game or at least this is the way I feel it. Let me explain.
With what I've described in the howto or what we discuss here at length, someone wiling to implement explicit proxy has enough inputs to do it, even with only partial understanding of how it works under the hood. However, I'm not saying "howto" will cover all different cases one may face. You have illustrated this with your very good example: explicit proxy may not work if you need to access non standard HTTPS port.
From this point, either you feel it doesn't work and you revert back to transparent proxy or you try to understand better and go one step further in case there is a solution using different configuration or even workaround.
What are you expecting from me here? Of course I should (and I will BTW) add a warning covering this aspect but I'm pretty sure you or someone else will come tomorrow writing: "hey, it doesn't work! I've been trying to to stack parent proxy with peer cache and since I'm facing performance issue!. I'll revert back to transparent proxy!"

So my goal is not to push anyone to go deploy explicit proxy rather than transparent one but to say:
- to me, explicit proxy has mode added value than drawback while transparent proxy is the opposite
- here is how to proceed if you want to deploy such design

and obviously, each time someone asks forum about feature not achievable because of use of transparent proxy, my obvious point is
"do you use transparent proxy  :P  ???"
but then admins have to make their own choice. This is not proselytism from my side  :P

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #27 on: May 21, 2012, 05:06:59 pm »
Following your [off topic] post:
- why do you want to use HTTP 301 and not 302?  301 is permanent redirect, thus the correct one. Sorry.
- why not chatting on IRC too  ;)

** edit ** mix-up between 301 and 302  :-[
« Last Edit: May 21, 2012, 05:24:15 pm by christian »

christian

  • Guest
Re: Zentyal's proxy in the real world
« Reply #28 on: May 21, 2012, 05:17:46 pm »
What am I missing? I'm on speaker-phone which isn't the best situation anyway, but all I get is mumble mumble A records mumble mumble. And then, but I can do the 301, it's just not how I would do it.

If I had to address this, I would:
- determine whenever users still have or can access the former web site
- understand whenever I can or not update web pages on the former web server
- based on above, change DNS to point, if this is the goal, to the new server. This has to be done carefully because there are some pitfalls as web site reached by user is not the one initially requested. BTW, is there any use of HTTPS here?

Sam Graf

  • Guest
Re: Zentyal's proxy in the real world
« Reply #29 on: May 21, 2012, 05:25:20 pm »
Following your [off topic] post:
- why do you want to use HTTP 301 and not 302?
- why not chatting on IRC too  ;)

Because the W3C recommend 301 for a permanent change of address--a permanent redirect, based on the HTTP 1.1 specification for redirection. That's all I know about it. :-[

Sam, we are pretty much in line but you play a kind of biased game or at least this is the way I feel it.
I apologize. I don't want to be biased. I want to be concise and thorough in one place, or at least have all resources refrenced in one place. But you may be right: I simply don't know enough to do this based on what's already available, and that leads to a sort of bias.