Zentyal Forum, Linux Small Business Server
Zentyal Server => Installation and Upgrades => Topic started by: Sam Graf on May 16, 2012, 05:07:43 pm
-
Zentyal provides two basic types of proxy implementation: transparent, and non-transparent--where the proxy configuration details have to be declared explicitly on client devices. Which implementation to choose, and why?
At least some portion of the Zentyal user base is not going to use the proxy at all. This topic is for admins who are not going to not know the answer to the basic "which proxy implementation?" question beyond the "it works!" test. They will not immediately know what the choice really means to their work-flow, to their ability to conform the technology to company policy, and so on.
For some of us, Zentyal's transparent proxy implementation is the correct "just works" choice. It involves minimal fuss for all devices on the network, including mobile and visitor devices, while providing robust Web content filtering and virus screening.
For some of us, that level of service is great as a starting point, but eventually we would like to accomplish other things--tracking individual users as they browse the Web (to determine who is violating an acceptable use policy, for instance), or implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service. How do I get from where I am and what I know to where I would like to be, based on a vague idea of what I want?
The problem in answering that question is that Zentyal's explicit proxy is non-trivial to use if one lacks knowledge of how proxies work in general and how Zentyal's implementation works in particular. For purposes of this topic, I'm going to assume a simple admin--he knows what he wants to accomplish in general terms (say, block objectionable Web content, track policy violations) but does not understand the underlying technologies well, if at all. Knowledgeable admins may use Zentyal's GUI-managed services as a convenient starting point and then routinely hand configure the rest. Other admins may have Web content filtering for the very first time only because Zentyal makes it remarkably easy to integrate that service into an easy-to-manage network infrastructure, all in a single product, and that's all they know about it. And they love it because they can do something they might not otherwise be able to do! But some day something new arises...
If an exploration-minded admin sets up an explicit proxy (she can get that far since the process it relatively straightforward) and feels very good about that first step because everything is working great--until a staff member tries to access their hosting cPanel, and can't get in...now what? Since there is no "allow cPanel" proxy option, she's stumped. Does she give up on her experiment and go back to the transparent proxy? Is that the best choice for her to make?
Hopefully this topic can amass the tips and tricks necessary to exploit Zentyal's proxy to whatever purpose a simple admin might want, including the fundamental tips on making the choice in the first place--what's gained an lost in each case.
-
Excellent starting point +++
If you don't mind, I would just add some explicit ;D statement for those not reading between the lines:
- access to cPanel example in Sam's description is because out-of-the-box, Zentyal proxy implementation may not support HTTPS access to port different from 443.
- notice that with transparent proxy, there is no such problem because HTTPS access is not done via proxy but directly via firewall.
-
Thank you for the explicit clarifications. ;D
But consider my poor admin's problem...she has no idea why cPanel access is suddenly blocked. She can't read between the lines because she has no idea what's going on. I was hoping we could take a minute to feel her predicament. :P
I used that example deliberately, out of my own experience, in keeping with my intent to see this stay true to the real world. We had people suddenly unable to access cPanel-based webmail. It wasn't really suddenly, it was just that a period of time had elapsed between the start of the explicit proxy experiment and the attempt to access webmail (on a machine without a proper e-mail client). So there was no immediate connection between the two in anybody's mind. "Try again tomorrow" didn't work, so then the mental machinery had to crank it up a notch, to connect dots that didn't have a very obvious relationship--other HTTPS connections work, after all, and we're explicitly ;D including the port in connection attempt ...
-
Much clearer.
For these Zentyal users, transparent proxy is the best choice. 8)
As you wrote, it works out-of-the-box. Let them use it like this until they have additional needs, maybe later nor never ;)
Again, people looking only for cache feature and potentially basic filtering on HTTP only can be satisfied with transparent proxy. Not need to look further, especially if it has to be looked from the end-user standpoint.
As previously discussed in another topic, some Zentyal "users" made this choice because of its simplicity and their willingness not look at the technical details. If trigger is this one, then transparent proxy is their obvious best choice.
For them, it doesn't matter if client has to resolve names or if there is not HTTPS filtering.
One detail, having in mind such users: if only few users are behind Zentyal box using transparent proxy with no specific "proxy feature", then they should rather not use proxy at all, they will get better performance:
- cache efficiency ratio, with only few users, is below '1"
- because of the way it works, transparent proxy is slightly slower than explicit proxy, thus slower than not proxy at all.
This said, the point is "how to address all the other cases" if any :D
-
One detail, having in mind such users: if only few users are behind Zentyal box using transparent proxy with no specific "proxy feature", then they should rather not use proxy at all, they will get better performance:
- cache efficiency ratio, with only few users, is below '1"
- because of the way it works, transparent proxy is slightly slower than explicit proxy, thus slower than not proxy at all.
Very good point.
This said, the point is "how to address all the other cases" if any :D
Let me suggest that for my simple admin, content filtering and virus screening likely are going to be an attraction. He may have downloaded Zentyal for that alone (my first interest in eBox was its VPN possibilities; I didn't care about the rest at first) and then discovered other wonderful tools in it. They may not care about caching or about authentication, at first. So transparent proxy seems the natural choice.
Given that interest, let's take it to the problem solving level. We had a local elementary school principal access child pornography using school equipment on the school network. The access was discovered during a routine audit, and he got dismissed for violating the school district's acceptable use policy. Some simple, just-getting-by non-profit admin works for an organization where all staff and volunteers have to pass a background check, so broadly speaking this kind of behavior is an issue, and he wonders about what happened at the school very seriously for the first time--I'm trying to block access, just like the school does, but would I know if somebody had gotten access anyway, just like happened at the school?
So he comes here, describes what he's after and why, and then asks: "Is there a way for me to know about Web access policy violations on an individual level? I don't see user names and site access connected together in the logs. Is the transparent proxy coupled with Zentyal subscription services my best option? Will that tell me what I need to know? Or will I need to use a non-transparent proxy, since I see something about user authentication there? Does anybody know exactly how the two (proxy and subscription services) work together?"
-
I don't know anything about subscription service and even don't understand why this jumps in our debate :-[
I've read your post already 3 times and still don't understand. Sorry. Do you mind elaborating a bit ???
-
Subscription service allows for extensive logging/reports, perhaps that is what Sam meant?
To elaborate a bit more, I came from a Windows box with Squid installed. For me, caching was and is the main reason to use Squid.
Internet speed is rather limited at my end, so making efficient use of our Internet connection is most important.
The network grew over time and I decided to change to a all-in-one Linux based router solution.
Zentyal is all that and it provides me with the web proxy. I have dedicated 10GB of disk space to the cache and it saves a heap of bandwidth every day.
Apart from the proxy cache, other goodies are the logs. Not so much per user based, but more like general stats of what type of content is being used on my network.
Then came the adzapper, I consider it a must-have. The content blocker also made it on my list.
One of the things that do not really interest me (at the moment) is authentication. We don't need to track policy violations either. Perhaps at a later point in time.
So far for my real world scenario, I hope it adds to this topic.
Cheers.
-
Subscription service allows for extensive logging/reports, perhaps that is what Sam meant?
Correct. For really real world examples of this type of question already discussed here, see:
http://forum.zentyal.org/index.php/topic,6180.0.html
http://forum.zentyal.org/index.php/topic,7628.msg30343.html#msg30343
-
Clearer, although link shown by Sam doesn't explain anything, at least to me, for what concerns subscription.
Let me explain: there is a link from sixstone to Zentyal website but this doesn't explain the detail of subscription neither how it works (and such link is not supposed to explain this level of detail, I agree).
However I understand now that debate is about getting detailed report about web usage (I focus on web usage as we discuss HTTP proxy, I'm sure subscription provides much more).
What I don't understand is why this debate about "internet usage analysis" while we discuss about transparent vs. explicit proxy. Is there any impact I do not perceive?
@Escorpiom: you explain benefit of use of proxy on your network. This is pretty clear and exactly what Sam meant with "real world". I understand also better what "subscription" can provide although this is a bit strange to me: you have limited internet bandwidth and decide to buy services consuming part of your bandwidth :o I'm a bit confused. Well, I know Zentyal doesn't provide anything to really look at log content. Reasons why we discussed stuff like Awstats in one of the links Sam showed. But here again, as this log analysis stuff any impact on proxy design choice? I don't understand.
Last but not least, if internet browsing performance is your real concern (which I can easily understand), then we are entering in another dimension because achieving top perf with Squid requires tuning that is much more trick than transparent vs. explicit and very far from end-user approach. It starts with high performance disks dedicated to proxy cache, tuning of cache content cleaning, memory cache... well, another world, far from our simple debate 8)
-
@Escorpiom: you explain benefit of use of proxy on your network. This is pretty clear and exactly what Sam meant with "real world". I understand also better what "subscription" can provide although this is a bit strange to me: you have limited internet bandwidth and decide to buy services consuming part of your bandwidth :o I'm a bit confused.
I'm confused too. What do you mean by buying services that consume part of my bandwidth? If this is about a subscription service, I only have the free subscription to peek at the Zentyal Cloud, but most of the time it's offline. I'm not buying anything that I'm aware of.
I do understand that we can take the whole proxy thing to the next level but neither my wallet nor my knowledge is sufficient to do this. I do have a 10K raptor disc for the cache :)
However, perhaps you're trying to get another point across: No proxy is faster than transparent or explicit proxy? If so, then I obviously don't agree.
Cheers.
-
Escorpiom,
If you are convinced or better, have measured that proxy improve internet browsing speed, this is very good.
I can explain why this result is seldom achieved, however I would like to start with another comment:
HTTP proxy is used for 2 main reasons:
- security
- performance
Security is almost self-explanatory:
- profiling, authorization, blacklist, ad removal etc...
Performance is more questionable because:
1 - your browser brings its own local cache
- HTTPS is not cached
- more and more ignorant web developer are wrongly using "pragma no-cache" >:(
2 - accessing web page directly will bring the page to your browser, directly (kind of) where it will stored in local cache(*).
- doing the same via proxy, you will send the request to proxy that will act as web client itself, retrieve page and forward it to you while storing it in proxy cache(*)
- doing the same with transparent proxy is like with explicit proxy except that client is not aware of proxy in the middle, thus sends request that is intercepted then redirected by proxy, so, slightly slower, even if you don't perceive it without measuring tool.
3 - proxy cache mechanism however as some benefit (do not think I'm trying to demonstrate that proxy is always slower ;)): when one given page has already been accessed by someone else in your organization, then you will get it from proxy. Because of this, proxy efficiency directly depends on the number of users, reason why I said: with few users, proxy is likely slower than no proxy.
If you are not convinced, I suggest you give a closer look, using tool like HTTP analyser.
On top of that, tuning proxy is not an obvious task, much more complex than debating about transparent vs. explicit proxy.
Large cache size, even on fast disk, is not enough.
- some other parameters have more impact
- too large cache size will make it slower because of the amount of file it will store.
-
True to some extend.
Not only HTTPS isn't cached, a lot of webpages (as you also pointed out) won't be cached either.
If we take a look at exactly what gets cached it becomes clear that proxy cache is surely not the holy grail.
I knew that from the beginning.
Furthermore, webpages as a whole seldom get cached. But certain elements on those pages might get cached.
But I'll give you an example of real world proxy cache:
It's that "Microsoft patch Tuesday". Around 40 computers on my net will get updated, these patches for both Windows and Office will sum anywhere from 10MB to 100MB in size.
Once Zentyal server has the patches in it's cache (depending on max. filesize given in the config file), other computers on the network can retrieve the updates directly from the cache and thus it's faster without the need to download over and over again.
You can calculate the savings in bandwidth. It may be necessary to adjust some settings in the Squid config file, but it is not hard to do.
We currently use a 3.5Mbit down, 850Kbit up Adsl2 connection. I know that some of you may be spoiled with Internet connections ranging from 10Mbits until 50Mbits or even 100Mbits.
The use of a proxy cache is perhaps less interesting having those Internet speeds available.
Cheers.
-
Very good point ::thumbs up::
In large companies, admins take care of this by handling OS update differently (because problem is exactly the same, whatever your bandwidth, if you have 1000, 2000 or 5000 PC performing Windows update)
Furthermore, with 40 PCs, you certainly benefit from proxy cache, no doubt about this.
My comment what in fact for SMBs and SOHO, maybe not clear enough.
Back to the (interesting) initial debate, launched by Sam:
we were comparing real world experiences because deploying explicit proxy was supposed to require too much technical understanding and manual actions not handled by Zentyal GUI.
As you are tuning your own Squid config, you are obviously very far from this ;D
-
What I don't understand is why this debate about "internet usage analysis" while we discuss about transparent vs. explicit proxy. Is there any impact I do not perceive?
Because I'm guessing that to take full advantage of the subscription service in the real world, I am going to have to run an explicit proxy. So for a simple admin, we will move from the frying pan and into the fire, all the time thinking we are solving the problem of detailed user tracking.
Now my simple admin is paying for a solution and he will find his system is in a place he wasn't expecting. Of course, I could have cut right to an explicit proxy, but you are already tempted to say he doesn't need it. So, I took the long way to make a point--that at least some Zentyal admins will wander into explicit proxy territory not because they decided up front that it is a better way to go, but because their evolving needs led them there even if they didn't know what they were getting into.
MY real point is that we need to do two things: lay out the advantages and disadvantages of Zentyal's proxy implementation, and we also need to understand that it's not necessarily that straightforward for admins who are just trying to solve a real world problem.
I'm not far outside my own real experience in everything I have written so far, including the road taken to even trying an explicit proxy in the first place (what a bad experience for a dummy!)
-
Hello,
Since there is no "allow cPan*l" proxy option[...]
This is configured in the firewall and not the proxy.
(My 2 pennies) This issue could be fixed by adding the list of official/unofficial ports list to the default configuration (or at least the most commonly used services).
Regarding the proxy administration;
The same idea could be applied in the default domain filtering rules.
[...]eventually we would like to accomplish other things--tracking individual users as they browse the Web (to determine who is violating an acceptable use policy, for instance)
(My 2 pennies) This is some other problem that need to be solved. I'm currently using a third party software to get daily/weekly reports on usage.
It may be necessary to adjust some settings in the Squid config file, but it is not hard to do.
(My 2 pennies) That should be done with the Zentyal GUI.
[...] implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service
(My 2 pennies) No easy way of making a custom page for the offending users is also something that I do find harsh. Most of the users don't know what Zentyal is and they don't need to know anything about it. Some companies prefer to let the user know that the requested domain is off limits with the company logo along with its policies.
I do have a 10K raptor disc for the cache
RAM would give you more bang for your bucks.
Best,
Marcus
-
Since there is no "allow cPan*l" proxy option[...]
This is configured in the firewall and not the proxy.
Hi Marcus. I don't think so, if my recollection is any good. In the case of an explicit proxy, I think there is hand configuration of the proxy involved, unless I'm mistaken.
[...] implementing company policy by blocking access through the company network to specific sites. The transparent proxy implementation may be inadequate to that kind of expectation of the proxy as a service
(My 2 pennies) No easy way of making a custom page for the offending users is also something that I do find harsh. Most of the users don't know what Zentyal is and they don't need to know anything about it. Some companies prefer to let the user know that the requested domain is off limits with the company logo along with its policies.
I hadn't thought about that and I think you make an excellent point.
-
I want to move forward on the idea that the explicit proxy, all by itself, isn't the goal of an administrator, I don't think. It's explicit proxy and something the explicit proxy makes possible--authentication, network objects, and so on.
So I want to start a list of things that happen in various scenarios, things that may take a simple admin by surprise. I'm going to start with explicit proxy plus authentication. The behavior of certain things changes:
- Automatic updates of various things may fail unless configured to authenticate (when possible)
- Mobil and guest devices become more difficult to manage, in the case of guest devices, especiaily
- Standard HTTPS port assignments that are not 443 may fail
- Re-authenitcation may be required over the course of the work day, causing some end user ill will :)
It would be nice, I think, to begin to develop the "workarounds" required to solve these "problems" (and any others people want to add) in detail, for those admins who might give up on using explicit proxy without them.
-
Sam,
I'm not very comfortable with your approach :-\
Of course, administrators should not select design because of the beauty of it but because of features or requirements. Given design is only what you have to implement so that requirements are covered.
Based on this, why do you force yourself to go in a direction that may not fit your requirements and that is also not aligned with what you feel to be the "standard design", i.e. transparent proxy ::)
Explicit proxy is not mandatory ;)
Each design has pros and cons and none is perfect, reason why I would suggest to start with requirement rather than design.
To elaborate a bit:
- Explicit proxy doesn't mean authentication (while the opposite is not true)
- Profiling means explicit proxy
- HTTPS filtering means explicit proxy
- Explicit proxy doesn't mean manual device configuration
If I try to rephrase it, first step can be to enable explicit proxy without authentication if you feel that prompting users for authentication is too much for them and also if authentication is not required.
There is no perfect workaround neither: one example is authentication
One can not ask for authentication because it brings some features at proxy and at the same time ask for SSO so that authentication occurs only once. Such approach will just show that security at workstation level is now more critical and not for proxy only.
Transparent proxy
Features:
HTTP filtering
No HTTPS filtering (control, if needed, has to be done at FW level)
No profiling (meaning no different proxy behaviour based on user) not user tracking
Pros
Easy deployment
Cons
Potential side effects because client is not aware of proxy and thinks to communicate directly with server
Difficult to analyse proxy log
Clients on LAN must resolve internet names
No cache for intranet servers
Explicit proxy
Features:
HTTP and HTTPS filtering
Profiling if authentication is enabled
Pros
Proxy log reflects internet usage
WPAD permits to control access to Intranet web server: direct or through proxy (useful for cache in case of server over WAN)
Cons
Requires to define proxy on each device if not "auto discovery service" is deployed
May require multiple settings per device for programs not using OS or browser settings
Non standard HTTPS ports have to be explicitly defined
To be completed...
-
I'm a little confused :-[.
I'm not trying to force anything. I feel like perhaps you assume all Zentyal admins can think of Zentyal's proxy in the abstract. For instance, until you brought it up the first time, I had never heard of WPAD. How can simple people like me be as intentional as to understand the pros and cons of this and that, as abstractions? Keep in mind that I installed eBox itself not because I clearly understood its pros and cons relative to other offerings or because I was convinced of the technical superiority of Linux, but because eBox had a VPN service that even I could use to solve a company problem. Not very scientific, but that's all you get when it comes to the real world admins I am hoping to help with this topic. The alternative is to tell people like me (which some have in fact suggested :) ) that people who aren't "real" admins shouldn't be managing SMB IT in the first place. I won't get into my response here. ;D
I bring up authentication not to force anything--well, the boss said, we want this, so I was tasked with making it happen. Authentication was the very first step to get the boss what she wanted. She hears things and reads things, even if it's just in the news, enough to be able to ask questions:
Q. If we had a user violating our AUP, would you know about it?
A. I would know that someone was violating our AUP, but not whom.
Q. Why not?
A. Because I'm not even close to being as smart as christian. :D
Q. I'll remember that at review time. In the meantime, what is the point of having an AUP when you can't even let me know if a particular person is violating it? Since it's better to get rid of you than the AUP, please don't leave me with only one option.
A. Yes ma'am.
OK, I'm exaggerating a little, but that's not far from how it really went.
So I deny I'm trying to force myself into anything. <rant>I "prefer" (because it is simply not true that I actually prefer a transparent proxy) a transparent proxy (when it comes to Zentyal) because the alternative isn't nearly so simple as just implementing WPAD. It's potentially half a dozen things I have to do by hand, and maintain by hand, just to do what the boss asked. And then there's the other things that I'm expected to manage by hand to do what we want ... the DHCP server for example. So at the end of the day, it's really Zentyal that pushed me in the direction of a transparent proxy, because the total cost of ownership of an explicit proxy keeps me from doing my other work.</rant>
That said, I don't mind at all following you as you take the approach to this that you think best :) . I am certain to learn something :) .
-
Sam,
For sure at the very beginning, for someone not understanding all the technical stuff but having only needs and boss with requests, asking to have theoretical approach to balance and discuss pros & cons has little meaning. I fully share.
However, we are discussing this since months, which give us time to:
- understand better technical stuff
- measure pros & cons of different choices
- refine requirements if needed
- at the end, make the right decision hopefully not to far from the ideal target, or at least understanding why the "perfect" world doesn't exists ;D
Goal is not to tell you that you have - or not - to manage SMB. You are in this position right now and have this responsibility. Goal is to exchange and share our view, share our knowledge and achieve the best result. You don't know everything, neither I do but working together, we should achieve better result because there are some aspects you know and some other I know. But this means that we have to move from our respective positions.
Rephrasing this, I mean that at some point, one has to learn some technical stuff in order to move ahead. If we keep debate at the end-user level, the is very little we can do :-[
Back to this technical stuff, I don't understand why you stick on this position, thinking that explicit proxy will need half a dozen of manually managed configuration options.
If I try to summarize what I currently have in mind:
1 - Once explicit proxy is enabled, auto-discovery is highly suitable to avoid managing clients manually. This can be done 100% using Zentyal GUI if option you select is DNS, now that SRV and TXT records are available in Zentyal interface.
However, pushing this via DHCP is very suitable too but not available in Zentyal GUI.
2 - WPAD server can be managed via Zentyal GUI but proxy.pac file (wpad.dat) has to me manually managed.
3 - in case of use of non standard HTTPS ports, squid conf has to be manually tweaked.
What else?
- You may have some (very few) devices not implementing auto-discovery.
- You may have (here again very few) programs not using OS or browser settings to determine whenever proxy has to be used or not
- if you need profiling or identification, then authentication is required. Difficult here to have something (authentication) and the opposite (not to be bothered by authentication) at the same time ::) and as I wrote, once SSO (thanks to Keberos) will be there, we will discuss at length about security on workstations ;D ;D ;D
but this is what you have to put in the balance to decide whenever explicit proxy is better than the few drawback. Once you have this in your hands, no one can decide for you because you are, at the end, the one operating and managing.
This is the way I perceive it 8)
Then we may hope that Zentyal team, in a next version, will improve their platform and include these few interfaces so that everything can be done using Zentyal GUI. do not take it wrongly: it will never prevent to understand a bit of technique in order to make the right choice.
-
We have been discussing this, true but I have miscommunicated. This isn't supposed to be about me, but about other admins who want to master what Zentyal offers. And it's about what Zentyal offers today, as is. It's about tips and tricks, with enough theory to explain the how and why, but with the emphasis on accomplishing real world tasks--because advanced planning presupposes the knowledge to plan. At bottom, it is about reducing any technical barriers to Zentyal adoption, if not eliminate them altogether.
Instead, it could be about generic drop-in scripts, detailed how-tos, and so on. It isn't about me and shouldn't be about me because my organization is committed to a different course. That's also something we have discussed before.
My goal was to point out that people like me need help not just understanding to make choices, but also enough handholding to make it work, at least the first time. Your earlier efforts at detailing the WPAD process are something of what I had in mind.
Keep in mind that people will have to role out their theory in production. I don't know of a single small business that can mirror production environments in a test setup. So very likely they will need to not only know what's ahead of them, but also just what's in front of them at the moment. They will learn more from their experience then from anything else, almost certainly. I say that as a person who is an educator by training, not a technician. Of course, I'm not doing too good at educating this community to think like a Linux noob and to see how Linux noobs can use Zentyal, with a little better help, so take what I say with a grain of salt. The proxy is a perfect case where more help is desperately needed, IMHO.
So, carry on as you see it. Keep in mind that so far, from how I see it, you are long on reassurances, but short on tips and tricks :) . As I said at the start, it might be pure ignorance that makes someone think they face a mountain when really it's not that bad. So your reassurances are encouraging. But I would still have no idea how to do what you're describing, personally speaking. So I would have hope if this were about me, but I would still be waiting for more information, the tips and tricks part to make the magic happen.
Carry on. :D
-
Keep in mind that so far, from how I see it, you are long on reassurances, but short on tips and tricks :)
;D ;D ;D ;D ;D
I can accept this, even understand but I can't really help because, although my doc is not perfect, I thought I already explained almost everything for someone wiling to deploy.
So in order to improve this, we need someone else to highlight what is missing, what is not clear and what is wrong so that we can update this HowTo.
However, as I tried to explain multiple times, such decision (deploying explicit proxy rather than transparent one) cannot be made without some investment on the technical side. do not expect me to produce a cookbook (e.g. like current Zentyal documentation) with screen-shot showing "click here and there" only because this is, to me, worst than nothing :-X You have the howto but you don't understand why and as soon as you face the first either problem or unavoidable side effect, you don't understand what happens, neither why. Best case, you need a lot of support from this forum :D Worst case, you revert back and decide that explicit proxy doesn't work >:(
Zentyal's goal (that is to provide interface to allow people with no IT admin technical background but in charge it to achieve something easily) is a very tricky one. It reminds me Windows compared to Linux some years ago when you had thousands of Windows admins having deployed Windows based platforms without single understanding of how it works. And most of the time, it doesn't really work ;D or at least poorly with security holes and poor quality. Is it because of Windows? No, of course. More than 20 years ago, Microsoft had the Orange book certification with Windows NT !
This is because of the approach that makes it happening too easily thank to GUI and "click and run" hiding the difficulty behind :o
I'm not promoting the command line with complex grep/pipe/awk ;D I'm fighting against the "cookbook only" documentation even if I do understand that one can not produce documentation re-explaining everything from scratch. To me there is something in the middle explaining the "how" and "why" so that beginner admin can make choices because he understand or can decide to learn a bit more of technique before diving.
So, what is missing with current HowTo preventing you to understand? 8)
-
Some quick comments:
However, as I tried to explain multiple times, such decision (deploying explicit proxy rather than transparent one) cannot be made without some investment on the technical side. do not expect me to produce a cookbook (e.g. like current Zentyal documentation) with screen-shot showing "click here and there" only because this is, to me, worst than nothing :-X You have the howto but you don't understand why and as soon as you face the first either problem or unavoidable side effect, you don't understand what happens, neither why. Best case, you need a lot of support from this forum :D Worst case, you revert back and decide that explicit proxy doesn't work >:(
Agreed, 100% in principle even if we are not yet agreeing on the how-to of a how-to. :D
I'm fighting against the "cookbook only" documentation even if I do understand that one can not produce documentation re-explaining everything from scratch. To me there is something in the middle explaining the "how" and "why" so that beginner admin can make choices because he understand or can decide to learn a bit more of technique before diving.
Again agreed, 100% in principle if nothing else. What I'm fighting is the tendency, as I perceive it, to push for the "buck up and learn Linux" type of documentation. You see it frequently here, in one form or another. "Issue sudo blah blah blah and see what happens." How is that any better than "point and click" stuff if I have no idea what sudo blah blah blah actually does? And since Zentyal itself keeps the command line out of my daily experience, how can anybody here expect it to become second nature to me? :o
I often can't decide which is more obtuse--the command line, or Perl. I generally end up thinking Perl is. ;D
The underlying problem is two-fold, I think. Those who understand the technology well often don't have patience to teach beginners. It takes too much time and concentration to conduct "special education classes for Linux dummies," especially in the context of a support forum.
Additionally, FOSS advocates aren't always very objective about their passion. I think Linus Torvalds said it best in a clip I heard recently (and I'm paraphrasing): "I would hope that people don't choose open source software because it is somehow the morally right thing to do. I would hope they choose open source software because it's better software." Some open source software is still very much a work in progress and isn't truly better yet. But if you "buck up and learn Linux," you can compensate. ::)
In any case, it is axiomatic in education that to teach, you must take a person from the known to the unknown. It's true that I must be willing to learn. It is also just as true that the teacher must be willing to teach. So in general, whatever we write as a how-to must start firmly within the known, and only then move to the unknown, to the new. I think a good general rule is to start with the Zentyal GUI and what it teaches (think of Zentyal as a teacher, because for good or ill, that's what it is), since that's reliable common ground. if the Zentyal GUI doesn't teach it, or teaches it poorly, then assume the student will not have a good grasp of the concepts. The whole proxy discussion is a textbook example, since the Zentyal GUI doesn't teach this concept particularly well, IMHO. Looked at this way, something like "Once explicit proxy is enabled, auto-discovery is highly suitable to avoid managing clients manually. This can be done 100% using Zentyal GUI if option you select is DNS, now that SRV and TXT records are available in Zentyal interface." can be perfectly clear and yet still fly right over the head of a student, since we may not have connected enough of the dots between the known and the unknown.
If we are unwilling to follow this known-to-the-unknown axiom as we write how-tos, then it is really unjust to criticize Zentyal users who "revert back and decide that explicit proxy doesn't work" as bad students. Only if they refuse to grapple with the technical side are they bad students. If we don't give them enough finger holds and toe holds as we drive them up what is to them a very steep mountain, they will naturally fall through no fault of their own.
-
1 - I do not criticize or, better expressed I hope, I try not to do and if it looks like I'm criticizing, this is only because of my broken English ;)
2 - I don't know Linux CLI and do not push anyone to use CLI first before GUI. I'm perhaps one of those using most often "man" command :D
3 - However when I use something like mail, proxy or VPN service, what I do need is to understand how it works, in term of concept and protocol. So what I'm pushing for is not to learn Linux but to learn concepts. Trust me ;) e.g. knowing pretty well LDAP, I've been teaching and correcting quite often Windows administrators because I understand some aspects of AD as this is yet another LDAP server 8)
4 - to me, Zentyal GUI will never teach anything. Not that I like text. I do prefer clear drawing but can't figure out how to learn from GUI. Well, you may make some guess but hardly more than this. Hopefully, if at the end is works, you will shift from guess and assumptions to noun.
5 - So we are now in a crazy loop: if I don't know what you know or don't know, I don't know where to start and definitely refuse to describe everything from scratch, although I'm not sure it will consume more time than debate we currently have ;D ;D But at least this debate is a funny one while writing doc is, for me, painful :-[
6 - I'll try to introduce some screen-shots in my howto in case it helps. Please tell me which ones you would like to see here.
Few more posts and we will together win the price for the longer thread with the fewer posts ;)
-
4 - to me, Zentyal GUI will never teach anything.
All experience teaches. Zentyal is the common experience here. We are in serious trouble if we cannot use this to our advantage, as teachers.
5 - So we are now in a crazy loop: if I don't know what you know or don't know, I don't know where to start
See above. Start with what we know. There is a reason why there are very few good technical writers in the world, because it is actually quite hard to think this way. Forget the physics for a minute; study Stephen Hawking's books for how they teach.
In this case (my example from above), it's not automatically clear what the proxy service and DNS service have in common, but to a certain point I can trust my teacher long enough to gain some desperately needed experience. So resort to the cookbook style documentation just long enough for me to know how to get the first steps behind me. Write the cookbook in such a way that I can't get any further than you want me to get, but at least get me to something new, some new system behavior that gives me the satisfaction of accomplishing something. The mountain in front of me automatically gets smaller. I can see that what you told me to do actually works, though I may not yet know exactly why. If you always put comprehension before experience, you will teach less. See Socrates. But, once I have had the experience of doing this and that and can see that it works and have had some reason for joy, then set me the task of understanding what I just did. It makes no difference if the tool was the command line or the GUI, the educational principle is still the same. I suspect even you didn't know all there is to know about mail servers before you had your first one up and running.
my broken English
Your English is very good. If we had to rely on my French, we could not talk at all. :-[
Few more posts and we will together win the price for the longer thread with the fewer posts ;)
If we ever get to the original purpose, we can split this. In the meantime, what is the prize? I love winning prizes!!
-
<evenmoreofftopicthantherestofmyposts>So I had a phone conference in the past hour with a Web host. I want to redirect a domain they control to a domain I control through a different host--an acquisition transition. We are going to let the domain name they control expire, but we have a contract with them until fall; I am going to honor the contract but not transfer a domain I don't want. The W3C recommend issuing HTTP 301 headers so that search engines and bookmarks are properly updated. The host's server administrator says to me that I have to do that. I say, well, no, I need your server to issue the permanent change of address as things stand right now. What am I missing? I'm on speaker-phone which isn't the best situation anyway, but all I get is mumble mumble A records mumble mumble. And then, but I can do the 301, it's just not how I would do it.
This is a teachable moment. We have in common a specific task--redirecting one domain to another on a permanent basis the right way. If there is a better way to do it than my request for a 301 header from his server, and I ask for that information, I am teachable. But my teacher would not teach me. And I am the paying customer. >:( </evenmoreofftopicthantherestofmyposts>
-
As explained in the HowTo, reason why DNS is involved in proxy design is because of this draft (http://www.wrec.org/Drafts/draft-cooper-webi-wpad-00.txt). I'm not aware it has ever been promoted from draft to RFC but this one is used everywhere.
Once you know it, this is just a matter of describing the right SRV and TXT or wpad.yourdomain entry (well known alias method).
All experience teaches. Zentyal is the common experience here. We are in serious trouble if we cannot use this to our advantage, as teachers.
So I am :-[ because I can't tell you what you will learn once I will have shown screen-shot showing that you have to click here and there :(
Sam, we are pretty much in line but you play a kind of biased game or at least this is the way I feel it. Let me explain.
With what I've described in the howto or what we discuss here at length, someone wiling to implement explicit proxy has enough inputs to do it, even with only partial understanding of how it works under the hood. However, I'm not saying "howto" will cover all different cases one may face. You have illustrated this with your very good example: explicit proxy may not work if you need to access non standard HTTPS port.
From this point, either you feel it doesn't work and you revert back to transparent proxy or you try to understand better and go one step further in case there is a solution using different configuration or even workaround.
What are you expecting from me here? Of course I should (and I will BTW) add a warning covering this aspect but I'm pretty sure you or someone else will come tomorrow writing: "hey, it doesn't work! I've been trying to to stack parent proxy with peer cache and since I'm facing performance issue!. I'll revert back to transparent proxy!"
So my goal is not to push anyone to go deploy explicit proxy rather than transparent one but to say:
- to me, explicit proxy has mode added value than drawback while transparent proxy is the opposite
- here (http://trac.zentyal.org/wiki/Documentation/Community/HowTo/SelectRightHTTPproxyDesign) is how to proceed if you want to deploy such design
and obviously, each time someone asks forum about feature not achievable because of use of transparent proxy, my obvious point is
"do you use transparent proxy :P ???"
but then admins have to make their own choice. This is not proselytism from my side :P
-
Following your [off topic] post:
- why do you want to use HTTP 301 and not 302? 301 is permanent redirect, thus the correct one. Sorry.
- why not chatting on IRC too ;)
** edit ** mix-up between 301 and 302 :-[
-
What am I missing? I'm on speaker-phone which isn't the best situation anyway, but all I get is mumble mumble A records mumble mumble. And then, but I can do the 301, it's just not how I would do it.
If I had to address this, I would:
- determine whenever users still have or can access the former web site
- understand whenever I can or not update web pages on the former web server
- based on above, change DNS to point, if this is the goal, to the new server. This has to be done carefully because there are some pitfalls as web site reached by user is not the one initially requested. BTW, is there any use of HTTPS here?
-
Following your [off topic] post:
- why do you want to use HTTP 301 and not 302?
- why not chatting on IRC too ;)
Because the W3C recommend 301 for a permanent change of address--a permanent redirect (http://www.w3.org/QA/Tips/reback), based on the HTTP 1.1 specification for redirection (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3). That's all I know about it. :-[
Sam, we are pretty much in line but you play a kind of biased game or at least this is the way I feel it.
I apologize. I don't want to be biased. I want to be concise and thorough in one place, or at least have all resources refrenced in one place. But you may be right: I simply don't know enough to do this based on what's already available, and that leads to a sort of bias.
-
What am I missing? I'm on speaker-phone which isn't the best situation anyway, but all I get is mumble mumble A records mumble mumble. And then, but I can do the 301, it's just not how I would do it.
If I had to address this, I would:
- determine whenever users still have or can access the former web site
- understand whenever I can or not update web pages on the former web server
- based on above, change DNS to point, if this is the goal, to the new server. This has to be done carefully because there are some pitfalls as web site reached by user is not the one initially requested. BTW, is there any use of HTTPS here?
No HTTPS. The former site will pass forever into obscurity, the domain name will expire. So the goal isn't to have DNS point to the new server as such, but for the old domain to redirect to the new. To me these are two different goals (I don't care about the DNS server, only the permanent change of address and conveying that to user agents), but I'm not confident I have all the facts to really know that. :-[
-
DNS and HTTP 301 are complementary, to mee.
Without DNS, users will always access the old site and be redirected until server disappears.
With DNS change + HTTP 301 permanent redirect, user without any cached entry (think about DNS cache here) will ask DNs and point directly to the new server while others will access the old server and then be redirected to the new one.
You may also have some hard-coded link (meaning IP address) somewhere, reason why HTTP 301 is mandatory.
-
Had he explained that to me, that yes, we want the 301, but have I considered the fact that for a few months people who visit the old site will always get the redirect, and that we could avoid that, I might have been able to follow his point. Instead, he starts with the idea that I have to do the redirect at my end. And I am immediately lost. I'm not sure it really makes a lot of practical difference in my case, but it does seem to me (now) that the complementary approach is the more elegant one.
Thank you for taking the time to patiently take me from the bit I do know to the bit I didn't know. You are a good teacher. :)
-
:P
-
Seriously.
I'm still not sure how that all works if DNS changes (I'm assuming the domain eventually shows up in cPanel?), but at least I could have asked a few more useful questions along that line.
By the way, a few months back I bought O'Reilly's TCP/IP Network Administration to try to remedy some of my ignorance. I even pulled it off the shelf to try to figure out what the guy was trying to tell me. So I got a little smarter about the reference to A records (I had a vague idea of where he was headed, but I was thinking in terms of DNS transfer, not DNS redirection), but I still missed the connection. Maybe this lack of common sense is common to part-time small business administrators, who have too much variety on their task list? :-[
-
Well, we are not discussing about "DNS redirect" here but to point www.olddomain.com to www.newdomain.com
This is a CNAME or A record that you have to change.
Pay attention that you will need server at the endpoint answering to such "www.olddomain.com" request ;)
What this bring is that even if the old server is not up, requests from clients will reach server YOU manage.
I made the assumption that DNS is the one from the registar but I might be wrong.
Another point to be taken in account is whenever target (old) server is used by other virtual servers. This may limit your flexibility to work at DNS level with A record.
-
DNS redirection wasn't the right term. Sorry. :-[
My thought was to let the server hosting olddomain.com to earn its keep by doing the redirect. As I mentioned, the contract expires in the fall. I have no other use for that server or account. It is a complete hosted solution, so I have no direct control over any aspect of the site, including content updates and corrections. And in my case, that matters because the old site needs to be almost completely rewritten from the ground up, in terms of content. The existing site is full of errors. It made (and makes) sense to me just to move everything permanently to the domain of our choosing and situate things so we can fix the content easily, and to let their server pay for itself by doing the redirect.
I'm assuming that the old server is a shared host. But it was their admin that said the change should be (I think), 1) change the A record, and 2) do the redirect on our host's server. If there are complications to that, he didn't make that clear (to me).
Maybe at the end of the day it was just simpler the way I did it. :'(
-
[Tentatively teaching mode on :P]
Main difference between HTTP 301 and DNS CNAME or A record pointing www.oldsdomain.com to www.newdomain.com
with A or CNAME record, browser asking for www.olddomain.com will either ask proxy (in case of explicit proxy) or resolve www.olddomain.com to get an IP address then will contact this IP and expect to reach www.olddoamin.com. This means that web server at this IP must answer to www.olddomain.com
In case of HTTP 301, client (or proxy) will also resolve www.olddomain.com, reach this IP and get HTTP 301 return code asking browser (or proxy) to redirect to www.newdomain.com
So what's the real difference?
HTTP 301 only looks simpler, no need to deploy temporarily web server for www.olddomain.com but it doesn't solve some aspects because HTTP 301 works at page level; you could redirect only few pages, not all. But this also means that in case someone accesses short cut to something else than main page where you put your HTTP 301, redirect will not occur.
On the other hand, DNS change will apply at server level.
[teaching - and joking - modes off]
-
No, I am learning. Wrong topic altogether (I will fix that eventually), but I am learning.
The cPanel "point and click" implementation is global in the sense that you redirect the entire domain to a new domain in one action, not select pages. Maybe via htaccess? Or maybe cPanel will change the DNS record too?
-
I don't know cPanel :-[
What I know is that if you replace one page with another one sending back HTTP 301, that you will have redirect when this page is reached.
Then perhaps cPanel does something magic here, replacing all pages ;)
changes at DNS using cPanel... hum, I have strong doubts, at least if you don't own DNS because managed by your ISP or registrar.
-
Well, June 1 we are redirecting a site through cPanel where both domains are on the same DNS server and the one cPanel has access to (cPanel provides "Simple DNS Zone Editor," where "This feature allows you to create and edit A and CNAME records"; I guessing that it can do automatically, if that's part of the deal, what it allows me to do manually).
I'll see if I can sort out what cPanel has done on June 1 and report back (in teacher mode).
-
just to be 100% exhaustive, if possible:
if you do DNS only, then you will have a (short) period of time during which some users will still access old serverafter June the 1st with no redirect to the new one because of DNS propagation delay and also cache effects.
If this matters and if you do need to avoid this, I would suggest you change TTL, at DNS level, few days in advance so that you speed up propagation.
-
Another (http://forum.zentyal.org/index.php/topic,10557.0.html) - different - look at the real world :P
To solve this request, explicit proxy is even not enough :o but WPAD does the trick ;D
If it could convince Zentyal team to implement WPAD "service" out-of-the-box.. ::)
-
I opt for Zentyal's HTTP Proxy implementation since it filters content, disallowing banned sites or content types.