Zentyal Forum, Linux Small Business Server

Zentyal Server => Installation and Upgrades => Topic started by: BrettonWoods on November 05, 2013, 06:55:48 am

Title: wan load balancing failover and dns
Post by: BrettonWoods on November 05, 2013, 06:55:48 am
I have just had a period of confusion and to be honest I am still trying to work it out.

I have two wan connections to two different isp's.

I thought I had network load balancing and wan failover configured.

Firstly can you have both network load balancing and wan failover configured or is it one or the other?

I am really puzzled by the settings in the wan failover.

I can only ever get one test to save and this often dissapears after a reboot or configuration change.

I got myself into trouble as I had forgot the router password I had applied so out came a paper clip and I did a hard reset.

Its also a new router and new ISP who has told me they are going to send config details which I am still waiting for and just wiped by doing a hard reset.

So I had one router dead and another working.

But my wan traffic stopped completely?

I thought I might get a 50/50 chance with the equal weights but nothing.

Then I have another question in network > dns has that now gone as I am showing no root dns and have no option to add any?

I ended up disabling the dead wan and adding the router ip as a forwarder and the internet came back to life.

Have I missed an update and now we have to configure forwarders and the automatic dns from dhcp clients is not in use anymore?

Sorry about all the questions but it wasn't until a failure that I noticed it doesn't seem to work?
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 06, 2013, 12:28:26 am
Where are the root dns entries now?

Anyone get similar to this ?

Title: Re: wan load balancing failover and dns
Post by: ctek on November 06, 2013, 11:31:04 am
Hi Bretton,
I have a similar config.
2 lines with two routers before zentyal.
In network/dns i've added the two ISP's nameservers and open dns.
I've added the ip's from both ISP's to the domain and also to the host srv01.
Domain.com: ip 1 and ip 2.
srv01.domain.com: ip1 and ip2.

After this i've set up wan failover and load balancing with same weight on both gw.
and enabled "WAN failover" in Events
let me know if your config is different.
Regards
Bogdan
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 06, 2013, 04:46:47 pm
Exactly the same.

I have fibre on one and adsl on the over via two Isp's.

Just as a question are you nics set up as dhcp so all the dns and gateways are automatic?

I have given up with the wan failover just will not save.

The load balancing weights I had set at 1:5 in favour of the the fibre

Things actually went OK until I changed the IP on the adsl via the router DHCP.
The router has a function via DHCP to apply a wan static ip of one of five.

Since then the whole internet runs like a bat without wings.

Then my confusion about root dns which I thought should be the IP's of my two routers (gone as in above pic)
I have tried the forwarders even added multiwan routing so dns queries to the correct dns go through the correct gateway.

Its sort of back to a reinstall get the networking going first and then add all my clients again...

 
Title: Re: wan load balancing failover and dns
Post by: christian on November 06, 2013, 05:26:16 pm
Running 2.2, I'm (almost) not using load balancing because there is not enough feature, from my standpoint, in term of rule and granularity nor enough documentation about sticky connection management.
I've only few rules so that I access web sites stored by each of my providers using gateway pointing to it, otherwise everything goes through the fastest link (FTTH) and ADSL is used only as fail-over.

My configuration is very similar to what ctek describes:
DNS:
Zentyal (localhost)
2 DNS for each of my ISPs
2 OpenDNS servers

I do not use any forwarders
I also don't understand what would be the purpose of:
Quote
Domain.com: ip 1 and ip 2.
srv01.domain.com: ip1 and ip2.
if serv01 is your Zentyal server and if these IPs are either public IPs or even external IPs
My Zentyal server has only one single internal interface and this is the one I store in Zentyal DNS

Fail-over works almost well. The only real issue I face is that from time to time FTTH is seen as down (perhaps my test is not the most clever) so Zentyal switches to ADLS gateway but also deactivate FTTH gateway and never try to reactive and test it while if I do it manually, it works each and every time...
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 06, 2013, 07:05:00 pm
Thanks Christian.

I am just confused and it might be memory but I thought the root DNS from the DHCP clients on my two external nics showed in the Network > DNS module.

I just have a blank page and a message about forwarders in the above picture.

I get confused with versions and I am not sure if the multigateway rules where in v2. You would be able to tell me.

Your network knowledge is way better than mine so I would be interested in you thoughts on additions.

I got the ppp username and password from my fibre provider today.

I have deleted much of my firefighting and I am back up and running.

The wan failover doesn't work from experience. The config just dissapears on reboot and when set still doesn't seem to do much.

I might add the google 8.8.8.8 as a forwarder but I still don't have a display of my root dns and I think I used to have?

When I turned off the faulty gateway and ran just on the single adsl all was OK.

Its just made me question the balancing as even when set to 1:1 it failed consistently and even though I had one wan down I thought I would get a 50% chance of it working.

Solutions was to disable the downed wan.

[edit]
currently both wan failures look like they have saved ?!

have not rebooted yet.
 
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 06:39:27 am
I am just confused and it might be memory but I thought the root DNS from the DHCP clients on my two external nics showed in the Network > DNS module.
I just have a blank page and a message about forwarders in the above picture.

As I don't run 3.x, I'm discovering this new (to me) DNS configuration approach.
To some extend, it makes sense because previous DNS configuration was confusing for lot of users as we have 2 different DNS roles:
- DNS client
- DNS server

So now this is clearer.  8)

On the other hand, what this tells, for those reading carefully, is that if you think about deploying Zentyal as internet gateway only, you will have, nevertheless, to deploy DNS server even if you don't need it  :-\
DNS module is no more an optional but now mandatory module. Well, this is no more a module but part of core Zentyal.
This also makes sense if you think about Microsoft and plan to deploy DC. DNS server is mandatory here.

hehe, the new Zentyal minimum install is bigger and bigger isn't it   :-X

Quote
I get confused with versions and I am not sure if the multigateway rules where in v2. You would be able to tell me.
Yes Zentyal 2.2 does bring rules here too although I can't compare.

Quote
I might add the google 8.8.8.8 as a forwarder but I still don't have a display of my root dns and I think I used to have?
You don't need it any more because of my above comment.
However, you should look at this post (http://forum.zentyal.org/index.php/topic,18701.0.html).
This guy has poor performances because of, to me, strange rules but also because DNS was swinging between the 2 gateways. We fixed it adding one rule for DNS.
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 11:03:18 am
Hi Christian,  Bretton
The zentyal server hostname is SRV01.
I've put both public ip's from the ISP to that host.
Also the same IP's i've used on the domain. all this is done in the DNS section.
This aproach is necessary so that the server will be reachable from intenet on both ISP. If i only let the local interface as set for the host this will create a whole bunch of issues. In fact the local ip does not appear in any setup and i do not want it to be propagated into the internet on a DNS query for my domain.

One of my interfaces (the Fiber one) is set with static IP, the other one is via PPPoE. I've had before a situation where the interfaces were connected to some home routers Dlink and Huawey but it still worked.

The rules for DNS sound ok in theory but i've seen that it does not play well in real life. Maybe some sort of BGP mode should be more suitable but this will be even more complex to implement.

Hope this will clarify more from this confusion with ambigous terms used Dns, Fowarders, local domain External domain etc :)

Regards
Bogdan
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 11:17:05 am
Hope this will clarify more from this confusion with ambigous terms used Dns, Fowarders, local domain External domain etc :)

At least from my side it unfortunately doesn't clarify anything.
Your Zentyal server has 3 IPs:
- one internal interface with private (RFC1918) IP
- Two external interfaces, each with public IP provided by your ISP.

Reaching your server from internet can be done using 2 different implementations:
- name server for your own domain is Zentyal => in such case, Zentyal DNS is used
- name servers are hosted by your registrar and his is where you define IPs to be used to reach your Zentyal server.

If you are relying on your registrar infrastructure (I believe most of us do this), then what Zentyal DNS contains doesn't really matter (at least for what concerns access from internet).
If you rely on Zentyal DNS, there is a couple of things you have to keep in mind:
- there is no split view, split DNS or whatever the way you want to call it. To make it short, the whole Zentyal DNS content is visible from internet, including internal IPs. These are under RFC1918 so not directly reachable but this may ease some attack from internet.
- if you also run Samba, editing Zentyal DNS content will have no impact for what concerns Zentyal host itself. Samba will keep synchronize DNS on your behalf and expose all IPs in DNS.

I can't see your point neither relationship between DNS client and BGP. Could you please elaborate on this?
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 12:29:37 pm
Hi Christian,
Right My zentyal has 3 IP's
1 LAN and 2 for WAN.

I do not rely on my ISP.
I do not use SAMBA and the lan IP does not show up on dig or nslookup.
The BGP implementation does not have anything to do with DNS but it has with load balance and wan fail-over.
To achieve real load balance you will have to make use of EiBGP or *BGP (take a brief look here  http://blog.ipspace.net/2013/06/eibgp-load-balancing.html (http://blog.ipspace.net/2013/06/eibgp-load-balancing.html)) so that the traffic will be correctly pointed to the interfaces. (this will fall into advanced routing and is not easily done with zentyal)

The only point where my ISP will be involved is with rDNS so that the reverse lookup will be corect.

The Wan fail-over aspect has two sides! Keep in mind that if you use Zential as a server and NOT as a gateway only the WAN (as an aggregate) has to be reachable on both ISP lines! Also that means that the domain will have to be set to "respond" for both IP's also the host (zentyal itself) will have to do the same.

The following setup in Zentyal DNS section is valid:

Domain.com ip: xxx.xxx.xxx.xxx; yyy.yyy.yyy.yyy;
HOSTS: srv01 ip: xxx.xxx.xxx.xxx; yyy.yyy.yyy.yyy;

if you query:
 nslookup srv01.domain.com

Non-authoritative answer:
Name:   srv01.domain.com
Address: xxx.xxx.xxx.xxx
Name:   srv01.domain.com
Address: yyy.yyy.yyy.yyy

so failover is achieved 
Hope this helps.

Best regards
Bogdan
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 01:30:46 pm
I think I understand some of the points your describe but still can't put everything in a perspective that makes sense to me. I'm not meaning you're wrong but this makes me totally puzzled.
I feel confusion is because we don't see "fail-over" from the same viewpoint.

If you don't use Zentyal as a gateway but as a server to be accessed from internet, then Zentyal "WAN fail-over" is not for you as this feature doesn't aim, if I understand well, at providing high availability "from internet" but high availability "from intranet to internet".
On top of this, I fully agree that Zentyal is not the wisest choice in case you need some control on routing but I also still don't understand why BGP would help if you don't use Zentyal as gateway (as of course, you do know that BGP stands for Border Gateway Protocol)

This being said, of course if your Zentyal server is only used as server on internet, as you don't use Samba, you can control what DNS exposes by not creating in this DNS any entry for internal servers or services.
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 07, 2013, 02:33:31 pm
Bogdan I think there was some confusion over ISP.

I think christian was talking about your domain name register where ever you hold your DNS entries.

Sorry but before going into the technicals, just make my simple mind a little more at ease.

Is the main objective to do some sort of round robin load balancing for two nics and isp's for a singular site.

I noticed you have set your rDNS so I am presuming email and the two nics and isp's are also for redundancy?

 
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 02:42:41 pm
Quote
The only point where my ISP will be involved is with rDNS so that the reverse lookup will be corect.

This is another aspect I don't understand. Again, this is perhaps feasible but I just don't understand howto  :-[
How can you have PTR handled by your ISP (I suppose you mean registrar or perhaps both are the same) with DNS managed on your side by Zentyal.
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 07, 2013, 05:59:58 pm
I think I understand some of the points your describe but still can't put everything in a perspective that makes sense to me. I'm not meaning you're wrong but this makes me totally puzzled.
I feel confusion is because we don't see "fail-over" from the same viewpoint.

If you don't use Zentyal as a gateway but as a server to be accessed from internet, then Zentyal "WAN fail-over" is not for you as this feature doesn't aim, if I understand well, at providing high availability "from internet" but high availability "from intranet to internet".
On top of this, I fully agree that Zentyal is not the wisest choice in case you need some control on routing but I also still don't understand why BGP would help if you don't use Zentyal as gateway (as of course, you do know that BGP stands for Border Gateway Protocol)

This being said, of course if your Zentyal server is only used as server on internet, as you don't use Samba, you can control what DNS exposes by not creating in this DNS any entry for internal servers or services.

I have a bit of it all, in that samba is used with two nics also serving mail and internet sites.
I really dont like the bind9 implementation for samba. samba has an internal simple dns that works.
I am forced to use a bind9 server publicly but cant split my information into reverse and forward zones which does have implications.

When it comes to your dns queries then as christain said with your domain registry you can assign the domain pointers mx records blah to both ips.
So in a way this is load balanced i am not sure who to implement a round robin method but it will pull from the first and if fails try the second.

my problem when i did lose a wan was also for some reason my failover rule dissapeared and the failover didnt work.

 
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 06:05:59 pm
ufff... :)

Ok let me explain a little.
The "outside":

the domain is declared at TLD with the two IP's as nameservers for my domain.
I have two ISP's that have given me two public IP's.

Zentyal is used for the following roles:
DNS; Mail; Webserver; Gateway;

at the network section of zentyal i've declared:

Eth 1 - IP from isp1
Eth 2 - IP from isp2
Eth 0 - Lan ip;

Enabled the wan failover monitor;
Declared the primary gateway from isp1;
Enable load balance on external interfaces with 50/50;

On the DNS section of zentyal:
Create the domain.com;
Add both ip's from ISP to the domain.com;
Add the forwarders from my ISP;

in the host section of the domain (SRV01) i've added the two IP's
In alias section i've added the proper aliases.

Then configured the firewall and that was it ... more or less:)

But now i have wan fail over for my lan subnet.
and the server is reachable from each ISP because it is responding with two IP when domain is asked.

For the rDNS i've asked the IPS's help so that the ip's alocated to me will resove properly on their end. so for example a traceroute will resolve to the proper ip/name from my ISP. (i'm not well versed in rDNS and  FCrDNS so that's why i've asked for their help)

I know what BGP stands for and is not so easily to implement. A few years i've did (with outside help of course since i'm not all-knowing ) a BGP configuration with  AS and everything, but that was when i needed proper load balancing between 4 ISP and it was for a small  neighborhood and I was the local ISP. But this is not the case and Zentyal is perfect for rr loadbalance.
I have both HA for reaching the servers on different isp and wan failover for LAN side.

The round robin is done internaly by zentyal you however can specify if you want, how much of the total queries can be on put  on one interface and how much on the other.

My DNS setup is done correct (i hope:) ) on my end but the rdns i've asked for help on my ISP (not registrar since it is registered ar TLD)

Best regards
Bogdan 



Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 06:23:31 pm
Cool, now I do understand your set up. This was only a matter of wording  ;D ;D
I do have exactly the same, almost 100%  ;)

If I understand well what you have and what you did, your Zentyal DNS is only used by internal users. All the external stuff is handled by DNS on ISPs side, including MX and PTR. There is nothing surprising neither wrong. I've the same here and that's very standard design.

Thus there is no need to declare your external Zentyal IPs in your local (Zentyal) DNS.
You also don't need split DNS because your DNS is not seen (in fact rather not used) from internet.

If your webserver is used from internet, high availability is partially achieved as you have two public IPs and external clients will get one IP then the other in round-robin mode, meaning this is not 100% achieved, e.g. for external web clients it may fail if one or you ADSL link is done.

This is somewhat different with mail because assuming you set same weight in MX records, in case MTA can't be reached at first IP, second MX will be used.

Quote
The round robin is done internaly by zentyal you however can specify if you want, how much of the total queries can be on put  on one interface and how much on the other.

Regarding this point, I would have loved some documentation from Zentyal describing the sticky connection. I'm not using it this way as my FTTH link is way faster than the ADSL one but I would not be surprised if balancing everything over the 2 equivalent links doesn't exhibit some side effects with web based applications e.q when connection to application server is seen from 2 different source IPs.
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 07, 2013, 06:49:25 pm
I dont think it would matter as its one ip (client) and apache would be listening on both ip's.

I cant see how you can do the round robin thing though as the client would just try the first dns if fail second.
As christian said with mail as which ip is hit is external to zentyal.

Glad you are talking about this as I have a very similar setup and the second isp is a new addition.

I dont know enough about apache session states to say if this will be a problem, i will probably do my usual suck it and see methodology.

going outwards have you ever tested the wan failover as in 3.2 i am not sure it works.

ps the rdns pointer is mainly for mail as many mail servers run a dns check of the source and reject to stop spam and aliases.

its just a matter for the owner of the ip not to return something like my provider  host81-148-01.btopenworld.com. which is still to be sent a rdns pointer request.

if you do a tracert to google.co.uk or what ever the second hop should return your domain name i think and not the isp. I cant remember it might still return the host. http://remote.12dt.com/lookup.php

I have to ask as I dont know but dns requests are they always returned in the same order with multiple ip's?
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 07:23:51 pm
Christian, Bretton, if you could join me in a skype session would be great since this is a more in depth discussion and maybe we can make some sort of documentation or recipie or how-to so that other users can benefit from it.

if you can PM me your skype id's i will be glad to continue this talk.

Best regards
Bogdan
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 07:30:56 pm
I dont think it would matter as its one ip (client) and apache would be listening on both ip's.

Problem I try to describe is not this one  :-\  or I don't understand your point.
If Zentyal doesn't bring any "sticky connection" stuff, when internal client using web browser accesses external web server, say web based application, this server sees connection from public IP. In case you have 2 different IPs, depending on how your application is written, it will see client's connection as 2 different connections, which may prevent application to work smoothly.

iproute brings some "sticky connection" but my understanding of how this impact load balancing efficiency is quite poor.

Quote
I cant see how you can do the round robin thing though as the client would just try the first dns if fail second.

For incoming flow:
This works for mail transfer.
This doesn't work for web browsing.
Say external browser or proxy tries to reach your server behind 2 different IPs. URL is resolved, returns one IP. Bad luck, this is the one matching your link that is down... connection fails..  :-[ there is no retry. If you ask another time (after you got the error message, you might be luck as DNS should return back the other IP.

For outgoing flow:
there is no round robin stuff but iproute
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 08:10:56 pm
if you can PM me your skype id's i will be glad to continue this talk.

No....  ;D ;D ;D
Not because I don't want but your mail address is not available so I can't send you any PM  :P

Furthermore, I frankly don't see any area we have not yet covered, except perhaps debate about the best way to test WAN availability.
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 08:21:37 pm
Regarding the dns, only the reverse is done by My ISP the rest is done by me local
:)
Testing the wan.... hmm
Well that would be, to have it get http headers with at least 75% rate :)

Regards
Bogdan 

Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 08:40:26 pm
Regarding the dns, only the reverse is done by My ISP the rest is done by me local

Can you explain this ? I'm totally clueless about how this could work  :-[
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 07, 2013, 08:46:17 pm
you where right about wording when i mentioned about one ip client I was talking about external client to zentyal.
I am presuming this will work as apache is listening on both ip's and looking for a server name header which will be the same.

going out through multiwan and to different ips might have problems but i think session states rather ips are more common.
I dont know as its common to use a browser session state. How much that will impact I dont know.

I think its good to talk on the forum and keep it public. I often go off on a wrong tangent and dont mind being wrong.
If we keep it on the forum then others can use it if useful.

christian do you get any problems with web auth and the balancing?
Title: Re: wan load balancing failover and dns
Post by: christian on November 07, 2013, 09:00:34 pm
christian do you get any problems with web auth and the balancing?

No because I do not load balance  ;D
One link is 100Mb/s while the other is 20Mb/s
I only use rules in order to access sites that are expecting (hard coded) one of my 2 IPs otherwise purpose if fail-over and HA for incoming flow.
Title: Re: wan load balancing failover and dns
Post by: ctek on November 07, 2013, 09:01:22 pm
Regarding the dns, only the reverse is done by My ISP the rest is done by me local

Can you explain this ? I'm totally clueless about how this could work  :-[

ok :) Christian I've made public my email address to the profile. I can show you the config and this i think should be more explanatory than i can put into words.
This is why i wanted to create a short how-to and maybe a "good practice" example.
For obvious reasons i can't make the screens available on the forum  ;D
Title: Re: wan load balancing failover and dns
Post by: BrettonWoods on November 07, 2013, 09:35:29 pm
No because I do not load balance  ;D
One link is 100Mb/s while the other is 20Mb/s
I only use rules in order to access sites that are expecting (hard coded) one of my 2 IPs otherwise purpose if fail-over and HA for incoming flow.

I have 50Mbs & 10Mbs so I will give it a go and report back. So I have a weight of 5:1 only testing will tell I guess.