The cluster has not any resources defined.

[illunis]

i guess my windows background is haunting me....

Give me some time...i have "left" the cluster to test my theory and i ll report back.

[christian]

while you are testing, let me add some inputs/thoughts:
- when having 2 Samba domain controllers, you don't need any cluster (for the domain controller feature) as both are running in parallel and synchronize smoothly. Users may authenticate against one or the other, is doesn't make any difference (assuming of course Kerberos is configured to take this in account.
- file sharing is very different: if you want high availability at file sharing level, then there is only one single file server that should be accessed using floating IP that points to the active node. You are not supposed to access the other node otherwise you will get replication conflicts. While editing your file stored on file server, lock is required... if you have 2 identical files on 2 different servers... 

This is the "Samba" example for Windows addict guys. Same apply for some services that can't scale-out but scale-up, meaning adding servers doesn't help, you have to add more powerful server.

What is annoying with current implementation, mainly because there is not yet any documentation (and also because I'm pretty sure next to come documentation will only show screen-shots ) is that this is difficult to guess what such service is supposed to offer and cover.

[illunis]

while you are testing, let me add some inputs/thoughts:
- when having 2 Samba domain controllers, you don't need any cluster (for the domain controller feature) as both are running in parallel and synchronize smoothly. Users may authenticate against one or the other, is doesn't make any difference (assuming of course Kerberos is configured to take this in account.

Totally agreed....simply wanted to check it out

- file sharing is very different: if you want high availability at file sharing level, then there is only one single file server that should be accessed using floating IP that points to the active node. You are not supposed to access the other node otherwise you will get replication conflicts. While editing your file stored on file server, lock is required... if you have 2 identical files on 2 different servers... 

Will try it out soon.

What is annoying with current implementation, mainly because there is not yet any documentation (and also because I'm pretty sure next to come documentation will only show screen-shots ) is that this is difficult to guess what such service is supposed to offer and cover.

THAT is the most important....actually we are blind testing...

So...did some QUICK tests...

When i left the cluster and after a reboot i can see both domain controllers. So i was right that i had made a mistake.
Manually adding the Floating IP didnt change anything and i dont know if it should change anything. It has to be clarified.

I will destroy the machines and take it a step at a time cause i did everything really fast to test the HA module. I ll come back to it in the next few hours or latest tomorrow (unless someone else does it before me...)

[christian]

Highlighted by Robb:
Introducing HA

Indeed this is more screen-shots collection (no surprise here  ) but at least it confirms:
- the active/ passive design (this was already clear in my mind  )
- there is no resource management.

It nevertheless open doors for a lot of technical debate about design choices.
- DNS cluster while another approach could have been to synchronize DNS content and define more than one DNS client side (thus getting something closer to active/active in term of feature).
- VPN cluster ? well, as VPN client, why not but as VPN server, one can already define multiple VPN servers client side so that you can connect to another server in case the first one fails.
- what's about other services like mail ?
- what's about data (like files sharing... and mailboxes too)

I suppose we will have to run some reverse engineering... 

[sixstone]

Hi guys!!

Thanks very much for testing HA module. Please, stress this out .

As stated in the article, a resource (name scheme borrowed from the Cluster Resource Managers ) is a the scope of this module: Floating IP addresses and DHCP module. Those resources are meant to be run in a single instance of the multi-node cluster. You can move resources using Promote and Demote buttons.

The replication only works for configuration settings of the following modules: DHCP, DNS, Firewall, HA, IPS, Network, Objects, Services, Squid, Trafficshaping, CA and OpenVPN. In CA module, the certificates, private keys and such are replicated. Likewise /etc/zentyal directory is synchronised.

- there is no resource management.

It is as explained above. Quite limited indeed.

It nevertheless open doors for a lot of technical debate about design choices.
- DNS cluster while another approach could have been to synchronize DNS content and define more than one DNS client side (thus getting something closer to active/active in term of feature).

Using the basis of the DNS module (without any kind of DNS replication), the configuration is completely active/active if no Samba AD is present .

- VPN cluster ? well, as VPN client, why not but as VPN server, one can already define multiple VPN servers client side so that you can connect to another server in case the first one fails.

You can have a cluster of VPN servers with the same configuration without a hassle using this module o:).

Best regards and happy testing,

[illunis]

Good morning all

Thanks Christian...havent seen that...it also confirms that SAMBA clustering is not (at least yet) supported

"if you require a custom resource configuration, you will have to use the pacemaker directly to configure using crm shell"

I ll keep on playing then and see what more can we destroy

[christian]

After some chat, I've got a much better understanding about Zentyal HA.
As very clearly written in above link, scope is FW, DNS, DHCP and OpenVPN.

As explained by sixstones, having VPN cluster may help to not duplicate configuration.

Regarding FW, I've to admit that I'm totally lost but thinking twice about this, if your FW fails, having another one handling same iptables rules without having other services like proxy, mail and other stuff switching to the secondary node, given the fact that services are reached quite often using CNAMES or host names that are added in DNS on your behalf by either Zentyal directly or DLZ  makes, at least for the time being, this FW HA stuff at least questionable. 

At least, scope being now crystal clear, I'm done with my tests 

[christian]

...it also confirms that SAMBA clustering is not (at least yet) supported

Although I just wrote that I'm done with this (and indeed I'm done) there is at least one comment I can't refrain myself to express, reacting to your point:
- Samba is now suffering from the exact same problem (and to me major drawback) as Microsoft Windows: this is all seen as a whole, single service while it provides many different services and features. Samba acts as domain controller, file server, DNS (to some extend via DLZ)
- thinking about HA and Samba without having clear understanding that these services are different and require different approach is, IMHO, a misconception.

DC HA exists, out of the box, when you configure additional DC. This brings de-facto "AD like" content synchronization (I'm discussing about Samba here, not Zentyal implementation). Therefore building cluster for this purpose (service) is not mandatory.
Same for DNS related stuff.

File server is another totally different story. File server is made of at least 2 layers:
- file server itself (as a service client is accessing)
- storage

In our implementation, this is often merged on same box but you may imagine this is done with 2 different layers, even when using one single box (e.g. look at Netapp implementation where clustering on one single box splits service layer and data layer).

So, to make a potentially long debate short, there is a need for file sharing HA that is very different from DC HA and even from Samba HA, reason why I react and suggest that you keep this in mind while addressing this point 

[sixstone]

indeed and now (this was not shown 2 days ago if I remember well), floating IPs are shown attached to interface that is in the same - network - range 

Funny enough, there is no control while adding floating IP but if you add one with IP that is not covered by existing interface, then this one is shown as "error" in the cluster status tab.

This is now potentially fixed in this pull request: https://github.com/Zentyal/zentyal/pull/1054

Thanks very much for your feedback!

Do not hesitate to open new issues in the new tracker if you find any other bug!

Happy testing!

[half_life]

Hopefully this is still on topic.   I have spent a good chunk of my weekend trying to get this working.  I started off with my previously installed and upgraded 3.4 testbed running under KVM on my local machine.  I was able to setup a cluster and add resources.  I added a second machine from the same daily ISO (2-22) and ran the upgrade.  On joining the cluster,  things started going wrong.  Long story short,  split-brain.  Each machine thinks the other is off-line.  Doing Zentyal restarts will result in a hung machine.


Starting clean from 13.10 server gives very bad results.  It will not finish the install and hangs on dpkg --configure zentyal-core until it exhausts memory and oom kills it.

Grr....   I have tried one network interface on the machine and 2.  I have tried going with the daily build ISO as well as straight from Ubuntu server.

I haven't tried sacrificing any chickens yet but maybe it would help 


Thoughts?
   

[peterpugh]

+1 for the chickens

[half_life]

This is what I get from a crm_mon -1.   Split brain,  right?

Code: [Select]

Last updated: Sat Mar  1 15:29:25 2014
Last change: Sat Mar  1 15:21:32 2014 via crmd on Campion
Stack: corosync
Current DC: Campion (1) - partition WITHOUT quorum
Version: 1.1.10-42f2063
2 Nodes configured
1 Resources configured


Online: [ Campion ]
OFFLINE: [ Starkey ]

 testing        (ocf::heartbeat:IPaddr2):       Started Campion


Code: [Select]

Last updated: Sat Mar  1 15:29:57 2014
Last change: Sat Mar  1 15:21:55 2014 via crmd on Starkey
Stack: corosync
Current DC: Starkey (2) - partition WITHOUT quorum
Version: 1.1.10-42f2063
2 Nodes configured
0 Resources configured


Node Campion (1): UNCLEAN (offline)
Online: [ Starkey ]



Sixstone stated that they were using quorum disk right?  I can find no evidence of that or any other quorum mechanism.

Code: [Select]

root@Campion:/var/log# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      3556/vsftpd     
tcp        0      0 192.168.163.3:53        0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 192.168.122.22:53       0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      984/sshd       
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      2557/haproxy   
tcp        0      0 127.0.0.1:62080         0.0.0.0:*               LISTEN      3468/apache2   
tcp        0      0 127.0.0.1:61443         0.0.0.0:*               LISTEN      3651/nginx.conf
tcp        0      0 0.0.0.0:390             0.0.0.0:*               LISTEN      3297/slapd     
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1054/mysqld     
tcp        0      0 127.0.0.1:62443         0.0.0.0:*               LISTEN      3468/apache2   
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      950/redis-server
tcp        0      0 127.0.0.1:6380          0.0.0.0:*               LISTEN      1200/redis-server
tcp        0      0 192.168.163.250:8880    0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 192.168.163.3:8880      0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 192.168.122.22:8880     0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 127.0.1.1:8880          0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 127.0.0.1:8880          0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42361     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56053         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56052         ESTABLISHED 3658/nginx: worker
tcp        0      0 192.168.122.22:443      192.168.122.1:42356     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42357     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56050         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0    768 192.168.122.22:22       192.168.122.1:39664     ESTABLISHED 2714/sshd: denny [p
tcp        0      0 192.168.122.22:443      192.168.122.1:42358     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42359     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42360     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56054         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56055         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:61443         127.0.0.1:56053         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56055         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56054         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:61443         127.0.0.1:56050         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56051         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56051         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56052         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp6       0      0 :::22                   :::*                    LISTEN      984/sshd     

[half_life]

+1 for the chickens

How do you prefer yours?  Grilled or Cacciatore ?

[peterpugh]

Had to look up Cacciatore but sounds like something I should try

Voodoo Peri Chicken solves all manner of technical difficulties, starts with a pain in the stomach and has a very hot exit but leaves a very clean system.

[sixstone]

Hi half_life,

We are not using quorum disk at this moment.

Probably, the keys are not synchronised. You'd better chhose a host, leave the cluster and re-join it to the cluster and save changes .

Best,