Zentyal Forum, Linux Small Business Server

Zentyal Server => Installation and Upgrades => Topic started by: illunis on February 26, 2014, 01:14:57 pm

Title: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 01:14:57 pm
Hi there
i am also beta testing the 3.4 Zentyal and specifically the HA module.
I have setup a cluster and everything seems to be replicating fine (judging from the logs) but i am trying to understand what is the "The cluster has not any resources defined." message and how can i define resources.

Thanks again
iLLUNiS
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 01:43:52 pm
UPDATE

I have defined a Floating IP address for the cluster and this has added this action as a "resource"

Are there any other actions that can be registered at the cluster as resources and therefore be managed bu the cluster?
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 01:46:17 pm
just curious (not helping as for the time being I didn't reach yet this stage):
how did you create floating IPs on various sides of Zentyal server ?
I mean where do you define that one floating IP is external and one or more are internal or for DMZ ?

On my up-to-date Zentyal 3.4 platform, although I've defined only one single node, everything is "OK", on-line cluster started on my (single) server with one resource that is my unique floating IP.

In pacemaker terminology, floating IPs are resources, correctly reflected by Zentyal.
If you add more floating IPs, you will see more resources  ;)

EDIT: you were posting while I was typing, so my post is almost meaningless  ;)
Still I don't understand how this is managed.
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 01:54:20 pm
Hi Christian
at the screenshot i am attaching u can see that i have setup the cluster ip to be 172.16.10.230 (i am connected at this IP with my browser) and the cluster servers are using 172.16.2.243 / 246

Still trying to find other "resources" :)
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 02:06:20 pm
indeed and now (this was not shown 2 days ago if I remember well), floating IPs are shown attached to interface that is in the same - network - range  ;D

Funny enough, there is no control while adding floating IP but if you add one with IP that is not covered by existing interface, then this one is shown as "error" in the cluster status tab.
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:09:34 pm
indeed and now (this was not shown 2 days ago if I remember well), floating IPs are shown attached to interface that is in the same - network - range  ;D
Yes indeed !....and also tried promoting the other server and changed "on the fly" Seems to be working very well (if only i can find what other resources exist...:P)

Funny enough, there is no control while adding floating IP but if you add one with IP that is not covered by existing interface, then this one is shown as "error" in the cluster status tab.

Havent tried that !
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 02:14:20 pm
so far so good, so it works... well at least on your system. On mine, I still can't enable DNS module  :(

Discussing about DNS: did you notice that Zentyal is automatically adding on your behalf entries in DNS for all your physical interfaces but none for floating IPs which are the ones to be used at the end...
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:28:04 pm
so far so good, so it works... well at least on your system. On mine, I still can't enable DNS module  :(
I have set up a domain and the DNS worked straight away

Discussing about DNS: did you notice that Zentyal is automatically adding on your behalf entries in DNS for all your physical interfaces but none for floating IPs which are the ones to be used at the end...

No i havent noticed that !...Could this be the reason that i cannot join a computer at the domain with the cluster IP?....dunno
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:37:51 pm
Further testing shows that there is a problem with the domain.

Active directory "sees" two domain controllers but i can only communicate with the primary. The secondary is always unavailable even after machine restart.

maybe i should open new thread about it?..or move it?....
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 02:44:12 pm
I have set up a domain and the DNS worked straight away

Domain ? what kind of domain ? DNS domain, Windows domain ?

My issue so far is (was ? I still don't know but need to test) that trying to enable DNS service locked my server, HTTP 504 error and no capability to restart Zentyal or get back access.

Moving to Windows (Samba in fact) domain: have a look at DNS content (dig) because floating IP may indeed be missing.
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:47:38 pm
I have set up a domain and the DNS worked straight away

Domain ? what kind of domain ? DNS domain, Windows domain ?

My issue so far is (was ? I still don't know but need to test) that trying to enable DNS service locked my server, HTTP 504 error and no capability to restart Zentyal or get back access.

Moving to Windows (Samba in fact) domain: have a look at DNS content (dig) because floating IP may indeed be missing.

I have created a Windows (SAMBA) domain so i havent faced the problem ur describing.
The floating IP is definetely not registered at the any of the DNS....i ll try to manually create a registration for it for BOTH the controllers and see what happens
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 02:51:07 pm
Sure, let us now.
Still I'm confused about your wording. To me, registering IP and (hot or C) name is a matter of DNS, not matter of Windows stuff  ???
Then DNS must have the right content in order for user to access services like Samba but this is the next step.
Well.... in my understanding  ;)
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:52:10 pm
NOPE....there is DEFINITELY a problem with the domain. There is NO replication between the two controllers. I tried manually and still nothing.

I have a suspision that the HA module is causing the "problem". I have two controllers (primary & secondary) on the same cluster and now that i think of it, it seems rather wrong :P
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 02:54:19 pm
Sure, let us now.
Still I'm confused about your wording. To me, registering IP and (hot or C) name is a matter of DNS, not matter of Windows stuff  ???
Then DNS must have the right content in order for user to access services like Samba but this is the next step.
Well.... in my understanding  ;)

What i meant is that since i created the windows domain, this took care of the DNS entries also...but like i said in my post above i must have made a mistake combining everything in the same cluster.
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 03:00:48 pm
OK, on the DNS side, we have definitely slightly different view, mainly because you look only as the windows side that is, thanks to DLZ, pushing some entries to DNS. It doesn't matter that much.

Another interesting point that you raise is the cluster type: you make the assumption that this is an active/active cluster (again, looking only as the "Windows like" stuff while I was rather thinking that such cluster must be active/passive.
Let's start discussing this  ;)
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 03:04:22 pm
i guess my windows background is haunting me....:P

Give me some time...i have "left" the cluster to test my theory and i ll report back.
Title: Re: The cluster has not any resources defined.
Post by: christian on February 26, 2014, 03:16:23 pm
while you are testing, let me add some inputs/thoughts:
- when having 2 Samba domain controllers, you don't need any cluster (for the domain controller feature) as both are running in parallel and synchronize smoothly. Users may authenticate against one or the other, is doesn't make any difference (assuming of course Kerberos is configured to take this in account.
- file sharing is very different: if you want high availability at file sharing level, then there is only one single file server that should be accessed using floating IP that points to the active node. You are not supposed to access the other node otherwise you will get replication conflicts. While editing your file stored on file server, lock is required... if you have 2 identical files on 2 different servers...  :o

This is the "Samba" example for Windows addict guys. Same apply for some services that can't scale-out but scale-up, meaning adding servers doesn't help, you have to add more powerful server.

What is annoying with current implementation, mainly because there is not yet any documentation (and also because I'm pretty sure next to come documentation will only show screen-shots :-X ) is that this is difficult to guess what such service is supposed to offer and cover.
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 26, 2014, 03:36:50 pm
while you are testing, let me add some inputs/thoughts:
- when having 2 Samba domain controllers, you don't need any cluster (for the domain controller feature) as both are running in parallel and synchronize smoothly. Users may authenticate against one or the other, is doesn't make any difference (assuming of course Kerberos is configured to take this in account.
Totally agreed....simply wanted to check it out ;)

- file sharing is very different: if you want high availability at file sharing level, then there is only one single file server that should be accessed using floating IP that points to the active node. You are not supposed to access the other node otherwise you will get replication conflicts. While editing your file stored on file server, lock is required... if you have 2 identical files on 2 different servers...  :o
Will try it out soon.


What is annoying with current implementation, mainly because there is not yet any documentation (and also because I'm pretty sure next to come documentation will only show screen-shots :-X ) is that this is difficult to guess what such service is supposed to offer and cover.

THAT is the most important....actually we are blind testing...:P

So...did some QUICK tests...

When i left the cluster and after a reboot i can see both domain controllers. So i was right that i had made a mistake.
Manually adding the Floating IP didnt change anything and i dont know if it should change anything. It has to be clarified.

I will destroy the machines and take it a step at a time cause i did everything really fast to test the HA module. I ll come back to it in the next few hours or latest tomorrow (unless someone else does it before me...:P)

Title: Re: The cluster has not any resources defined.
Post by: christian on February 27, 2014, 09:11:03 am
Highlighted by Robb:
Introducing HA (http://labs.zentyal.org/high-availability-in-zentyal/)

Indeed this is more screen-shots collection (no surprise here  :-\) but at least it confirms:
- the active/ passive design (this was already clear in my mind  8))
- there is no resource management.

It nevertheless open doors for a lot of technical debate about design choices.
- DNS cluster while another approach could have been to synchronize DNS content and define more than one DNS client side (thus getting something closer to active/active in term of feature).
- VPN cluster ? well, as VPN client, why not but as VPN server, one can already define multiple VPN servers client side so that you can connect to another server in case the first one fails.
- what's about other services like mail ?
- what's about data (like files sharing... and mailboxes too)

I suppose we will have to run some reverse engineering...  ;D
Title: Re: The cluster has not any resources defined.
Post by: sixstone on February 27, 2014, 09:48:58 am
Hi guys!!

Thanks very much for testing HA module. Please, stress this out :).

As stated in the article, a resource (name scheme borrowed from the Cluster Resource Managers ;)) is a the scope of this module: Floating IP addresses and DHCP module. Those resources are meant to be run in a single instance of the multi-node cluster. You can move resources using Promote and Demote buttons.

The replication only works for configuration settings of the following modules: DHCP, DNS, Firewall, HA, IPS, Network, Objects, Services, Squid, Trafficshaping, CA and OpenVPN. In CA module, the certificates, private keys and such are replicated. Likewise /etc/zentyal directory is synchronised.

- there is no resource management.
It is as explained above. Quite limited indeed.

Quote
It nevertheless open doors for a lot of technical debate about design choices.
- DNS cluster while another approach could have been to synchronize DNS content and define more than one DNS client side (thus getting something closer to active/active in term of feature).

Using the basis of the DNS module (without any kind of DNS replication), the configuration is completely active/active if no Samba AD is present :).

Quote
- VPN cluster ? well, as VPN client, why not but as VPN server, one can already define multiple VPN servers client side so that you can connect to another server in case the first one fails.
You can have a cluster of VPN servers with the same configuration without a hassle using this module o:).

Best regards and happy testing,
Title: Re: The cluster has not any resources defined.
Post by: illunis on February 27, 2014, 09:50:44 am
Good morning all :)

Thanks Christian...havent seen that...it also confirms that SAMBA clustering is not (at least yet) supported

"if you require a custom resource configuration, you will have to use the pacemaker directly to configure using crm shell"

I ll keep on playing then and see what more can we destroy :)
Title: Re: The cluster has not any resources defined.
Post by: christian on February 27, 2014, 10:16:06 am
After some chat, I've got a much better understanding about Zentyal HA.
As very clearly written in above link, scope is FW, DNS, DHCP and OpenVPN.

As explained by sixstones, having VPN cluster may help to not duplicate configuration.

Regarding FW, I've to admit that I'm totally lost but thinking twice about this, if your FW fails, having another one handling same iptables rules without having other services like proxy, mail and other stuff switching to the secondary node, given the fact that services are reached quite often using CNAMES or host names that are added in DNS on your behalf by either Zentyal directly or DLZ  makes, at least for the time being, this FW HA stuff at least questionable.  :-X

At least, scope being now crystal clear, I'm done with my tests  ;D
Title: Re: The cluster has not any resources defined.
Post by: christian on February 27, 2014, 10:38:53 am
...it also confirms that SAMBA clustering is not (at least yet) supported

Although I just wrote that I'm done with this (and indeed I'm done) there is at least one comment I can't refrain myself to express, reacting to your point:
- Samba is now suffering from the exact same problem (and to me major drawback) as Microsoft Windows: this is all seen as a whole, single service while it provides many different services and features. Samba acts as domain controller, file server, DNS (to some extend via DLZ)
- thinking about HA and Samba without having clear understanding that these services are different and require different approach is, IMHO, a misconception.

DC HA exists, out of the box, when you configure additional DC. This brings de-facto "AD like" content synchronization (I'm discussing about Samba here, not Zentyal implementation). Therefore building cluster for this purpose (service) is not mandatory.
Same for DNS related stuff.

File server is another totally different story. File server is made of at least 2 layers:
- file server itself (as a service client is accessing)
- storage

In our implementation, this is often merged on same box but you may imagine this is done with 2 different layers, even when using one single box (e.g. look at Netapp implementation where clustering on one single box splits service layer and data layer).

So, to make a potentially long debate short, there is a need for file sharing HA that is very different from DC HA and even from Samba HA, reason why I react and suggest that you keep this in mind while addressing this point  ;)
Title: Re: The cluster has not any resources defined.
Post by: sixstone on February 27, 2014, 10:37:01 pm
indeed and now (this was not shown 2 days ago if I remember well), floating IPs are shown attached to interface that is in the same - network - range  ;D

Funny enough, there is no control while adding floating IP but if you add one with IP that is not covered by existing interface, then this one is shown as "error" in the cluster status tab.

This is now potentially fixed in this pull request: https://github.com/Zentyal/zentyal/pull/1054

Thanks very much for your feedback!

Do not hesitate to open new issues in the new tracker if you find any other bug!

Happy testing!
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 01, 2014, 10:04:44 pm
Hopefully this is still on topic.   I have spent a good chunk of my weekend trying to get this working.  I started off with my previously installed and upgraded 3.4 testbed running under KVM on my local machine.  I was able to setup a cluster and add resources.  I added a second machine from the same daily ISO (2-22) and ran the upgrade.  On joining the cluster,  things started going wrong.  Long story short,  split-brain.  Each machine thinks the other is off-line.  Doing Zentyal restarts will result in a hung machine.


Starting clean from 13.10 server gives very bad results.  It will not finish the install and hangs on dpkg --configure zentyal-core until it exhausts memory and oom kills it.

Grr....   I have tried one network interface on the machine and 2.  I have tried going with the daily build ISO as well as straight from Ubuntu server.

I haven't tried sacrificing any chickens yet but maybe it would help  :D


Thoughts?
   
Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 01, 2014, 10:30:08 pm
+1 for the chickens :)
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 01, 2014, 10:42:54 pm
This is what I get from a crm_mon -1.   Split brain,  right?




Code: [Select]
Last updated: Sat Mar  1 15:29:25 2014
Last change: Sat Mar  1 15:21:32 2014 via crmd on Campion
Stack: corosync
Current DC: Campion (1) - partition WITHOUT quorum
Version: 1.1.10-42f2063
2 Nodes configured
1 Resources configured


Online: [ Campion ]
OFFLINE: [ Starkey ]

 testing        (ocf::heartbeat:IPaddr2):       Started Campion

Code: [Select]
Last updated: Sat Mar  1 15:29:57 2014
Last change: Sat Mar  1 15:21:55 2014 via crmd on Starkey
Stack: corosync
Current DC: Starkey (2) - partition WITHOUT quorum
Version: 1.1.10-42f2063
2 Nodes configured
0 Resources configured


Node Campion (1): UNCLEAN (offline)
Online: [ Starkey ]



Sixstone stated that they were using quorum disk right?  I can find no evidence of that or any other quorum mechanism.


Code: [Select]
root@Campion:/var/log# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      3556/vsftpd     
tcp        0      0 192.168.163.3:53        0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 192.168.122.22:53       0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      984/sshd       
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      1963/named     
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      2557/haproxy   
tcp        0      0 127.0.0.1:62080         0.0.0.0:*               LISTEN      3468/apache2   
tcp        0      0 127.0.0.1:61443         0.0.0.0:*               LISTEN      3651/nginx.conf
tcp        0      0 0.0.0.0:390             0.0.0.0:*               LISTEN      3297/slapd     
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1054/mysqld     
tcp        0      0 127.0.0.1:62443         0.0.0.0:*               LISTEN      3468/apache2   
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      950/redis-server
tcp        0      0 127.0.0.1:6380          0.0.0.0:*               LISTEN      1200/redis-server
tcp        0      0 192.168.163.250:8880    0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 192.168.163.3:8880      0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 192.168.122.22:8880     0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 127.0.1.1:8880          0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 127.0.0.1:8880          0.0.0.0:*               LISTEN      3332/kdc       
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42361     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56053         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56052         ESTABLISHED 3658/nginx: worker
tcp        0      0 192.168.122.22:443      192.168.122.1:42356     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42357     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56050         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0    768 192.168.122.22:22       192.168.122.1:39664     ESTABLISHED 2714/sshd: denny [p
tcp        0      0 192.168.122.22:443      192.168.122.1:42358     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42359     ESTABLISHED 2557/haproxy   
tcp        0      0 192.168.122.22:443      192.168.122.1:42360     ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:56054         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56055         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:61443         127.0.0.1:56053         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56055         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56054         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:61443         127.0.0.1:56050         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56051         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp        0      0 127.0.0.1:61443         127.0.0.1:56051         ESTABLISHED 3658/nginx: worker
tcp        0      0 127.0.0.1:56052         127.0.0.1:61443         ESTABLISHED 2557/haproxy   
tcp6       0      0 :::22                   :::*                    LISTEN      984/sshd     
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 01, 2014, 10:45:05 pm
+1 for the chickens :)

How do you prefer yours?  Grilled or Cacciatore ?
Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 01, 2014, 10:52:57 pm
Had to look up Cacciatore but sounds like something I should try :)

Voodoo Peri Chicken solves all manner of technical difficulties, starts with a pain in the stomach and has a very hot exit but leaves a very clean system.
Title: Re: The cluster has not any resources defined.
Post by: sixstone on March 03, 2014, 12:48:44 pm
Hi half_life,

We are not using quorum disk at this moment.

Probably, the keys are not synchronised. You'd better chhose a host, leave the cluster and re-join it to the cluster and save changes :).

Best,
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 03, 2014, 04:21:24 pm
CA being active and configured wouldn't happen to be a pre-req would it?
Title: Re: The cluster has not any resources defined.
Post by: sixstone on March 03, 2014, 04:31:15 pm
No, it is a shared-secret scheme.

FYI, the key exchange is crypted :).
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 09, 2014, 01:09:37 am
I am still where I was last time I wrote.  I am unable to get the cluster to do anything but be split brained.  I dunno and the logs aren't giving me any clues.  Anyone have any thoughts before I put this back down and wait for more time to play with it?

Title: Re: The cluster has not any resources defined.
Post by: sixstone on March 09, 2014, 09:35:45 am
Hello,

If I were you, I'd select a node and do as follows: Leave the cluster, join the cluster again and save changes. This should do the trick and you will not lose any information (conf replicated).

Only if you are interested in getting to know what's wrong, check the corosync auth file (/etc/corosync/authkey) is equal in both nodes and they are reachable each other. Running corosync-quorumtool -l and such it may help to debug the issue.

Best,
Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 09, 2014, 04:08:41 pm
Dunno if anybody else fancies having a look at HA.

http://sourceforge.net/projects/zentyal/files/Incoming/

I have 3.4 loaded and so far its been a pretty painless affair. OPenChange and SOGo work great but still to have confirmation if anybody else struggles.

Samba seems to work but today I will be adding a few OU's and users with group policies. Just to do some further testing its a requirement of Openchange and I don't really expect anything undue.

I have my main system going with 3.4 and have already added the proxy which seems fine but haven't tried the SSO and filter groups yet.

Only thing seems to be that the bundle part of the openvpn server is missing, prob just an oversight with the new distro.

I will say that I really don't like Half-Lifes title of bughunter and it is nothing to do with Half-Life as is he is prob the best on here at it.
I think anyone can be a bug hunter and you don't need to get techy, just posting where at what things went wrong is enough.
If you can include a screen shot or better the log then that just add's more.
Being a noob is extremely important as your experience will tell much.

If there are any talented log sniffers out there who have limited time maybe if a bug is posted you could just confirm and post logs and stuff so its concise for the devs.

Half-Life I said it before about "Bug-Hunter" honestly we have a singular bughunter Doh! Lol wish they would give you a proper title maybe beta co-coordinator or QA co-coordinator. If titles are a must?
I cringe every time I see that title :) Our singular bug hunter seems to set a tone to me, but hey it is me!

Back to HA I deliberately created a separate thread on HA because I wanted to document from start to finish.
Hopefully even though a bit long winded it can act as a quick start guide to start.

With a little co-ordination I am sure many heads would make light work. I stopped at a point because I really don't understand how it can work without a quorum disk.
I am not saying it can not work without a quorum disk but personally I just don't understand how?

Would be great when it comes to HA that maybe someone would slap there meatware on the table and explain a little more.

Coffee, bacon butty, visit to the bank as low on supplies and will have a look at the info Sixstone has provided.

My reading when I get back http://clusterlabs.org/doc/en-US/Pacemaker/1.1-plugin/html/Clusters_from_Scratch/ch05s03.html
Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 09, 2014, 07:22:54 pm
Hello,
Only if you are interested in getting to know what's wrong, check the corosync auth file (/etc/corosync/authkey) is equal in both nodes and they are reachable each other. Running corosync-quorumtool -l and such it may help to debug the issue.
Best,

node1 authkey
€áÓ¹Á|†•àêÀ›3˜šfÀ!Nç)WçÀ;ðrK
%Õ¨²Í,0õ­1´W±Xy‹+‹g@TôÖÞE|Ѹ ×ôˆ\‡´6/3"ؒ=~¶…å+; ’áH™éþĸ–:à¸mtwæxF¼Õä–É64oޓƒ^®YQ

node2 authkey
€áÓ¹Á|†•àêÀ›3˜šfÀ!Nç)WçÀ;ðrK
%Õ¨²Í,0õ­1´W±Xy‹+‹g@TôÖÞE|Ѹ ×ôˆ\‡´6/3"ؒ=~¶…å+; ’áH™éþĸ–:à¸mtwæxF¼Õä–É64oޓƒ^®YQ

node 1 corosync-quorumtool -l
Membership information
----------------------
    Nodeid      Votes Name
         1          1 192.168.3.1 (local)

node2 corosync-quorumtool -l
Membership information
----------------------
    Nodeid      Votes Name
         2          1 192.168.3.4 (local)

zentyal@zent1:~$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Sun Mar  9 18:30:38 2014
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          1
Ring ID:          56
Quorate:          No

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      1
Quorum:           2 Activity blocked
Flags:           

Membership information
----------------------
    Nodeid      Votes Name
         1          1 192.168.3.1 (local)


zentyal@zent2:~$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Sun Mar  9 18:29:44 2014
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          2
Ring ID:          16
Quorate:          No

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      1
Quorum:           1 Activity blocked
Flags:            2Node WaitForAll

Membership information
----------------------
    Nodeid      Votes Name
         2          1 192.168.3.4 (local)

Still confused as each quorum is expecting 2 votes but only getting one.
This is what I don't get if you havent got a shared quorum disk.
I presume there is some form but local to each machine which means that each node thinks its alone?!

This time before I down or reboot a VM I am going to remember to disable HA.





Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 09, 2014, 10:39:12 pm
Had a browse around and still no thoughts then I suddenly thought "Hold on the port in Zentyal is the replication port", "What is the HA corosync port?"

So enabled 5405 UDP on both servers.

zentyal@zent1:~$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Sun Mar  9 21:39:31 2014
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          1
Ring ID:          148
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2 
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
         1          1 192.168.3.1 (local)
         2          1 192.168.3.4


zentyal@zent2:~$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Sun Mar  9 21:35:54 2014
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          2
Ring ID:          148
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           1 
Flags:            2Node Quorate WaitForAll

Membership information
----------------------
    Nodeid      Votes Name
         1          1 192.168.3.1
         2          1 192.168.3.4 (local)

Still got the red box on save though?

Looking at the logs not sure?

Also only a small thing but the demote and promote actions would seem to be the wrong way round.

Have a look at the image

Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 09, 2014, 10:56:09 pm
Now I am confused, now what? :) what do I do?
Title: Re: The cluster has not any resources defined.
Post by: half_life on March 09, 2014, 10:56:42 pm
I had understood your direction and had already tried leaving and rejoining to no avail.

Checking the files:
md5sum says they are the same.
dfbc5288d0e0d8d0ea79b7dc965fe91c  authkey-campion
dfbc5288d0e0d8d0ea79b7dc965fe91c  authkey-starkey

This is one unit:

Membership information                                                                                                                                                                                                                                                         
----------------------                                                                                                                                                                                                                                                         
    Nodeid      Votes Name                                                                                                                                                                                                                                                     
         2          1 192.168.163.4 (local) 




The other unit is currently frozen in a /etc/iniit.d/zentyal restart at the HA point in the sequence.  I suppose I will do another tear down and clean rebuild using the latest daily build on my next day off.


 I am very interested in this.
Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 09, 2014, 11:06:06 pm
I was exactly the same with the joining node. If it fails do not reboot or it seems to get stuck. Disable HA first (passive node anyway).

Using a VM I just cloned the first and changed the IP and host name. Deleted the dead server.

Adding a service HA with UDP 5405 (both nodes)

Then allowing on the firewall.

I was looking on the HA screen and kept thinking 443 a funny port for a default.
It suddenly dawned on me this is purely for replication shipping and is the webadmin port, doh! obviously.
That is why it says blocked because it is.

My question is OK, It works and I have a floating IP.

But showing my complete ignorance in conjunction with services I am totally unsure what to do next?
External ports have me totally baffled?

Adding another float auto updates across nodes.

OK Euston, guide us in!

ps I was browsing http://clusterlabs.org/wiki/Debian_Lenny_HowTo

ps ps Community QA (much better title)

The leaving and rejoining.

The floating IP sets something somewhere.
On the designated node whilst module enabled.
Remove all floats.
leave cluster and create new.
add float

node2
join

Title: Re: The cluster has not any resources defined.
Post by: peterpugh on March 10, 2014, 05:56:11 am
I sort of had to think about HA again apols for being slow.

I am going to let Half-Life continue with his thread here.

I am going to jump back to my Beta HA post in the below url. Still a few things that I am unsure of.
https://forum.zentyal.org/index.php/topic,20921.msg80385.html#msg80385

Apols from jumping in but it was just to utilise the info from sixstone

Title: Re: The cluster has not any resources defined.
Post by: sixstone on March 10, 2014, 07:07:52 pm
Hi all,

The other unit is currently frozen in a /etc/iniit.d/zentyal restart at the HA point in the sequence.  I suppose I will do another tear down and clean rebuild using the latest daily build on my next day off.

I suppose you allow all connections in your private network between both nodes, it is not that case. You must let heartbeat packets from corosync pass in the firewall. The corosync server is listening on UDP/5405 as peterpugh has explained above.

I'd better to let the firewall open between cluster nodes to leave entropy away.

Best regards,