Zentyal 7. I can't restart dns, I can't update and the web interface gives 504

[fcortes]

Hi guys

my zentyal ad controller went out today. A power outage crippled the server and now I can´t load the web interface and I seem to be unable to get dns (bind9), nginx or samba services restarted.

things I tried

1. tried to restart some services, some I was able to restart, other just simply hang
sudo zs dns restart - hangs indefinetely
sudo systemctl restart  - hangs indefinetely
sudo systemctl start zentyal.webadmin-nginx.service worked ok
sudo systemctl start zentyal.webadmin-uwsgi.service worked ok
after issuing the last commands I get a 504 ngnix error when trying to load the page

2. I figure I'd try to update the system but that didn´t help
first of all sudo apt update was not able to connect without dns and for some reason I didn´t have a default gateway but that I was able to resolve by adding the gateway manually with sudo ip route add default via <gateway ip> dev eth0, so I can ping out to an ip address but i haven´t be able to figure out how to get dns working.
I did an sudo apt autoremove -y.. but this process gets stucked at 99% after restarting zentyal module webadmin

3. I read I could try to reconfigure the packages so I did sudo -configure -a but that get's stuck in configuring zentyal-core (7.0.9)...

4. I tried to lookup some info in the logs
- zentyal.log shows nothing that may lead to a resolution.. no error in particular
- syslog shows some error related to samba and dnsupdate_nameupdate_done failed dns update with exit code 1
- software.log says there's an issue connecting to local mysql server through socket /var/run/mysqld/mysql.sock' when processing package zentyal-core, I tried to restart mysql and and that did work as well as this command: mysqlcheck -u root -p$(cat /var/lib/zentyal/conf/zentyal-mysql.passwd) \
   --all-databases

5. try sudo dpkg-reconfigure zentyal and the same for bind9 and the command completed with no error
but when I try to do sudo sudo zs dns start it hangs.. and when i try to do sudo systemctl start bind9 it fails with code exited status =1/failure ... failed to start Bind Domain name server
and journalctl -u bind9 shows no entries

6. samba  (smbd) appears dead.. and when I try to do sudo systemctl start smbd if failed after sometime (it times out)...  checking sudo journalctl -u smbd says failed to start samba smb daemon but nothing that would tell me why.
I tried to check samba with samba-tool dbcheck -cross-ncs and --fix and all appears good but if I check if the service is up and running with sudo systemctl start smbd It fails and I get a message that smbd.service is masked as the reason. I tried to unmask it but still not able to get the service up and running again.
I looked at var/log/zentyal/zentyal.log and I get something that says
'ldap.::safeconnect - fatal : could not connect to samba ldap server connect permission denied  at /usr/share/perl5/ebox ldap.pm line 219, found this post https://forum.zentyal.org/index.php?topic=35172.0
, I did as suggested but I can't still start bing9 or samba

Any help or guidance would be greatly apprciated it

[Siroco]

There was a critical bug in the DNS module that probably is causing your issue.

- https://github.com/zentyal/zentyal/issues/2173
- https://github.com/zentyal/zentyal/issues/2175

Before applying the fix, make sure that there is not any lock file related to the DNS, you can do this as follows:

1. Get the DNS lock file:   
   

Code: [Select]

sudo lslocks | grep 'dns.lock'
2. Kill the lock file process if present:   
   

Code: [Select]

sudo kill -9 PID-OF-LOCK   

[fcortes]

I think I saw this in my search and troubleshooting attempts and tried it but the restart of dns just hangs
nevertheless, I went ahead and tried it again:
I stopped the dns lock process found and
I went ahead and tried the solution by
doing sudo nano /usr/share/zentyal/stubs/dns/named.conf.option.mas
and left this on that file as suggested
// DNSSEC configuration
dnssec-validation yes;

problem is that
sudo zs dns restart
again, just hangs... been waiting for a while now but no error is thrown nor the command completes.

after 10 minutes of waiting I opted to restart the server and give it another go.
but again. no dns resolution. (ping google.com temporary failure in name resolution) but ping to 8.8.8.8 succeeds
any idea where I might be able to see what else is going on?

[fcortes]

I found something

when I do
sudo journalctl -u named -n 50
I see a red line that says /etc/bind/anmed/conf.options:20 option 'dnssec-enable' no longer....
and when I go to that file I see that option is there.. so comment it and save it..
but then when I try to restart the service with sudo systemctl restart bind9
it fails and if I go look up again journal, the same line appears in red.. and somehow the file that I edited before it's again with the problem line uncommented... what is going on ?

[fcortes]

almost at my wits' end
dns never got back up again
I finally figured out why options file was not taking my changes after I finally got the proper fonfiguration for bind9/named I never got dns working again.. so I have a server that I can´t update

Somehow zs webadmin restart worked and I got the web interface back but not for too long:
radius which I use was down, dns was down and so ldap
I tried to enabled modules but some failed samba and dns (of course)
I decided to updated the damn zentyal-dns with the wrong .deb package (I have 7.0.9) but I started dkpg with 7.1.x. I went back and got the 7.0.3 .deb package but that installation never succeeded.. i gets stuck in the confugraiton and i ended cancelling out... of course at that point I have packages broken..
tried to do a dkpg --configure -a and of course it gets stuck from the get go on the dns-zentyal configuration.. i never finishes.. so once again ctrl-c out of that.. more stuff broken..
now zs webadmin won´t start.. so Iḿ back to square one but worst as
sudo apt install -f is a no go.
Iḿ getting fired

[fcortes]

Holly shit

ok.. this is what I ended up doing

- removing/changing a few lines of the options.mas or .conf tied to bind (dns/named) as suggested by dev given the bug found.. did get my dns working ...so
I remove dns.. and that took out samba (dc/ad) and radius, which I use.. I said fuck it..
then I resintall the damn zentyal dns (if you don´t know the names apt search zentyal should give you the names of the services needed)
a restart after... lo and behold.. I was finally able to get dns working again.... proceed to updated from 7.0.9 all the way up to 8 (last version available)... in the process after getting 7.1 working and having the system updated.. I installed zentyal-samba and zentyal-radius... since the configs were there.. it appears like I didn´t loose users...
fingers cross my users will be able to authenticate... but well.. this is a huge step forward.