Author Topic: Dead Primary DC, Working Secondary DC - How Create Primary again?  (Read 1935 times)

chris.holmes

  • Zen Apprentice
  • *
  • Posts: 8
  • Karma: +4/-0
    • View Profile
Zentyal Version 6.1.6 running only as a domain controller / DNS server.
Primary and Secondary DNS Servers. NOT using roaming profiles. Have all my scripts and the workstation group policy backed up.

Problem 1: My Primary domain controller (PDC) is dead.
Secondary Domain Controller is functional (SDC), domain authentication is working. The license key is the only thing left of the PDC.
What do I need to do to create a new Primary Domain Controller for my domain so I don't loose all the user accounts, connected computers etc.?

I'm assuming turn the SDC into a PDC and create a new SDC, but documentation on that is mainly on migrating from a Windows PDC.

Problem 2: (which lead to the dead PDC)
DNS not updating automatically. Got the following error after adding the noexpiry flag to the dns-<PDC> account.

Exit value: 1 at root command kinit -k -t /var/lib/samba/private/dns.keytab dns-zentyal failed.
Error output: kinit: Password incorrect

How do I properly set the password in the dns.keytab file to get DNS updating properly again?

Explaination of Problem 2:
The password for the dns-<PDC> was manually changed via the Users and Computer Management screen. The fix I found to reset the password on the dns-<PDC> account was the start of the cause of Problem 1.
THIS IS BAD DO NOT DO - (samba_upgradedns --dns-backend=local then back to BIND9_DLZ)

This is me putting down the shovel to get out of the hole. Thank you in advance.

doncamilo

  • Zen Samurai
  • ****
  • Posts: 478
  • Karma: +165/-1
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #1 on: November 20, 2020, 04:56:49 pm »
:)

1 Backup!

1.1 Backup!

1.2  Read https://wiki.samba.org/index.php/Transferring_and_Seizing_FSMO_Roles
(Zentyal provides the /usr/share/zentyal-samba/ad-migrate that manages this operation)

Bear in mind that the Webadmin Domain panel will be outdated Don't change anything here! https://doc.zentyal.org/en/directory.html#total-migration

2 Your initial problem was probably fixable by creating a new dns.keytab with this command:

Code: [Select]
samba-tool domain exportkeytab dns.keytab --principal=dns-$(hostname)

(I've never seen before the setexpiry command crashing a kerberos principal. Could you paste here the logs?)

In order to fix your issue read this: https://wiki.samba.org/index.php/Dns_tkey_negotiategss:_TKEY_is_unacceptable In spite of your previous experience, temporarily changing the DNS Back End, should fix your issue (I have done it in a VM right now and it fixed the dns-$(hostname) which password I changed manually just before) Try it in a VM before to proceed in production!

You can check the fix this way:

Code: [Select]
samba_dnsupdate --verbose --all-names
Cheers!





- Do my pigeons bother you passing over your land?
- They block the sun!

G. Guareschi., Don Camillo.,

chris.holmes

  • Zen Apprentice
  • *
  • Posts: 8
  • Karma: +4/-0
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #2 on: November 20, 2020, 10:19:58 pm »
Thank you for the response. Yes Backup but I need to take my own advice in that an untested backup is not a backup.
I had the PDC virtualize on a ZFS volume and my snapshots are corrupt, and so are my offsite replications.
This is very alarming with ZFS and something I'm looking into. The system my SDC is on a different machine and doesn't have this issue.

Sorry if I was unclear about what caused the main issue but it wasn't the set expiry command, it was the "samba_upgradedns --dns-backend=local" then setting it back to  BIND9_DLZ. I might of been able to save it if I ran "samba-tool domain exportkeytab dns.keytab --principal=dns-$(hostname)" first, but there were many other issues.

Regarding the dns.keytab fix - Worked like a charm. Thank you. "samba-tool domain exportkeytab dns.keytab --principal=dns-$(hostname)"

Domain Controller fix:

1. Show the list of who owns the rolls by using:
samba-tool fsmo show

2. Seize all the FSMO roles to the SDC by running this command on the SDC:
samba-tool fsmo seize --role=all

3. Demote the broken domain controller - https://wiki.samba.org/index.php/Demoting_a_Samba_AD_DC

4. Don't change anything in the Zentyal WebAdminconsole Domain Panel

5. Create a new SDC and join it to the domain.

Questions:
1. Is there a way to get the WebAdminconsole Domain Panel updated on the now PDC?

I will be backing up the SDC as is (via the Zentyal WebAdminconsole and a tested snapshot of the system) and will try this out over the weekend.
Will post the outcome.

Thank you, I've gone from panic to hope.



 



doncamilo

  • Zen Samurai
  • ****
  • Posts: 478
  • Karma: +165/-1
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #3 on: November 25, 2020, 10:23:58 am »
 :)

Regarding your question, I think that there's no easy way of updating the Webadmin > Domain panel :P https://doc.zentyal.org/en/directory.html#total-migration

Cheers!
- Do my pigeons bother you passing over your land?
- They block the sun!

G. Guareschi., Don Camillo.,

chris.holmes

  • Zen Apprentice
  • *
  • Posts: 8
  • Karma: +4/-0
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #4 on: December 24, 2020, 12:40:42 am »
I was able to transfer the roles and demote the server as described.
Creating a new Secondary Domain Controller.

This will leave me in a position that I will never have a working Domain panel for a Primary Domain Controller. Hmmm....

Thank you for your help.

chris.holmes

  • Zen Apprentice
  • *
  • Posts: 8
  • Karma: +4/-0
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #5 on: January 19, 2021, 05:34:04 pm »
My system is too far gone. I was able to export the Users and Groups and the sysvol directory.
Rebuilding a new Primary Domain Controller, imported users and groups.
Note: Exporting doesn't sets user passwords to "password". If you have end user passwords, you can change them before import.

I was able to migrate user profiles to the remade domain but login into the local workstation and using this tool:
User Profile Transfer Wizard
http://www.forensit.com/downloads.html
It also joins the new domain at the same time. Huge time saver.

I have my server running in a VM with snapshots replicating offsite. I've been able to roll back my DC as a test.

doncamilo

  • Zen Samurai
  • ****
  • Posts: 478
  • Karma: +165/-1
    • View Profile
Re: Dead Primary DC, Working Secondary DC - How Create Primary again?
« Reply #6 on: January 25, 2021, 03:58:00 pm »
 :)

It's an interesting tool.

Actually, if it weren't for the need to join the machines to the domain, more of the times I would prefer to migrate the data from old servers to a fresh install instead of upgrading them(especially if you have your shares in a dedicated disk)

Thank you!
- Do my pigeons bother you passing over your land?
- They block the sun!

G. Guareschi., Don Camillo.,