Author Topic: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!  (Read 23954 times)

J. A. Calvo

  • Zentyal Staff
  • Zen Hero
  • *****
  • Posts: 1986
  • Karma: +67/-3
    • View Profile
    • http://blogs.zentyal.org/jacalvo
Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« on: December 27, 2012, 12:25:47 pm »
Hi Zentyal 2.2 users,

We know that many of you were expecting a way to migrate your configuration from a Zentyal 2.2 to the new Zentyal 3.0 server and we're glad to let you know that a migration guide has now been published!

It is important note that due to the mayor changes introduced in Zentyal 3.0 (new distribution base, Kerberos, deep changes in some modules like HTTP Proxy, etc.), migrating everything is not possible. However, the migration guide provides some helpers to ease the migration process and helps to avoid manual configuration from the scratch.

You can find all the information at:

http://trac.zentyal.org/wiki/Documentation/Community/Migrate_2.2_to_3.0

If you are missing some specific information, please do request it through the Forum and we'll do our best to include it. When additional migration information is available, it will be documented on this same page.

We hope you will soon enjoy your new Zentyal 3.0!
Zentyal Server Lead Developer

Escorpiom

  • Zen Hero
  • *****
  • Posts: 897
  • Karma: +25/-1
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #1 on: December 27, 2012, 01:19:00 pm »
A huge "thank you"!
I find this configuration export much safer than the previous migration tool. This wil yield cleaner results.

Cheers.
Marcus' Rule:
Blanks & capitals = avoid it and you'll avoid problems...

ian

  • Community Council Member
  • Zen Samurai
  • *****
  • Posts: 296
  • Karma: +10/-1
    • View Profile
    • Familie site
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #2 on: December 27, 2012, 03:24:01 pm »
Hi,

Thanks for the scripts.

Is it also possible to upgrade from release 1.4.9 ?
I suppose that there are changes needed then because teh home structure of 1.4.9 is not the sam as in 2.2.

Only importing the users and groups is al a great advance.

Thanks and best regards,
Ian


bufke

  • Zen Apprentice
  • *
  • Posts: 14
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #3 on: December 28, 2012, 06:18:13 am »
Nice, but can't migrate passwords? Unless your deployment is trivial that's a pretty big deal. I would think one could copy over the hashes manually.

http://lists.samba.org/archive/samba-technical/2012-June/084758.html

Quote
The unicodePwd appears to be same hash as for sambaNTPassword in
samba3 ldap just using base 64 encoding rather than direct hex
printing of the value previously.

So copy the Samba NT password to the samba 4 ldap schema then copy userPassword for plain old ldap. Has anyone tried this? Am I missing something? I'll see if I can do this next week.

Of course if LM passwords are enabled (which is default) you can just use rainbow tables, yay security. Sadly I don't have LM passwords for all my users.

half_life

  • Bug Hunter
  • Zen Hero
  • *****
  • Posts: 867
  • Karma: +59/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #4 on: January 03, 2013, 11:11:49 pm »
Hi Zentyal 2.2 users,

We know that many of you were expecting a way to migrate your configuration from a Zentyal 2.2 to the new Zentyal 3.0 server and we're glad to let you know that a migration guide has now been published!

It is important note that due to the mayor changes introduced in Zentyal 3.0 (new distribution base, Kerberos, deep changes in some modules like HTTP Proxy, etc.), migrating everything is not possible. However, the migration guide provides some helpers to ease the migration process and helps to avoid manual configuration from the scratch.

You can find all the information at:

http://trac.zentyal.org/wiki/Documentation/Community/Migrate_2.2_to_3.0

If you are missing some specific information, please do request it through the Forum and we'll do our best to include it. When additional migration information is available, it will be documented on this same page.

We hope you will soon enjoy your new Zentyal 3.0!

I had a bout with the flu so am just now trying this.  In your documentation you say:
sudo perl dump-users-from-backup confbackupname.tar users.csv
Did you mean?
sudo perl dump-users-from-backup confbackupname.tar >users.csv


bufke

  • Zen Apprentice
  • *
  • Posts: 14
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #5 on: January 04, 2013, 11:33:34 pm »
Ok so LDAP password migration is not so bad. This doesn't include Samba!

1. In your 2.2 ldap server you need to get the userPassword attributes for all users. You can find gui tools or do cli like:
ldapsearch -D 'cn=ebox,dc=domain,dc=org' -w 'password_from_ldap_settings' -b 'dc=domain,dc=org'
That dumps out everything but what you want is the uid and userPassword. Keep in mind userPassword is a base64 of the hash.

2. Import your users as the guide suggests. Now they have bad passwords.

3. Decode passwords like this

echo crazy_base64_thing_you_got_from_ldapsearch | base64 -d

This decodes it but it's still a hash. It should start with something like {SHA} but could be some other type of hash depending on your setup.

4. Use ldapmodify on the 3.0 server to set the userPassword. By default the userPassword is just a base64 of {K5KEY} which isn't the password at all. I'm guessing it just saying use kerberos?
To do this make a ldif file like

dn: uid=username_here,ou=Users,dc=domain
changetype: modify
replace: userPassword
userPassword: {SHA}fdslfjdsffljsdfljsdfll

where userPassword is what you did base64 -d to. Run it like

ldapmodify -D 'cn=zentyal,dc=domain' -w 'password_from_ldap_settings' -p 390 -h yourhostname -f your_file.ldif

Verify it worked by running the ldapsearch command like you did in 2.2.

Now users can log into User's Corner, use pam_ldap, any other systems that depend on the ldap auth backend. In User's Corner people could change their own passwords and it updates everything! But this still sucks and forcing all employees to reset passwords is a great way to make people hate you.

I'll play with Samba and report back. Maybe I'll even make a script.

bufke

  • Zen Apprentice
  • *
  • Posts: 14
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #6 on: January 06, 2013, 12:40:30 am »
Ok samba guide. This does not include kerberos just basic samba authentication. Please note this is a proof of concept, I haven't tried this in production and neither should you without sufficient testing.

Get a list of uid and SambaNTPassword from Zentyal 2.2 as described in the above ldap guide.

I used python since we need to manipulate the hash a bit. Execute this on the 3.0 server. Fill in the username, password, and domain of course

Code: [Select]
import base64
import binascii
import subprocess

sam_file = "/var/lib/samba/private/sam.ldb.d/DC\=DOMAIN.ldb"
user_dn = "CN=<username goes here>,CN=Users,DC=domain"
ldap_samba_nt_password = "<SambaNTPassword goes here>"
b64_hash = base64.b64encode(binascii.a2b_hex(ldap_samba_nt_password))

cmd = """ldbmodify -H %s --controls=local_oid:1.3.6.1.4.1.7165.4.3.12:0 <<EOF
dn: %s
changetype: modify
replace: unicodePwd
unicodePwd:: %s
EOF""" % (sam_file, user_dn, b64_hash)
subprocess.call(cmd, shell=True)

You can test with smbclient.

I'll probably improve the script to handle a big list of users and passwords. Is there any reason this isn't a good migration strategy? If users weren't using kerberos before, why would they miss it with 3.0? And if they want it they can just change their password at their leisure.

A Linux client with pam_ldap who is mounting samba shares should never notice the change (in theory). I'm less sure about Windows clients, will they attempt to use kerberos and then fail?

jp.otto

  • Zen Apprentice
  • *
  • Posts: 25
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #7 on: January 21, 2013, 01:37:25 pm »
I run into problems while trying to follow the instructions.

When doing the import-basic-modules I get:
Code: [Select]
root@hostname:~# perl import-basic-modules hostname_2013-01-21-101112.tar
Use of uninitialized value in uc at /usr/share/perl5/EBox/Types/DomainName.pm line 76, <GEN0> line 726.
Use of uninitialized value $value in substitution (s///) at /usr/share/perl5/EBox/Types/Host.pm line 169, <GEN0> line 736.

dump-users-from-backup results in:
Code: [Select]
root@hostname:~# perl dump-users-from-backup hostname_2013-01-21-101112.tar users.csv
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 120.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 545.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 589.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 633.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 677.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 721.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 765.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 809.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 853.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 897.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 941.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 985.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 1029.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1047.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1080.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1097.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1115.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1133.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1151.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1168.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1186.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1203.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 1635.
write_file 'users.csv' - sysopen: Permission denied at dump-users-from-backup line 110.

The old machine is on 2.2:
Code: [Select]
root@hostname:~# dpkg -l |grep zentyal
[...]
ii  zentyal-antivirus                    2.2.2                                 Zentyal - Antivirus
ii  zentyal-common                       2.2.3                                 Zentyal - Common Library
ii  zentyal-core                         2.2.7                                 Zentyal - Core
ii  zentyal-ebackup                      2.2.5                                 Zentyal - Backup
ii  zentyal-firewall                     2.2.1                                 Zentyal - Firewall
ii  zentyal-network                      2.2.9                                 Zentyal - Network Configuration
ii  zentyal-objects                      2.2                                   Zentyal - Network Objects
ii  zentyal-printers                     2.2.2                                 Zentyal - Printer Sharing Service
ii  zentyal-samba                        2.2                                   Zentyal - File Sharing Service
ii  zentyal-services                     2.2                                   Zentyal - Network Services
ii  zentyal-software                     2.2.7                                 Zentyal - Software Management
ii  zentyal-users                        2.2.8                                 Zentyal - Users and Groups

The new one is a newly installed 3.0.1:
Code: [Select]
root@hostname:~# dpkg -l |grep zentyal
[...]
ii  zentyal                              3.0.1                                   Zentyal - Core metapackage
ii  zentyal-antivirus                    3.0                                     Zentyal - Antivirus
ii  zentyal-common                       3.0.5                                   Zentyal - Common Library
ii  zentyal-core                         3.0.10                                  Zentyal - Core
ii  zentyal-dns                          3.0.4                                   Zentyal - DNS Service
ii  zentyal-ebackup                      3.0.4                                   Zentyal - Backup
ii  zentyal-firewall                     3.0.1                                   Zentyal - Firewall
ii  zentyal-network                      3.0.1                                   Zentyal - Network Configuration
ii  zentyal-ntp                          3.0                                     Zentyal - NTP Service
ii  zentyal-objects                      3.0                                     Zentyal - Network Objects
ii  zentyal-printers                     3.0.1                                   Zentyal - Printer Sharing Service
ii  zentyal-samba                        3.0.12                                  Zentyal - File Sharing and Domain Services
ii  zentyal-services                     3.0.1                                   Zentyal - Network Services
ii  zentyal-software                     3.0.3                                   Zentyal - Software Management
ii  zentyal-users                        3.0.7                                   Zentyal - Users and Groups

Any ideas?

half_life

  • Bug Hunter
  • Zen Hero
  • *****
  • Posts: 867
  • Karma: +59/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #8 on: January 21, 2013, 05:22:43 pm »
I run into problems while trying to follow the instructions.

When doing the import-basic-modules I get:
Code: [Select]
root@hostname:~# perl import-basic-modules hostname_2013-01-21-101112.tar
Use of uninitialized value in uc at /usr/share/perl5/EBox/Types/DomainName.pm line 76, <GEN0> line 726.
Use of uninitialized value $value in substitution (s///) at /usr/share/perl5/EBox/Types/Host.pm line 169, <GEN0> line 736.

dump-users-from-backup results in:
Code: [Select]
root@hostname:~# perl dump-users-from-backup hostname_2013-01-21-101112.tar users.csv
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 120.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 545.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 589.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 633.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 677.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 721.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 765.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 809.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 853.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 897.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 941.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 985.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 1029.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1047.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1080.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1097.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1115.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1133.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1151.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1168.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1186.
Use of uninitialized value $description in join or string at dump-users-from-backup line 157, <GEN0> line 1203.
Use of uninitialized value $description in join or string at dump-users-from-backup line 131, <GEN0> line 1635.
write_file 'users.csv' - sysopen: Permission denied at dump-users-from-backup line 110.

The old machine is on 2.2:
Code: [Select]
root@hostname:~# dpkg -l |grep zentyal
[...]
ii  zentyal-antivirus                    2.2.2                                 Zentyal - Antivirus
ii  zentyal-common                       2.2.3                                 Zentyal - Common Library
ii  zentyal-core                         2.2.7                                 Zentyal - Core
ii  zentyal-ebackup                      2.2.5                                 Zentyal - Backup
ii  zentyal-firewall                     2.2.1                                 Zentyal - Firewall
ii  zentyal-network                      2.2.9                                 Zentyal - Network Configuration
ii  zentyal-objects                      2.2                                   Zentyal - Network Objects
ii  zentyal-printers                     2.2.2                                 Zentyal - Printer Sharing Service
ii  zentyal-samba                        2.2                                   Zentyal - File Sharing Service
ii  zentyal-services                     2.2                                   Zentyal - Network Services
ii  zentyal-software                     2.2.7                                 Zentyal - Software Management
ii  zentyal-users                        2.2.8                                 Zentyal - Users and Groups

The new one is a newly installed 3.0.1:
Code: [Select]
root@hostname:~# dpkg -l |grep zentyal
[...]
ii  zentyal                              3.0.1                                   Zentyal - Core metapackage
ii  zentyal-antivirus                    3.0                                     Zentyal - Antivirus
ii  zentyal-common                       3.0.5                                   Zentyal - Common Library
ii  zentyal-core                         3.0.10                                  Zentyal - Core
ii  zentyal-dns                          3.0.4                                   Zentyal - DNS Service
ii  zentyal-ebackup                      3.0.4                                   Zentyal - Backup
ii  zentyal-firewall                     3.0.1                                   Zentyal - Firewall
ii  zentyal-network                      3.0.1                                   Zentyal - Network Configuration
ii  zentyal-ntp                          3.0                                     Zentyal - NTP Service
ii  zentyal-objects                      3.0                                     Zentyal - Network Objects
ii  zentyal-printers                     3.0.1                                   Zentyal - Printer Sharing Service
ii  zentyal-samba                        3.0.12                                  Zentyal - File Sharing and Domain Services
ii  zentyal-services                     3.0.1                                   Zentyal - Network Services
ii  zentyal-software                     3.0.3                                   Zentyal - Software Management
ii  zentyal-users                        3.0.7                                   Zentyal - Users and Groups

Any ideas?

See my question above.  My question was actually a gentle "fix the documentation" type question.  They left out the > in the command.

Code: [Select]
sudo perl dump-users-from-backup confbackupname.tar >users.csv

jp.otto

  • Zen Apprentice
  • *
  • Posts: 25
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #9 on: January 22, 2013, 04:02:35 pm »
[...]
Any ideas?

See my question above.  My question was actually a gentle "fix the documentation" type question.  They left out the > in the command.

Code: [Select]
sudo perl dump-users-from-backup confbackupname.tar >users.csv

Thank you, that resulted at least in a users.csv, though the errormessages stay the same.

The "only" remaining problem is, that import-basic-modules seems not to do what I expected it to...
Or is it not its job to configure file sharing, shares and printers with their access rights?

half_life

  • Bug Hunter
  • Zen Hero
  • *****
  • Posts: 867
  • Karma: +59/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #10 on: January 23, 2013, 01:40:16 am »
I am sorry,  it has been too long since I ran the upgrade to remember which modules it carried over but you will have to do significant re-work in the file sharing module for sure.  Wish I had better news but I can see where writing an upgrade tool would have been a huge effort in the samba module.

jp.otto

  • Zen Apprentice
  • *
  • Posts: 25
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #11 on: January 23, 2013, 09:34:28 am »
Too bad, but thank you anyways  :o

graylion

  • Zen Apprentice
  • *
  • Posts: 32
  • Karma: +0/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #12 on: January 24, 2013, 06:54:36 pm »
folks, this is ridiculous. Migration needs to be part of the plan, not something you stick on after.

What about Zarafa for instance? Yes I can migrate emails via IMAP client if need be (gee, thanks), what about address books and calendar info? And no upgrade-in-place? Disappointed I am. If you are planning to do better than the competition, you need to do better than the competition. :(

friedmar

  • Zen Apprentice
  • *
  • Posts: 44
  • Karma: +0/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #13 on: January 24, 2013, 08:01:47 pm »
totally agree!

This is not a migration!

bufke

  • Zen Apprentice
  • *
  • Posts: 14
  • Karma: +1/-0
    • View Profile
Re: Migration guide from Zentyal 2.2 to Zentyal 3.0 published!
« Reply #14 on: January 25, 2013, 12:22:20 am »
Yea I'm still trying to plan my migration. Their old desktop debian package would set up pam_ldap for you but in 3.0 it's not compatible because they changed the ldap port, restricted anon searching, and do groups in a very different schema. So even if I redid the entire zentyal server I would have to change each and every client too. I'm no longer recommending Zentyal as a reasonable alternative to AD though. Hope to have a real migration guide for you all soon. I don't use Zarafa though so you are on your own with that. Good to know I should never consider it.