Author Topic: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users  (Read 8378 times)

jopeme

  • Zen Warrior
  • ***
  • Posts: 126
  • Karma: +0/-0
    • View Profile
Hello.
This script i got from some thread of this forum works for me on Zentyal 3.0. Now i have testing 3.2 version and it doesn't works. Script is :
#!/usr/bin/perl

use strict;
use warnings;

use EBox;
use EBox::UsersAndGroups::User;
use EBox::Global;

EBox::init();

my $users = EBox::Global->modInstance('users');


my @users;

open (my $USERS, 'users');

while (my $line = <$USERS>) {
    chomp ($line);
    my $user;
   
    my ($username, $givenname, $surname, $password, $grupo) = split(',', $line);
    $user->{'user'} = $username;
    $user->{'givenname'} = $givenname;
    $user->{'surname'} = $surname;
    $user->{'password'} = $password;   
       
    my $u= EBox::UsersAndGroups::User->create($user, 0);
    my $group = $users->group($grupo);
    $group->addMember($u);                         
}
   
close ($USERS);

1;

And i get following error:
Can't locate EBox/UsersAndGroups/User.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl .) at ImportUsersInBulk line 7.
BEGIN failed--compilation aborted at ImportUsersInBulk line 7.


How do i solve this?
Thanks.

jbahillo

  • Zentyal Staff
  • Zen Hero
  • *****
  • Posts: 1444
  • Karma: +77/-2
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #1 on: September 20, 2013, 02:06:21 pm »
API for Users module has changed in 3.2. If you dare to deal with it you can read the code of this module and then adapt the script to the new version ;)

jopeme

  • Zen Warrior
  • ***
  • Posts: 126
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #2 on: September 23, 2013, 06:00:09 pm »
I'm afraid my knowedge is not enought.  :'(

Miko10

  • Zen Apprentice
  • *
  • Posts: 13
  • Karma: +2/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #3 on: September 24, 2013, 09:46:23 am »
New version of script is already here

http://doc.zentyal.org/en/appendix-c.html#importing-configuration-data

But I'm missing adding users to group. Can anybody add to script goup adding? I'm not so good yet to make it myself.

Thanks.

znac049

  • Zen Apprentice
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #4 on: September 27, 2013, 10:40:36 am »
Hi, I'm having similar problems with bulk user creation in 3.2. I need to import a lot of users but it's important to me to give the new users the same unix uid (LDAP uidNumber) as in the old system. I looked at the code for EBox::Users::User::create and can see that I can pass many items to it, including uidNumber. I ended up with this code which runs and creates users:

Code: [Select]
#!/usr/bin/perl
 
 use strict;
 use warnings;
 
 use EBox;
 use EBox::Users::User;
 
 EBox::init();
 
 my $parent = EBox::Users::User->defaultContainer(1);
 
 while (<>) {
     chomp;
     my ($username, $pw, $uid, $gid, $givenname, $surname, $fullname, $email) = split(':');
     my $userObj = EBox::Users::User->create(
         uid => $username,
         password => $pw,
         uidNumber => $uid,
         fullname => $fullname,
         givenname => $givenname,
         surname   => $surname,
         description => 'Auto imported',
         mail => $email,
         parent => $parent
         );
 }
 
 1;

All other aspects of the new user are exactly as I require, but the uidNumber does not get set as I expect.  Here is the input the program reads:

Code: [Select]
ajith:sndwedoli:2044:1901:Ajith:Rija:Ajith Rija:ajith.rija@someplace.org
I expect the new uidNumber to be 2044, but instead it is 3000059 (and increments for each new user created). Does anyone have any ideas what's going on here?

Thanks,
-Bob

christian

  • Guest
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #5 on: September 28, 2013, 05:59:23 am »
That's pretty strange that you get uidnumber greater than 32767  :o
 I can't help regarding root cause because I did upgrade my 2.1 sandbox machine to 3.2 in order to look closer and I'm getting, although migration when smoothly, very strange behaviour.
I can't create neither accounts, groups or OU.

Code: [Select]
Can't call method "isContainer" on an undefined value at /usr/share/perl5/EBox/Samba/OU.pm line 118

jopeme

  • Zen Warrior
  • ***
  • Posts: 126
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #6 on: September 30, 2013, 10:17:18 am »
Quote
New version of script is already here

http://doc.zentyal.org/en/appendix-c.html#importing-configuration-data

But I'm missing adding users to group. Can anybody add to script goup adding? I'm not so good yet to make it myself.

Thanks.

I think this script is not right. I always get this error:
sudo ./ImportUsersInBulk2
Ocurri� un error actualizando LDAP: The server cannot find an object specified in the request
.<br/> Operation parameters:'delete []

Miko10

  • Zen Apprentice
  • *
  • Posts: 13
  • Karma: +2/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #7 on: October 02, 2013, 03:29:06 pm »
I think this script is not right. I always get this error:
sudo ./ImportUsersInBulk2
Ocurri� un error actualizando LDAP: The server cannot find an object specified in the request
.<br/> Operation parameters:'delete []

I don't have this problem. Script is working on my Zentyal 3.2. It's clean install or upgrade?

But as I said, it would be great to add group while adding users. I have to import over 200 users and don't wont to edit them manualy.

jopeme

  • Zen Warrior
  • ***
  • Posts: 126
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #8 on: October 03, 2013, 01:51:12 pm »
It is a new installation. I even returned to perform an new installation with the user only needs to be created for installation, then you install the modules i  need. After I run the script and this is the error that shows me:
sudo ./importusersinbulk2
Error en la creaci�n del usuario LDAP:  The client attempted to add an entry that already exists. This can occur as
a result of

 * An add request was submitted with a DN that already exists

 * A modify DN requested was submitted, where the requested new DN already exists

 * The request is adding an attribute to the schema and an attribute with the
given OID or name already exists
.<br/> Operation parameters:'add []


I don't understand what happen. There is no users except as i said default user (admin) and my users.csv file contains:

ub12mprof, Profesor manana,Aula B12,ub12mprof
ub12m01,Alumno manana,Aula B12,ub12m01
ub12m02,Alumno manana,Aula B12,ub12m02
ub12m03,Alumno manana,Aula B12,ub12m03

If i go to graphics interface (users and machines) i can see in users only two of them "ub12m01" and "ub12mprof".

Any idea?


znac049

  • Zen Apprentice
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #9 on: October 04, 2013, 09:26:27 pm »
Ok, my bulk import users script now works as per my example above. I've just used it to import ~200 users from Zentyal 2.2 into Zentyal 3.2 and I successfully imported the unix uid (LDAP uidNumber) which had not been working.

How did I do this? Well, read on...

I spent a while crawling over the perl modules in /usr/share/perl5/EBox... and discovered an unexpeced behaviour. Maybe it's documented somewhere, but I haven't seen it. What happens when you call EBox::Users::User::create(...) is that a basic user is created with the absolute minimum of information. Then zentyal builds a list of the remaining ldap attributes to be set *BUT* before it actually writes it to the ldap db, it calls a special function called 'preAddUser' on a number of other EBox modules that have an interest in users. Exactly which of these gets called depends on which modules you have installed. This means that each of these preAddUser functions gets the chance to override any of the ldap attributes. In my case, the EBox::SambaLdapUser module was trying to be clever (too clever, IMHO). It was overriding the uidNumber supplied by my script with the Samba xidNumber (extended uid) attribute. As far as I can tell, these two items don't need to be the same, so I don't think EBox::SambaLdapUser should be overriding it. Anyway, that's not for me to say.

So, I reinstalled Zentyal 3.2 with only minimum modules and definitely no File Sharing (Samba), imported my users, and then installed and enabled File Sharing. So far all seems to be working well.

Here is the script I used to dump data from Zentyal 2.2 and generate a random pronounceable password for each user:
Code: [Select]
#!/usr/bin/perl

use strict;
use warnings;

use EBox;
use EBox::Global;

use Text::Password::Pronounceable;

EBox::init();

sub escape_text {
    my ($txt) = @_;
    $txt =~ s/:/_+_/;
    return $txt;
}

my $usersMod = EBox::Global->modInstance('users');
my $users = $usersMod->usersList();

foreach my $user (@$users) {
    my $detail = $usersMod->userInfo($user->{'user'});
    my $email = $detail->{givenname} . "." . $detail->{surname} . '@somedomain.com';

    $email =~ tr/A-Z/a-z/;

    print join ":", escape_text($detail->{username}), Text::Password::Pronounceable->generate(8,10),
        escape_text($detail->{uid}), escape_text($detail->{group}), escape_text($detail->{givenname}),
        escape_text($detail->{surname}), escape_text($detail->{fullname}), $email;
    print "\n";
}

1;

and the script to bulk import into 3.2:

Code: [Select]
#!/usr/bin/perl

use strict;
use warnings;

use EBox;
use EBox::Users::User;

use Data::Dumper;

EBox::init();

my $parent = EBox::Users::User->defaultContainer(1);

while (<>) {
    chomp;
    my ($username, $password, $uid, $gid, $first, $last, $full, $email) = split(':');

    my $userObj = EBox::Users::User->create(
        uid => $username,
        password => $password,
        uidNumber => $uid,
        fullname => $full,
        givenname => $first,
        surname   => $last,
        description => 'Auto imported',
        mail => $email,
        parent => $parent
    );
}

1;

Hope this is of some use to folks trying to do something similar to me.

-Bob

christian

  • Guest
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #10 on: October 05, 2013, 11:10:36 am »
It does help, for sure (thanks a lot) and also confirms, if needed, that having multiple repositories to store similar information (here accounts stored in different LDAP servers due to technical constraint) makes things much more complex, unfortunately.

I'm pretty convinced, given what I heard during summit, that the right way to go, at least for those looking in this direction, is to have Samba part of Zentyal core modules (no more option with or without file sharing) so that only one single LDAP server is used.
- no more master / slave but one single way of having multiple Zentyal servers
- no more internal sync
 - easiest (perhaps) OpenChange integration

and last but not least, lighter Zentyal server  ;)

thiagopredebon

  • Zen Apprentice
  • *
  • Posts: 5
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #11 on: March 27, 2014, 01:10:45 am »
and the script to bulk import into 3.2:

Code: [Select]
#!/usr/bin/perl

use strict;
use warnings;

use EBox;
use EBox::Users::User;

use Data::Dumper;

EBox::init();

my $parent = EBox::Users::User->defaultContainer(1);

while (<>) {
    chomp;
    my ($username, $password, $uid, $gid, $first, $last, $full, $email) = split(':');

    my $userObj = EBox::Users::User->create(
        uid => $username,
        password => $password,
        uidNumber => $uid,
        fullname => $full,
        givenname => $first,
        surname   => $last,
        description => 'Auto imported',
        mail => $email,
        parent => $parent
    );
}

1;

Hope this is of some use to folks trying to do something similar to me.

-Bob
[/quote]

Need Help adding group name on the importing code. can you help me ? zentyal 3.4

f3rn4nd0q

  • Zen Apprentice
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: ImportUsersInBulk doesn't works in Zentyal 3.2 to create a lot of users
« Reply #12 on: April 29, 2014, 03:48:51 pm »
API for Users module has changed in 3.2. If you dare to deal with it you can read the code of this module and then adapt the script to the new version ;)
Hello,
Where can i find documentation to use the new API ?

Regards

kerridge0

  • Zen Monk
  • **
  • Posts: 57
  • Karma: +1/-0
    • View Profile
Hi, I have looked into the code of 3.2 and got this groups script working for me, and if 3.4 is the same api, which I guess it could well be, then you could do the following:-

  • Change the group name in the groups csv to be the "group DN"
    this is basically the name of the group followed by the "Groups DN"
    you can get the Groups DN by Going to Users and Computers ‣ LDAP Settings
    mine was ou=Groups,dc=worthing,dc=futures
    then, for example, my group named non-staff has a DN of:-
    CN=non-staff,OU=Groups,DC=worthing,DC=futures
    (I confirmed this by using the ldbsearch command, but it should be pretty standard)
  • in order to put that into the CSV file, I had to change the delimiter. this is because otherwise the group name wouldn't be parsed correctly as it already has commas in it. So I replaced the existing commas with the ~ (tilde) character before adding in the group DN
    e.g. first line:-
    desk1~user~desk 1~sunshine~CN=non-staff,OU=Groups,DC=worthing,DC=futures~
  • So I changed the BULKUSERS script as follows:-
Code: [Select]
#!/usr/bin/perl

use strict;
use warnings;

use EBox;
use EBox::Users::User;

EBox::init();

my $parent = EBox::Users::User->defaultContainer();

open (my $USERS, 'users.csv');

while (my $line = <$USERS>) {
    chomp ($line);
    my ($username, $givenname, $surname, $password, $grp) = split('~', $line);
    EBox::Users::User->create(
    uid => $username,
        parent => $parent,
        givenname => $givenname,
        surname => $surname,
        password => $password
    );
        my $group = new EBox::Users::Group(dn => $grp);
        $group->addMember(EBox::Users::User->new(uid => $username));

}

close ($USERS);

1;



« Last Edit: May 26, 2014, 07:30:34 pm by kerridge0 »

jbahillo

  • Zentyal Staff
  • Zen Hero
  • *****
  • Posts: 1444
  • Karma: +77/-2
    • View Profile
Hello kerridge0 :

Thanks a lot for contributing with this !
;)