Lost access to admin interface after upgrade 2.0 > 2.2.

[Escorpiom]

It looks like I could use a hand here...
Overnight I did the upgrade from my old 2.0. server to the Zentyal 2.2. Seemingly there were little problems, but when the migration tool finished,
there was no access to the admin interface anymore.
The last line from the migration tool was:

Code: [Select]

* Starting Zentyal module: apache                                       [fail]
The other error I got during migration was with moving the ssl cert to the new name.

Code: [Select]

mv: cannot stat '/etc/apache/ssl/apache.pem:: No such file or directoryAfter that I copied and renamed the .pem file by hand, but I never had the "etc/apache/ssl" dir but instead "etc/apache2/ssl dir".
So far I so good - but still no Zentyal admin interface.
I looked up robb's post and did the grep command, and sure - nothing listening on port 443.

Code: [Select]

netstat -tlnp | grep apache2
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      23121/apache2
tcp        0      0 0.0.0.0:8888            0.0.0.0:*               LISTEN      9956/apache2
Next, tried robb's suggestion dpkg --reconfigure zentyal
and got "package not installed".
So I did "apt-get install zentyal" and it installed ok.

Trying to restart zentyal apache and got this error:

Code: [Select]

/etc/init.d/zentyal apache restart
 * Restarting Zentyal module: apache                                     [fail]
root command /usr/share/zentyal/change-hostname ~ failed.
Error output: sed: -e expression #1, char 11: unknown option to `s'
 sed: -e expression #1, char 11: unknown option to `s'
 hostname: the specified hostname is invalid

Command output: .
Exit value: 1
I checked the hostname file in /etc and it has the correct hostname inside so I do not understand what is happening?
BTW, It seems that other functions like filesharing and printer sharing are working.

Looking at the Zentyal log, there are a few ugly entries:

Code: [Select]

11/16 03:02:45 DEBUG> Network.pm:1257 EBox::Network::setIfaceStatic - Interface eth0:vif0110 does not exist.
2011/11/16 03:02:45 WARN> Network.pm:192 EBox::Network::__ANON__ - Network configuration import failed

2011/11/16 03:05:32 DEBUG> UsersAndGroups.pm:1394 EBox::UsersAndGroups::userInfo - user name root does not exist.
2011/11/16 03:05:32 DEBUG> UsersAndGroups.pm:1394 EBox::UsersAndGroups::userInfo - user name hans does not exist.

2011/11/16 03:08:46 INFO> Service.pm:716 EBox::Module::Service::restartService - Restarting service for module: apache
2011/11/16 03:08:46 ERROR> Sudo.pm:213 EBox::Sudo::_rootError - root command /usr/share/zentyal/change-hostname ~ failed.
Error output: sed: -e expression #1, char 11: unknown option to `s'
 sed: -e expression #1, char 11: unknown option to `s'
 hostname: the specified hostname is invalid

Command output: .
Exit value: 1
2011/11/16 03:08:46 ERROR> Service.pm:721 EBox::Module::Service::__ANON__ - Error restarting service: root command /usr/share/zentyal/change-hostname ~ failed.
Error output: sed: -e expression #1, char 11: unknown option to `s'
 sed: -e expression #1, char 11: unknown option to `s'
 hostname: the specified hostname is invalid
and finally the output from change-hostname:

Code: [Select]

#!/bin/bash

NEW=$1

if [ -z $NEW ]
then
    echo "Usage: $0 <new_hostname>"
    exit 1
fi

CURRENT=`hostname`

sed -i "s/$CURRENT/$NEW/" /etc/hostname
sed -i "s/$CURRENT/$NEW/g" /etc/hosts
hostname $NEW
In short, it seems that the last steps from the post-upgrade script got interrupted, and now the zentyal admin interface is not starting.
I'm close to jumping of a bridge at the moment, all was working fine before but it got borked. Please help!

Cheers.

[c4rdinal]

Hi,

Try to recreate this file (/etc/apache/ssl/apache.pem) and ensure proper permission and file ownership. Your error shows:

mv: cannot stat '/etc/apache/ssl/apache.pem:: No such file or directory

Also, try to restart apache service and try again.

HTH

[Escorpiom]

Thanks c4rdinal,
I've recreated the apache.pem and also made a copy named "ssl.pem". It should have been solved, normal Apache webserver starts fine because I can access our webpage and even the zoneminder page that is running on the same server.
Everything seems to be working except the Zentyal admin interface.

This is what happens: Zentyal Apache won't start because on startup a "root command" named change-hostname needs to be executed. This root command needs the "hostname" parameter like this:

Code: [Select]

change-hostname myhostI tested the change-hostname command in this format giving it a valid hostname and it works fine, so the conclusion is that when starting Zentyal Apache, the invalid hostname parameter "~" is passed to the change-hostname command and of course it errors out, ~ is not a valid hostname.

Code: [Select]

2011/11/16 21:12:34 ERROR> Service.pm:721 EBox::Module::Service::__ANON__ - Error restarting service: root command /usr/share/zentyal/change-hostname ~ failed.
Error output: sed: -e expression #1, char 13: unknown option to `s'
 sed: -e expression #1, char 13: unknown option to `s'
 hostname: the specified hostname is invalid

Command output: .
Exit value: 1
I've since turned my Zentyal box inside out to find where this hostname parameter is being specified, I guessed it could be in some configuration file, and actually there is a file "Apache.bak" that contains the string "hostname='~'.
Changed it to my actual hostname but it had little effect, as it is a .bak file it's probably an old file not in use on Zentyal 2.2. I could find no other strings containing "hostname='~'".

If anyone can help out please try, it's my production server and I'm locked out. Can access the server using Putty and WinSCP only. I'm even willing to pay for support, reinstall is not an option atm.

Cheers.   

[half_life]

There is a directory /var/lib/ebox?  Maybe renamed to /var/lib/zentyal.   Under it is a file apache2.conf.  This file contains the info for the admin web page.  Start there.

[c4rdinal]

Post the content of the file /var/lib/zentyal/conf/apache2.conf here so we can see. Have you checked if apache is running? Can you access it from a remote browser? Can you access the default apache page?

what's the result of
hostname -f

[Escorpiom]

Thanks for helping out, I really appreciate it.
Found the requested info:
I have both "/var/lib/ebox" and "/var/lib/zentyal" folders. From the logs (after migration) I understand that "/var/lib/ebox" was to be deleted, but it said "not empty" so it wasn't deleted.
The apache2.conf is present in both "/var/lib/ebox/conf/" and in /var/lib/zentyal/conf/". The files are identical, same size.
I can see that this apache2.conf file is still pointing to the the old ebox dirs, while it should point to the var/lib/zentyal dir. But I'm hesitant to edit the file.
The file has not been modified after migration, it was last modified before I migrated the system.
 

Code: [Select]

Timeout 300
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15
AddDefaultCharset utf-8

PidFile /var/lib/ebox/tmp//apache.pid

<IfModule mpm_prefork_module>
    StartServers          1
    MinSpareServers       1
    MaxSpareServers       1
    MaxClients            1
    MaxRequestsPerChild   200
</IfModule>

# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_worker_module>
    StartServers         1
    MaxClients          1
    MinSpareThreads      1
    MaxSpareThreads      1
    ThreadsPerChild      1
    MaxRequestsPerChild   200
</IfModule>

PerlInterpMaxRequests 200

Include /etc/apache2/mods-available/auth_basic.load
Include /etc/apache2/mods-available/authn_file.load
Include /etc/apache2/mods-available/authz_default.load
Include /etc/apache2/mods-available/authz_groupfile.load
Include /etc/apache2/mods-available/authz_host.load
Include /etc/apache2/mods-available/authz_user.load
Include /etc/apache2/mods-available/autoindex.load
Include /etc/apache2/mods-available/cgi.load
Include /etc/apache2/mods-available/deflate.conf
Include /etc/apache2/mods-available/deflate.load
Include /etc/apache2/mods-available/dir.conf
Include /etc/apache2/mods-available/dir.load
Include /etc/apache2/mods-available/env.load
Include /etc/apache2/mods-available/mime.load
Include /etc/apache2/mods-available/negotiation.load
Include /etc/apache2/mods-available/setenvif.load
Include /etc/apache2/mods-available/rewrite.load
Include /etc/apache2/mods-available/ssl.conf
Include /etc/apache2/mods-available/ssl.load
Include /etc/apache2/mods-available/status.load
Include /etc/apache2/mods-available/perl.load

Listen 443
User ebox
Group ebox

ServerAdmin webmaster@localhost
ServerName localhost

DocumentRoot /usr/share/ebox/www/

<Directory />
    Options SymLinksIfOwnerMatch
    AllowOverride None
</Directory>


<Directory /usr/share/ebox/www/>
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

<Directory /var/lib/ebox/dynamicwww>
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>



UseCanonicalName Off
TypesConfig /etc/mime.types
DefaultType text/plain

<IfModule mod_mime_magic.c>
    MIMEMagicFile /usr/share/misc/file/magic.mime
</IfModule>

HostnameLookups Off

ErrorLog /var/log/ebox/error.log
LogLevel warn

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{forensic-id}n\"" combined

CustomLog /var/log/ebox/access.log combined

<IfModule mod_backtrace.c>
 EnableExceptionHook On
</IfModule>

<IfModule mod_whatkilledus.c>
 EnableExceptionHook On
</IfModule>

ServerSignature Off
ServerTokens Min

<IfModule mod_ssl.c>
SSLEngine on
SSLProtocol all
SSLCipherSuite HIGH:MEDIUM

SSLCertificateFile /var/lib/ebox/conf/ssl/ssl.pem
</IfModule>

<IfModule mod_setenvif.c>
    BrowserMatch "Mozilla/2" nokeepalive
    BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
    BrowserMatch "RealPlayer 4\.0" force-response-1.0
    BrowserMatch "Java/1\.0" force-response-1.0
    BrowserMatch "JDK/1\.0" force-response-1.0
</IfModule>

PerlWarn On

# PerlRequire "/var/lib/ebox/conf/startup.pl"

PerlModule EBox::Auth
PerlSetVar EBoxPath /
PerlSetVar EBoxLoginScript /ebox/Login/Index
PerlSetVar EBoxSatisfy Any
PerlSetVar AuthCookieDebug 0

<Files LOGIN>
AuthType EBox::Auth
AuthName EBox
SetHandler perl-script
PerlHandler EBox::Auth->login
</Files>

<Directory /usr/share/ebox/cgi/>
        <IfModule mod_ssl.c>
   SSLOptions +StdEnvVars
        </IfModule>

AuthType EBox::Auth
AuthName EBox
PerlAuthenHandler EBox::Auth->authenticate
PerlAuthzHandler  EBox::Auth->authorize
require valid-user
    SetHandler perl-script
    PerlHandler ModPerl::Registry
    PerlSendHeader On
    AllowOverride None
    Options +ExecCGI
    Order allow,deny
    Allow from all
</Directory>

RewriteEngine On

# From /ebox to /ebox/ and redirect
RewriteRule ^/ebox$ /ebox/
# From / to /ebox and redirect
RewriteRule ^/$ /ebox/
# Map /ebox/ebox.cgi to the right Perl CGI and redirect
RewriteRule ^/ebox/ebox.cgi$ /ebox/
# From /data/ to / and finish
RewriteRule ^/data(.*) $1 [L]
# From /dynamic-data/ to the right directory in FS and finish
RewriteRule ^/dynamic-data(.*) /var/lib/ebox/dynamicwww$1 [L]
RewriteRule ^/ebox/(.*) /usr/share/ebox/cgi/ebox.cgi [E=script:$1,L]

SSLCACertificatePath /var/lib/ebox/conf/ssl-ca/
The service apache2 is running, as I can access our webpage (port 80).
But Zentyal Apache does NOT run:

Code: [Select]

service zentyal apache start
 * Restarting Zentyal module: apache                                     [fail]
Should I edit the apache2.conf by hand to update all paths?

The migration console window showed this detail when setting up zentyal webserver:

Code: [Select]

Setting up zentyal-webserver (2.2.2) ...
Installing new version of config file /etc/init/ebox.apache2-user.conf ...
Listening SSL port is empty.I looked up the config file "/etc/init/ebox.apache2-user.conf":

Code: [Select]

pre-start script
    invoke-rc.d apache2 stop || true
    test -d /var/run/apache2 || mkdir /var/run/apache2
end script
exec /usr/share/zentyal-webserver/apache2-start
respawnAs you can see it points to "/usr/share/zentyal-webserver/apache2-start", so I looked up that folder, and it seems there is something missing.
The only file in that folder is called "enable-module" 192 Bytes long. No apache2-start file!

The output of hostname -f is

Code: [Select]

root@anubis:~# hostname -f
anubis.redcomputronics.com
Cheers.

[c4rdinal]

Backup your apache2.conf and post the contents of the default Zentyal 2.2 apache2.conf as attached here.

Code: [Select]


Timeout 300
KeepAlive Off
MaxKeepAliveRequests 500
KeepAliveTimeout 15
AddDefaultCharset utf-8

PidFile /var/lib/zentyal/tmp//apache.pid

<IfModule mpm_prefork_module>
    StartServers             1
    MinSpareServers          1
    MaxSpareServers          1
    MaxClients               1
    MaxRequestsPerChild  10000
</IfModule>

# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_worker_module>
    StartServers             1
    MinSpareThreads          1
    MaxSpareThreads          1
    ThreadsPerChild          1
    MaxClients               1
    MaxRequestsPerChild  10000
</IfModule>

PerlInterpMaxRequests 10000

Include /etc/apache2/mods-available/auth_basic.load
Include /etc/apache2/mods-available/authn_file.load
Include /etc/apache2/mods-available/authz_default.load
Include /etc/apache2/mods-available/authz_groupfile.load
Include /etc/apache2/mods-available/authz_host.load
Include /etc/apache2/mods-available/authz_user.load
Include /etc/apache2/mods-available/autoindex.load
Include /etc/apache2/mods-available/cgi.load
Include /etc/apache2/mods-available/deflate.conf
Include /etc/apache2/mods-available/deflate.load
Include /etc/apache2/mods-available/dir.conf
Include /etc/apache2/mods-available/dir.load
Include /etc/apache2/mods-available/env.load
Include /etc/apache2/mods-available/mime.load
Include /etc/apache2/mods-available/negotiation.load
Include /etc/apache2/mods-available/setenvif.load
Include /etc/apache2/mods-available/rewrite.load
Include /etc/apache2/mods-available/ssl.conf
Include /etc/apache2/mods-available/ssl.load
Include /etc/apache2/mods-available/status.load
Include /etc/apache2/mods-available/perl.load


Listen 443
User ebox
Group ebox

ServerAdmin webmaster@localhost
ServerName localhost

DocumentRoot /usr/share/zentyal/www/

<Directory />
    Options SymLinksIfOwnerMatch
    AllowOverride None
</Directory>

<Directory /usr/share/zentyal/www/>
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

<Directory /var/lib/zentyal/dynamicwww>
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>



UseCanonicalName Off
TypesConfig /etc/mime.types
DefaultType text/plain


Listen 443
User ebox
Group ebox

ServerAdmin webmaster@localhost
ServerName localhost

DocumentRoot /usr/share/zentyal/www/

<Directory />
    Options SymLinksIfOwnerMatch
    AllowOverride None
</Directory>

<Directory /usr/share/zentyal/www/>
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

<Directory /var/lib/zentyal/dynamicwww>
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>



UseCanonicalName Off
TypesConfig /etc/mime.types
DefaultType text/plain

<IfModule mod_mime_magic.c>
    MIMEMagicFile /usr/share/misc/file/magic.mime
</IfModule>

HostnameLookups Off

ErrorLog /var/log/zentyal/error.log
LogLevel warn

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{for$

CustomLog /var/log/zentyal/access.log combined

<IfModule mod_backtrace.c>
    EnableExceptionHook On
</IfModule>

<IfModule mod_whatkilledus.c>
    EnableExceptionHook On
</IfModule>

ServerSignature Off
ServerTokens Min

<IfModule mod_ssl.c>
    SSLEngine on
    SSLProtocol all
    SSLCipherSuite HIGH:MEDIUM
    SSLCertificateFile /var/lib/zentyal/conf/ssl/ssl.pem
    SSLCACertificatePath /var/lib/zentyal/conf/ssl-ca/
</IfModule>

PerlWarn On

PerlModule EBox::Auth
PerlSetVar EBoxPath /
PerlSetVar EBoxLoginScript /Login/Index
PerlSetVar EBoxSatisfy Any
# FIXME: disabled to avoid error after upgrade (apache restart)
#PerlSetVar EBoxCookieName Zentyal
PerlSetVar AuthCookieDebug 0

<Files LOGIN>
        AuthType EBox::Auth
        AuthName EBox
        SetHandler perl-script
        PerlHandler EBox::Auth->login
</Files>

<Directory /usr/share/zentyal/cgi/>
    <IfModule mod_ssl.c>
        SSLOptions +StdEnvVars
    </IfModule>

    AuthType EBox::Auth
    AuthName EBox
    PerlAuthenHandler EBox::Auth->authenticate
    PerlAuthzHandler  EBox::Auth->authorize
    require valid-user
    SetHandler perl-script
    PerlHandler ModPerl::Registry
    PerlSendHeader On
    AllowOverride None
    Options +ExecCGI
    Order allow,deny
    Allow from all
</Directory>

RewriteEngine On

# Compatibility with old URLs (permanent redirects)
RewriteRule ^/ebox(.*) /$1
RewriteRule ^/zentyal(.*) /$1
# skip rewrites for favicon and login
RewriteCond %{REQUEST_FILENAME} ^/favicon.ico$ [OR]
RewriteCond %{REQUEST_FILENAME} ^/LOGIN$ [OR]
RewriteCond %{REQUEST_FILENAME} ^/slave$
RewriteRule .? - [S=100]
# Map /ebox.cgi to the right Perl CGI and redirect
RewriteRule ^/ebox.cgi$ /
# From /data/ to / and finish
RewriteRule ^/data(.*) $1 [L]
# From /dynamic-data/ to the right directory in FS and finish
RewriteRule ^/dynamic-data(.*) /var/lib/zentyal/dynamicwww$1 [L]
RewriteRule ^/(.*) /usr/share/zentyal/cgi/ebox.cgi [E=script:$1,L]


Restart apache and see if this fixed it.

[Escorpiom]

Sorry to report it did not get fixed.
I found the problem, during migration some characters in config files were replaced by "~" causing this problem.
After a reboot, Squid would not start anymore complaining about "bungled config at line 57 (timeHours)" with the same strange character ~
I have now spend two days on this, got no sleep and have to reinstall for the second time in half a year.

Sorry guys, Zentyal is not mature, to many problems.

Cheers.

[c4rdinal]

I'm sorry for your hard time. Anyway, if you find time, please post here what you did or how the Zentyal team be able to reproduce the error so they can fix them. This way you can help the community by reporting any possible bugs you've found.

I'm also planning to upgrade my Zentyal 2.0 to 2.2 next week but I'm having a second thought...

I hope your problem can be fixed by then.

Good luck.

[Escorpiom]

OK, a little follow up on this. I am now posting this from my fresh installed Zentyal 2.2. The 2.0 was beyond recovery.
I spend about 3 hours getting it al up again and still have to create te user accounts and the lockdown stuff. Oh well.

Please forgive me for sounding negative. The Zentyal box is rather important for what we do here and I just can't have downtime.
In the end it became clear that it would be faster to reinstall than wait for someone to provide the "golden hint".
I'm tired and in a bad mood (because I couldn't solve it) but at least I can browse again.
 
What I did to get me in this situation? Well, the las reinstall was in March of this year. I've put Zoneminder on it, Opera browser and some other tiny stuff.
It was working fine and there were no problems with the 2.0 config. After running the migration tool, I lost the Zentyal web interface. After a reboot, I lost the proxy.
The logs provided one detail: Invalid characters "~" as described in my previous post. The migration tool must have borked some config files.
 
The good:
New Zentyal 2.2 is a good deal faster, both on boot and the admin interface. I only peeked at it for a while but I saw a lot of new goodies.
I had a backup from the users home dirs and var/www (forget about Ubuntu Live cd, no permissions) so no data lost, only a looooot of time.

Cheers.

[robb]

Good to hear your box is up and running again. Too bad you ran into all this trouble. For what it's worth: What I heard at Zentyal summit in Zaragoza last weekend, intentions are to provide upgrade from 2.2 to 3.0 next year by standard upgrade commands like upgrading from 1 version of ubuntu to a next. Let's hope the Zentyal devs can make this happen.

[Tupteq]

I have the same problem as Escorpiom.
Unfortunately I didn't make a server backup before upgrade and it end in the identical state - all zentyal components work, except of Web Admin Interface. I tried to analyze scripts, but I'm not an expert in both Linux and Perl, so it's a bit hard for me.
I also tried dpkg-reconfigure, starting/stopping daemons, copying config files - nothing helped.
I'd renistall Zentyal on other machine, but I'm not sure how to move existing configuration to the new server (I have some email accounts, firewall rules, web server and not much more).
Is there an easy way to repair Zentyal, or to reinstall on-place, or to backup and then restore all user/config data (all without use of Web Interface)?

[Josep]

If you are accessing the administration interface from another computer, you may be blocked by the firewall.
For these situations, I access the server via SSH and run a local text web browser Lynx.
This way, even if I manage to block myself with the firewall I have a handy way to run the interface and get me back in.