Admin interface unreachable - help request

[thorsten]

Hi,

 I kindly ask for any help on unusuall behavioir of my Zental 3 server, which was running perfectly. the last thing i did was to change the directory for backup files. When I quit the admin interface, the was a claIm for unsaved changes, but I am shure there were none so I ignored this and closed the browser.

After a while, some dhcp cloents were not able to get an IP from Zentyal, so I tried to log in, BUT the login screen told me that I was logged off due to A time out. ok, I did a reboot.

Now it is not possible to get the logon screen of the admin interface at all, all webservices seem to be down, but apache seems to run. All other services are not available to clients (dhcp, dns, samba, whatever) A further reboot did not helü. During reboot I was able to see that mysql service failed to star, but I can not restart, e.g. By service mysql start.

Can somebody help. please?

Thx
Thorsten

[half_life]

I have been where you are at on a 2.2 system.  Can you manually start dhcp?

[thorsten]

Hi,

how do I restart dhcp within zentyal? Usuall way /etc/init.d/dhcp restart does not work - there is no dhcp3 or any dhcp file within that directory. service dhcp restart/start/stop or service dhcp3 restart/start/stop do not work eiter.

THX
Thorsten

[half_life]

/etc/init.d/zentyal dhcp start

[half_life]

If you would like to see what is available for you to control within the /etc/init.d/zentyal script just give it a bad parameter such as:

/etc/init.d/zentyal  test status


If you just call /etc/init.d/zentyal  (start or stop)  it calls all of the startup or shutdown services that zentyal controls.

[thorsten]

Any command

/etc/init.d/zentyal dhcp start
/etc/inid.d/zentyal test status

results in:

Error in tempfile() using /var/lib/zentyal/tmp/XXXXXXXXXX: Could not create temp file /var/lib/zentyal/tmp/KwKLuDkhfP: No space left on device at /usr/share/perl5/EBox/Module/Base.pm line 999.

Just the file name behind /tmp/xxx changes....

Best regards
Thorsten

[half_life]

Sounds like your root partition is full. 

Run

Code: [Select]

df -h
and paste the results back to here.

[thorsten]

mmm... root itself looks fine, but var is full. But what is stored in there?

I do not use that - is there something I can whipe out? Also an increase by means of lvm would be possible...

filesystem                      Size  Used Avail Use% Mounted on
/dev/mapper/VG--01-LV--01_root   14G  6,1G  7,1G  47% /
udev                            3,9G  4,0K  3,9G   1% /dev
tmpfs                           1,6G  536K  1,6G   1% /run
none                             50M     0   50M   0% /run/lock
none                            3,9G     0  3,9G   0% /run/shm
cgroup                          3,9G     0  3,9G   0% /sys/fs/cgroup
/dev/mapper/VG--01-LV--03_tmp    23G  172M   22G   1% /tmp
/dev/mapper/VG--01-LV--04_srv    33G  413M   31G   2% /srv
/dev/mapper/VG--01-LV--02_var    33G   32G     0 100% /var
/dev/mapper/VG--02-LV--05_home  1,4T   80G  1,2T   7% /home

[half_life]

Your /var mountpoint  is full. You can move all of the old logs off /var/log to gain some room.  I am suspecting that you use Zarafa?

Anyways, first things first.  You will need to create some free space on /var to return to normal operations.  Once you have gotten the machine back on its feet,  you will need to root cause this event.  I am guessing it will relate to the mysql server given how much room you have allotted for /var.  Thus my question about Zarafa.

[thorsten]

Hi,

OK, the enemy is /var/cache/zentyal/duplicity with four directories and two of them with 9.9 GB ... is this what you suppose?

[half_life]

I suggest

Code: [Select]

mv /var/log/*.gz /home to move all of the old logs.

I am curious as to how big your mysql installation is so try

Code: [Select]

du -sh /var/lib/mysql
Unless you feel like re-installing Zentyal today, don't randomly delete things stored under the /var directory.  Adding space via lvm is an option but there may be safer ways to approach this.

[half_life]

That would be it!   Move those directories to /home and then see what is what.

[half_life]

Before I quit monitoring this thread for updates on the off chance you still need help,  have you gotten things straightened out?

[thorsten]

Sorry, I will have visitors today and can not work on the problem. /var/logs is quite smal, just some 100 MB compared to the LVM device size of 30 GB. I will move the files from /var/cache/zentyal/duplicity ..., is this any problem?

also:
/var/lib/mysql is just 1.2 GB.

THX in advance, will give you a PM in case I will need further help on this thread.

Thorsten

[half_life]

I see no problem moving the two directories under /var/cache/zentyal/duplicity .   They are used to keep track of changes made to portions of the system that are backed up.  Removing them will trigger a full backup to the best of my knowledge. It should re-create one directory when it runs.  Longer term,  it probably would be best to make a little more room on the var mountpoint to avoid this issue