[RESOLVED] How to recover zarafa after a server crash?

[acon]

I recently surfered a server (disk) crash. I was running zentyal 2.2.
I have now installed Zentyal 3 and recovered all users data from backups. Mail server is now running, but without old mail.
I have backed up from crashed disk the /var/lib/mysql and /var/lib/zarafa folders. All files are fine except for /var/lib/mysql/ibdata1 wich has a damaged cluster (4096 bytes). I have recovered the remaining data in file with ddrescue, replacing damaged cluster with zeroes.
I have installed a virtual machine with zentyal 2.2 and mail an zarafa configured, and with same users as the creashed server.
I plan to replace the mysql and zarafa folders with those rescued from crashed disk, to access and copy old mail.
I have now replaced these folders and mysql start without any error, but zarafa dos not start and i get:

Code: [Select]

Unable to connect to database: Access denied for user 'zarafa'@'localhost' (using password: YES)In /var/log/zarafa/server.log.

Any ideas? Yes i know it a good question for zarafa forums....

[half_life]

Mysql has crashed. I have bad news,  the data was stored in innodb.  Innodb is much harder to corrupt but also much harder to fix if it does break. You might try here http://www.percona.com/software/mysql-innodb-data-recovery-tools if you are desperate.  Nightly backups and/or mysql clustering are your best defense here.  I am heading towards clustering myself to avoid large losses of data when mysql crashes.  It is weird,  I have worked with mysql for years and have never experienced a non recoverable crash until the last six months. For me it started after the zentyal 2.0 to 2.2 upgrade back in April.

[acon]

Hi Half_life. I am realy a lucky gui today 
I have found where the password for zarafa db is storred in zebtyal config and i can connect to the zarafa virtual machine.
I im realy surpised because i have recovered 100% of my data, even after the ibdata1 file corruption.
I have migrated the data to the new server using an outlook client to store them in PST and then to new server.
So im up again, and now i know how to backup-recover zarafa database.

[robb]

Any chance you can provide a step-by-step (n00b-style) howto on your recover adventure?

[half_life]

Hi Half_life. I am realy a lucky gui today 
I have found where the password for zarafa db is storred in zebtyal config and i can connect to the zarafa virtual machine.
I im realy surpised because i have recovered 100% of my data, even after the ibdata1 file corruption.
I have migrated the data to the new server using an outlook client to store them in PST and then to new server.
So im up again, and now i know how to backup-recover zarafa database.

Good to hear.  Xtrabackup allows hot backups and so far (very preliminary) testing looks good.  Disk I/O is the bottleneck for me but this http://www.mysqlperformanceblog.com/2012/06/20/percona-xtradb-cluster-reference-architecture-with-haproxy/ might be the solution that I am looking for (not tested). I started a thread here http://forum.zentyal.org/index.php/topic,13071.msg54032.html#msg54032 talking about my issues.

Sorry for the red herring in my last post but I assumed that you had the password but mysqld was not responding (like it would if a table was corrupted). 

[acon]

Hi Robb, the process is just as explained in my post.
The failed disk has servo problems. So with lots of retryes i managed to recover the full /var and /usr, except for the ibdata1 file, wich had a bad cluster next to begining of file. To read the remaining file, i used (from the same folder)

Code: [Select]

ddrescue ibdata1 /home/temp/ibdata1.bckThis copyed good cluster anf fill unreadable cluster with 0.
I installed a virtual machine with a fresh zentyal 2.2 (same version as failed server), configured mail, zarafa and users same way as failed server.
Then i stopped zentyal and mysql and replaced /var/lib/mysql and /var/lib/zarafa folders with the recovered ones.
At this point, there was an error in zarafa logs about bad credentials to acces zarafa mysql database. I fixed this by editing a file (cant remember name).
Zarafa whas then started and mail is available in webaccess.
I configured outook connector to copy all data (mail, contacts, calendar, notes...) to a local PST.
Then i reconfigured outlook connector to connect to my new server and copied all data stored in local PST to zarafa.
All this whas possible because i only need to recover 2 maiboxes. Remaining users has local mail or is not important.
I do sql dumps to backup zarafa, but for mail, daily backup is not enought.

Half-Life, after finding the damaged sector in ibdata1, y said to users that recovery is not possible, but continued with testings.
It whas an absolute surprise to find that i can start mysql and access mail.
The only explanation i can find is: there was one more database in old server and maybe the damaged part of file is for this database (im not an sql expert)...
Anyway the second database whas recovered from backup, so im up again.

[half_life]

What I was getting at was that mysql will stop processing a transaction when it finds a corrupted record.  I would think that all zeros would qualify but hey it worked so thumbs up.