how to backup 350Gb

[michele]

I'm trying to backup my Zentyal (v. 2.2.7) file server using eBackup module. I'm using FTP method to another Zentyal File Server on my lan. I need to backup 350 Gb of files and folders, but after one day backup is not complete and still working.
This is first full backup.

My settings are:
Full Backup Frequency: monthly
Incremental Backup Frequency: daily
Keep previous full copies: limit by number (1)

What do you suggest to speed up backup process?
What's the best backup configuration for 350 Gb?

Thanks

[robb]

What connection do you have between the 2 servers? Is that Gb or 100Mb? (no-brainer that for such amounts of data Gb is highly recommended)

Now doing the math: 350GB over 24 hours
That translates to (350 x 8 x 1024) / (24 x 3600) Mb/s = 33Mb/s (= +/- 4 MB/s)
To me it seems that throughput could be better.

Can you give some more details on the adapters of those servers too?

[christian]

Can you give some more details on the adapters of those servers too?

And also some details about storage one each side: how many spindles? RAID?
This said, even if throughput is twice what you currently have, it will still require quite a lot of time if you don't have fast file system and network

[michele]

Unlikely, the connection between these 2 Zentyal servers is 100Mb. I know this can be a limit, but not big as it is (backup started two days ago at 21:00 and until now it's not finished!!!)

Some hardware details about Zentyal file server:
Model IBM X3400 M3 with one Intel Xeon Quad-Core 2.4 GHz 64bit, RAM 4 GB DDR3, 6 HD IBM SAS 10K on RAID5

and some hardware details about Zentyal FTP server:
Model Dell Poweredge SC420 with one Intel Celeron 2.5 GHz 32bit, RAM 512 MB DDR2 ECC, 2 HD Dell SATAII 7200rpm RAID0

Now my question are:
- will Zentyal backup finish his work giving me after 2 days the first full backup?
- what can i do to stop backup in progress? (i really don't want to reboot file server!)
- is it better to use an external HD to make backup faster?
- in case Zentyal backup one day will finish first full backup, i think incremental backup will run faster but it's better to use the option "only first time" for full backup. Is it right?

[christian]

Interesting points & questions raised in your post 

1 - given your configuration, throughput is limited by either network (100Mb/s is not that fast) or because of target (FTP) storage. RAID0 will be slightly slower than single disk that might not be able to sustain much more than 40Mb/s
2 - DAS might be faster... USB attached drive too if using USB3 but here also, it depends on HDD speed which might not be faster than your SATA II 7200rpm disk.

The very interesting point here is about "incremental" backup concept.
One have to distinguish between incremental and differential backups.

Incremental would mean that from one full backup, next backup will save only objects modified since previous backup, then next incremental will save object modified since... previous backup, which was incremental. Which means that in order to restore, you will have to restore last full back then restore ALL incremental backups.

Differential backup will save all objects modified since last full backup. in case of restore, you restore last full and last differential.

both have pros and cons but none prevents to perform on very regular basis full backup.

BTW I don't remember what Zentyal offers 

[michele]

Thanks Christian, but i think your reply was trunked...
Could you please post again your entire reply?
In addiction, do you think SCP or RSYNC method instead of FTp will run faster?

[christian]

Thanks Christian, but i think your reply was trunked...
Could you please post again your entire reply?

Done, sorry I typed the wrong key 

[christian]

In addiction, do you think SCP or RSYNC method instead of FTp will run faster?

I don't know  but running some performance test should not be that difficult 
I'm using RSYNC and don't face any performance issue (although my target system is not that fast) but I do not backup this large amount of data and I also exclude some useless folers (like /var/log or /tmp)

[michele]

Interesting points & questions raised in your post 

Incremental would mean that from one full backup, next backup will save only objects modified since previous backup, then next incremental will save object modified since... previous backup, which was incremental. Which means that in order to restore, you will have to restore last full back then restore ALL incremental backups.

Differential backup will save all objects modified since last full backup. in case of restore, you restore last full and last differential.

both have pros and cons but none prevents to perform on very regular basis full backup.

BTW I don't remember what Zentyal offers 

Yes, please, could someone explain better how Zentyal Backup module works?
For example, in a configuration like mine with:
- one monthly full backup
- daily incremental backup
- number of full backup to keep: 1
how Zentyal backup module is working?

[christian]

This will keep one monthly full backup plus 29 or 30 daily incremental backups.
Quite simple.

In case you have to restore from scratch, for what I understand, you will have to perform one full restoration plus 29 or 30 restoration for all daily backups.

[ichat]

running a fullback and  30 incremental backups is not very nice,  and if only mothly fullbackups,  i would recoment using diferenttial...

or  you do  weekly diferentiall and daily incremental... 

in the last case  you store a lot less backup data,   than if you do a full weekly backup,  but still dont have to apply 31 backups...   but instead  a maxium number of    9... which is still a lot...

other than that, i would advice,   usb3 or esata,   specially  with  esata  you can  add cheap and still verry quick backup storage to you system... 

[michele]

running a fullback and  30 incremental backups is not very nice,  and if only mothly fullbackups,  i would recoment using diferenttial...

Sorry, but where's differential option? In Zentyal i can configure full backup, incremental backup and copies of full backup to mantain. What about differential? What combination is?
Having 350 Gb to backup i think i need to find the faster way to execute backup. That's why i set monthly full backup (because only 12 times a year will backup the entire 350 Gb). I think incremental backup will be much faster. What do you suggest?

Yes, with eSate or USB3 backup will be faster but i need to mantain an external hard disk connected all the time in the same plece where file server is. With network backup i can mantain backup in a differente location.

[christian]

There is no differential backup with Zentyal.
The only option is incremental.

Your point about aSATA or USB 3 vs. remote server is correct but you can't have both advantages. If you want fast back-up over network, you have to ensure that source and remote file systems and network are fast.

100Mb/s is definitely not very fast and only one spindle on target system is not very fast too but this should not be so bad. You could expect about 50 to 60Mb/s (best case)
Are both source and target on same LAN ?

[michele]

Yes Christian, they are both in the same LAN.
I'm still having problem trying to make my first full backup. I tried with SCP but i got same error wrote here http://forum.zentyal.org/index.php/topic,12762.msg52751.html#msg52751
I'd like to try with RSYNC but i never used it before and i don't know how to configure both Zentyal File Server and Zentyal FTP server. I know you're currently using this method, could you please help me?
Thanks.

[christian]

In order to configure RSYNC, you obviously need target server on which you have enable RSYNC. Mine is OpenMediaVault server. You can do it on your FTP server but this will need some extra option (I mean to say that FTP is not used here).

You will find a lot of rsyncd related tutorial on internet. This is quite simple 

Zentyal side, you just need to define target as "your-target-server::target directory" with valid (target server side) account and password.