Recover a file with a ' in the path name

[quentin.jodhaa]

Hi guys,

I have an up-to-date Zentyal server used mainly for sharing files.

I came across a problem yesterday when I tried to recover a folder named like this : /home/myuser/samba/shares/myshare/subfolder/workin'/mywork

I need to recover the "mywork" folder, but I get an error about a non authorized character, and I guess it is the ' in workin'

Questions :

1) Is there a quick workaround ?
2) Is it a bug ?
3) Would someone know how to recover that folder in CLI with duplicity ? (where I guess I can use workin\' or "myfullpath")

Many thanks for your help, and keep up the good work !

QJ

[J. A. Calvo]

Yes, it is a bug that should be already fixed in Zentyal 2.1, and we will probably release a backport for 2.0. In the meanwhile, you can try to patch the code manually.

Here's the changeset:

http://trac.zentyal.org/changeset/21563

As you can see, you only need to add a line to /usr/share/perl5/EBox/EBackup/Model/RemoteFileList.pm

Let us know if it solves your issue!

[quentin.jodhaa]

Hi,

Thanks for your quick reply.

I've added the line as mentionned, I rebooted and checked that the change was still here, it was.

When I tried to recover my folder, I get a first paragraphe, with a grey background, saying :

root command /usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 1305576008 --file-to-restore '/home/samba/shares/myshare/subfolder/workin'/mywork' file:///BACKUP --no-encryption '/home/samba/shares/myshare/subfolder/workin'/mywork' failed. Error output: Command line error: Expected 2 args, got 0 Enter 'duplicity --help' for help screen. Command output: . Exit value: 2

I have to mention that I have some spaces in the path as well. The full path with the first trailing slash is 90 characters long, including 5 spaces and 1 '

Then below this quote I have this :

To show technical details click here. [nothing happens when I click]

Trace
root command /usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 1305576008 --file-to-restore 'home/samba/shares/myshare/subfolder/workin'/mywork' file:///BACKUP --no-encryption '/home/samba/shares/myshare/subfolder/workin'/mywork' failed.
Error output: Command line error: Expected 2 args, got 0
Enter 'duplicity --help' for help screen.

Command output: .
Exit value: 2 at /usr/share/perl5/Error.pm line 182
Error::throw('EBox::Exceptions::Sudo::Command', 'cmd', '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...', 'output', 'ARRAY(0xb8c4aea8)', 'error', 'ARRAY(0xb858f9c0)', 'exitValue', 2, ...) called at /usr/share/perl5/EBox/Sudo.pm line 212
EBox::Sudo::_rootError('/usr/bin/sudo -p sudo: /var/lib/ebox/tmp/Q0KzsGVP4c.cmd 2> /v...', '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...', 512, 'ARRAY(0xb8c4aea8)', 'ARRAY(0xb858f9c0)') called at /usr/share/perl5/EBox/Sudo.pm line 181
EBox::Sudo::_root(1, '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...') called at /usr/share/perl5/EBox/Sudo.pm line 136
EBox::Sudo::root('/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...') called at /usr/share/perl5/EBox/EBackup.pm line 201
EBox::EBackup::restoreFile('EBox::EBackup=HASH(0xb96844f8)', '/home/samba/shares/myshare/subfolder/workin\'...', 'Mon May 16 22:00:08 2011') called at /usr/share/perl5/EBox/EBackup/Model/RemoteFileList.pm line 221
EBox::EBackup::Model::RemoteFileList::setTypedRow('EBox::EBackup::Model::RemoteFileList=HASH(0xba4ab518)', 21315, 'HASH(0xbd5c41b0)', 'force', undef, 'readOnly', undef) called at /usr/share/perl5/EBox/Model/DataTable.pm line 1306
EBox::Model::DataTable::setRow('EBox::EBackup::Model::RemoteFileList=HASH(0xba4ab518)', undef, 'date', 'Mon May 16 22:00:08 2011', 'filter', 'mywork', 'file', '/home/samba/shares/myshare/subfolder/workin\'...', 'id', ...) called at /usr/share/perl5/EBox/CGI/Controller/DataTable.pm line 121
EBox::CGI::Controller::DataTable::editField('EBox::CGI::Controller::DataTable=HASH(0xbb2f2748)') called at /usr/share/perl5/EBox/CGI/Controller/DataTable.pm line 226
EBox::CGI::Controller::DataTable::_process('EBox::CGI::Controller::DataTable=HASH(0xbb2f2748)') called at /usr/share/perl5/EBox/CGI/ClientRawBase.pm line 174
EBox::CGI::ClientRawBase::run('EBox::CGI::Controller::DataTable=HASH(0xbb2f2748)') called at /usr/share/perl5/EBox/CGI/Run.pm line 120
EBox::CGI::Run::run('EBox::CGI::Run', 'EBackup/Controller/RemoteFileList', 'EBox') called at /usr/share/ebox/cgi/ebox.cgi line 35
ModPerl::ROOT::ModPerl::Registry::usr_share_ebox_cgi_ebox_2ecgi::handler('Apache2::RequestRec=SCALAR(0xbae56098)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
eval {...} called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
ModPerl::RegistryCooker::run('ModPerl::Registry=HASH(0xbbbc0998)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 170
ModPerl::RegistryCooker::default_handler('ModPerl::Registry=HASH(0xbbbc0998)') called at /usr/lib/perl5/ModPerl/Registry.pm line 31
ModPerl::Registry::handler('ModPerl::Registry', 'Apache2::RequestRec=SCALAR(0xbae56098)') called at -e line 0
eval {...} called at -e line 0

How to report this problem

    Download the log file with additional information by clicking here.
    If the problem has ocurred right after installing or upgrading Zentyal, please download this file.
    Create a new ticket in the Zentyal trac by clicking here.
    Write a short description of the problem in the summary field.
    Write a detailed report of what you were doing before this problem ocurred in the description field.
    Do not forget to attach the downloaded file in the ticket.

I've tried to replace the real values with precision.

Many thanks for your help.

Bests,

QJ

[Josep]

So, now Zentyal is not checking for "unsafe" chars, which means that when the values are passed on to the actual command that will process them the final line that will be executed is messed up.
Duplicity would try to restore to

Code: [Select]

'/home/samba/shares/myshare/subfolder/workin'/mywork'The right thing to do when passing arguments to duplicity would be to avoid this kind of collisions and escape any single quote with \'.
The previous line should be:

Code: [Select]

'/home/samba/shares/myshare/subfolder/workin\'/mywork'
My guess is that the culprit may be in http://trac.zentyal.org/browser/tags/2.0-series/ebackup-2.0.13/src/EBox/EBackup.pm, lines 182 and 187, (sub restoreFile), although I may be wrong, of course:

Code: [Select]

182         $rFile = shell_quote($rFile);
183     }
184
185     # shell quote does not work well for espaces with duplicity...
186     $destination =~ s:\ :\\\ :g;
187     $destination = shell_quote($destination);
$rFile and $destination should be escaped.

[quentin.jodhaa]

Hi Josep,

Thanks for helping.

I've read your reply, but I didn't see what I had to change. Did I miss something ?

QJ

[Josep]

No, you didn't miss a thing. I just pointed where the problem may be. But I am not an expert in Perl and I don't feel comfortable enough to suggest anyone to apply changes on a production server, sorry. I was expecting some Zentyal programmer to jump in and propose some quick test.

[Josep]

All right, I have created a short script as a proof of concept:

Code: [Select]

#!/usr/bin/perl
$a = q(abc'def"gh\'i);

$a =~ s/\\/\\\\/g;
$a =~ s/\'/\\\'/g;

print $a."\n";

With this script, the input string abc'def"gh\'i becomes abc\'def"gh\\\'i.
I assume that Duplicity would accept an escaped string lilke that.

How can you apply it? By editing the file /usr/share/perl5/EBox/EBackup.pm. On lines 182-187, where you have

Code: [Select]

    $rFile = shell_quote($rFile);
}

# shell quote does not work well for espaces with duplicity...
$destination =~ s:\ :\\\ :g;
$destination = shell_quote($destination);

Replace with

Code: [Select]

    $rFile =~ s/\\/\\\\/g;
    $rFile =~ s/\'/\\\'/g;
    $rFile = shell_quote($rFile);
}

$destination =~ s/\\/\\\\/g;
$destination =~ s/\'/\\\'/g;
# shell quote does not work well for espaces with duplicity...
$destination =~ s:\ :\\\ :g;
$destination = shell_quote($destination);

I have not tested the results so make no guarantees, use it at your own risk!
Having said that, I would make those changes and try to restore the file.
If it works, post it here and it is up to you to decide whether you keep the changes or not.
If it doesn't work, post it here with whatever error messages you get and undo the changes in the file.

Good luck.

[quentin.jodhaa]

Thanks Josep,

I'll try it later today when no one is working on the server, and I'll keep you posted for sure.

Thanks for your time and your expertise.

QJ

[quentin.jodhaa]

Hi,

I just tried, here are the results :

To be more accurate, the full path will now be : /home/samba/shares/MARKETING/Products Index/AA0062 Workin'Hard/EXP 0.4/Technical Sheet/

First paragraph :

root command /usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 1305576008 --file-to-restore 'home/samba/shares/MARKETING/Products\\ Index/AA0062\\ Workin\'\''Hard\\ EXP\\ 0.4/Technical\\ Sheet' file:///BACKUP --no-encryption '/home/samba/shares/MARKETING/Products\ Index/AA0062\ Workin\'\''Hard\ EXP\ 0.4/Technical\ Sheet' failed. Error output: Command line error: Expected 2 args, got 7 Enter 'duplicity --help' for help screen. Command output: . Exit value: 2

Then :

root command /usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 1305576008 --file-to-restore 'home/samba/shares/MARKETING/Products\\ Index/AA0062\\ Workin\'\''Hard\\ EXP\\ 0.4/Technical\\ Sheet' file:///BACKUP --no-encryption '/home/samba/shares/MARKETING/Products\ Index/AA0062\ Workin\'\''Hard\ EXP\ 0.4/Technical\ Sheet' failed.
Error output: Command line error: Expected 2 args, got 7
Enter 'duplicity --help' for help screen.

Command output: .
Exit value: 2 at /usr/share/perl5/Error.pm line 182
Error::throw('EBox::Exceptions::Sudo::Command', 'cmd', '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...', 'output', 'ARRAY(0xb9e1dea0)', 'error', 'ARRAY(0xb9762968)', 'exitValue', 2, ...) called at /usr/share/perl5/EBox/Sudo.pm line 212
EBox::Sudo::_rootError('/usr/bin/sudo -p sudo: /var/lib/ebox/tmp/NbBMevQg3N.cmd 2> /v...', '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...', 512, 'ARRAY(0xb9e1dea0)', 'ARRAY(0xb9762968)') called at /usr/share/perl5/EBox/Sudo.pm line 181
EBox::Sudo::_root(1, '/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...') called at /usr/share/perl5/EBox/Sudo.pm line 136
EBox::Sudo::root('/usr/share//ebox-ebackup/ebox-duplicity-wrapper --force -t 13...') called at /usr/share/perl5/EBox/EBackup.pm line 206
EBox::EBackup::restoreFile('EBox::EBackup=HASH(0xba94da58)', '/home/samba/shares/MARKETING/Products Index/AA0062 Workin\'...', 'Mon May 16 22:00:08 2011') called at /usr/share/perl5/EBox/EBackup/Model/RemoteFileList.pm line 221
EBox::EBackup::Model::RemoteFileList::setTypedRow('EBox::EBackup::Model::RemoteFileList=HASH(0xbb67dcf8)', 21313, 'HASH(0xbcdf8408)', 'force', undef, 'readOnly', undef) called at /usr/share/perl5/EBox/Model/DataTable.pm line 1306
EBox::Model::DataTable::setRow('EBox::EBackup::Model::RemoteFileList=HASH(0xbb67dcf8)', undef, 'date', 'Mon May 16 22:00:08 2011', 'filter', 'PR0062', 'file', '/home/samba/shares/MARKETING/Products Index/AA0062 Workin\'...', 'id', ...) called at /usr/share/perl5/EBox/CGI/Controller/DataTable.pm line 121
EBox::CGI::Controller::DataTable::editField('EBox::CGI::Controller::DataTable=HASH(0xb9fdfc80)') called at /usr/share/perl5/EBox/CGI/Controller/DataTable.pm line 226
EBox::CGI::Controller::DataTable::_process('EBox::CGI::Controller::DataTable=HASH(0xb9fdfc80)') called at /usr/share/perl5/EBox/CGI/ClientRawBase.pm line 174
EBox::CGI::ClientRawBase::run('EBox::CGI::Controller::DataTable=HASH(0xb9fdfc80)') called at /usr/share/perl5/EBox/CGI/Run.pm line 120
EBox::CGI::Run::run('EBox::CGI::Run', 'EBackup/Controller/RemoteFileList', 'EBox') called at /usr/share/ebox/cgi/ebox.cgi line 35
ModPerl::ROOT::ModPerl::Registry::usr_share_ebox_cgi_ebox_2ecgi::handler('Apache2::RequestRec=SCALAR(0xbc051650)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
eval {...} called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 204
ModPerl::RegistryCooker::run('ModPerl::Registry=HASH(0xbc051e90)') called at /usr/lib/perl5/ModPerl/RegistryCooker.pm line 170
ModPerl::RegistryCooker::default_handler('ModPerl::Registry=HASH(0xbc051e90)') called at /usr/lib/perl5/ModPerl/Registry.pm line 31
ModPerl::Registry::handler('ModPerl::Registry', 'Apache2::RequestRec=SCALAR(0xbc051650)') called at -e line 0
eval {...} called at -e line 0

To bu sure, here is the extract from /usr/share/perl5/EBox/EBackup.pm after I've added your modifications :

my $rFile;
    if ($file eq '/') {
        $rFile = undef;
    } else {
        $rFile = $file;
        $rFile =~ s:^/::; # file must be relative

        # shell quote does not work well for espaces with duplicity...
        $rFile =~ s:\ :\\\ :g;
        $rFile =~ s/\\/\\\\/g;
        $rFile =~ s/\'/\\\'/g;
        $rFile = shell_quote($rFile);
    }

        $destination =~ s/\\/\\\\/g;
        $destination =~ s/\'/\\\'/g;

    # shell quote does not work well for espaces with duplicity...
    $destination =~ s:\ :\\\ :g;
    $destination = shell_quote($destination);

What do you think about it ?

[Josep]

How come there is a single quote unescaped in 'home/samba/shares/MARKETING/Products\\ Index/AA0062\\ Workin\'\''Hard\\ EXP\\ 0.4/Technical\\ Sheet'?

You have Workin\'\''Hard in the result. That (third) unescaped quote confuses Duplicity. What was the original name?

[Javier Amor Garcia]

Hello,
 I am trying too Joseph's solution and it works with more file than previously.

By the way, quentin, do you don't had the complete patch; in your coe you have lsot the space escape for $file. This is why it complains of "6 arguments instead 2".

However, the problem with the thrird quote is real. This is the command which is executed after the substitions:

Code: [Select]

root@z21:/etc/alternatives# /usr/share//zentyal-ebackup/duplicity-wrapper --force -t 1307348273 --file-to-restore 'etc/alternatives/MARKETING/Products\ Index/AA0062\ Workin\'\''Hard/EXP\ 0.4/Technical\ Sheet/ea'  file:///ymp/ea --no-encryption '/etc/alternatives/MARKETING/Products\ Index/AA0062\ Workin\'\''Hard/EXP\ 0.4/Technical\ Sheet/ea'
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Jun  6 10:17:53 2011
Error '[Errno 2] No such file or directory: "/etc/alternatives/MARKETING/Products Index/AA0062 Workin'Hard/EXP 0.4/Technical Sheet/ea"' processing .

[Javier Amor Garcia]

Ok, I have debugged a bit more and the extra quotes are put by shell_quote. I think it gets confused with a so complex string. I will mail to the author explaining the problem and I will commit Joseph's changes becasue they fix a lot of path strings.

Thanks Joseph!/

[quentin.jodhaa]

Josep :

How come there is a single quote unescaped in 'home/samba/shares/MARKETING/Products\\ Index/AA0062\\ Workin\'\''Hard\\ EXP\\ 0.4/Technical\\ Sheet'?

Huh, I don't know ! I matched this path to the real one. Did you see anything wrong in my config file due to a typo from me ?

Javier :

By the way, quentin, do you don't had the complete patch; in your coe you have lsot the space escape for $file. This is why it complains of "6 arguments instead 2".

I'm sorry but I don't understand, 'coe', 'lsot' ? Can you explain it differently please ?

Thank you for debugging and submitting, how can I know when an official update is available ?

In the meantime, I will do the Josep changes again and make sure everything is fine.

I'll let you know if I see any positive change.

Bests,

QJ

[Javier Amor Garcia]

Sorry, I mean that in your code you apply the substitution just to the $rFile variable, you should apply also to the $destination variable.

Josep's patch is committed to the repository, likely the next version of ebox-ebackup will have it but I cannot give you any date.

[quentin.jodhaa]

Wow, this is really good news, many thanks to you guys, and bravo for the reactivity !