[SOLVED] Suddenly unable to send pdf through mail (egroupware)

[auerhaan]

Out of the blue (maybe upgrade from 1.2 tot 1.4 ?) user can't mail pdf files anymore.

They get the following reply-mail from the ebox-server:

BANNED CONTENTS ALERT

Our content checker found
    banned name: multipart/mixed | application/pdf,.pdf,kosten
      infrastructuur.pdf

in email presumably from you <xxx@xxx.xx>
to the following recipient:
->xxx.xxx@xxx.xx

Our internal reference code for your message is 30657-05/stHFw58Cw4pA

According to a 'Received:' trace, the message originated at: [127.0.0.1],
  ubuntu01 localhost [127.0.0.1]

Return-Path: <xxx@xxx.xx>
Message-ID: <4c65d58d59c4a73b4400aab07e2f63d2@ubuntu01>
Subject: Facturen telefonie Auerhaan

Delivery of the email was stopped!

The message has been blocked because it contains a component
(as a MIME part or nested within) with declared name
or MIME type or contents type violating our access policy.

To transfer contents that may be considered risky or unwanted
by site policies, or simply too large for mailing, please consider
publishing your content on the web, and only sending an URL of the
document to the recipient.

Depending on the recipient and sender site policies, with a little
effort it might still be possible to send any contents (including
viruses) using one of the following methods:

- encrypted using pgp, gpg or other encryption methods;

- wrapped in a password-protected or scrambled container or archive
  (e.g.: zip -e, arj -g, arc g, rar -p, or other methods)

Note that if the contents is not intended to be secret, the
encryption key or password may be included in the same message
for recipient's convenience.

We are sorry for inconvenience if the contents was not malicious.

The purpose of these restrictions is to cut the most common propagation
methods used by viruses and other malware. These often exploit automatic
mechanisms and security holes in more popular mail readers (Microsoft
mail readers and browsers are a common target). By requiring an explicit
and decisive action from the recipient to decode mail, the danger of
automatic malware propagation is largely reduced.

In UTM -> Mail filter -> Files ACL I have added application/pdf and multipart/mixed as allowed.

Looks like this changed after upgrade to 1.4 from 1.2

Any ideas ?

Update: I have the same problem with incoming mail now    

[Javier Amor Garcia]

Could you post the complete email with al lthe headers, please?

[auerhaan]

Here is the complete original email-message. I only truncated the mime-lines to save (a lot) of space.

Date: Tue, 9 Feb 2010 12:06:10 +0100
Return-Path: xxx@xxxx.xx
To: xxx.xxx@xxxx.xx
From: Paul Wirl <xxx@xxxx.xx>
Subject: Facturen telefonie Auerhaan
Message-ID: <4641cd85868cb5c987e429d33dbc3d4e@ubuntu01>
X-Priority: 3
X-Mailer: PHPMailer (phpmailer.codeworxtech.com) [version 2.1]
X-Mailer: FeLaMiMail
Disposition-Notification-To: xxx@xxxx.xx
Organization: Auerhaan B.V.
MIME-Version: 1.0
Content-Type: multipart/mixed;
   boundary="b1_4641cd85868cb5c987e429d33dbc3d4e"


--b1_4641cd85868cb5c987e429d33dbc3d4e
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Geachte heer de Gerth,

Hierbij onze facturen wat betreft telefonie.
Ik stuur van de laatste 3 maanden de facturen met gesprekskosten mee omdat
december niet erg representatief is. In die maand zijn we namelijk altijd
bijna 2 weken dicht, waardoor de gesprekskosten een stuk lager liggen.

Met vriendelijke groet,


Paul Wirl
Automatisering
Auerhaan B.V.=20

--b1_4641cd85868cb5c987e429d33dbc3d4e
Content-Type: application/pdf; name="gesprekskosten dec 09.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="gesprekskosten dec 09.pdf"

JVBERi0xLjQKJRamipIKNCAwIG9iago8PC9UeXBlL1hPYmplY3QKL1N1YnR5cGUvSW1hZ2UKL1dp
...
OTQKJSVFT0YK


--b1_4641cd85868cb5c987e429d33dbc3d4e
Content-Type: application/pdf; name="gesprekskosten nov 09.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="gesprekskosten nov 09.pdf"

JVBERi0xLjQKJRamipIKNCAwIG9iago8PC9UeXBlL1hPYmplY3QKL1N1YnR5cGUvSW1hZ2UKL1dp
...
byAzIDAgUgovUm9vdCAyIDAgUgo+PgpzdGFydHhyZWYKMTA4OTU1MgolJUVPRgo=


--b1_4641cd85868cb5c987e429d33dbc3d4e
Content-Type: application/pdf; name="gesprekskosten okt 09.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="gesprekskosten okt 09.pdf"

JVBERi0xLjQKJRamipIKNCAwIG9iago8PC9UeXBlL1hPYmplY3QKL1N1YnR5cGUvSW1hZ2UKL1dp
...
c3RhcnR4cmVmCjEwMDcyNTAKJSVFT0YK


--b1_4641cd85868cb5c987e429d33dbc3d4e
Content-Type: application/pdf; name="kosten infrastructuur.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="kosten infrastructuur.pdf"

JVBERi0xLjQKJRamipIKNCAwIG9iago8PC9UeXBlL1hPYmplY3QKL1N1YnR5cGUvSW1hZ2UKL1dp
...
IDAgUgo+PgpzdGFydHhyZWYKMzA4MzI2CiUlRU9GCg==


--b1_4641cd85868cb5c987e429d33dbc3d4e--

The complete reply messages is:

Return-Path: <MAILER-DAEMON>
Delivered-To: xxx@xxxx.xx
Received: from localhost (localhost [127.0.0.1])
   by ubuntu01.localdomain (Postfix) with ESMTP id 4B82319E20E
   for <xxx@xxxx.xx>; Tue,  9 Feb 2010 12:06:11 +0100 (CET)
Content-Type: multipart/report; report-type=delivery-status;
 boundary="----------=_1265713571-30657-1"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Subject: BANNED message from you (multipart/mixed |
 application/pdf,.pdf,kosten infrastructuur.pdf)
In-Reply-To: <4c65d58d59c4a73b4400aab07e2f63d2@ubuntu01>
Message-ID: <VSstHFw58Cw4pA@ubuntu01.auerhaan.nl>
From: "Content-filter at ubuntu01.auerhaan.nl" <postmaster@ubuntu01.auerhaan.nl>
To: <xxx@xxxx.xx>
Date: Tue,  9 Feb 2010 12:06:10 +0100 (CET)

This is a multi-part message in MIME format...

------------=_1265713571-30657-1
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

BANNED CONTENTS ALERT

Our content checker found
    banned name: multipart/mixed | application/pdf,.pdf,kosten
      infrastructuur.pdf

in email presumably from you <xxx@xxxx.xx>
to the following recipient:
-> xxx.xxx@xxxx.xx

Our internal reference code for your message is 30657-05/stHFw58Cw4pA

According to a 'Received:' trace, the message originated at: [127.0.0.1],
  ubuntu01 localhost [127.0.0.1]

Return-Path: <xxx@xxxx.xx>
Message-ID: <4c65d58d59c4a73b4400aab07e2f63d2@ubuntu01>
Subject: Facturen telefonie Auerhaan

Delivery of the email was stopped!

The message has been blocked because it contains a component
(as a MIME part or nested within) with declared name
or MIME type or contents type violating our access policy.

To transfer contents that may be considered risky or unwanted
by site policies, or simply too large for mailing, please consider
publishing your content on the web, and only sending an URL of the
document to the recipient.

Depending on the recipient and sender site policies, with a little
effort it might still be possible to send any contents (including
viruses) using one of the following methods:

- encrypted using pgp, gpg or other encryption methods;

- wrapped in a password-protected or scrambled container or archive
  (e.g.: zip -e, arj -g, arc g, rar -p, or other methods)

Note that if the contents is not intended to be secret, the
encryption key or password may be included in the same message
for recipient's convenience.

We are sorry for inconvenience if the contents was not malicious.

The purpose of these restrictions is to cut the most common propagation
methods used by viruses and other malware. These often exploit automatic
mechanisms and security holes in more popular mail readers (Microsoft
mail readers and browsers are a common target). By requiring an explicit
and decisive action from the recipient to decode mail, the danger of
automatic malware propagation is largely reduced.


------------=_1265713571-30657-1
Content-Type: message/delivery-status; name="dsn_status"
Content-Disposition: inline; filename="dsn_status"
Content-Transfer-Encoding: 7bit
Content-Description: Delivery error report

Reporting-MTA: dns; ubuntu01.auerhaan.nl
Received-From-MTA: smtp; ubuntu01.localdomain ([127.0.0.1])
Arrival-Date: Tue,  9 Feb 2010 12:06:10 +0100 (CET)

Original-Recipient: rfc822;xxx.xxx@xxxx.xx
Final-Recipient: rfc822;xxx.xxx@xxxx.xx
Action: failed
Status: 5.7.0
Diagnostic-Code: smtp; 554-5.7.0 Reject, id=30657-05 - BANNED: multipart/mixed
 554 5.7.0 | application/pdf,.pdf,kosten infrastructuur.pdf
Last-Attempt-Date: Tue,  9 Feb 2010 12:06:10 +0100 (CET)
Final-Log-ID: 30657-05/stHFw58Cw4pA

------------=_1265713571-30657-1
Content-Type: text/rfc822-headers; name="header"
Content-Disposition: inline; filename="header"
Content-Transfer-Encoding: 7bit
Content-Description: Message headers

Return-Path: <xxx@xxxx.xx>
Received: from ubuntu01 (localhost [127.0.0.1])
   by ubuntu01.localdomain (Postfix) with ESMTP id 1DCF619E204
   for <xxx.xxx@xxxx.xx>; Tue,  9 Feb 2010 12:06:09 +0100 (CET)
Date: Tue, 9 Feb 2010 12:06:09 +0100
To: xxx.xxx@xxxx.xx
From: Paul Wirl <xxx@xxxx.xx>
Subject: Facturen telefonie Auerhaan
Message-ID: <4c65d58d59c4a73b4400aab07e2f63d2@ubuntu01>
X-Priority: 3
X-Mailer: PHPMailer (phpmailer.codeworxtech.com) [version 2.1]
X-Mailer: FeLaMiMail
Disposition-Notification-To: xxx@xxxx.xx
Organization: Auerhaan B.V.
MIME-Version: 1.0
Content-Type: multipart/mixed;
   boundary="b1_4c65d58d59c4a73b4400aab07e2f63d2"

------------=_1265713571-30657-1--

[auerhaan]

In UTM -> Mail Filter -> Files ACL the file extension pdf is allowed

[Javier Amor Garcia]

After seeing the headers is clear that is not eBox mailfilter the one banning the PDF extensions. (there are not amavis headers there).  So we can forget about the settings in TM -> Mail Filter -> Files ACL.

Is possible that you have another filtering software in your mail's way out?.
The "ubuntu01.auerhaan.nl" host is your eBox server or is another thing?. Any additional software running on it?.

[auerhaan]

I've installed ubuntu and ebox 1.2 with the cd-image downloaded from this site. Last friday I changed the repository to upgrade to 1.4.

Today I noticed that we weren't able to send pdf's anymore, so I checked the mailfilter config in ebox which seems oke.

ubuntu01.auerhaan.nl is indeed our ebox server.

I'm not aware that I installed other filtering software.

Yesterday I only installed winbind and pptp according to this post http://forum.ebox-platform.com/index.php?topic=844.0
It was needed to grant to a software supplier access to a server in our network with their software (ERP) on it.

Incoming mail has the same problem, so it is filtering both ways.

[auerhaan]

Temporarily disabled the mail filter in the webinterface and email doesn't get blocked anymore. So the problem is somewhere in the ebox mailfilter.

Already tried sudo /etc/init.d/ebox mailfilter restart and sudo /etc/init.d/ebox restart

[Javier Amor Garcia]

Mmmm.. Strange
In Mail->General, filter tab what setting do you have?

[auerhaan]

Mail -> General -> Mail filter options

Filter in use: none (was eBox internal mail filter)
Custom filter's mail forward port: 10025
Custom filter's IP address: 127.0.0.1
Custum filter's port: 10024

I set filter in use temporarily to none to let my users send pdf files to clients.

[auerhaan]

I found the text of the reply message in /etc/amavis/en_US/template-virus-sender.txt so it is definitely amavis that blocks the messages.

Could it be a problem with the configuration in /etc/amavis/conf.d/amavisd.conf ?

In /etc/amavis/conf.d.old/ the configuration is in debian-style like this:

-rw-r--r-- 1 root root  1640 2008-03-11 06:21 01-debian
-rw-r--r-- 1 root root   692 2008-03-11 06:21 05-domain_id
-rw-r--r-- 1 root root   428 2008-03-11 06:21 05-node_id
-rw-r--r-- 1 root root 13907 2008-03-11 06:21 15-av_scanners
-rw-r--r-- 1 root root   554 2008-03-11 06:21 15-content_filter_mode
-rw-r--r-- 1 root root  9248 2008-03-11 06:21 20-debian_defaults
-rw-r--r-- 1 root root   573 2008-03-11 06:21 25-amavis_helpers
-rw-r--r-- 1 root root  2130 2008-03-11 06:21 30-template_localization
-rw-r--r-- 1 root root   318 2008-03-11 06:21 50-user

[auerhaan]

This is in /var/log/mail.info when an email is blocked.

Feb 10 16:15:02 ubuntu01 postfix/cleanup[27888]: F293719E21B: message-id=<7f087a431002100714u4f7671fetcaf1f20adca5c4c2@mail.gmail.com>
Feb 10 16:15:03 ubuntu01 postfix/qmgr[27702]: F293719E21B: from=<sender@example.com>, size=425137, nrcpt=1 (queue active)
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) ESMTP::10024 /var/lib/amavis/amavis-20100209T213758-14517: <sender@example.com> -> <eboxuser@domain.tld> SIZE=425137 Received: from ubuntu01.localdomain ([127.0.0.1]) by localhost (ubuntu01.auerhaan.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP for <eboxuser@domain.tld>; Wed, 10 Feb 2010 16:15:03 +0100 (CET)
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) Checking: n0JViHcHPWh8 <sender@example.com> -> <eboxuser@domain.tld>
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) p.path BANNED:1 eboxuser@domain.tld: "P=p004,L=1,M=multipart/mixed | P=p003,L=1/2,M=application/pdf,T=pdf,N=kosten infrastructuur.pdf", matching_key="(?i-xsm:inf)"
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) spam_scan: not wasting time on SA, message longer than 153600 bytes: 1539+418124
Feb 10 16:15:03 ubuntu01 postfix/smtpd[27839]: connect from localhost[127.0.0.1]
Feb 10 16:15:03 ubuntu01 postfix/smtpd[27839]: AF4F919E210: client=localhost[127.0.0.1]
Feb 10 16:15:03 ubuntu01 postfix/cleanup[27832]: AF4F919E210: message-id=<VSn0JViHcHPWh8@ubuntu01.auerhaan.nl>
Feb 10 16:15:03 ubuntu01 postfix/smtpd[27839]: disconnect from localhost[127.0.0.1]
Feb 10 16:15:03 ubuntu01 postfix/qmgr[27702]: AF4F919E210: from=<>, size=5492, nrcpt=1 (queue active)
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) SEND via SMTP: <> -> <sender@example.com>,ENVID=AM..20100210T151503Z@ubuntu01.auerhaan.nl 250 2.6.0 Ok, id=14517-03, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as AF4F919E210
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) BANNED name/type (multipart/mixed | application/pdf,.pdf,kosten infrastructuur.pdf), <sender@example.com> -> <eboxuser@domain.tld>, quarantine n0JViHcHPWh8, Message-ID: <7f087a431002100714u4f7671fetcaf1f20adca5c4c2@mail.gmail.com>,
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) Hits: -
Feb 10 16:15:03 ubuntu01 amavis[14517]: (14517-03) Blocked BANNED (multipart/mixed | application/pdf,.pdf,kosten infrastructuur.pdf), <sender@example.com> -> <eboxuser@domain.tld>, Hits: -, tag=0, tag2=5, kill=5, L/Y/0/0
Feb 10 16:15:03 ubuntu01 postfix/smtp[27833]: F293719E21B: to=<eboxuser@domain.tld>, orig_to=<eboxuser_alias@domain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.9, delays=1.7/0/0/0.24, dsn=2.5.0, status=sent (250 2.5.0 Ok, id=14517-03, BOUNCE)
Feb 10 16:15:03 ubuntu01 postfix/qmgr[27702]: F293719E21B: removed
Feb 10 16:15:03 ubuntu01 postfix/smtp[27889]: AF4F919E210: to=<sender@example.com>, relay=mailrelay[xxx.xxx.xxx.xxx]:25, delay=0.11, delays=0.03/0/0.02/0.06, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as BCC97DE8886)
Feb 10 16:15:03 ubuntu01 postfix/qmgr[27702]: AF4F919E210: removed

[auerhaan]

YEAH found out what the problem is. I guess it is an ebox bug.

In /etc/amavis/conf.d/amavisd.conf file extensions are blocked the wrong way after the update to 1.4

$banned_filename_re = new_RE(
#  qr'^UNDECIPHERABLE$',  # is or contains any undecipherable components
   qr'\.[^.]*\.(exe|vbs|pif|scr|bat|cmd|com|dll)$'i, # some double extensions
   qr'[{}]',     # curly braces in names (serve as Class ID extensions - CLSID)
   qr'^message/partial$'i,  # rfc2046. this one is deadly for Outcrook
   qr'sys'i,
   qr'bat'i,
   qr'com'i,
   qr'pif'i,
   qr'inf'i,
   qr'wsh'i,
   qr'reg'i,
   qr'cpl'i,
   qr'scr'i,
   qr'dll'i,
   qr'chm'i,
   qr'vbs'i,
   qr'exe'i,
   qr'cmd'i,
);

Is what ebox generates as rules now. So every filename with one of these parts is blocked. So qr'com'i blocks program.com, but computer.doc too.

$banned_filename_re = new_RE(
#  qr'^UNDECIPHERABLE$',  # is or contains any undecipherable components
   qr'\.[^.]*\.(exe|vbs|pif|scr|bat|cmd|com|dll)$'i, # some double extensions
   qr'[{}]',     # curly braces in names (serve as Class ID extensions - CLSID)
   qr'^message/partial$'i,  # rfc2046. this one is deadly for Outcrook
   qr'\.sys$'i,
   qr'\.bat$'i,
   qr'\.com$'i,
   qr'\.pif$'i,
   qr'\.inf$'i,
   qr'\.wsh$'i,
   qr'\.reg$'i,
   qr'\.cpl$'i,
   qr'\.scr$'i,
   qr'\.dll$'i,
   qr'\.chm$'i,
   qr'\.vbs$'i,
   qr'\.exe$'i,
   qr'\.cmd$'i,
);

Is what ebox used to generate in 1.2

Is there a template where I can fix this ?

[Javier Amor Garcia]

Thanks a lot for you feedback. I was looking in the wrong places..

I will work on the fix now

[Javier Amor Garcia]

Ok, we have the fix uploaded to the repository.  It will be present in the next release of ebox-mailfilter.

If you dont want to wait you can patch your server with this instructions:

1. Grab the new FileExtensionsACL.pm file at http://trac.ebox-platform.com/export/16671/trunk/client/mailfilter/src/EBox/MailFilter/Model/FileExtensionACL.pm
2. In your Ebox use it to overwrite /usr/share/perl5/EBox/Mailfilter/Model/FileExtensionACL.pm
3. Restart the mailfilter system and the ebox webserver to commit the changes. You can use this two commands:
 /etc/init.d/ebox mailfilter restart
 /etc/init.d/ebox apache restart

Thanks again to auerhaan for helping eBox development with his bug hunt

[auerhaan]

Javier, thank you for the fast bug fix. I've successfully patched my system and amavisd.conf is generated correctly now.

I'm glad I could get you on the right track to fix this problem. At least I've learned a lot about amavis this way.

My compliments to the EBox team !!!

I will surely keep bug hunting to help make EBox better and better.