Author Topic: Where did External mail recieve go?  (Read 3586 times)

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Where did External mail recieve go?
« on: July 16, 2014, 03:33:12 pm »
Hey Guys!

I've just noticed that the external mail recieve is missing from the Mail module in Zentyal 3.5! Where did it go? I've read a few words that sais fetchmaili is still in and working, but how and where?

Cheers!

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #1 on: July 16, 2014, 05:20:30 pm »
fetchmail is still installed but the interface to add the external emails has gone.

Create a file /etc/fetchmailrc

Code: [Select]
set daemon 180
set syslog
set postmaster postmaster@zent1.office.zentyal.lan


poll my.bumbox.com proto POP3
   user "quarkybeddingspot" with password "theansweris42" is test1@zentyal.lan here  ;

Just add a list of external accounts with the full local email address and that worked for me with 3.5 and standard mail settings.

Haven't tried it with openchange but because it sits on top I presume its the same.

There are loads of examples for fetchmail

http://www.zarafa.com/wiki/index.php/Fetchmail_to_Zarafa
http://www.howtoforge.com/debian_etch_fetchmail

I would prob use the zarafa example but use the above fetchmailrc. Create fetchmail.sh and cronjob.

create a cronjob and thats it just add each email address like the one above.

Manual Reference Pages  - fetchmail (1)
http://www.fetchmail.info/fetchmail-man.html

Frequently Asked Questions About Fetchmail
http://www.fetchmail.info/fetchmail-FAQ.html

Fetchmail-users -- general discussion on fetchmail, its use, and support
https://lists.sourceforge.net/lists/listinfo/fetchmail-users
« Last Edit: July 21, 2014, 12:46:36 pm by StuartNaylor »

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #2 on: July 18, 2014, 11:01:47 am »
Jesus, god damn it. Why did they remove it? I'm using Zentyal because it is user friendly and I can learn ubuntu server and setting slowly while still operating it as a "noob". Are they gonna bring it back somewhen? BTW: I have to use OpenChange, so it will be 2x more problematic for me. Thanks for the link, I'm observing and studying them.

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #3 on: July 18, 2014, 11:12:34 am »
Question: Can I ( or is it adviceable to ) set up the fetchmail with IMAP instead of POP3 because I wish to keep the mails on the external accounts server as well. I don't want to download it and delete.

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #4 on: July 18, 2014, 11:42:34 am »
I am going to by shares in webmin. I bang on about it enough :)

If you have webmin and Zentyal then for a noob much is possible.

I guess I am a noob as I use them all the time and have learnt loads.

Read http://www.zarafa.com/wiki/index.php/Fetchmail_to_Zarafa

They didn't remove fetchmail just the script and datastore (which might of been openldap)

Its very easy to create some cron jobs in webmin.

Its very easy to download zentyal 3.2 and have a look what was going on.

imap or pop3 you can have yes.

If you need any help either post or email on the provision you write up a community document if satisfactory.

I just wish generally on each module Zentyal had blank custom fields that would allow many fixes and hackz.

As an idea each user could have a file in there home folder and these are scanned to create the account list?

Or a sysadmin just writes them out in one fretchmailrc
« Last Edit: July 18, 2014, 11:46:19 am by StuartNaylor »

Torsten73

  • Zen Warrior
  • ***
  • Posts: 174
  • Karma: +6/-1
    • View Profile
Re: Where did External mail recieve go?
« Reply #5 on: July 18, 2014, 03:38:23 pm »
Webmin has been remove from ubuntu due to security reasons. It can break your system.

I am not sure if it is a good idea to modify without the WebIF of zentyal. These changes can affect other parts of zentyal. Or better i say, every modification without zentyal can brake it espessially for a noob.

the fetchmailrc has some lines i dont understand, do i need to modify "set postmaster ..."?
can you give us your modified files as full example?

Anyway i can´t believe that we need to do such quirks to get basic functionality back.

 
--------------------------------------------------------------
Zentyal 3.5 (offline) unter Ubuntu12.04.3 YAVDR 0.5 als KVM Host
Action Pack Abo with a running Exc. 2013 :-)

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #6 on: July 18, 2014, 04:11:26 pm »
I am not going to go into it but Torsten the tooth fairy is as real as santa claus.

I dunno where these urban myths come from especially about webmin.

webmin like the command line do many things that the zentyal interface can not.

Change the network settings in zentyal and you can end up with a broken system

Don't as if you think about it you could argue using a windows desktop you could break the system, are we going to say don't use it or be scared?

I spoke to James Cameron about it who is the main developer and he just hasn't bothered submitting it.

He is a good guy and has helped me with a few things.

He is happy with http://www.webmin.com/deb.html

Code: [Select]
apt-get install perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl apt-show-versions python
wget http://prdownloads.sourceforge.net/webadmin/webmin_1.690_all.deb
dpkg --install webmin_1.690_all.deb

create a network service webmin port 10000
allow just like the zentyal webadmin in the firewall

Its a lot easier for the noob for many tasks than the cli.

https://library.linode.com/email/fetchmail there are loads of good examples for fetchmail.

Some people seem to be stuck and in desperate need of fetchmail functions so I posted some info on how to do it.

I can't help that the fetchmail part of zentyal is missing until we get an update.

I am sure Zentyal will add it, but until then.
The easiest way to do your cronjobs is through webmin which zentyal just doesn't have. Its a bit like a general gui for linux admin.
It also has a file manager which is great as ftp isn't part of zentyal anymore either.
So its also good for creating the fetchmailrc file and setting its permissions.

I suggest for a noob to install webmin and ask the community if unsure.

Just my tip, if you have an alternative method please say.

set postmaster: Give the name of the last-resort mail recipient (default: user running fetchmail, "postmaster" if run by the root user). In case there is no delivery route

Also at the end of each mail poll you can add...
You will notice that the poll lines have different endings (e.g. nofetchall (default), fetchall, keep, nokeep). The meanings are as follows:

nofetchall: Retrieve only new messages (default). If nothing else is specified (e.g. fetchall, keep), this means nofetchall.
fetchall: Fetch all messages whether seen or not.
keep: Don't delete seen messages from server.
nokeep: Delete seen messages from server.

imap, pop3 or whatever protocol the account allows


Manual Reference Pages  - fetchmail (1)
http://www.fetchmail.info/fetchmail-man.html

Frequently Asked Questions About Fetchmail
http://www.fetchmail.info/fetchmail-FAQ.html

Fetchmail-users -- general discussion on fetchmail, its use, and support
https://lists.sourceforge.net/lists/listinfo/fetchmail-users
« Last Edit: July 21, 2014, 12:46:54 pm by StuartNaylor »

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #7 on: July 21, 2014, 09:01:46 am »
You gave me a very good and detailed example of fetchmail, thank you for it. However I have an issue. I'm using OpenChange ( can't use Zarafa, must use OpenChange ). I've made a simple set up for one account but I don't know where to set the time how often should it download new messages. That would be needed. Google-in it right now. :) I think it will work cause I've recieved ONE email to my original mail address and THAT has appeared in the newly created one. But that was on the weekend. I haven't recieved anything that was sent today. Damn, why did Zentyal remove this? Along with the old virtualization. That was very useful and awsome. I'm affraid that Zentyal is beginning to look like a very plain server system with purchaseable modules. Am I alone with this feeling?

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #8 on: July 21, 2014, 09:55:14 am »
Just use a cron job.

/etc/fetchmailrc has the line set daemon 180 which is the poll frequency

The cron job will make sure fetchmail is running

I think you have got Zentyal at a feature low point because it has made some fundemental base changes.

Install webmin as honest its a perfect complement as one can not do what the other can. Together they are just superb.

Be carefull with the server settings but also its great for seeing what server settings are in zentyal.

Set up a cron job and you choose how often you download.


Code: [Select]
apt-get install -y perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl apt-show-versions python
wget http://prdownloads.sourceforge.net/webadmin/webmin_1.690_all.deb
dpkg --install webmin_1.690_all.deb

In zentyal>network>services create a webmin service and add port 10000
Allow on internal firewall section.

Makes making cronjobs much easier.
webmin>system>scheduled cron jobs
also
webmin>other>file manager makes things mush easier.

Have a go with those two honest they will not bite.
Apols Torsten but the don't run with a knife in your hand, don't stick your fingers in a plug socket crowd wind me up.
Apols if I sounded severe.

Quote
Next create a file called: /etc/cron.domain/fetchmail.sh and add the following to that file: (This file will be called via crontab to initiate the fetchmail collection every 5 minutes or what ever you specify.)
/etc/cron.domain/fetchmail.sh

   #!/bin/bash
   #
   # Add this to a cron process that runs once every 15 mins
   # just to make sure fetchmail is up and running
   #
   fetchmailpid=`/sbin/pidof fetchmail`
   #
   # If fetchmail is not running then start it
   #
   if [ $? -ne 0 ]; then
     /usr/bin/fetchmail -F -K -L /var/log/fetchmail
   fi
Please note: The '-K' option above will delete retrieved messages from the remote mailserver. If you want to keep the emails on the external POP3 server, then i would suggest removing the '-K'.
Parameter Breakdown from above:
   -K - Delete retrieved messages from the remote mailserver. This option forces retrieved mail to be deleted
   -L - Specify the log file to which fetchmail should log to
   -F - Delete old (previously retrieved) messages from the mailserver before retrieving new messages
Set the permissions on the /etc/cron.domain/fetchmail.sh to 777:
   chmod 777 /etc/cron.domain/fetchmail
Now we setup the cron to collect the mail every 5 minutes:
   */5 * * * * /etc/cron.domain/fetchmail.sh
If there any errors check the /var/log/fetchmail.log for more information.

I always forget the cron format so just use webmin years of gui's I guess


Manual Reference Pages  - fetchmail (1)
http://www.fetchmail.info/fetchmail-man.html

Frequently Asked Questions About Fetchmail
http://www.fetchmail.info/fetchmail-FAQ.html

Fetchmail-users -- general discussion on fetchmail, its use, and support
https://lists.sourceforge.net/lists/listinfo/fetchmail-users
« Last Edit: July 21, 2014, 12:47:22 pm by StuartNaylor »

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #9 on: July 21, 2014, 11:27:29 am »
Sounds very easy to do it with webmin. I'll give it a go, but ATM I'm very close to make fetchmail working as it is. There are 2 things I need to solve still. Oh and this is only a mail server. Nobody is going to log in directly, only the sysadmin.
1; Use IMAP and not POP3. I think this is why it grabs down the mails now correctly but deleting it as well from the original server
2; I want to enable this for multiple users

+1; I need to download the already opened messages as well once. Basicly copy the whole profile with subfolders and all messages inside it. Any tipp?

Here is my fetchmailrc:
Code: [Select]
set syslog;
set daemon 10;
set postmaster "postmaster@***.hu";
set logfile /home/rendszergazda/fetchmail.log;

defaults
   proto imap
   keep

poll i**e.be***an@***.hu
   via mail.t****e.hu
   user "i**e.be***an@***.hu"
   pass "********";
smtphost "z****l.****.hu smtpname "i**e.be****an@***.hu";

I know it is messed up, but I've found soooooooo many variations. I don't know wich one to use or wich is better, etc....
« Last Edit: July 21, 2014, 11:29:51 am by bertalanimre »

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #10 on: July 21, 2014, 11:31:44 am »
Its OK as these things can be a nightmare.

Logs is where it is all at. Just try and read you logs.

I had this format and found the FQ email address did the job on my server.

/etc/fetchmailrc

Code: [Select]
set daemon 180
set syslog
set postmaster postmaster@zent1.office.zentyal.lan


poll my.bumbox.com proto POP3
   user "quarkybeddingspot" with password "theansweris42" is test1@zentyal.lan here fetchall, nokeep

poll mail.there.com proto IMAP
   user "marvintheparanoidandroid" with password "zaphod" is test2@zentyal.lan here nofetchall, keep

  ;

Each account can have its own protocol. set daemon 180 is the global repeat poll time.
Your cron job just sets it going and keeps it going.

Haven't tried but think I got the two lines right.
Fetchall, nokeep grabs everything and cleans the source
nofetchall, keep just grabs new.

So both should stop duplication with the 2nd keeping a backup store.

I don't know how clever fetchmail is and how nofetchall would work on pop3

Apols as never even looked at fetchmail until I read the forum posts. In production I am still running 3.2 so personally have never used it.
« Last Edit: July 21, 2014, 06:56:11 pm by StuartNaylor »

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #11 on: July 21, 2014, 12:32:31 pm »
Getting there. :)

It does fetch all email and keeps them on the source server as well. Magnificant, thank you. Now, ther is still one problem. :D It keeps fetching it non-stop and now in my test account I have 2 messages but each 20 times ( before I stopped fetchmail ). I don't know if there is any way of forcing the fetchmail to compare the mails and see if it already exists. If there is not, then I'll try to solfe it like this:

1stly on Friday I'll reset all users password so I can add them all to the fetchmailrc and start fetchmail so by Monday morning it will finnish with all users .( thinking about it, it is not a good idea because if it finnishes in 10 hours, then all users will have all their emails but 4 times.... ) damn, need to rethink again....

*Bert is brainstorming.........

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #12 on: July 21, 2014, 12:42:24 pm »
Dunno I am not sure if we just put the full email address that it just sends like a sendmail without any checks.

Easiest way would be to delete them nokeep

Below is the Zarafa example and a MTA is defined which might keep control the uid's of the emails.

/root/.fetchmailrc:
Code: [Select]
   set daemon 60
   defaults
           proto pop3
           nokeep
   poll user@domain.com.au
           via mail.domain.com.au
           user "user@domain.com.au"
           pass "password"
           mda "/usr/bin/zarafa-dagent user"
           flush
           no ssl
There are a number of other parameters you can use, find more here: Redhat Fetchmail Commands
Then you repeat the 'poll' section to 'no ssl' for each user. Obviously changing the details.
Next create a file called: /etc/cron.domain/fetchmail.sh and add the following to that file: (This file will be called via crontab to initiate the fetchmail collection every 5 minutes or what ever you specify.)

mda "/usr/libexec/dovecot/dovecot-lda -d test@domain.org.uk"
mda "/usr/lib/dovecot/deliver -d mail -m /srv/mail/user1/"


/etc/cron.domain/fetchmail.sh

Code: [Select]
   #!/bin/bash
   #
   # Add this to a cron process that runs once every 15 mins
   # just to make sure fetchmail is up and running
   #
   fetchmailpid=`/sbin/pidof fetchmail`
   #
   # If fetchmail is not running then start it
   #
   if [ $? -ne 0 ]; then
     /usr/bin/fetchmail -F -K -L /var/log/fetchmail
   fi
Please note: The '-K' option above will delete retrieved messages from the remote mailserver. If you want to keep the emails on the external POP3 server, then i would suggest removing the '-K'.
Parameter Breakdown from above if you want global fetch options:
   -K - Delete retrieved messages from the remote mailserver. This option forces retrieved mail to be deleted
   -L - Specify the log file to which fetchmail should log to
   -F - Delete old (previously retrieved) messages from the mailserver before retrieving new messages

Manual Reference Pages  - fetchmail (1)
http://www.fetchmail.info/fetchmail-man.html

Frequently Asked Questions About Fetchmail
http://www.fetchmail.info/fetchmail-FAQ.html

Fetchmail-users -- general discussion on fetchmail, its use, and support
https://lists.sourceforge.net/lists/listinfo/fetchmail-users
« Last Edit: July 21, 2014, 01:41:24 pm by StuartNaylor »

bertalanimre

  • Zen Monk
  • **
  • Posts: 87
  • Karma: +3/-0
    • View Profile
Re: Where did External mail recieve go?
« Reply #13 on: July 21, 2014, 01:42:16 pm »
What if.....

I make a batchfile that runs fetchmail for all users ( line by line after each other ) and fetches all emails. If I'm not doing this with the daemon it should do this only once am I right?
IF that has been done and all mails have arrived, then I can start the daemon but to recieve only the new messages. Then it wouldn't spam the mailbox with duplicated mails ( since the old ones were downloaded once and it wasn't made with the daemon ) and would still download all newly recieved messages. Right?

StuartNaylor

  • Guest
Re: Where did External mail recieve go?
« Reply #14 on: July 21, 2014, 01:56:01 pm »
Can you just not delete the read mail? Why are you keeping it if its in there new mailbox?

I am not sure depends on the manner you end your poll line, "nofetchall, keep"

Or the run parameters of  fetchmail -F -K -L /var/log/fetchmail for global

Setting up a cron job can run as any user and any schedule.

Will leave you to try and report back.
« Last Edit: July 21, 2014, 02:01:55 pm by StuartNaylor »