Sendmail & Fetchmail for e-mail access.

[Binestar]

Having for years used sendmail and fetchmail to access all my e-mail needs from anywhere on the internet with ssh access, I had a bit of trouble getting it to work on Gentoo. So I figured I'd write up a little Tips & Tricks.

Lets first start with what this would be used for:

Lets say you want to make a Gentoo box that handles all your e-mail, so that all your outgoing mail gets sent to your ISP's mail server, and all your incoming mail can get handled by your own spam filters on procmail or via SpamAssassin, or even just be read by Mozilla.

So we need a few things to start.

#1: Fetchmail
#2: Sendmail
#3: Procmail
#4: E-mail Client (I use pine)

so:

Code: Select all

emerge sendmail
emerge m4
emerge procmail
emerge fetchmail

After those 4 things are installed we are ready to begin configuration.

First thing we want to do is to configure sendmail to accept e-mail only from the localmachine, as well as sending e-mail only through your ISP's mail server.

You need to edit the /etc/mail/sendmail.mc file to include the following:

Code: Select all

divert(-1)
#
# Copyright (c) 1998, 1999 Sendmail, Inc. and its suppliers.
#       All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#

divert(0)dnl
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
define(`confDEF_USER_ID',``8:12'')dnl
VERSIONID(`$Id: sendmail.mc,v 1.2 2002/07/04 04:55:29 g2boojum Exp $')dnl
OSTYPE(linux)dnl
DOMAIN(generic)dnl
define(`SMART_HOST',`mail.yourisp.net')dnl
MASQUERADE_AS(yourisp.net)dnl
FEATURE(`allmasquerade')dnl
FEATURE(`masquerade_envelope')dnl
define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnl
FEATURE(`smrsh',`/usr/sbin/smrsh')dnl
FEATURE(`local_lmtp',`/usr/sbin/mail.local')dnl
FEATURE(`local_procmail')dnl
define(`PROCMAIL_MAILER_FLAGS',`procmail -Y -m $h $g $u')dnl
define(`PROCMAIL_MAILER_ARGS',`procmail -Y -m $h $g $u')dnl
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl
MAILER(procmail)dnl
MAILER(local)dnl
MAILER(smtp)dnl
Cwlocalhost.localdomain

What the above does is configures sendmail to forward all outgoing mail through your ISP's mail server, and all incoming mail will be sent through the procmail mailer (if there is a .procmailrc file), or through local mail/internet mail depending on destination. It also makes sure that sendmail is *ONLY* listening on the local interface (I.E. no one from another machine on the internet can relay mail through your server.)

Now that you have a sendmail.mc file you need to use the program m4 to convert it to sendmail.cf format.

Code: Select all

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Once you do that Sendmail is ready to be run.

Code: Select all

/etc/init.d/sendmail start

Make sure that sendmail started properly, you can check the logs in /var/log/

The next thing you want to do is to make sure it only is listening on the local interface.

Code: Select all

telnet localhost smtp

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mhonline.net ESMTP Sendmail 8.12.6/8.12.6; Fri, 22 Nov 2002 11:44:56 -0500
ehlo mhonline.net
250-mhonline.net Hello localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-DELIVERBY
250 HELP
quit
221 2.0.0 mhonline.net closing connection
Connection closed by foreign host.

Next you need to make sure it is *NOT* listening on your ethernet interface or ppp interface:

Code: Select all

telnet 123.456.789.101 smtp
Trying 123.456.789.101...
telnet: Unable to connect to remote host: Connection refused

Obviously you would use your ethernet ip address for that test. Once you are sure that it starts properly, You can add it to your default runlevel:

Code: Select all

rc-update add sendmail default

Now that sendmail is installed, configured, secured, and running, you can move onto the configuration of fetchmail.

If you have just a few users, you can setup fetchmail with a user specific configuration file.

~/.fetchmailrc

Code: Select all

set postmaster "username"
set bouncemail
set properties ""
set daemon 300
poll mail.yourisp.net with proto POP3
       user "ispusername" there with password "isppassword" is username here options fetchall warnings 3600

What this does, is sends any error messages to the local user you set as postmaster (I usually set it as your username so you can get any errors delivered to you)
It also sets fetchmail to run in the background, checking your e-mail every 300 seconds (5 minutes)

The poll option tells fetchmail to poll your isp's mail server with the POP3 protocol (That can be changed to other protocol's that fetchmail supports)
Then it says that "ipsusername" has a password of "isppassword" and to deliver that username's mail to "username" on the local server.

The extra options say to fetch everything on the server and then delete the messages.

You can add more usernames to the fetchmailrc file if you wish, you would just add another user line to the ISP poll line, or if you have multiple POP/IMAP/etc accounts you can put another poll line in to do multiple servers.

Now you can run fetchmail from a shell.

Then you can access the mail however you prefer to. (Pine, Mutt, Mozilla Mail, etc)


Hope that helps someone,

Binestar
chrisf@mhonline.net

[Binestar]

I'd like to mention, that this configuration will only work if you run your mail client on the machine that you did this configuration on. If you want to use a mail client that is for example on a LAN behind the Gentoo box, you would need to also bind sendmail to the LAN IP address, and install either a POP3 server or an IMAP server.

Then you would have to add your LAN subnet to the ACCESS file for sendmail and rebuild the file.

If anyone wants help with that it's very simple and I can write up a howto for that as well.

Binestar
chrisf@mhonline.net

[jecepede]

I have installed this too but ehhrr.....

I have my own domain and I send to user1@mydomian.nl and user2@mydomain.nl but fetchmail only delivers to root@localhost and not to user1 and user2 ?

I added all the users in this way :

useradd -m user1
passwd user1

but fetchmail doest deliver to the different users


Greets

Jessy

[Binestar]

Fetchmail *CAN* be configured to deliver to different users.

For example: You want one global fetchmailrc file that can get 5 different e-mail addresses from 2 different ISP's and send it to 3 different users on your system.

An example fetchmailrc file would be:

Code: Select all

set postmaster "johndoe"
set bouncemail
set properties ""
set daemon 300

poll mail.isp1.net with proto POP3
       user "johndoe" there with password "password1" is johndoe here options fetchall warnings 3600
       user "janedoe" there with password "password2" is janedoe here options fetchall warnings 3600
       user "johnd" there with password "password3" is johndoe here
options fetchall warnings 3600

poll mail.isp2.com with proto POP3
       user "jasondoe" there with password "password4" is jasondoe here
options fetchall warnings 3600
       user "marydoe" there with password "password5" is jasondoe here
options fetchall warnings 3600

obviously users johndoe janedoe and jasondoe would need to be on the local system.


Binestar

[jecepede]

Thanx Binestar, that really did the trick

I now do something else too, I configured .forward - file to send the mail after scanning (with amavis) to another pop box..

However, can I make it so that I get an e-mail notification on my phone ? I need to make a new mail like "You have new mail from johndoe@domain.nl"
When I send with a random mail-thingy (like hotmail or yahoo) to myphonenumberr@gin.nl, the Gin-company forwards it to my phone....

Can that be made ????


Greets


Jessy

[Binestar]

PLEASE IGNORE THIS POST -- THIS IS A REALLY BAD METHOD TO DO WHAT YOU ASKED. PLEASE READ THE NEXT POST FOR A BETTER METHOD


This is a very convoluted method of getting what you want to work to work, but it works.

You need the mail program:

Code: Select all

emerge mailx

Then you need to create a new user who will e-mail you each time you receive an e-mail.

in this example I will use "notify" for a user:

Code: Select all

adduser notify

You need to then create a home directory for notify:

Code: Select all

mkdir /home/notify
chown notify.users /home/notify
chmod 755 /home/notify

Now you need to add a procmailrc file and a perl script to notify's home directory:

the file /home/notify/.procmailrc should have 0644 permissions and it should read:

Code: Select all

LOGFILE=/home/notify/procmail.log
MAILDIR=/home/notify/mail
FORMAIL=/usr/bin/formail

:0 fhw
| $FORMAIL -I "From " -a "From "

:0: fhw
| $FORMAIL -rtzxTo: | /home/notify/notify.pl

and the file /home/notify/notify.pl should have 0755 permissions and it should read:

Code: Select all

#!/usr/bin/perl
        $notify = <STDIN>;
        system("/bin/mail -s \"Email from $notify\" yourphonenumber\@gin.nl");

next you need to edit your /etc/mail/aliases file to include:

Code: Select all

notifyme:   yourusername, notify

then you need to run the newaliases command:

Code: Select all

newaliases

Then change your .fetchmailrc to deliver to "notifyme" instead of "yourusername" and sendmail will then deliver all messages that fetchmail sends to "notifyme" to "userusername" and "notify".

Code: Select all

poll mail.isp1.net with proto POP3
       user "johndoe" there with password "password1" is notifyme here options fetchall warnings 3600

All notify does is strips the From: Header out and sends it to your phone. The real e-mail goes to your regular username.


I know that is confusing, so let me know if you understand what I was explaining.

Thanks,

Binestar

[Binestar]

I did a little research last night and realized that you can do all this with a simple procmail recipe as your own user that calls a perl script.


No need to follow the directions above, but I'm not sure if I can delete it or edit it, plus it might give someone an idea on things... so I'll leave it.

This post has a much better method.

first you do still need mailx

Code: Select all

emerge mailx

You need to add a simple recipe to your *OWN* .procmailrc file:

Code: Select all

:0 c
| $FORMAIL -rtzxTo: | /home/username/notify.pl

and your notify.pl file should have permissions of 0755 and have the lines:

Code: Select all

#!/usr/bin/perl
        $notify = <STDIN>;
        system("/bin/mail -s \"Email from $notify\" yourphonenumber\@phonedomain.com");

Much easier than going through the process of adding new users and changing delivery.

the ":0 c" portion of the procmail recipe tells procmail to make a carbon copy of the e-mail and process it with the recipe, then to pass the original on to the next recipe. I just didn't think of looking at the man page yesterday when I was helping you. =)

Hope this helps more,

Binestar

[jecepede]

Maybe a silly question but ehhrr ahum ahum what is mailx ?

And I used sendmail in combination with fetchmail ?

And where should I type :

:0 c
| $FORMAIL -rtzxTo: | /home/username/notify.pl


And... I don't want a cc to my phone, just a new one-liner mail to my phone. Did you ever send a 80Kb gif file to a phone ? that is about 69 sms !!!!


Greets

Jessy

[Binestar]

mailx is the package that includes the program 'mail' which is a commandline e-mailler.

That is used by the perl script I gave you which is called by procmail.

For the exact places to put the information:

You need to create a file in your home directory called ".procmailrc"

That file should have the following:

Code: Select all

LOGFILE=/home/username/procmail.log
MAILDIR=/home/username/mail
FORMAIL=/usr/bin/formail 

:0 c
| $FORMAIL -rtzxTo: | /home/username/notify.pl 

Then you also need to create a file in your home directory called "notify.pl"

That file should have the following lines:

Code: Select all

#!/usr/bin/perl
        $notify = <STDIN>;
        system("/bin/mail -s \"Email from $notify\" yourphonenumber\@phonedomain.com");

That perl script just sends an e-mail to your phone with a subject of: "E-mail from whoever@thierisp.com"

It doesn't forward the whole message, it just gets the from address from the e-mail and forwards it to your phone.

Hope that helps,

Binestar

[jecepede]

Okidokie...

I made the two files, how can I start this now ? Or is the fact the I type "emerge mailx" enough ?
Should I not use a line like "rc-update add mailx default" or so ??


Greets

Jessy

[Binestar]

If you installed sendmail and procmail as above, you'll be all set. Mailx is *NOT* a service. It is just a commandline mail CLIENT. It still uses sendmail as the MTA.

So as long as those files are created, you should be all set. I would test it with a message to make sure you don't lose any mail, then set it up and see if it works to your liking.

Binestar

[jecepede]

Binestar, I am afraid I lost you completely now ?

Procmail and sendmail ? I have fetchmail and sendmail ? How can I run AND procmail AND sendmail ? They both use port 25 ??? I think it will not work like you think it will....

Or do you mean I have to replace my sendmail with procmail ????


Greets


Jessy

[Binestar]

OK, this has gotten completely on a tangent of the original Tips & Tricks I posted, but here goes.

procmail
mailx
sendmail

All different programs that serve different functions.

Please read the man pages for procmail, mail, and sendmail to see how they have different functions:

Basically:

procmail is an autonomous mail processor. Whats this mean? It means it filters and files mail according to recipe's. (One of which I had you put into your .procmailrc file in your home directory)

mailx is a package of programs that includes the mail program. mail is a program that can be used as a scripted e-mail client (Which is what I have done in the above examples for you)

sendmail is an MTA (Mail Transport Agent) That is the program that listen in daemon mode on port 25.

All 3 of these programs serve different functions, just as fetchmail is used to pull the e-mail from the server to your local machine.

Having procmail and sendmail in no way overlaps thier functionality, in fact, they are *VERY* complimentory to each other. Procmail can't send and receive e-mail across the internet like sendmail does, but it does have some very nice mail sorting abilities, whereas sendmail on the otherhand just dumps everything from a single user into a single mailbox. Procmail would then read the mailbox and sort/filter it based on it's recipe's. (Which at the moment for you they are very basic -- just enough to send an e-mail to your phone saying that it has received a new message.)

I've tried to be as basic as possible, but in doing so I think I've kept you from reading the instructions and therefore the purposes for each of these items... so i say again, Please read the man pages for the following:

mail
procmail
sendmail
fetchmail
perl


Thank you,

Binestar

[ebrostig]

I personally use fetchmail to retrieve all my emails from out imap server, then delete the mail from the imap server and then deliver it with sendmail to my local user.

Since I use Kmail, configuring filtering is easy and I have also included the use of spamassassin to move all junk to a trash can where I can briefly verify that it has correct status before beeing deleted.

Why am I doing this?

Well, by using fetchmail, I never run out of space on the central email server. I can also filter all incoming email and move them to their correct location (mailinglists gets sorted correctly). This is something you can not do on an imap server.

Another good point about fetchmail is that you can consolidate several external mail accounts into one local (very nice if you use hotmail with several usernames)

Both fetchmail and sendmail seems to be complicated to configure, but a normal user only needs a few of the options available.

I can also recommend spamassassin, it has really made my email managable again

Erik

[jecepede]

I now really lost you binestar ?

A simple mail notification, how difficult can it be ????????????

[Binestar]

first you need to make sure you install sendmail and fetchmail as in my first post. Once sendmail, fetchmail, and procmail are working as in my first post you can then go onto installing the notification.

The notification install goes like this:

Code: Select all

emerge mailx

Now create a file in your home directory called "notify.pl"

In that notify.pl file you should put:

Code: Select all

#!/usr/bin/perl
        $notify = <STDIN>;
        system("/bin/mail -s \"Email from $notify\" yourphonenumber\@phonedomain.com");

Once the file is created you need to change it to executable:

Code: Select all

chmod 755 notify.pl

Next, create a file in your home directory called: ".procmailrc"

In the .procmailrc file you will enter the following lines.

Code: Select all

LOGFILE=/home/notify/procmail.log
MAILDIR=/home/notify/mail
FORMAIL=/usr/bin/formail

:0 c
| $FORMAIL -rtzxTo: | /home/notify/notify.pl

:0
/var/spool/mail/username 

After that, you are done. And should be able to send a test message and have it appear in your mailbox, with a notification that you have received a message going to your phone.


It's not that difficult, the instructions are exactly as listed.

Binestar

[DArtagnan]

I'd like to ask some about all these email/network/handy/dandy

Soon. my ISP announced us that they will block the SMTP forward (??? not exactly understand them, can't connect to the SMTP server from outside like ssmtp does???) so is it possible to send/get emails from my gentoo without beeing dependant by my ISP?

Thanks in advance

[Binestar]

If you follow this guide, Sendmail already forwards all outgoing mail through your ISP's server, and all incoming mail is delivered locally from your ISP's server via fetchmail.

The outgoing mail and incoming mail of course is dependant on your ISP in this instance, but if you wanted to have your own domain hosted on your box you will probably have to get a mail server that is outside your ISP.

There is really not much you can do to get around that without an outside mail server.

You can set that outside mail server to listen on a port other than 25 and use that for sending and retrieving mail (i.e. 26)

If you just want to have a sendmail server on your local machine and don't mind forwarding everything through your ISP then this setup will work fine.

If you don't want the mail to ever touch the ISP's server you'll have to work around thier filters (Which is probably against the AUP.)


Binestar

[DArtagnan]

If you follow this guide, Sendmail already forwards all outgoing mail through your ISP's server, and all incoming mail is delivered locally from your ISP's server via fetchmail.

The outgoing mail and incoming mail of course is dependant on your ISP in this instance, but if you wanted to have your own domain hosted on your box you will probably have to get a mail server that is outside your ISP.

There is really not much you can do to get around that without an outside mail server.

You can set that outside mail server to listen on a port other than 25 and use that for sending and retrieving mail (i.e. 26)

If you just want to have a sendmail server on your local machine and don't mind forwarding everything through your ISP then this setup will work fine.

If you don't want the mail to ever touch the ISP's server you'll have to work around thier filters (Which is probably against the AUP.)


Binestar

Thank you

[jecepede]

Okie, Im gonna try sir... (or is it ma'am)

Im gonna emerge mailx, then procmail and test...

See you in a while....

first you need to make sure you install sendmail and fetchmail as in my first post. Once sendmail, fetchmail, and procmail are working as in my first post you can then go onto installing the notification.

The notification install goes like this:

Code: Select all

emerge mailx

Now create a file in your home directory called "notify.pl"

In that notify.pl file you should put:

Code: Select all

#!/usr/bin/perl
        $notify = <STDIN>;
        system("/bin/mail -s "Email from $notify" yourphonenumber\@phonedomain.com");

Once the file is created you need to change it to executable:

Code: Select all

chmod 755 notify.pl

Next, create a file in your home directory called: ".procmailrc"

In the .procmailrc file you will enter the following lines.

Code: Select all

:0 c
| $FORMAIL -rtzxTo: | /home/username/notify.pl

After that, you are done. And should be able to send a test message and have it appear in your mailbox, with a notification that you have received a message going to your phone.


It's not that difficult, the instructions are exactly as listed.

Binestar

Jessy

[jecepede]

Allllllrighty Binestar !!!

Itcha worinking : -)))))))))))))))))))))) I get a oneliner mail on my phone saing :

Email from however@wherever.com

only one sad thing

I can;t read my mail via mutt anymore, it suddeny delivers the mail in douzends of small files in /home/jecepede/.mail/new instead of /var/spool/mail/jecepede ???
I can read the mail amually by cat-ting the little file, but mutt says that there si no mail ?

Also, it doest forward anything anymore ???

I have the little file

.forward

in my homedir but it seems to ignore it ?????


Greets Jessy...

[Binestar]

To fix that you just need to change your .procmailrc file to:

Code: Select all

LOGFILE=/home/notify/procmail.log
MAILDIR=/home/notify/mail
FORMAIL=/usr/bin/formail

:0 c
| $FORMAIL -rtzxTo: | /home/notify/notify.pl 

:0
/var/spool/mail/username

Of course put your username where it says username, and it should work. Let me know how it tests out.


Binestar
(Christopher Fisk)

[jecepede]

Aloha Binestar !



I just wanted to type : I already tried that but it doesnt work ???
But then I saw I typed /var/spool/mial


Hihihihi thank you anyway....


Jessy

[jecepede]

cat .procmailrc

LOGFILE=/var/log/procmail.log
MAILDIR=/var/spool/mail
FORMAIL=/usr/bin/formail

:0 c
| $FORMAIL -rtzxTo: | /home/username/notify.pl

:0
/var/spool/mail/username



cat notify.pl

#!/usr/bin/perl
$notify = <STDIN>;
system("/bin/mail -s \"you have email from $notify\" username\@mydomain.nl");

Hi Binestar

I use this script now quite sucsesfully but :

When I now send mail to for example testuser@mydomain.nl the sms-mail that is being send is "made" by testuser@mydomain.nl ? So if somone sees that adres they could send to that and ehhr I don't want....

Can I change that into justauser@mydomain.nl ? coz if I send to justauser@mydomain.nl it will be not deliverd....

So in short :

1 I send to myuser@mydomain.nl

2 the mail will be forwarded to myuser@myisp.nl

3 the sms-notification will be sent to myphone@mysmsgateway.nl in the name of myuser@mydomain.nl

so can I change the myuser@mydomain.nl from step 3 into another e-mail adress ????



Many Greets :

Jessy

[eyevee99]

i've just done a fresh 1.4 install.

I'm setting up fetchmail etc as described in this thread.

When I try to emerge sendmail there is a conflict with net-mail/ssmtp

Code: Select all

[ebiuld    N   ] net-dns/hesiod-3.0.2
[blocks   B   ] net-mail/ssmtp ("virtual/mta" from pkg net-mail/sendmail-8.12.6-r1
[ebiuld    N   ] net-mail/sendmail-8.12.6-r1

Any idea what this means and how to fix it?