Email System For The Home Network - Version 2.1

[beowulf]

Email System For The Home Network
For Gentoo Linux
Beowulf <beowulf_agate AT imap DOT cc>

Version 2.1 - Fixed an error that resulted in sasl using sasldb, added link to AV solution.
Version 2.0 - Complete Rewrite to ease SSL, SASL, OE and general setup time.
Version 1.5 - Added Outlook Express 6, updated Apache/PHP setup to Apache2, small fixes.
Version 1.4 - software version updates, misc enhancements/clarifications. Added Troubleshooting section.
Version 1.3 - Typo corrections, Added Spam Solutions thanks to (puddpunk,proteus)
Version 1.2 - Fixed some errors, re-wrote some sentances for clarity. Added Squirrelmail
Version 1.1 - Fixed a number of errors. Added pam config for courier-imap
Version 1.0 - Initial guide.

Abstract:

This guide will help you create a fully functional email service within a home network. You will run servers to allow you to both send and receive email from all over the world. We will use free services to facilitate this such as DynDNS[1].

This guide can be used as full blown mail server provided you have an MX record pointing to your mail server. No changes are necessary.

[1] DynDNS - A DNS service offering up to 5 hosts.

1. Introduction
2. Preperation
3. Sending Email
4. Filtering Email
5. Providing IMAP Email Access
6. Fetching Email From External Sources
7. Email Client Setup
8. Squirrelmail Webmail Setup
9. Bogofilter Mail Filtering Solution
10. Spam Assassin Mail Filtering Solution
11. Troubleshooting
12. Resources

1. Introduction:

There's already quite a number of email systems available on the net, on Gentoo's web site and even in this very forum. Here's yet another email setup. Created since I could never find a middle ground for setting up email, either they were too simple or too robust for my needs. Perhaps if you find yourself in this position, this guide can help.

We will create a fully functional email service within a home network. We will become our very own SMTP, POP3/IMAP provider using free services, free software and a free operating system. We use SMTP Auth through Cyrus-SASL so that we can force users to authenticate before using our service.

1.1 Sending:

Email Client->Cyrus-SASL[2]->Postfix[3]->Internet

This setup allows you to use this SMTP server from anywhere in the world, provided your ISP does not block port 25 incoming. This may not be what you want though, so we have another method of sending, taking a few extra steps to allow you to forward (or relay) your email to your ISP's SMTP server. You would use this method if you don't want port 25 open to the Internet and you only need to access it inside your lan. It is a safer since you can block all port 25 connections from the Internet using IPTables[4]. So just to recap, the above method is used if you want to become your own full-blown SMTP server, the method just below is used for an internal SMTP server with access to your ISP's SMTP server.

Email Client->Cyrus-SASL->Postfix->Cyrus-SASL->ISP SMTP Server->Internet

[2] Cyrus-SASL - Enables SMTP Auth
[3] Postfix - A robust Mail Transport Agent.
[4] IPTAbles - A stateful firewall.

1.2 Receiving:

ISP IMAP/POP3 Server->Fetchmail[5]->Procmail[6]->Courier-IMAP[7]->Email Client

This is a standard setup covered extensively on the forum, gentoo's site and on the Internet. This is the setup you would choose if your ISP blocks port 25 (incoming) or you need to get email from external email services.

The second method makes your email service inside your lan function just like the big boys. You'll be able to use your own domain name (or free service) and create your own cool email address. For example, beowulf AT apparition DOT ath DOT cx is my custom email address.

Internet->Postfix->Procmail->Courier-IMAP->Email Client

[5] Fetchmail - a full-featured remote-mail retrieval utility
[6] Procmail - Mail delivery agent/filter
[7] Courier-IMAP - An IMAP daemon designed for maildirs

1.3 Software Versions Used:

Please note, this guide was written using these versions of the software. If by the time you read this, a newer version of the software has been released, I encourage you to check the developer's web site and read the changelog or find changes in behaviour.

• net-mail/fetchmail-6.2.3
• net-mail/procmail-3.22-r6
• net-mail/courier-imap-2.1.2-r1
• net-mail/postfix-2.0.11
• dev-libs/cyrus-sasl-2.1.14 [8]
• net-mail/squirrelmail-1.4.2-r1 [9]
• net-mail/bogofilter-0.11.2 [10]
• dev-perl/Mail-SpamAssassin-2.55-r1 [11]
• dev-libs/openssl-0.9.7c-r1

[8] Cyrus-SASL - Cyrus Simple Authentication and Security Layer
[9] SquirrelMail - A PHP web mail for use with maildir
[10] BogoFilter - A Bayesian spam filter tuned for speed
[11] SpamAssassin - A program to filter spam.

1.4 What This Guide Doesn't Do Well:

This email system does not scale well. I can't imagine managing more than 5 accounts with the current set up as it will just become cumbersome in my opinion. When sending email using your own SMTP server, or this setup; some POP server's may regard your mail as spam and it will either be blocked entirely or filtered into a spam folder. Yahoo! is one such email service Thanks to dteisser for the info. I haven't encountered this problem yet so it may be rare. All the same, it is something you should be wary about.

2. Preparation:

Since we're dealing with 2 computers, we must designate one of them a server. We'll refer to the two computer's workstation and server. We will assume that you already have your hostname setup (should have been done during your install process[12]). So all that is needed is to find the information.

2.1 Local Servers:

Below is an ASCII chart of what is needed, and the possible values that could be used. Of course your network setup may differ, and i do encourage you to find out all information needed before you continue.

Code: Select all

Chart 2.1
.--------------------------------------------,
| Needed    ||     Server    |  Workstation  | Chart 2.1 - Server Info
|===========||===============================|
| Network   ||         192.168.2.0/24        |  - Copy paste this code
|-----------||-------------------------------|    block into a text editor
| IP        ||  192.168.2.2  |  192.168.2.3  |    editor for reference
|-----------||---------------|---------------|    later on.
| Hostname  ||    Chimera    |   Illusion    |
|-----------||-------------------------------|  - Substitute the values
| Domain    ||      apparition.ath.cx        |    here with your values
|-----------||-------------------------------|
| Username  ||   21s-beo     |      N/A      |
|-----------||---------------|---------------|
| Password  ||  21s-pass123  |      N/A      |
'--------------------------------------------'

Since we're a home user, we probably don't have a dns server running with an MX record pointing to our server. Therefore to facilitate the need to access this server from other places, I suggest a free IP service. I have used "apparition.ath.cx" as the domain name provided by DynDNS[13], however No-IP.com[14] is another solution. Therefore my FQDN for my server is: Chimera.apparition.ath.cx.

In case you have skipped it, please enter your FQDN in /etc/hosts substituting your values that you've recorded in chart 2.1 with the one's I have used.

Code: Select all

root@server # echo 'Chimera' > /etc/hostname && echo 'apparition.ath.cx' > /etc/dnsdomainname
root@server # vi /etc/hosts

127.0.0.1       localhost
192.168.2.2     Chimera.apparition.ath.cx               Chimera

[13] DynDNS - A free IP redirection service offering 5 free entries
[14] No-IP - A free IP redirection service offering lots of free entries.

2.2 Remote Email Services:

You should obtain this information from your ISP/Email service provider. We will use 3 different examples as designated in chart 2.2 (below). Copy the chart to the same text file and label it accordingly. We'll be referring to it later in the guide.

Code: Select all

Chart 2.2
.----------------------------------------------------------------,
| Needed    ||      SMTP     |   IMAP / SSL      | POP3 / No SSL |
|===========||===============|===================|===============|
| Server    || smtp.isp.com  | imap.fastmail.com |  pop.huah.com |
|-----------||---------------|-------------------|---------------|
| User      ||    beo739     |    beo_agate      |  beowulf_999  |
|-----------||---------------|-------------------|---------------|
| Password  ||  rsmtp-pass   |   rimap-pass      |   rpop-pass   |
'----------------------------------------------------------------'

2.3 Installing The Software:

It's about time we did something. Since we use portage and benefit from the Gentoo build system, this step is easy. Don't worry about editing make.conf as we'll set the flags we need on the command line. If you are installing this system on a system without portage, you should "./configure --help" to find out the configure flags needed to match our use flags. Please SSH into your server now, or physically walk over there.

NOTE: If you already have a MTA such as Sendmail[15] or ssmtp[16], you may receive a block message from portage. Simply unmerge the package before continuing.

Code: Select all

root@server # USE="ssl pam nls maildir sasl gdbm berkdb -mysql -ldap \
      -mbox -postgres -kerberos -java -static" emerge courier-imap \
      cyrus-sasl fetchmail postfix -pv
root@server # emerge procmail -pv

What we've done is set our use flags on the command line to avoid editing /etc/make.conf. Since we won't be using mysql or postgres with this email setup, we explicitly tell portage not to compile support for them. This command simply pretends and shows the use flags associated with each package. Once you accept it, simply remove the -pv switch. Do not start any service or add anything to run-time yet since we need to set everything up.
[15] Sendmail - A popular MTA used everywhere
[16] SSMTP - An extremely simple MTA installed as a dependancy to *cron when you installed.

3. Sending Email:

Let's set up Postfix to send email out. This can be the hardest section of the guide. Let's get it out of the way.

3.1 Postfix Main Configuration:

We'll use a base configuration before we get into any configuring. Please make sure that your file matches mine so that we can all start with the same base. If you don't see an option in the following code block, it means it should be commented. Thanks to requiem for pointing out the mailbox_command variable to me.

Code: Select all

root@server # vi /etc/postfix/main.cf

queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix
myorigin = $myhostname
mydestination = $myhostname, localhost.$mydomain $mydomain
unknown_local_recipient_reject_code = 450
mynetworks_style = subnet
mynetworks = 127.0.0.0/8 192.168.2.0/24
mailbox_command = /usr/bin/procmail
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail
newaliases_path = /usr/bin/newaliases
mailq_path = /usr/bin/mailq
setgid_group = postdrop
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.0.16-r1/sample
readme_directory = /usr/share/doc/postfix-2.0.16-r1/readme
default_destination_concurrency_limit = 2
alias_database = hash:/etc/mail/aliases
local_destination_concurrency_limit = 2
alias_maps = hash:/etc/mail/aliases
home_mailbox = .maildir/

The mynetworks variable needs to be changed to match your internal network. If you experience hostname problems, please fill out the variables myhostname and mydomain with the appropriate information. This however should not be needed since postfix tries to get the information automatically making any declaration on our part a bit redundant.

3.2 Adding SMTP-AUTH to Postfix:

We'll add SMTP-AUTH to postfix by way of Cyrus-SASL. Since everything is already configured, let's simply configure it. New in v2.0 of this guide, we use saslauthd and auth against shadow. This should cut down on some confusion generated from earlier versions of this guide.

The first thing to do is edit /etc/smtpd.conf and tell SASL the method and mechanisms we intend to use for auth. Make sure your file matches this one exactly.

Code: Select all

root@server # vi /etc/sasl2/smtpd.conf

pwcheck_method:saslauthd
mech_list: plain login

NOTE: You may also be required to edit this file with the same information, I strongly urge you to do this. Thanks to Woolong for pointing this out.

Code: Select all

root@server # vi /usr/lib/sasl2/smtpd.conf

pwcheck_method:saslauthd
mech_list: plain login

What we have specified is that we will use saslauthd (daemon) for authentication, thus no longer relying on sasldb and it's quirky ways. Next up, we will have to edit the conf file for the daemon start up. Let's go and do that now.

Code: Select all

root@server # vi /etc/conf.d/saslauthd

SASL_AUTHMECH=shadow

SASL_RIMAP_HOSTNAME=""
SASL_TIME_OF_DAY_LOGIN_RESTRICTIONS=yes

SASLAUTHD_OPTS="-a ${SASL_AUTHMECH}"

You'll notice I have removed the gentoo generated line. The reason for this is because the current version of SASL that I have would not start with the default line. I have commented out everything and suggest you do the same so that your file matches mine exactly. We've stated in this file that saslauthd should use shadow as the auth mechanism.

All we have to do now is to tell postfix that you want to use sasl. Let's do that now:

Code: Select all

root@server # vi /etc/postfix/main.cf

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes
smtpd_client_restrictions = permit_sasl_authenticated, reject

This tells Postfix that we want to use SASL to provide SMTP-AUTH and that any user who can't authenticate against SASL should be rejected.

3.3 Postfix TLS Support:

A section that has caused more than it's fair share of trouble, this has now become fairly easy since postfix now provides some default key's for us. No more editing the CA.pl file, no more -nodes.

Simply copy this code block exactly down in your /etc/postfix/main.cf file.

Code: Select all

root@server # vi /etc/postfix/main.cf
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/ssl/postfix/server.key
smtpd_tls_cert_file = /etc/ssl/postfix/server.crt
smtpd_tls_CAfile = /etc/ssl/postfix/server.pem
smtpd_tls_loglevel = 3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

We have told postfix to always use TLS for authentication or reject the mail. Since we've decided to use plain text to send our passwords, we must use encryption to ensure any network sniffer doesn't get our password.

3.4 Making Postfix a Relay to our ISP:

As was mentioned in section 1.1, we can use Postfix to be an email relay and send any mail to our ISP's SMTP server before it hits the Internet. Please keep in mind, that this step is NOT needed if you intend to use Postfix as a full-blown MTA. This step is optional and should NOT be used if you have an MX record.

Email Client->Cyrus-SASL->Postfix->Cyrus-SASL->ISP SMTP Server->Internet

This is easily accomplished as well. Although it's not really needed, it may be needed with some ISP's or some network setups so I've included for anyone who may need it.

First you must create a file that holds our ISP's SMTP Server authentication information. The format is rather simple "[server] [user]:[pass]". Here's what one could look like if we used the information in chart 2.2 under the SMTP heading:

Code: Select all

root@server # vi /etc/postfix/saslpass

smtp.isp.com          beo739:rsmtp-pass

After you've completed that, let's protect the file and hash it so postfix can work with it. We do this with the following commands:

Code: Select all

root@server # /bin/chown root:root /etc/postfix/saslpass
root@server # /bin/chmod 600 /etc/postfix/saslpass
root@server # /usr/sbin/postmap hash:/etc/postfix/saslpass

Next, all we must do is tell Postfix that we want it to relay the email using SASL to our ISP's SMTP server. Let's do that now.

Code: Select all

root@server # vi /etc/postfix/main.cf

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpass
smtp_sasl_security_options = noanonymous

3.5 Adding Anti-Virus Protection:

I have not done this myself (yet) but there is a detailed guide by john5211 on how he got clam-av working on his server. If you are interested, I would recommend clicking here for more information. axxackall also adds a few more comments in his post which is on the same page.

3.6 Automating and Finalizing:

The sending section is just about done. We should add/edit our aliases that we want to use on our system. Let's do that now:

Code: Select all

root@server # vi /etc/mail/aliases

# Well-known aliases -- these should be filled in
 root:          2ls-beo

You'll notice I aliased root to my username on my server (information found in chart 2.1 under the "Server" column. Let's create the alias db and check our postfix configuration:

Code: Select all

root@server # /usr/bin/newaliases
root@server # /usr/sbin/postfix check

If all went okay, and no errors arose we can continue. If an error occured during the "postfix check" procedure. Double check your main.cf file for spelling errors, syntax errors. One note to keep in mind, any option that is prefixed with a blank space will produce an error. Thanks to paulfl for pointing this out.

Now let's just automate the server's startup and start the actual server. Your ouput should match mine exactly:

Code: Select all

root@server # /etc/init.d/saslauthd start
 * Starting saslauthd...                                                  [ ok ]
root@server # /etc/init.d/postfix start
 * Starting postfix...                                                    [ ok ]
root@server # rc-update add saslauthd default
 * saslauthd added to runlevel default
 * Caching service dependencies...                                        [ ok ]
 * rc-update complete.
root@server # rc-update add postfix default
 * postfix added to runlevel default
 * Caching service dependencies...                                        [ ok ]
 * rc-update complete.

Remember, the username and password you use to authenticate to your SMTP server is the same pair that we listed in chart 2.1. It is the same information that is found in /etc/passwd.

4. Filtering Email:

We made mention to Procmail in the Sending secion so it's only fitting that we set that up next. Procmail is a powerful piece of software that is very stable. Procmail uses rules (or recipes) similar in idea to the rules used in email clients, the difference however is we sort everything on the server side and deliver the email to various mail directories. Let's create our procmail file now.

First thing to do is drop out of root and go to our regular user.

Code: Select all

user@server $ cd ~
user@server $ touch .procmailrc
user@server $ vi ~/.procmailrc

MAILDIR=$HOME/.maildir/
DEFAULT=$MAILDIR

#
## Begin recipes
#

# put cron job emails in my aptly named cron-jobs maildir
:0
* ^Subject:.Cron*
.cron-jobs/

# Deliver Gentoo Specific email to our special maildir's
:0
* ^List-Id:.*gentoo-announce\.gentoo\.org
.gentoo-announce/

:0
* ^List-Id:.*gentoo-gwn\.gentoo\.org
.gentoo-gwn/

# Catch email from Gentoo not related to the lists (IE: Forums,Bugs)
:0
* ^From:.*gentoo\.org
.gentoo/

# Catch all email directed to my business email address:
:0
* ^To:.*myrealname\@apparition\.ath\.cx
.business/

## All the rest of our email will be delivered to our default INBOX
## so no additional rule is needed

As you can see, I have a very simple procmailrc file. You could do real special things with procmail such as set up autoresponders, automatically forwarding email, parse the email and call external applications. It's really a powerful piece of software, but for our needs, this example file works nicely. Make sure any maildir you wish to filter to is preceded with a dot (.) and that a forward slash (/) follows. This will deliver email in maildir format.

We only need to make our base maildir, procmail will create any other directory structure you need. Let's make our default maildir.

Code: Select all

user@server $ maildirmake ~/.maildir/

5. Providing IMAP Email Access:

We use Courier-IMAP as the server to provide access to our email from anywhere on the Internet or in our LAN. We chose this piece of software since it's designed to work with maildir's. We've already emerged the software, so let's configure it.

5.1 Setting up Authentication:

First thing to do is change to root and check that authdaemon is running with the appropriate method:

Code: Select all

user@server $ su -
Password:
root@server # vi /etc/courier-imap/authdaemond.conf

AUTHDAEMOND="authdaemond.plain"

We've told the authdaemond to use a plain method. It simply means we aren't going to use some of the more robust solutions such as mysql or ldap. After that variable has been set, we need to edit the conf file for authdaemond. Let's make sure that the authmodulelist is using pam. Again, as mentioned in the beginning of this guide, this setup is not for hundreds of users, so pam fits the bill nicely.

Code: Select all

root@server # vi /etc/courier-imap/authdaemondrc

authmodulelist="authpam"

Since we're authing against pam, please make sure the imap pam file matches mine exactly. Now these values should be there by default, but just in case, they are provided here.

Code: Select all

root@server # vi /etc/pam.d/imap

# PAM setup for
auth       required     pam_nologin.so
auth       required     pam_stack.so service=system-auth
account    required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

5.2 Adding SSL Support:

As mentioned, we want to only use SSL to connect to our IMAP server. Since we have chosen a safer method of authentication, it requires a bit more work. Let's do it now while we're still as root:

Code: Select all

root@server # vi /etc/courier-imap/imapd.cnf

[ req_dn ]
C=CA
ST=ON
L=Toronto
O=Mail Server
OU=Automatically-generated IMAP SSL Key
CN=localhost
emailAddress=root@localhost

As you can see, I've changed the variables to match my network and location. I recommend you do the same. It doesn't really matter, but you should do it anyways. You can find all the variables to change in the "[ req_dn ]" section of the file. After you've done that, we can make our certificate file:

Code: Select all

root@server # cd /etc/courier-imap && mkimapdcert

5.3 Automating and Finalizing:

We've created the base maildir in secion 4, all that's left is adding the servers to our default runlevel and starting the servers. Since we don't want IMAP to authenticate without SSL, we have only started the appropriate server. Make sure that authdaemond.plain started as a dependancy.

Code: Select all

root@server # /etc/init.d/courier-imapd-ssl start
 * Starting authdaemond.plain...                                          [ ok ]
 * Starting courier-imapd-ssl...                                          [ ok ]
root@server # rc-update add courier-imapd-ssl default
 * courier-imapd-ssl added to runlevel default
 * Caching service dependencies...                                        [ ok ]

Please remember, the username and password combination that you use to authenticate here is the same pair found in chart 2.1. It is the same username / password you use to login to the server.

6. Fetching Email From External Sources:

Fetchmail is a program that allows a user to fetch email from various external servers. It's a great little program that can handle just about any protocol (IMAP/S - POP3). Fetchmail does not need to run as root, so let's not have any more programs running as super user than needed. First thing to do is drop out of root.

6.1 Setting up the Configuration File:

First we will go to our home directory, create the file and then add a configuration. We'll discuss what goes where and how to customize this file to your unique setup after. First let's look at the commands and template-like view of the fetchmail file.

Code: Select all

user@server $ cd ~
user@server $ touch .fetchmailrc
user@server $ vi .fetchmailrc

set postmaster "[SERVER-USERNAME]"
poll [IMAP-SERVER] with proto IMAP user "[IMAP-USER]" there with password "[IMAP-PASSWORD]" is [SERVER-USERNAME] here options warnings 3600

As you can see, the options are surrounded with square brackets ([]). In chart 2.1 you have recorded your server's username. Substitute [SERVER-USERNAME] with your username. In chart 2.2 we gave two examples of servers which we could fetch email from. They are under the headings "IMAP/SSL" and "POP3/No SSL". Let's assume that this is my fetchmail file and the server I am fetching email from (polling) is under the "IMAP/SSL" heading. Here's what my .fetchmailrc file would look like:

Code: Select all

set postmaster "2ls-beo"
poll imap.fastmail.com with proto IMAP user "beo_agate" there with password "rimap-pass" is 2ls-beo here options warnings 3600

Let's take a look at another example, this time a POP3 server without SSL support found in chart 2.2 under the "POP3/No SSL" heading:

Code: Select all

set postmaster "2ls-beo"
poll pop.huah.com with proto POP3 auth password user "beowulf_999" there with password "rpop-pass" is 2ls-beo here options warnings 3600

As you can see, we added "auth password" to our poll line. This tells fetchmail not to use SSL when trying to fetch the email.

Chances are some of you have more than one email account that you'd like to fetch. Luckily, fetchmail can handles this with ease. Here's our two examples above combined into one file:

Code: Select all

set postmaster "2ls-beo"
poll imap.fastmail.com with proto IMAP user "beo_agate" there with password "rimap-pass" is 2ls-beo here options warnings 3600
poll pop.huah.com with proto POP3 auth password user "beowulf_999" there with password "rpop-pass" is 2ls-beo here options warnings 3600

Now that we've configured fetchmail, let's change it's permissions. Fetchmail is picky about the permissions of this file, so to meet it's requirements, we must chmod our file:

Code: Select all

user@server $ chmod 710 ~/.fetchmailrc

If you are a HotMail user, you might also consider installing GotMail. A simple solution to this problem is detailed by marienZ can be found by clicking here. I have not tested this as I do not have a HotMail account, but from what I've heard here and elsewhere, gotmail works fine. Even as a replacement.

6.2 Automating and Finalizing:

Since we're using Fetchmail in non-daemon mode, we'll use cron to emulate it. Here's the correct cron line, however I have found vcron tends to choke on it.

Code: Select all

*/10 * * * * /usr/bin/fetchmail -a -s -m "/usr/bin/procmail -d \%T"

The above code block adds a cron job that will run every 10 minutes and fetch email and then passes it directly to procmail. Thanks to requiem for the correct crontab line. If you're like me though, you'll want a script that you can run whenever you want. That way, if somebody tells you to check your email, you simply ssh into your server and run the script. This script should also be used in your cron in case cron chokes on the line above. As a nomral user, enter this:

Code: Select all

user@server $ mkdir ~/bin
user@server $ chmod 700 ~/bin
user@server $ echo -e "\043\041/bin/bash\n/usr/bin/fetchmail -a -s -m \"/usr/bin/procmail -d
%T\"" > ~/bin/getmyemailnow
user@server $ chmod +x ~/bin/getmyemailnow

Now all you have to do is add a line to cron similar to this:

Code: Select all

*/10 * * * * $HOME/bin/getmyemailnow

NOTE: If you're using an anti-virus system, you should instead have fetchmail redirect to port 25 (the default option). You will need to edit /etc/postfix/main.cf and edit this line:

Code: Select all

smtpd_recipient_restrictions = permit_sasl_authenticated, reject

... So that it reads like this:

Code: Select all

smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject

This will allow fetchmail to send email through postfix so that it can be scanned. Thanks to Advo for letting me know about this caveat.

7. Client Email Setup:

If you haven't already done so, close the SSH session to your server and let's setup your workstation. The username and passwords you use to authenticate can be found in chart 2.1 under the Server heading.

7.1 Kmail

Kmail[17] is the default Email client that is emerged with KDE[18]. It is a fully functional email client with support for every protocol we'll use and many we don't. In it's latest incarnation (3.2) it is lightning fast and very stable. It is the client I use and have the most experience with. For this setup, we'll assume you have already set up your Identities section.

1. Create a new network Configuration (Settings->Configure Kmail) and make sure you're viewing the "Sending" tab.
2. Click on "Add..." button. A new window will open offering you a choice of Sendmail or SMTP. Select SMTP.
3. Choose a name to enter in the Name field. The host field should contain the [IP] of your server (in my example: 192.168.2.2).
4. Check the box that says "Server requires authentication", then enter the username and password you have recorded in chart 2.1.
5. Click on the "Security" tab and click the button that reads: "Check What the Server Supports." In my case it was TLS for encryption and PLAIN for authentication method. Now click OK.
6. Still in the Networking Options, click on the Receiving tab. Now click "Add..." and when a new window opens up, choose "IMAP"
7. Enter the exact same information you used before, same host, user and password.
8. In the security tab, click on the button: "Check What the Server Supports" and wait until the options are changed. In my case it was "Use SSL for secure mail download" and "Plain" as the authentication method.

Click on the OK button and you're all set. Apply the changes and test the email by sending yourself an email and receiving it.

[17] Kmail - The default Email client
[18] KDE - A fully featured powerful desktop environment.

7.2 Sylpheed-Claws

Sylpheed-Claws[19] is a GTK+ application based on the Sylpheed[20] email client. It can be referred to as the bleeding edge version. Although it is bleeding edge, it is very stable. This is also the client preferred by several anti-bloat people over Evolution
list=1]
[*] First thing we do is create a new account (Configuration->Create new account...).
[*] Fill out the Personal Information and set a name for the account.
[*] In the server information, change Protocol to IMAP4.
[*] Change the server for both receiving and sending to [IP] or in my example: 192.168.2.2
[*] Fill in User ID and Password with the Server user's username and password found in chart 6 under "Local IMAP server"
[*] Under the "Send" tab, make sure to check off the SMTP Authentication (SMTP AUTH) option. Next fill in the the username and password found in chart 2.1 under the "Server" heading.
[*] Under the "SSL" tab, check off "Use SSL for IMAP4 connection" option and below that, check off Use STARTTLS command to start SSL session
[/list]
Apply the changes and test the email setup by sending yourself an email and then check to see that it was received correctly.

[19] Sylpheed-Claws - A solid lightweight GTK+ email client
[20] Sylpheed - The stable branch of the Sylpheed family
[21] Evolution - A GNOME groupware application

7.3 Outlook Express 6

Outlook Express is the default email client used on most Windows Machine. It comes pre-installed on every Windows version and is freely updated at Windows Update. NOTE: that I cannot test what happens with the million and one types of anti-virus programs out there and their outbound email scanning. Since we've changed the mechanisms for authentication (from CRAM-MD5 to PLAIN), Outlook Express should have no problem authenticating now. Here's what you need to do:

1. Start Outlook Express and go to the menu option Tools->Accounts.
2. When the new window opens, click on the "Mail" tab, then click on the "Add" button followed by "Mail". You will be presented with a wizard, enter the data as you see fit until you reach the 3rd page where you're asked for your Sending and Receiving servers. Please enter the IP address of your server (in my example it's 192.168.2.2) in both text fields and use the pull down to select "IMAP".
3. Next you'll be asked for your IMAP server's username/password. Please enter the information you found in chart 6 under the "Local IMAP" heading.
4. The wizard will finish, but you must open up the properties of that connection again. Under the "Mail" tab you should see your connection, highlight it with a single click and select the "Properties" button off to the right. A new window will open
5. Click on the "Advanced" tab and check the checkboxes so that you are enabling SSL for both sending and receiving. You'll notice the port for your IMAP server has changed to 993.
6. Click on the "IMAP" tab and under the Root folder path option, enter the word "INBOX" and make sure that "Check for new messages in all folders" is enabled.
7. Click on the "Servers" tab and enable the box that says "My server requires authentication", then click on the "Settings..." button.
8. You'll presented with a new window with radio buttons. Click on the option that says "Log on using", thus enabling the text fields below and enter your account information found in chart 6 under the "Local SMTP" heading.

Click the Apply/OK button and you're all set. Check your email and send out a test email to yourself.

--

Grover reports that new in Gkrellm's mail checker there is support for SSL so you can monitor your inbox now without the stunnel workaround.

8. Squirrelmail Webmail Setup:

We're going to set up Squirrelmail Webmail. Although this step is optional, it may be of use to some. If you're a fan of webmail, this piece of software is a godsend. Let's continue.

I will assume you have a working Apache and PHP setup. First thing to do is SSH into your server again and become root.


8.1 Emerge the Software:

Sounds simple right? Well thanks to portage, this section is short and to the point:

Code: Select all

root@server # emerge squirrelmail

8.2 Configuration:

This software is rather simple to set up. It comes with it's own ncurses? configuration utility. So let's run that now:

Code: Select all

root@server # cd /var/www/localhost/htdocs/squirrelmail/config
root@server # perl conf.pl

You should be presented with a screen showing menu choices allowing you to enter a number. Let's start with Organization Data by pressing 1. As you can see, every option is prefixed with a number, so to edit the option you would type in the corresponding number and enter a new value.

Once you're finished, press "R" to return to the main menu.

Now press "2" to configure the server settings.

Here are what my options look like, and perhaps you are able to extrapolate what yours should be.

Code: Select all

General
-------
1.  Domain                 : apparition.ath.cx
2.  Invert Time            : false
3.  Sendmail or SMTP       : Sendmail

IMAP Settings
--------------
4.  IMAP Server            : localhost
5.  IMAP Port              : 993
6.  Authentication type    : login
7.  Secure IMAP (TLS)      : true
8.  Server software        : courier
9.  Delimiter              : detect

As you can see, it's fairly straight forward. I entered my domain name after pressing "1", after pressing "3" I told squirrelmail to use Sendmail as opposed to SMTP. I then pressed "A" to get the IMAP settings available. I changed the port, TLS and server software to match this setup.
But wait! We never installed sendmail. Here's the beauty of Postfix, it seamlessly replaces sendmail even creating a link named sendmail in /usr/sbin/ for programs expecting the binary. Pretty clever of Postfix eh? Nothing to worry about.

Once you're done, press "S" to save your settings and then press "Q" to quit the configuration utility.

8.3 Finalizing and Automating:

Point your browser to this address "https://192.168.2.2/squirrelmail" and login using the same data you have written down in chart 2.1. Just remember to replace the IP in the URL that matches your Server.

Now, you must tell SquirrelMail that you wish to subscribe to various folders. Click on the link "Folders" that runs along the top of the page. Once the new page has loaded, simply highlight the folders listed in the select box such as "INBOX.gentoo and click on the Subscribe button. Refresh your folder listing and you'll see how Procmail has sorted all your email.

Send an email to yourself and receive it... you should know the drill.

All that's left is to add apache to your default runtime:

Code: Select all

root@server # rc-update add apache2 default

9. Bogofilter Mail Filtering Solution
By Chris Smith

This guide was written so that bogofilter[15] may be implimented in the "Email System for the Home Network" Guide. This guide proves that bogofilter can be used in client AND in server side filtering solutions, still leaving the user in total control.

The script contained in this guide depends on most of this guide being followed word for word. Feel free to edit and modify my guide and script for your own use, just post on this thread and let us know what your doing with it. We're very interested to see where this goes

All code contained in this documentation is released under the GPL Public Licence. Of course Right... Here we go!

Code: Select all

root@server # emerge bogofilter

9.1 Bogofilter Instructions

1. Make the spam maildirs:

   Code: Select all

   $ cd ~/.maildir
   $ mkdir .Spam{,.False-Positives,.False-Negatives}
   $ mkdir .Spam{,.False-Positives,.False-Negatives}/{cur,tmp,new}

   NOTE: If you change these, I hope you know python, as you will need to hack the script so it knows which maildirs to treat as spam.
2. Load your mail client and move ALL your spam mail out of your normal directories, and into the Spam directory.
3. OPTIONAL: If you have a LOT of mail (i.e. thousands), and not just spam either, all mail, you may choose to have a "Ham" directory, which you can put a selection of a few hundred messages in.
   
   You may choose to do this, because the script currently walks through all your directories (that aren't spam!) and commits all that mail to bogofilter as "Good" mail. If you have a lot of messages, this will take quite a while (but not _that_ long ), but bogofilter will be more thourughly trained. Do this only in special cases:
   1. Create ham directory:

      Code: Select all

      mkdir .Ham
      mkdir .Ham/{cur,tmp,new}

   2. Move a selection of a few hundred good messages into the new Ham directory
   The script will auto-detect the precense of a .Ham directory, so it won't walk all your maildirs.
4. Copy the following script, and name it as:

   Code: Select all

   ~/Bin/bogotrainer

   Code: Select all

   #! /usr/bin/python
   import os, os.path
   
   #Configuration entries. Not much ATM. More if needed.
   bogodir = "~/.bogofilter/"
   maildir = "~/.maildir/"
   
   #Leave everything below here unless you want to do some hacking :)
   needdbs = 0
   bogodir = os.path.expanduser(bogodir)
   maildir = os.path.expanduser(maildir)
   
   def cleanhamdirs(dir):
   	#We don't want Spam in the hamdirs :)
   	if dir[len(maildir):len(maildir) + 5] == ".Spam":
   		return 0
   	#The maildirs of the inbox, must be handled especially
   	if dir[len(maildir):len(maildir) + 3] == "cur":
   		return 0
   	if dir[len(maildir):len(maildir) + 3] == "tmp":
   		return 0
   	if dir[len(maildir):len(maildir) + 3] == "new":
   		return 0
   	#If you threw it away, you obviously don't want it :)
   	if dir[len(maildir):len(maildir) + 6] == ".Trash":
   		return 0
   	return 1
   
   if os.path.isdir(bogodir):
   	print "Bogofilter directory found"
   	#I'm just assuming if the spamlist.db exists, goodlist.db does too
   	#Program will die if goodlist.db doesn't exist anyway.
   	if os.path.isfile(os.path.join(bogodir, "spamlist.db")):
   		print "Databases found"
   	else:
   		print "Databases NOT found. Generating..."
   		needdbs = 1
   else:
   	print "Bogofilter directory NOT found. Generating..."
   	needdbs = 1
   
   if needdbs:
   	print "Generating databases:"
   	print "Regestering spam messages from", os.path.join(maildir,".Spam/cur")
   	spamlist = os.listdir(os.path.join(maildir,".Spam/cur"))
   	for spam in spamlist:
   		spampath = os.path.join(maildir,".Spam/cur/",spam)
   		print "- ", spampath
   		os.system("bogofilter -s < " + spampath)
   	if os.path.isdir(os.path.join(maildir, ".Ham")):
   		#If a specific .Ham dir exists, use that.
   		print "Regestering ham messages from", os.path.join(maildir,".Ham/cur")
   		hamlist = os.listdir(os.path.join(maildir,".Ham/cur"))
   		for ham in hamlist:
   			hampath = os.path.join(maildir,".Ham/cur",ham)
   			print "- ", hampath
   			os.system("bogofilter -n < " + hampath)
   	else:
   		#Or else, use everything that isn't spam!
   		print "Registering ham messages from", os.path.join(maildir,"cur")
   		hamlist = os.listdir(os.path.join(maildir,"cur"))
   		for ham in hamlist:
   			hampath = os.path.join(maildir,"cur",ham)
   			print "- ", hampath
   			os.system("bogofilter -n < " + hampath)
   		maildirs = [os.path.join(maildir,dir) for dir in os.listdir(maildir)]
   		maildirs = filter(os.path.isdir, maildirs)
   		maildirs = filter(cleanhamdirs, maildirs)
   		for dir in maildirs:
   			print "Regestering ham messages from", dir
   			hamlist = os.listdir(os.path.join(dir,"cur"))
   			for ham in hamlist:
   				hampath = os.path.join(dir,"cur",ham)
   				print "- ", hampath
   				os.system("bogofilter -n < " + hampath)
   
   # So, everything exists, this must be an "updating run", easy!
   # First, correct misdetected ham from the false-positives directory,
   # and move it into the inbox.
   print "Correcting ham messages from", os.path.join(maildir,".Spam.False-Positives")
   hamlist = os.listdir(os.path.join(maildir,".Spam.False-Positives/cur"))
   for ham in hamlist:
   	hampath = os.path.join(maildir,".Spam.False-Positives/cur",ham)
   	print "- ", hampath
   	os.system("bogofilter -Sn < " + hampath)
   	#Feed it back through procmail :)
   	os.system("/usr/bin/procmail -d $USER < " + hampath)
   	os.remove(hampath)
   
   # Now, correct misdetected spam, and put it in the Spam maildir :)
   print "Correcting spam messages from", os.path.join(maildir,".Spam.False-Negatives")
   spamlist = os.listdir(os.path.join(maildir,".Spam.False-Negatives/cur"))
   for spam in spamlist:
   	spampath = os.path.join(maildir,".Spam.False-Negatives/cur",spam)
   	print "- ", spampath
   	os.system("bogofilter -Ns < " + spampath)
   	#Don't bother procmailing it, put it in spam! :)
   	os.rename(spampath, os.path.join(maildir,".Spam/cur",spam))

5. Now, make the script executable:

   Code: Select all

   chmod +x ~/Bin/bogotrainer

6. If you have a previous training of bogofilter, the script won't overwrite it (so it's cronjob-able) but it's a good idea to start a fresh.

   Code: Select all

   rm -rf ~/.bogofilter

7. Run the script and wait while it takes in all of your mail and builds its databases. Bogofilter is quite fast, so it shouldn't take too long and you get to see it's progress!

   Code: Select all

   ~/Bin/bogotrainer

8. Add these recipies before all your other recipies:

   Code: Select all

   #Bogofilter filtering solution.
   :0fw
   | bogofilter -u -e -p
   
   :0e
   { EXITCODE=75 HOST }
   
   :0:
   * ^X-Bogosity: Yes,
   .Spam/

9. Add this line to your crontab:

   Code: Select all

   user@server $ crontab -e
   
   * 23 * * * ~/Bin/bogotrainer >/dev/null 2>&1

   This sets it to run once a day at 11pm, you can change it. Once a day is about right.
10. Done! Now you have 2 sub spamdirs which you can use to train bogofilter as you see fit, right from your mail client.
    
    When you recieve a mail that bogofilter moves to your spam directory, but isn't actually spam, move it into the False-Positives dir in your email client. You can either run the script immediately, or wait until the cronjob triggers. It retrains bogofilter correctly, then feeds the mail back through procmail for proper classicification. If it happens again, don't ignore, put it back in the False-Positives dir and run the script again until bogofilter learns it correctly!
    
    When you recieve a spam in your inbox, move it into the False-Negatives directory. Next time the script is run, it will retrain bogofilter to recognise that mail as spam then the mail is moved into your .Spam maildir.
    
    When you feel that your bogofilter is 100% accurate (when it comes to false-positives, you don't want to lose any mail) you can edit your .procmailrc so that when bogofilter detects a mail as spam, it moves it to /dev/null (deleting it). Use with caution! But with that method, you don't even have to look at the filth!

9.2 Conclusion

Well, I think that's about it for this. If there is anything I've forgotten, don't hesitate to drop me a PM. I will give out my email over PM if needed. I may look at updating and streamlining the script soon, so check back here in a little while.

9.3 Thanks and References

Thanks a lot to beowulf for creating this awesome guide, and all the other active participants on this thread (Proteus in particular ). The community is what makes Gentoo thrive!

The sites I used researching this little project are as follows:

• MairasWiki - Anti Spam System
• Bayesian Filtering with Bogofilter and Sylpheed-Claws
• Re: [Evolution] Built-in spam filtering?
• Spam Filtering with Bogofilter

10. Spam Assassin Mail Filtering
By Proteus

I have managed to get SpamAssassin[16] 2.55-r1 - this version has bayesian filtering, too.

I implemented it in a very simple way (basically combining the .procmailrc file from this guide and the example file that comes with SA, setting up a .spam maildir and setting up cronjobs to let SA learn the difference between spam and other emails):

10.1 Emerge Spam Assassin

First thing we do is emerge the program. It has a few perl dependencies, but shouldn't take that long.

Code: Select all

root@server # emerge Mail-SpamAssassin

9.2 Edit Your .procmailrc File

Open up your .procmailrc file which is located in your user's home directory. You will need to add the following:

Code: Select all

#set up a Spam maildir where all the spam goes for teaching SA spam vs. non-spam
#and to be sure that no mail - even if detected as spam - gets lost (like when you pipe it to /dev/null)

SPAM_FOLDER= $MAILDIR/.spam/

#pipe mails through SA (this is basically from the example files
#but I use a higher limit, every mail up to 512 kB is filtered)
#spamc is the client programm for the daemonized
#version of SA (designed to keep load and overhead down)
#If you don't run SA as a daemon change "spamc" to "/usr/bin/spamassassin"
#If you do use spamc here you must add spamd to your runlevel
#like this: rc-update add spamd default

:0fw: spamassassin.lock
* < 524288
| spamc

#All mail tagged as spam (eg. with a score higher than the set threshold)
#is moved to ".spam".

:0:
* ^X-Spam-Status: Yes
$SPAM_FOLDER

#Work around procmail bug: any output on stderr will cause the "F" in
#"From" to be dropped.  This will re-add it.
#(This is taken directly from the SA example file)

:0
* ^^rom[ ]
{
  LOG="*** Dropped F off From_ header! Fixing up. "

  :0 fhw
  | sed -e '1s/^/F/'
}

Try your best to leave the rest of the file as it is described above.

10.3 Setup Spam Maildir

Code: Select all

user@server $ maildirmake -f spam ~/.maildir

10.4 Configure Spam Assassin

This can be done automatically (almost) by using a script you can find here:
http://www.yrex.com/spam/spamconfig.php
Place the config file here: /etc/mail/spamassassin

If you setup SA with bayesian scanning enabled you must teach it to detect spam first.

This is done by putting all detected spam in the .spam maildir
(when some spam gets through, put it there manually, so SA can adapt)
and then letting SA learn from those mails and from those mails (considered good) in your .inbox.

You can do this by hand or - as I did - use a cronjob to do it.

SA will only start to use the bayesian scan after learning from at least 200 mails.

If you only use SA in standard mode or just merge the "stable" version (i.e. without using ACCEPT_KEYWORD="~x86") you do not need to do the next steps. The current stable version is 2.44 as of this writing and does not contain bayesian filtering at all...
(As it seems you can add bogofilter for this task instead, but I have no clue about that, yet.)

10.5 Setup Cronjob for sa-learn (bayesian filter teaching program):

Please enter the following into your crontab. In the code block below, make sure you substitute the home directory with one more appropriate to your server. For instance, mine would read: /home/beowulf/.maildir/.spam - yours will be different.

Code: Select all

user@server $ crontab -e

#This scans for spam and for good mails every half hour.
#Set the interval (30 minutes) appropriatly for your convenience and the amount of mails you get.

*/30 * * * *    sa-learn --dir --spam /home/user/.maildir/.spam > /dev/null 2>&1
*/30 * * * *    sa-learn --dir --ham  /home/user/.maildir/ > /dev/null 2>&1

10.6 Conclusion and Testing

So, I hope I haven't left out anything but I think this is all needed to enable spam-filtering with SpamAssassin.

You can check whether or not an email has been scanned by looking at the mail headers, there should be some looking similar to those when it has been scanned:

Code: Select all

X-Spam-Status: No, hits=2.1 required=5.0
	tests=HTML_00_10,HTML_MESSAGE,NO_REAL_NAME
	version=2.55
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)

----------------------------------------

The rest of the guide is continued here:

http://forums.gentoo.org/viewtopic.php?p=570280#570280

This post has reached the maximum size allowed and I cannot keep it all on one page anymore...

[maj]

running through it now cheers - couldnt get the official one to work myself, and i dont need all the virtual mail server stuff etc, hopefully this will work for me

cheers again

EDIT: hrm, having same troubles as before - wont let me authenticate - kmail says server rejected the password, but was able to grab all the security certs and what not..

EDIT^2: and pine running on the server claims that .maildir is not a slectable folder! even though i can change into it and see mail in the new dir where fetchmail has just dumped mail from my uni mailserver!

[beowulf]

running through it now cheers - couldnt get the official one to work myself, and i dont need all the virtual mail server stuff etc, hopefully this will work for me

cheers again

EDIT: hrm, having same troubles as before - wont let me authenticate - kmail says server rejected the password, but was able to grab all the security certs and what not..

EDIT^2: and pine running on the server claims that .maildir is not a slectable folder! even though i can change into it and see mail in the new dir where fetchmail has just dumped mail from my uni mailserver!

Hey glad you gave it a shot... when using Kmail, you should be using the username/password that you use to SSH into the server. Since it auth's against PAM, it will need to match /etc/passwd.

I don't know much about pine... does it need something special to work with .maildir/ ?

In any case, let me know if the problems persist... i'll do my best to help out...

[rogue]

When I tried to run the emerge line, I got a problem with procmail trying to install ssmtp as it's MTA. This caused a conflict when postfix was trying to be emerged. I resolved it by using emerge -O, but I haven't ran through the rest of the install. Hopefully it will work

[maj]

running through it now cheers - couldnt get the official one to work myself, and i dont need all the virtual mail server stuff etc, hopefully this will work for me

cheers again

EDIT: hrm, having same troubles as before - wont let me authenticate - kmail says server rejected the password, but was able to grab all the security certs and what not..

EDIT^2: and pine running on the server claims that .maildir is not a slectable folder! even though i can change into it and see mail in the new dir where fetchmail has just dumped mail from my uni mailserver!

Hey glad you gave it a shot... when using Kmail, you should be using the username/password that you use to SSH into the server. Since it auth's against PAM, it will need to match /etc/passwd.

I don't know much about pine... does it need something special to work with .maildir/ ?

In any case, let me know if the problems persist... i'll do my best to help out...

Heh Im not that bad!! using the correct username/password

would appear so with the pine issue - emerged pine-maildir and it works fine, just need remote access

[beowulf]

rogue:
you're correct... I'm going to have to change the emerge line to make it two so that procmail is emerged after postfix is finished.

maj:
I assume you mean for receiving email, when authenticating to Courier-IMAP....

Hmm, what methods of authentication does it currently say in Kmail when you go to:

1. Settings->Configure Kmail...->Network->Receiving
2. Modify (modify the connection you have made).
3. Click on the security tab..

What is listed for both Encryption and Authentication mode?

Is the port that Kmail is trying to connect to 993?

does these commands print out the following:

Code: Select all

# cat /etc/courier-imap/authdaemonrc | grep -v ^# | grep authmodulelist=

authmodulelist="authpam"

# cat /etc/courier-imap/authdaemond | grep -v ^# | grep AUTHDAEMOND=

AUTHDAEMOND="authdaemond.plain"

If it does, and i'm pretty sure it does, try changing DEBUG_LOGIN=0 to 1 in /etc/courier-imapd/imapd then watching the logs...

[maj]

rogue:
you're correct... I'm going to have to change the emerge line to make it two so that procmail is emerged after postfix is finished.

maj:
I assume you mean for receiving email, when authenticating to Courier-IMAP....

Hmm, what methods of authentication does it currently say in Kmail when you go to:

1. Settings->Configure Kmail...->Network->Receiving
2. Modify (modify the connection you have made).
3. Click on the security tab..

What is listed for both Encryption and Authentication mode?

Is the port that Kmail is trying to connect to 993?

does these commands print out the following:

Code: Select all

# cat /etc/courier-imap/authdaemonrc | grep -v ^# | grep authmodulelist=

authmodulelist="authpam"

# cat /etc/courier-imap/authdaemond | grep -v ^# | grep AUTHDAEMOND=

AUTHDAEMOND="authdaemond.plain"

If it does, and i'm pretty sure it does, try changing DEBUG_LOGIN=0 to 1 in /etc/courier-imapd/imapd then watching the logs...

Code: Select all

gimli root # cat /etc/courier-imap/authdaemonrc | grep -v ^# | grep authmodulelist=
authmodulelist="authcustom authcram authuserdb authpgsql authpam"
gimli root # cat /etc/courier-imap/authdaemond.conf | grep -v ^# | grep AUTHDAEMOND=
AUTHDAEMOND="authdaemond.plain"

Use SSL and Clear Text are selected in KMail for Encryption/Authentication mode, and KMail is trying to connect on port 993

changed the debug_login line to 1, restarted courier-imapd-ssl attempted to check mail with kmail again and it failed, but nothing has appeared in /var/log/mail/current, /var/log/pwdfail/cuirrent or /var/log/everything/current

[beowulf]

Okay, i think this is the problem... Right now the authdaemon is using the wrong authmodulelist. Try changing authmodulelist="authpam" in teh authdaemonrc file... i made a typo in my guide... going to fix that right now...

Code: Select all

root@server # vi /etc/courier-imap/authdaemonrc

authmodulelist="authpam"

Also, it appears there's an emtpy line... i'd recommend getting rid of the line that reads only... it may cause trouble... it may not...
authmodulelist=

do the same for /etc/courier-imap/authdaemond.conf in regards to the line that reads:
AUTHDAEMOND=

Then, run the two cat | grep | grep commands again, and try to match the result that i have shown exactly...

[maj]

made the changes, results are exactly as yours, restarted the servers still wont let me in

[beowulf]

Did you restart authdaemond? Since that would be the daemon whose files we edited... I mention this only because authdaemond starts as a dependancy to courier-imapd.... here's the output:

Code: Select all

root@server # /etc/init.d/courier-imapd stop
 * Stopping courier-imapd...                                              [ ok ]
root@server # /etc/init.d/courier-imapd-ssl stop
 * Stopping courier-imapd over SSL...                                     [ ok ]
root@server # /etc/init.d/authdaemond stop
 * Stopping authdaemond.plain...                                          [ ok ]
root@server # /etc/init.d/courier-imapd start
 * Starting authdaemond.plain...                                          [ ok ]
 * Starting courier-imapd...                                              [ ok ]
root@server # /etc/init.d/courier-imapd-ssl start
 * Starting courier-imapd over SSL...                                     [ ok ]

If this doesn't work, i really don't know what went wrong... Check and see what happened in /var/log/mail.info or /var/log/mail.warn or /var/log/mail.err

Let me know what happens..

[rogue]

when i'm using kmail i'm getting a connection broken error. I have it download mail fine on the server using fetchmail..if i use the wrong password in kmail it gives me a wrong password error so i know that's working right...but when it tries to get the mail it just dies. this is what mail.info says:

May 30 02:00:54 hrothgar imapd-ssl: Connection, ip=[192.168.0.114]
May 30 02:00:54 hrothgar imapd-ssl: LOGIN: DEBUG: ip=[192.168.0.114], command=CAPABILITY
May 30 02:00:54 hrothgar imapd-ssl: LOGIN: DEBUG: ip=[192.168.0.114], command=LOGIN
May 30 02:00:54 hrothgar imapd-ssl: LOGIN: DEBUG: ip=[192.168.0.114], username=rbattle
May 30 02:00:55 hrothgar imapd-ssl: LOGIN, user=rbattle, ip=[192.168.0.114]

I'm not quite sure what's going on as the logs don't seem to show a disconnection error or anything..I believe I have it set up exactly as specified in the most recent edit of the original post.

authdaemonrc:
authmodulelist="authpam"

authdaemond.conf:
AUTHDAEMOND="authdaemond.plain"

[beowulf]

I think i might have found the file that is the culprit...

Code: Select all

# vi /etc/pam.d/imap

#%PAM-1.0
#
# $Id: system-auth.authpam,v 1.1 2001/02/02 05:42:57 mrsam Exp $
#
# Copyright 1998-2001 Double Precision, Inc.  See COPYING for
# distribution information.
#
# This is a sample authpam configuration file that uses pam_stack
# (circa linux-pam 0.72).

auth       required     pam_nologin.so
auth       required     pam_stack.so service=system-auth
account    required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

This might be it, and if it is... i have missed a step in my guide... check for the existence of that file, and if different, or holds different information, create a file that looks exactly like that...

If this is the case, let me know so i can add it to the guide and fix it up... sorry for my errors...

[maj]

I think i might have found the file that is the culprit...

Code: Select all

# vi /etc/pam.d/imap

#%PAM-1.0
#
# $Id: system-auth.authpam,v 1.1 2001/02/02 05:42:57 mrsam Exp $
#
# Copyright 1998-2001 Double Precision, Inc.  See COPYING for
# distribution information.
#
# This is a sample authpam configuration file that uses pam_stack
# (circa linux-pam 0.72).

auth       required     pam_nologin.so
auth       required     pam_stack.so service=system-auth
account    required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

This might be it, and if it is... i have missed a step in my guide... check for the existence of that file, and if different, or holds different information, create a file that looks exactly like that...

If this is the case, let me know so i can add it to the guide and fix it up... sorry for my errors...

Sweet! that did it - had the file BUT the 4 bottom lines were commented out and what was in there from the virtual mailhosting guide was in there!, works fine now

[beowulf]

Ahhh great to hear! I've edited the guide to include the pam.d information.

My appologies to you guys... i missed such an integral part of the guide. Everything should be a-okay now... again, sorry.

[maj]

Ahhh great to hear! I've edited the guide to include the pam.d information.

My appologies to you guys... i missed such an integral part of the guide. Everything should be a-okay now... again, sorry.

nothing to be sorry about - guide would have worked if i had not previously tried to do the virtual mailhosting guide

[rogue]

i found the answer to my problem..i had everything right, except i had no mail, so my .maildir was empty so it was having problems logging in. i deleted the .maildir directory and just did "maildirmake .maildir" and it worked fine

[dtessier]

Cool! I will be trying this today or tomorrow. Now that I've upgraded Evolution to 1.4 rc1 it stopped being able to get mail from my (admittedly broken) ISP's POP server, so now's a good time to give it a shot. I'll report later on how it went.

[dtessier]

Well, I'm done with the install, and things are working, though not exactly as I'd expect. First, a few comments on the procedure:

root@server # vi /etc/ssl/openssl.cnf

countryName_default = CA
stateOrProvinceName_default = Ontario
localityName_default = Toronto
0.organizationName_default = beowulf inc.

emailAddress_default = root@localhost

When I generated my certificates, it complained that I was missing commonName. I continued anyway, but later on the SMTP connection failed. I regenerated the certificates with commonName_default set to "Postmaster", and restarted postfix. That fixed the problem.

root@server # cp deomCA/cacert.pem /etc/postfix

I took me a second or two to realize that there was a typo, it's actually "demoCA"...

Finally, what does not quite work as I'd expected. I sent myself an e-mail from my Yahoo! account, which I got fine, and then replied to it, which also worked fine. However, in my Yahoo! account, the e-mail was flagged as SPAM! I checked out the headers, and I noticed this:

Code: Select all

X-Apparently-To:	 dtessier2@yahoo.com via 66.163.169.96; 31 May 2003 23:02:36 -0700 (PDT)
X-YahooFilteredBulk:	68.4.79.151
Return-Path:	<dan.tessier@cox.net>
Received:	from 68.4.79.151 (EHLO hobbes.oc.cox.net) (68.4.79.151) by mta150.mail.scd.yahoo.com with SMTP; 31 May 2003 23:02:35 -0700 (PDT)
Received:	from hobbes.oc.cox.net (hobbes.oc.cox.net [192.168.0.100]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by hobbes.oc.cox.net (Postfix) with ESMTP id 8C4B11B1EC2 for <dtessier2@yahoo.com>; Sat, 31 May 2003 23:02:44 -0700 (PDT)
Subject:	Re: test
From:	"Daniel Tessier" <dan.tessier@cox.net> | This is not spam | Add to Address Book
To:	"Daniel Tessier" <dtessier2@yahoo.com>

Yikes! 68.4.79.151 is my router's WAN IP address. It went straight to Yahoo!, and not to my ISP first. Is that how it's supposed to work? And later on, it says it's received from hobbes.oc.cox.net [192.168.0.100]!!! That's my internal LAN IP address on the server! That can't be right, can it? I double-checked my ISP's server name, as well as user name and password in /etc/postfix/saslpass, and they were correct. Any thoughts?

Well that's it for now. Thanks for all the help.

[beowulf]

Thanks for the feedback. I will correct the typo's and add the commonName_default flag of which i forgot.

Regarding the email headers showing your internal ip. I had always assumed that was expected, as postfix would write that info down no matter what. The email should go directly to Yahoo! since it would appear you aren't using your ISP's smtp server. So in that respect, it should be expected that your external IP be logged as well. However, what i did not know, nor plan on was Yahoo! treating it as spam.

I have never seen the "X-YahooFilteredBulk: 68.4.79.xxx" header before. With your permission, i would like to send you an email using the same system to determine if it's the local email system that is causing the spam warning, or someone on your IP block is a known spammer. Let me know either via pm or here and the address to use...

Thanks again for the corrections and the information regarding Yahoo!

[dtessier]

You can e-mail me at dtessier2@yahoo.com. I'll let you know what happens. I think I may also go back to using my ISP's SMTP server directly. I just figured out I don't really need the extra layer (I can always use the ISP's webmail to send e-mails while not on their network).

[beowulf]

Email's sent, let me know what happens... And yeah, the extra layer is somewhat unneeded. However, in specific situations, it can be helpful hehe Thanks again

[edit: thanks for the heads up. I have edited the original post appropriately. /]

[dtessier]

I sent you a reply. The same thing happened. I wonder if it's because there's a 192.xxx.xxx.xxx IP address in the header?

Also, in the instructions you wrote

Sending:

Email Client->Postfix[1]->ISP's SMTP Server

so I had assumed that Postfix would send e-mails to my ISP's SMTP server, not directly to the recipient's SMTP server.

[Proteus]

I sent you a reply. The same thing happened. I wonder if it's because there's a 192.xxx.xxx.xxx IP address in the header?

Also, in the instructions you wrote

Sending:

Email Client->Postfix[1]->ISP's SMTP Server

so I had assumed that Postfix would send e-mails to my ISP's SMTP server, not directly to the recipient's SMTP server.

I thought the same as dtessier.

Is it possible to reconfigure the system to work as mentioned?

[beowulf]

To use your ISP's SMTP server, you would need to put that information in the file: /etc/postfix/saslpass

For example, since my ISP is Rogers.com i would enter Rogers as my SMTP server. I will change that line to simply read SMTP Server instead of ISP....

Sorry for the confusion....

An example of my file:
smtp.xxx.xxx.net.cable.rogers.com [isp-user]:[isp-pass]

An example of a file using Yahoo! as the SMTP server:
smtp.mail.yahoo.com [yahoo-user]:[yahoo-pass]

[jordant]

beowulf:

Thanks for the excellent tutorial. I just setup a second server on my home network for this exact purpose. After following through the tutorial, most of the stuff worked great (SMTP server doesn't appear to be running at all though?). IMAP is working awesome.

Right now I've set fetchmail up to grab from two separate e-mail accounts. Do you know if I can setup (either client side or on the server) to have it reply using that identity depending on what account I'm using? Or do I have to setup two separate users on the box and have two IMAP connections with diff logins in my e-mail client?

Once again, thanks for the great tutorial.

--jordant