Email System For The Home Network - Version 2.1

[beowulf]

miha can you please post the output of this command:

Code: Select all

root@server # grep -v "^#" /etc/postfix/main.cf | sed '/./,/^$/!d'

Also, what error does Sylpheed give? the error message you posted, what is that from (not the one with fetchmail) and how did it trigger?

You said it works fine, but you asked what is up with the errors you outputted. Is it that you cannot send email? You cannot authenticate to send email? You cannot connect to the server?

A little bit more information of what you did and at what point you received those errors are needed.... I haven't seen those errors before so I can't just at a glance know what is going on...

JHuizingh - Ahh... well let me know how it turns out... It never even occured to me that a colon would be used.... hehe... if it doesn't work, the only suggestion I have is to perhaps use a backslash "\" before it... but who knows... In any case, I hope you keep me/us posted....

[wheelspin]

First, I'd like to thank Beowulf and the rest who have contributed to this guide. Its awesome.

I think I have most everything working properly, but I can't send email. It seems to send it but every thing comes back with things similar to this in the log file:

Code: Select all

Dec 12 10:17:36 vette postfix/smtp[27726]: 0D4E53B6F: to=<me@mydomain.com>, relay=smtp.comcast.net[216.148.227.125], delay=1691, status=bounced (host smtp.comcast.net[216.148.227.125] said: 550 [PERMFAIL] mydomain.com requires valid sender (in reply to RCPT TO command))
Dec 12 10:17:36 vette postfix/cleanup[12815]: BF0C83B84: message-id=<20031212151736.BF0C83B84@vette.home.mydomain.com>

This happens when I send email to all the accounts I have. Obviously its something to do with the receiving email system. my current main.cf has relayhost set to my ISP smtp server. I tried getting sasl to work but got this message:

Code: Select all

Dec 12 09:49:25 vette postfix/smtp[4306]: warning: SASL authentication failure: No worthy mechs found
Dec 12 09:49:25 vette postfix/smtp[4306]: 0D4E53B6F: to=<bill@mydomain.com>, relay=mydomain.com[205.243.144.68], delay=0, status=deferred (Authentication failed: cannot SASL authenticate to server mydomain.com[205.243.144.68]: no mechanism available)

I would like to be able to send valid email to other sites and have them accept it. Is there something I can look at to figure out why this is happening? Any help is cool.

Thanks
Wheelspin

[beowulf]

Hey Wheelspin, glad it's almost working.... the first error is a result of the second error hehe

setting relayhost in main.cf will only work if your ISP's SMTP server requires NO authentication.... if you have a username/pass you must use SASL for this to work and unset the relayhost option by commenting it out.... Now the "no mech found" issue...

Please follow this code block and see if the output matches:

Code: Select all

root@Chimera(/etc/) # ls -l /usr/lib/sasl2/smtpd.conf
lrwxrwxrwx    1 root     root           29 Oct  6 00:47 /usr/lib/sasl2/smtpd.conf -> ../../../etc/sasl2/smtpd.conf
root@Chimera(/etc) # cat /etc/sasl2/smtpd.conf
pwcheck_method: sasldb
root@Chimera(/etc) # /etc/init.d/saslauthd status
 * status:  stopped
root@Chimera(/etc) # ls -l /etc/postfix/saslpass
-rw-------    1 root     root          196 Oct  6 01:47 /etc/postfix/saslpass
root@Chimera(/etc) # sasldblistusers2
beowulf@FQDN: userPassword
root@Chimera(/etc/) # cat /etc/postfix/main.cf | grep sasl
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpass
smtp_sasl_security_options = noanonymous,noplaintext
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
broken_sasl_auth_clients = yes
smtpd_client_restrictions = permit_sasl_authenticated,reject

Assuming your output matches mine for the most part.... Check the contents of the file /etc/postfix/saslpass and make sure your ISP's information is there with the correct settings in the proper format.

Another possibility, check the file /etc/sasl2/sasldb2 and make sure that it's only readable by user postfix, no group or world rights at all...

If after checking all that and you're still experiencing problems, reply back with the output of this command, preferably wrapped in [code} tags...

Code: Select all

root@server # grep -v "^#" /etc/postfix/main.cf | sed '/./,/^$/!d'

[wheelspin]

After all this time you are still answering questions about this guide. That is what I love about Gentoo and Linux. Coming from the Windows world and the larger Linux distro's its a breath of fresh air. I really appreciate your help.

On to the task at hand.

The output you asked for:

Code: Select all

queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix
mydomain = home.billrucker.com
inet_interfaces = $myhostname, localhost
mydestination = $myhostname, localhost.$mydomain

unknown_local_recipient_reject_code = 450
mynetworks = 192.168.0.0/24, 127.0.0.0/8
relay_domains = $mydestination
alias_maps = hash:/etc/mail/aliases
alias_database = hash:/etc/mail/aliases
home_mailbox = .maildir/
mailbox_command = /usr/bin/procmail -a $DOMAIN

local_destination_concurrency_limit = 2
default_destination_concurrency_limit = 10
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail
newaliases_path = /usr/bin/newaliases
mailq_path = /usr/bin/mailq
setgid_group = postdrop
manpage_directory = /usr/share/man
sample_directory = /etc/postfix/sample
readme_directory = /usr/share/doc/postfix-2.0.11

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpass
smtp_sasl_security_options = noanonymous,noplaintext

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
broken_sasl_auth_clients = yes
smtpd_client_restrictions = permit_sasl_authenicated, reject
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/postfix/newreq.pem
smtpd_tls_cert_file = /etc/postfix/newcert.pem
smtpd_tls_CAfile = /etc/postfix/cacert.pem
smtpd_tls_loglevel = 3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_daemon_random_source = dev:/dev/urandom
tls_random_source = dev:/dev/urandom

Also, my regular ISP doesn't require authentication, but every email I send gets returned with the same message I posted before when I send through that system. I also have a web host that provides smtp services and that server is the one I'd like to use. But its also the one that doesn't allow my authentication to work. I can authenticate to it and send fine using KMail. The settings I have configured for that are using TLS and Login.

I checked all the other items you mention and they all match perfectly. I can log into the IMAP server from another workstation on my network and get all the emails that have been received with no problem.

Thanks again,

Wheelspin

Keep your tires on the trail.

[beowulf]

Just to help me work out what we're doing in my head.

You have the option of 2 SMTP servers, one provided by your ISP, the other is a free web service. Neither of these SMTP servers require you to use SSL/TLS.

The ISP SMTP server requires no authentication. If you wish to attempt to use this, you must remove this from your main.cf, preferably commenting them out.

Code: Select all

smtp_sasl_auth_enable = yes 
smtp_sasl_password_maps = hash:/etc/postfix/saslpass 
smtp_sasl_security_options = noanonymous,noplaintext

Then uncomment the relay_host option you previously set. The error produced (1st in your post) means your client is not sending a proper RCP TO line, meaning the sender email address is not valid.... I'm curious what your client is? Is the sender email address provided by your ISP. For example, if my isp is isp.com but the email address I attempt to use is "beowulf@anotherisp.com", I will receive that error.

The Web SMTP server requires authentication and does not use SSL/TLS. When using this SMTP server, with the main.cf (as it is posted right now) produces this error:

Code: Select all

Dec 12 09:49:25 vette postfix/smtp[4306]: warning: SASL authentication failure: No worthy mechs found 
Dec 12 09:49:25 vette postfix/smtp[4306]: 0D4E53B6F: to=<bill@mydomain.com>, relay=mydomain.com[205.243.144.68], delay=0, status=deferred (Authentication failed: cannot SASL authenticate to server mydomain.com[205.243.144.68]: no mechanism available)

Could you tell me what web service this is so that I may test it out and find out why SASL can't use their SMTP server?

Hopefully we can get this all sorted out soon as you're really close and your main.cf file looks correct....

[wheelspin]

Just to help me work out what we're doing in my head.

You have the option of 2 SMTP servers, one provided by your ISP, the other is a free web service. Neither of these SMTP servers require you to use SSL/TLS.

That is correct. The one for my web host allows encryption and requires authentication.

The error produced (1st in your post) means your client is not sending a proper RCP TO line, meaning the sender email address is not valid.... I'm curious what your client is?

I am using mutt. And I'm no expert at that either. Its perfectly possible that it is not configured correctly. I will work through that and report back.

Could you tell me what web service this is so that I may test it out and find out why SASL can't use their SMTP server?

The server is running at billrucker.com. Its not really a free service I do have to pay for it.

Hopefully we can get this all sorted out soon as you're really close and your main.cf file looks correct....

That was my thought and I was just hoping somebody who knows more than I do about this stuff might be able to help. Not to say that that is a difficult request, cuz I don't know that much.

In the meantime of anything you may be able to find, I'll try and get mutt set up right, if I can figure out how and see how that works.

Thanks

Wheelspin

[wheelspin]

The address that is being used as the "From" is bill@home.mydomain.com when it should be bill@mydomain.com. myhome.domain.com will not resolve to a valid email address because my server is set up as a subdomain of a valid domain for which I am not running the dns for.

There must be a way to force the MAIL FROM to a valid email to get to Postfix and hence be sent to the remote domain. Maybe I'm wrong, but I can't find anything that helps on the web. I tried a couple of different things with Mutt that didn't make any difference.

I wonder if it would make any difference if I were using completely invalid domain at home? It seems to me I'd have the same problem so I'm not inclined to move that direction right now.

Wheelspin

[beowulf]

I'm at a loss as to why SASL can't auth..... as for editing the From field in Mutt, I found this page that may help you:

http://www.mutt.org/doc/manual/manual-2.html#ss2.4

You can probably set a default From email address in your ~/.muttrc file.... Something along the lines of "set From = a@b.c".

This should fix the problem... hope this helps

[Fragbeestje]

Tnx for this great tutorial!

Almost everything works great

I added some rules similar to the one below to my .procmailrc file :

Code: Select all

# filter fragbeestje mail to fragbeestje folder
:0
* ^To:.fragbeestje@myispmail\.com
.fragbeestje/

The .fragbeestje dir exists under my ~/.maildir/ folder but it doesn't show up in my IMAP account when I check it with my email client.
I looked into the specific subfolders and noticed that they did not contain the following files:
drwx------ 2 cannibal users 4096 Dec 8 16:56 courierimapkeywords
-rw-r--r-- 1 cannibal users 693 Dec 8 16:56 courierimapuiddb
Another inbox subfolder, which I created from my email client, does contain these courierimap files and this folder shows in my email client.

To see if it would lead to any changes,I removed the dirs, and recreated them with

Code: Select all

maildirmake -f fragbeestje ~/.maildir/

However the subfolders still don't contain the courierimap subfile and folder and do not show up in my Imap account.

Am I forgetting something? I read through the thread a few times now and I can't figure out where I went wrong.
I probably overlooked something, as it seems to work for most people by just adding the rules to the .procmailrc file.


**** FIXED ****

Had to add these subfolders to the file

"~/.maildir/courierimapsubscribed"

and restart the mailserver.

[wheelspin]

Well, it looks like I solved the issue with the rejected email because of domain name. It ended up being a Postfix config issue. Not what I would call a mistake, just the way Postfix works. The guide suggests setting up the $mydestination as the default for Postfix which is cool. This appears to be used as the default domain for all mail leaving the server. So the "RCPT TO" sent to the remote MTA is user@host.$mydomain in Postfix speak. If the domain resolves but the host (in my case it failed on the host) doesn't resolve, it is assumed to be spam and is rejected.

The way to override this setting is to use the $myorigin setting in the main.cf for Postfix. I made this the real top level domain that does actually resolve to a real MX record in DNS and the problems disappeared. I can even send mail to Yahoo users. YaaHooo!!

Wheelspin

[beowulf]

Fragbeestje: Good to hear it's all working out for you

Well, it looks like I solved the issue with the rejected email because of domain name. It ended up being a Postfix config issue. Not what I would call a mistake, just the way Postfix works. The guide suggests setting up the $mydestination as the default for Postfix which is cool. This appears to be used as the default domain for all mail leaving the server. So the "RCPT TO" sent to the remote MTA is user@host.$mydomain in Postfix speak. If the domain resolves but the host (in my case it failed on the host) doesn't resolve, it is assumed to be spam and is rejected.

The way to override this setting is to use the $myorigin setting in the main.cf for Postfix. I made this the real top level domain that does actually resolve to a real MX record in DNS and the problems disappeared. I can even send mail to Yahoo users. YaaHooo!!

Wheelspin

Great news! I'll add this to the next update.... it's been bugging me just a little, but not enough to look into it.... I assume however you have an MX record pointing to your server.... So perhaps I'll make a note about if you have an MX record, you should do this "...". Thanks for sharing your fix with the rest of us

[grover]

Great guide beowulf I've had my mailserver running perfectly for a while now.

Gkrellm's mail checker now has support for SSL so it can monitor your inbox without having to setup stunnel.

http://web.wt.net/~billw/gkrellm/Changelog2

[beowulf]

Guide updated with the addition of a few troubleshooting options, misc fixes/clarifications and of course Outlook Express 6 configuration.

Thanks to all who have replied in this thread, I've tried my best to include notes where your problems were found and what the solutions were.

[GeoffOs]

Great Document, managed to get myself up and running nicely now, except for getting mail into the .spam dir for SpamAssassin to learn from as known spam.
How would I do this?
I have a collection of about 3000 spam messages that exist in a mail folder visible in squirrelmail, but I am uncertain as to the mail dir structure.

Help please.

[GeoffOs]

ooops, duplicate posting, sry

[Proteus]

I am not using Squirrelmail but I suppose it uses the standard .maildir format for its email storage?

If so it should be easy to just point sa-learn to the dir where the spams are stored.

Like this:

Code: Select all

sa-learn --dir --showdots --spam /.maildir/.spam/

This way sa-learn should automatically search the spam directory in the maildir and all its standard subdirectories (cur/tmp/new).

Tell me if this works. If it does not it may help to post the error message here.[/quote]

[numerodix]

I've tried to follow the guide 100%, v1.5 and I'm having trouble sending mail. I'm using Outlook Express as a client and whether I turn off SSL (in which case there's no error but the mail never reaches the receiver) or keep it on (454 TLS not available due to temporary reason', Port: 25, Secure(SSL): Yes, Server Error: 454, Error Number: 0x800CCC7F) it's no good.

Code: Select all

# cat /etc/postfix/main.cf
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix

#inet_interfaces = $myhostname, localhost
inet_interfaces = localhost
mydestination = $myhostname, localhost.$mydomain
unknown_local_recipient_reject_code = 450
mynetworks = 10.0.1.0/24, 127.0.0.0/8
relay_domains = $mydestination
alias_maps = hash:/etc/mail/aliases
alias_database = hash:/etc/mail/aliases
home_mailbox = .maildir/
mailbox_command = /usr/bin/procmail -a $DOMAIN
local_destination_concurrency_limit = 2
default_destination_concurrency_limit = 10
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail
newaliases_path = /usr/bin/newaliases
mailq_path = /usr/bin/mailq
setgid_group = postdrop
manpage_directory = /usr/share/man
sample_directory = /etc/postfix/sample
readme_directory = /usr/share/doc/postfix-2.0.11

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpass
smtp_sasl_security_options = noanonymous,noplaintext

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
broken_sasl_auth_clients = yes
smtpd_client_restrictions = permit_sasl_authenticated, reject


smtpd_use_tls = yes
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/postfix/newreq.pem
smtpd_tls_cert_file = /etc/postfix/newcert.pem
smtpd_tls_CAfile = /etc/postfix/cacert.pem
smtpd_tls_loglevel = 3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_daemon_random_source = dev:/dev/urandom
tls_random_source = dev:/dev/urandom

Also, postfix will start but gives an error while stopping. The log prints:

Code: Select all

Jan  5 21:46:42 [postfix/postfix-script] starting the Postfix mail system
Jan  5 21:46:42 [postfix/master] fatal: bind 127.0.0.1 port 25: Address already in use

Netstat shows port 25 locally is listening for connections. It's also available remotely according to nmap.

I would test the certs but I don't know how. They "look" correct.

What I want ultimately is a system to fetch mail from multiple accounts, sort it into folders, then send mail (ie. replies) based on where the message is located, thus selecting the correct reply-to address, possibly relaying through the respective smtp servers but that's not as important. Is that even possible you think?

[beowulf]

Hey numerodix, your conf file looks good...

I'm also assuming you've setup OE correctly.... there is one problem that I've found when I googled your error string. Are you running Norton Anti-Virus? Apparently the outgoing email scan is interfering with SMTP over TLS and can be fixed by telling Norton not to scan outgoing email (but to keep scanning incoming)....

Here's the search link I used... Of course if that isn't the case, let me know and I'll look further into it.

As to your question, yes this is possible, but it's dependant on the client. I use Kmail for email, and I have it setup so that any email that is delivered to the ".personal/" maildir is replied using my email address (or Identity) that is closer to my real name. I'm pretty sure something like this can be set up, but I don't have access to my windows box anymore as it was re-wiped and back to serving an intranet site for my lan....

[wilddev]

Hey beowulf, inspired by your great example, I've posted my short guide to setting up cyrus-imapd with postfix and spamassassin here. Keep up the good work dude!

[numerodix]

Thanks, that did help get rid of the error. I now get this instead:
The connection to the server has failed. Account: 'kramer-alex', Server: 'kramer', Protocol: SMTP, Port: 25, Secure(SSL): Yes, Socket Error: 10061, Error Number: 0x800CCC0E

I was thinking maybe I could somehow get procmail to rewrite the headers to include a to: [the relevant account], for instance when it's from a mailing list. Then email clients would most likely try to use that address for outgoing mail, no?

EDIT: I know kmail usually works the best of all my email clients so I tried it with the setup. When sending a message, I get this:

Code: Select all

Jan  6 19:15:28 [postfix/smtpd] TLS connection established from frasier.matusiak.lan[10.0.1.11]: TLSv1 with cipher RC4-MD5 (128/128 bits)
Jan  6 19:15:28 [postfix/smtpd] warning: SASL authentication problem: unable to open Berkeley db /etc/sasl2/sasldb2: Permission denied
                - Last output repeated twice -
Jan  6 19:15:28 [postfix/smtpd] warning: SASL authentication failure: no secret in database
Jan  6 19:15:28 [postfix/smtpd] warning: frasier.matusiak.lan[10.0.1.11]: SASL DIGEST-MD5 authentication failed
Jan  6 19:15:29 [postfix/smtpd] disconnect from frasier.matusiak.lan[10.0.1.11] 

Code: Select all

ls -l /etc/sasl2/sasldb2 
-rw-r-----    1 root     root        12288 Jan  5 21:33 /etc/sasl2/sasldb2

EDIT: turned out to be a dns problem. I now have kmail working (yey!). I don't know what I'm going to do about that Windows error but at least it works at all.

EDIT: I learnt that the server wants to use digest-md5 for authentication. Both kmail and evolution are fine with it but Outlook Express doesn't have any options for authentication, so that's probably why it doesn't work.

[puke]

This thread is great!

Is there any way I can restrict the IP addresses of clients who connect via courier-imap-ssl, other than using iptables?

[beowulf]

wilddev - Nice guide!

numerodix

The reason that error is showing up in your logs is because postfix can't read the sasldb2 file..... The fix is this:

Code: Select all

root@server # /bin/chown postfix /etc/sasl2/sasldb2
root@server # /bin/chmod 600 /etc/sasl2/sasldb2

This should allow Outlook to authenticate since it was complaining that it couldn't read/open the file....

Outlook Express 6 supports MD5-Cram if you enable "Secure Password Authentication".... Is this option enabled in your client? or what version of OE are you using?

puke - Unfortunately I don't think you can restrict based on IP from the server. IP tables will allow you to easily do this however by dropping port 993.

hope this helps guys.

[numerodix]

numerodix

The reason that error is showing up in your logs is because postfix can't read the sasldb2 file..... The fix is this:

Code: Select all

root@server # /bin/chown postfix /etc/sasl2/sasldb2
root@server # /bin/chmod 600 /etc/sasl2/sasldb2

This should allow Outlook to authenticate since it was complaining that it couldn't read/open the file....

Outlook Express 6 supports MD5-Cram if you enable "Secure Password Authentication".... Is this option enabled in your client? or what version of OE are you using?

I'm not sure that is what it was, I set the permissions the first time. But I think the hostname/dns name was set wrong and fixing solved the problem.

My settings correspond exactly to those you published and to my great surprise today I was able to send mail I know that I didn't change anything from last night so this behavior I find quite puzzling. But satisfying all the same.

Thanks so much for all your help, beowulf!!! I tried this guide once before and I didn't get anywhere, I gave it another try now and viola, determination conquers.

[beowulf]

Great to hear you stuck with it!

[numerodix]

The only oddity currently in existence is Outlook Express asking me to pass a username, password and domain (?) for every session, required to send mail. Obviously, the login info is stored in the settings so I don't know what this is about but once it gets the values, sending mail works fine.

I also picked up a tip on the forum about adding "always_bcc = " to the postfix main.cf, thus enabling me to sort my sent mail in foders corresponding to the account it was sent from.

When you mentioned virus scanning, you said one should direct the mail to the mta at port 25, any pointers on how to do that?