Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
exim/dovecot upgrade to sieve filtering and SRS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
msst
Apprentice
Apprentice


Joined: 07 Jun 2011
Posts: 213

PostPosted: Sun Dec 29, 2013 9:48 pm    Post subject: exim/dovecot upgrade to sieve filtering and SRS Reply with quote

I just used some quiet Christmas days to do some overdue upgrades on my small mailserver. I had already planned to do this some time but shied back as some of these things are not really well documented or with contradicting descriptions etc.

The two things I upgraded and which I will describe here for (hopefully) the benefits of other (and if you find something to improve shoot away as well) are:

1) Upgrading the system to use sieve / managesieve for mailfiltering (previously manually edited .forward files)
2) Enabling SRS workarounds for those servers that are radical enough to reject on SPF fail

Both is connected as 2) is needed to have mailforwarding work well with these providers. Fortunately most are not so brain-dead but e.g. gmail is.

1) Upgrading to sieve filtering (server side filtering manageable by each end user):
-------------------------------------------------------------------------------------------------

The first thing to realize is that this requires two components: sieve filtering support and managesieve so that you do not have to edit files manually. There are severals ways to get this working and the biggest work was actually getting a reasonable overview and decide what is easiest and has least overhead. These things are not very well documented so that took a bit for me.

I am using exim as mailserver and dovecot for IMAP. Both packages support sieve. Only dovecot supports also an internal managesieve server. If you want to use exclusively exim then you need pysieved to manage your filter files. As I am not in favour of using python for servers and as dovecot has all the encryption code and has a good security reputation I decided to go that way. Getting the support is easily done by adding sieve and managesieve as use flags and recompile.

Configuration of dovecot is also relatively easy if you have already the split configuration in conf.d in use. I still had the old monolithic conf, so the biggest work was putting that into then conf.d files.
You need to edit 20-managesieve.conf and 90-sieve.conf to enable both services (easy and well described). You also need to enable LDA to use dovecot as local delivery agent, as it otherwise cannot do the filtering. This is configured in 15-lda.conf. This is the easy part. Restart dovecot and voila.

A bit more work is getting exim to use it:

a) Add a new transport in exims transport section:
Code:
dovecot_delivery:
  driver = pipe
  command = /usr/libexec/dovecot/dovecot-lda -f $sender_address
  message_prefix =
  message_suffix =
  log_output
  delivery_date_add
  envelope_to_add
  return_path_add
  group = mail
  temp_errors = 64 : 69 : 70: 71 : 72 : 73 : 74 : 75 : 78


b) Change your localuser router in the router section. Here order matters! Put it at the same place:
Code:
localuser:
  driver = accept
  check_local_user
#  transport = local_delivery
##This using dovecots delivery method instead of the above
  transport = dovecot_delivery
  cannot_route_message = Unknown user


There is no need to change the redirect routers around - the sieve filtering happens at delivery now and for all local delivery dovecot is used now. You may or may not disable your .forward redirect router.

This can be tested (and used) e.g. from Thunderbird after downloading the sieve filter plugin. Use version sieve 0.2.3.d with Thunderbid 24.2 - at least this works here. Up to here you have a fully functional system already.

Now at this point there is one slighthly more complicated 2nd option, which I think helps decreasing the overhead. Dovecot has a pretty good and complete support for sieve filtering, but it has its overhead and makes exim dependant on dovecot for delivery. Exim has sieve support as well, just a bit more limited (but sufficient for most cases) and less well documented. There are also a few glitches that need to be worked around. Here is the configuration needed:

Quote:
sievefilter:
debug_print = "R: sieve filter in exim for $local_part@$domain"
driver = redirect
check_local_user
require_files = $home/.dovecot.sieve
no_verify
no_expn
check_ancestor
sieve_vacation_directory = $home/.vacation
allow_filter
data = "#Sieve filter\n${sg{${readfile{$home/.dovecot.sieve}}}{\r}{}}"
file_transport = sieve_delivery
pipe_transport = address_pipe
reply_transport = address_reply


The important and non-intuitive line is the data = which tweaks the .dovecot.sieve file to start with the required (for exim) "#Sieve filter" line and fixes a CRLF issue. If you use dovecots standard config to place these filters in the user home directory it will work out of the box. If you wanna place them in a central place you may need to tweak file permissions.

You need to also provide a sieve_delivery transport, as the standard address_file transport won't work:
Code:
sieve_delivery:
  driver = appendfile
  file = ${if eq{$address_file}{inbox} \
           {/var/mail/$local_part/inbox} \
           {${if eq{${substr_0_1:$address_file}}{/} {$address_file} \
              {/var/mail/$local_part/$address_file} \
           }} \
         }
  delivery_date_add
  envelope_to_add
  return_path_add
  group = mail
  #mode = 0660  #Don't be too picky


Notice I have my mail files under /var/mail/$username. The if sequence checks if an absolute path is given and otherwise uses this line. It also places "inbox" in the right place.


Last edited by msst on Sun Dec 29, 2013 10:04 pm; edited 2 times in total
Back to top
View user's profile Send private message
msst
Apprentice
Apprentice


Joined: 07 Jun 2011
Posts: 213

PostPosted: Sun Dec 29, 2013 9:49 pm    Post subject: Reply with quote

2) Enabling SRS workarounds for those servers that are radical enough to reject on SPF fail
----------------------------------------------------------------------------------------------------------

Background (short):
Let's not get too political, but SRS (sender rewriting scheme) is in the end a horrible hack to allow people to forward their mail to providers that reject mail based on a failed SPF verification only. This is nonsense and most providers don't do it, but as gmail (google) is among those who do not care about breaking their users forwarding (possibly deliberately with second thoughts!), you probably have to implement SRS and bypass that.
Otherwise the SPF check will check if your server is allowed to send mails for <whatever domain the incoming mail is from>. Some providers publish a SPF record and your server in between is most certainly not listed. If they publish their SPF record with an -all flag, google will refuse your mail forward. So you need to rewrite the envelope sender address to bypass that.
Notice that I consider SPF next to useless for spam filtering so I neither publish an SPF record for my server nor check SPF before rewriting. This is against the SRS standard, but I do not care as my own spam filtering is way more efficient than gmail with SPF checks. This means if you want SPF validation you need to add that and have a bit more complex setup! Also such a setup is only reasonable with working spam filtering.

a) Enabling SRS in exim:
This requires setting the srs, mysql and exiscan-acl (the latter two were already added so I did not check). Then reemerge.

b) Configuring exim to use it - first router
This is the harder part and lousily documented. I am using a setup with mysql as I anyway use that for user management.

Add this at the start of your exim.conf and adapt the fields in <>:
Code:

#You may already have this, otherwise add it
hide mysql_servers = localhost/<table>/<user>/<password>

# SRS hack
hide srs_secrets = <long random>
srs_maxage = 8
srs_hashlength = 10


Don't worry for the short maxage, its mainly needed for bounces. Use a good hashlength and random to reduce risks. Someone breaking this may use your server as open relay. Beware.
Then add the following routers right at the top of your router list:
Quote:
srs_forwarding_router:
driver = redirect

# Only email to these listed TARGET domains get rewritten
domains = /etc/exim/srs-domains

# Don't rewrite if it's a bounce, or from one of our own addresses.
senders = ! : ! *@+local_domains

# Is the sender a domain using spf?
condition = ${if match{${lookup dnsdb{defer_never,txt=$sender_address_domain}}}{v=spf1}}

caseful_local_part
srs = forward
srs_dbinsert = ${lookup mysql{servers=localhost; INSERT INTO `srs` \
(`key`, `address`, `time`) VALUES \
('${srs_db_key}', '${srs_db_address}', NOW())}}
srs_alias = <your_senderdomain>
data = ${quote_local_part:$local_part}@$domain
headers_add = X-SRS-Rewritten: By $primary_hostname for $sender_address.


I am only using SRS for the domains listed in /etc/exim/srs-domains (you need to create and maintain that file!). These are the providers rejecting on SPF fail that I care to support. I am deliberately not doing SRS for every domain that in principle supports it. This is consistent with only using SRS where really needed.
This is the redirect router that does most of the work. Most will work with this alone. Set your own domain in srs_alias line!

c) Configuring exim to use it - second router for bounces:
This is needed below the above router to backroute bounces. Well configured mailservers should not bounce much, but we support it for a few days:
Quote:

srs_router:
driver = redirect

#This must only match a target address with SRS in local part
condition = ${if match{$local_part}{^SRS[0-9]\=.*}}
caseful_local_part
srs = reverseandforward
srs_dbinsert = ${lookup mysql{servers=localhost; INSERT INTO `srs` \
(`key`, `address`, `time`) VALUES \
('${srs_db_key}', '${srs_db_address}', NOW())}}
srs_dbselect = ${lookup mysql{servers=localhost; SELECT `address` FROM `srs` WHERE \
`key` = '${srs_db_key}' AND \
`time` > SUBDATE(NOW(), INTERVAL 8 day) LIMIT 1}}
data = ${srs_recipient}
headers_add = X-SRS-Forwarded: By $primary_hostname for $sender_address.


This will forward bouces of the SRS-rewritten addresses. Don't worry, users will normally not see all this magic.

d) Create the table 'srs' in your mysql mail database:
Quote:
CREATE TABLE IF NOT EXISTS `srs` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`key` tinytext NOT NULL,
`address` tinytext NOT NULL,
`time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='For sender rewriting scheme tracking' AUTO_INCREMENT=1 ;

You may also create a MYSQL event to purge old entries in that table but that is mostly cosmetic.

------------------------------------

Phew, that was quite some work. So far this configuration does what I want and has passed all my tests. Suggestions welcome. This is a bit experimental as there is almost no documentation that fits for gentoo systems.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum