Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How to set up an email server with postfix/cyrus
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sat Apr 23, 2011 12:12 am    Post subject: Reply with quote

So that was the culprit. I just knew it was going to be a small thing.

I'm now ready to tackle throwing spamassassin and clam into the mix! :D

Thanks once again, dude. You rock!
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sat Apr 23, 2011 12:15 am    Post subject: Reply with quote

cach0rr0 wrote:

EDIT:
Looks like we're not alone.
http://www.mail-archive.com/info-cyrus@lists.andrew.cmu.edu/msg37017.html

That seems very broken. Easy to work around, but very broken.
Remove 'defaultdomain', and set:
Code:

servername: serverdef.audiodef.com

in /etc/imapd.conf


Is adding servername necessary? It seems to be working without it.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Sat Apr 23, 2011 12:38 am    Post subject: Reply with quote

audiodef wrote:

Is adding servername necessary? It seems to be working without it.


I've come to that same conclusion from reading more (and testing)
One thing it does do, according to that thread, is disable hostname lookups entirely. I don't know that this would be relevant at all, aside from a marginal performance improvement should someone mistakenly type in 'user' instead of 'user@domain' (which of course would be a failed login anyway).

servername seems to do what defaultdomain was *supposed* to do.
However, defaultdomain just breaks things entirely, regardless of any other settings.

Either way, safe to leave servername out from what I can gather. Just need to ensure 'defaultdomain' is not set, that seems to be all that matters.

I'm just really annoyed defaultdomain is not functioning as it's supposed to function per the doc.

Oh well - long as it works! I've just flat-out removed all reference to that in the doc, which should be sufficient since people following that guide won't even know that setting exists.

I'd say kick the tires for a day or two, but now that IMAP logins are working (and well, POP logins should work as well), it should be good to go.

At some point I would also recommend you disable the non-ssl protocols in cyrus.conf, so that you only have IMAPS and POP3S.
We've already set Postfix to only allow auth over an encryption session, so that's fine, but obv you dont want your users sending plaintext passwords over an unencrypted channel.

They should only be connecting to 993 or 995 (IMAPS/POP3S respectively), using 'SSL/TLS' and not 'STARTTLS'
For SMTP, theyll connect to plain old port 25, using STARTTLS
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sat Apr 23, 2011 3:02 am    Post subject: Reply with quote

Adding domains and users sure is easy. Add an entry in maildb, make sure the domain is listed in main.cf, go to registrar, point MX to audiodef.com - voila!

I can survive without web mail, but my gf would like it so she can check email while out on short trips where she doesn't bring her laptop. Any suggestions on setting up a web mail interface?
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Sat Apr 23, 2011 3:33 am    Post subject: Reply with quote

audiodef wrote:

I can survive without web mail, but my gf would like it so she can check email while out on short trips where she doesn't bring her laptop. Any suggestions on setting up a web mail interface?


A few cursory suggestions:

-you're primarily deciding on which interface is going to be "pretty". I've tried horde, atmail, roundcube, squirrelmail, and a handful of others.

atmail: pretty, but sometimes buggy rendering
roundcube: pretty, but clunky
squirrelmail: very plain looking, but rock solid
horde: decent looking, pretty damn solid

If you think she'll be finicky about aesthetics, I'd probably lean towards Horde. Most of the clients that are AJAX heavy seem to have issues from what I've seen.

-webmail clients typically have a horrid security record. To alleviate that risk, you should password-protect the directory where you store all of the php nonsense for your webmail client (typically done with an .htaccess file if using apache, but i recommend using mod_auth_imap - more on that in a sec). Now, instead of you having to maintain two different sets of usernames and passwords, there's actually an Apache IMAP authentication module - user browses to site, is prompted for username and password, they enter their login, and only if THAT is successful, can they ever even so much as touch the webmail app (at which point theyll have to provide the user/pass again - the first time they provide it to Apache to get access to the webmail software, the second they provide it to the webmail software itself). Basically, you want an attacker to have to get past a username/password prompt before they can attempt to exploit some known vuln in your webmail client software.

the IMAP auth module is not included in apache by default; you'll need to merge www-apache/mod_auth_imap2, edit /etc/conf.d/apache2 and add -D AUTH_IMAP, then setup something like this in /etc/apache2/modules.d/10_mod_auth_imap.conf

Code:

<IfDefine AUTH_IMAP>
LoadModule auth_imap_module modules/mod_auth_imap.so

<Directory /www/mail.whitehathouston.com/htdocs>
        AuthBasicAuthoritative Off
        Auth_IMAP_Enabled on
        AuthName "mail.whitehathouston.com"
        AuthType Basic
        Require valid-user
        Auth_IMAP_Authoritative on
        Auth_IMAP_Server renee.whitehathouston.com
        Auth_IMAP_Port 143
        Auth_IMAP_Log on
</Directory>
</IfDefine>


I've set /www/mail.whitehathouston.com/htdocs as the DocumentRoot for 'mail.whitehathouston.com' in my Apache vhost settings, and the settings above say that anyone wishing to access that directory must first provide apache with a username/pass, which it will check against the IMAP server located at renee.whitehathouston.com on port 143 (I block all but SSL connections at the firewall, however i leave a non-SSL instance of cyrus listening on 143 so that servers inside my firewall - including my web server - can connect plaintext to Cyrus. This should be fine for you too, since it's not an external user connecting on 143, but rather Apache, which is located on the server itself)

-just as a general rule, not simply for webmail clients, but for the web in general, any of your sites that accept user input at all, should ideally be hidden behind such a password protection scheme, whether the password backends to IMAP, or a standard htpaswd file. Doing otherwise opens you up for exploitation, and means you have to be damn vigilant about keeping whichever webapp updated. When I have the option, I disable file uploads in PHP (breaks webmail clients if you want to attach files to emails), I build sites in pure HTML/CSS, and dont build them to accept user input. For any sites that say, backend to a database, or that have dynamically generated content (basically, if they generate content conditionally based upon the URL the user provides), I make sure anyone browsing to such a site must first provide apache with a username/password in order to get to the actual site content. For many commercial sites this isn't feasible, but for most personal/informational sites that are a bit smaller in nature, this is definitely workable, and a small price to pay for peace of mind.

-in general, I steer away from installing webapps through portage. It just ends up being easier to manually download the source, untar it to whichever directory, and go from there.

-youll want to set up a CNAME record in godaddy's DNS, something like 'mail.herdomain.com', and have it point to audiodef.com.

-I only have this site served via SSL. This is done by setting up a virtualhost entry for her domain ONLY in 00_default_ssl_vhost.conf, and not in 00_default_vhost.conf. Anyone browsing to http://mail.herdomain.com would just end up at the default vhost - audiodef.com - whereas anyone browsing to https://mail.herdomain.com, would be routed to that vhost, at which point theyd have to give apache their login details, then provide the webmail software their login details.

And on that note, I have only one working light bulb left in the entire house. Just bought a new stash, time to replace them and clean this pig sty
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Sat Apr 23, 2011 3:34 am    Post subject: Reply with quote

good lord, i didnt think id be typing that much when i started that post.

whoops! probably didnt come out as particularly organized thought, but hopefully you can decrypt that (this is why for long docs, i *still* go through and do an outline first!)
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sat Apr 23, 2011 2:27 pm    Post subject: Reply with quote

Well, it was a very well organised and lucid thought! :)

Yeah, I manually install webapps, too. I got tired of looking for where portage put them, and then they're not even the latest. Anything that goes in htdocs I just DIY.

I'm going to go over your explanation above some time next week. Gf's domains on Godaddy expire soon, so I'm going to get her site on my vhost and find a new registrar. I hear Godaddy bites like a bulldog, so she may have to extend service with them and then switch so there's no gap in her service/business. Hopefully, I'll find a good registrar that doesn't bullshit their customers.

Hm, maybe you need a supply of candles, for next time you run out of light bulbs. :P

That reminds me, I need to dig up that info on building a small solar battery. Thought of it last year and am hoping to build it this year to provide partial power to my studio. Should be fun.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sat Apr 23, 2011 5:13 pm    Post subject: Reply with quote

One thing I've noticed is much faster performance, and I assume this is at least in part because I'm running mail for only two people, instead of thousands. 8)
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Sun Apr 24, 2011 2:45 am    Post subject: Reply with quote

audiodef wrote:
One thing I've noticed is much faster performance, and I assume this is at least in part because I'm running mail for only two people, instead of thousands. 8)


Even though I haven't the slightest need for scalability with my little home setup, part of the reason I've opted for the combination of Cyrus and Postfix, is because if the exceptional scalability.

It will certainly be speedy and snappy with a small number of users. But because of the way, especially Cyrus, indexes mailboxes, stores metadata, etc, the speed sticks around even when your needs grow. What you have is truly an "enterprise class" mail setup.

We've really only scratched the surface of what these two are capable of, but both are flexible and scalable enough I'd doube any need to look elsewhere in the near future.

Just waiting to see if they decide to include the autocreate functionality in Cyrus 2.5, which is where it is on the current roadmap (2.4 is already out, so it wont make it in there)
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
AaronPPC
Guru
Guru


Joined: 29 May 2005
Posts: 522
Location: Tucson, AZ

PostPosted: Sun Apr 24, 2011 8:05 am    Post subject: Reply with quote

I love to recognize excellent threads and this is definitely one that riveted me. I think it will help many people. I feel motivated to get off my ass and build that email server I want to build. :)
_________________
--Aaron
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Sun Apr 24, 2011 2:43 pm    Post subject: Reply with quote

AaronPPC wrote:
I love to recognize excellent threads and this is definitely one that riveted me. I think it will help many people. I feel motivated to get off my ass and build that email server I want to build. :)


All the credit goes to cach0rr0, but I'm glad I could be a part of shaping an excellent mail server guide. 8)
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Mon Apr 25, 2011 3:00 pm    Post subject: Reply with quote

I first looked up Horde. It looked like overkill, so I checked out Squirrelmail and decided on that one. My gf is a pragmatist when it comes to tech: she just wants it to work well, so she'll have no trouble with a simple interface like this.

I just did a basic setup. Tomorrow, I'll go through your mini guide on configuring security. 8)
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Mon Apr 25, 2011 4:06 pm    Post subject: Reply with quote

OK, set up with auth_imap. It was insanely easy. Makes me wonder why it's not standard to have at least this level of security. Then again, I wonder about a lot of things. :P
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Mon Apr 25, 2011 4:43 pm    Post subject: Reply with quote

cach0rr0 wrote:

-I only have this site served via SSL. This is done by setting up a virtualhost entry for her domain ONLY in 00_default_ssl_vhost.conf, and not in 00_default_vhost.conf. Anyone browsing to http://mail.herdomain.com would just end up at the default vhost - audiodef.com - whereas anyone browsing to https://mail.herdomain.com, would be routed to that vhost, at which point theyd have to give apache their login details, then provide the webmail software their login details.



If I have the mail interface on audiodef.com/squirrelmail (I don't, but thought it safer to use a fake example in a public venue), how would I make that url (just that subdir, not the TLD) only accessible via SSL?

EDIT: Come to think of it, is there really any reason why I should not just make audiodef.com an SSL site? I don't sell anything and there's no sensitive information, but it is dynamically generated.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Mon Apr 25, 2011 7:54 pm    Post subject: Reply with quote

I finally got around to following the Content Filtering section. My gf asked how she would know whether the RBL's (which she doesn't know about, I just told her I set some filtering config options) are picking up email she wants to get by mistake, and I realised I don't know the answer to that.

I used the config you specified for RBL's, verbatim.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Mon Apr 25, 2011 8:19 pm    Post subject: Reply with quote

audiodef wrote:
I finally got around to following the Content Filtering section. My gf asked how she would know whether the RBL's (which she doesn't know about, I just told her I set some filtering config options) are picking up email she wants to get by mistake, and I realised I don't know the answer to that.

By examining the logs to see what is getting rejected.

I must admit that my life is so much simpler now that I've moved myself and my clients to Google Apps. Keeping up with anti-spam tweaks can get to be a full-time job. One could spend hours for weeks (maybe months) on end and still not come close to what Google/Postini provides out-of-the-box (unbelievably low false positives or false negatives). Google Apps also provides DKIM support, even for the free version.
My Postfix/Cyrus installation is basically used for archives now.
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Mon Apr 25, 2011 11:42 pm    Post subject: Reply with quote

audiodef wrote:

If I have the mail interface on audiodef.com/squirrelmail (I don't, but thought it safer to use a fake example in a public venue), how would I make that url (just that subdir, not the TLD) only accessible via SSL?


you can forcibly redirect to https via mod_rewrite, but this is among the reasons i think it's easier to just have a secondary vhost with its own alias (e.g. mail.domain.com)

audiodef wrote:

EDIT: Come to think of it, is there really any reason why I should not just make audiodef.com an SSL site? I don't sell anything and there's no sensitive information, but it is dynamically generated.


Certificate errors. Your users may not feel like dealing with certificate errors, as well it's a small increase in resource usage doing SSL. Avoiding cert errors means paying for a cert realistically (I mean yeah, people can just accept your cert as trusted, but that only works with regular users of the site).

You may prune out a fraction of the blind "spray and pray" type attacks out there by shifting to SSL, but it's a nominal gain in terms of protecting input. What you're more likely to stop, is the handful of bots that can be used for one attack or another, as it's not always the case an author will want to account for the extra bloat/overhead of including the necessary SSL libs in their bot.

audiodef wrote:
My gf asked how she would know whether the RBL's (which she doesn't know about, I just told her I set some filtering config options) are picking up email she wants to get by mistake, and I realised I don't know the answer to that.


Basically, what darkphader said. Rejections will be very visibly logged in mail.log - and among the things i do like about postfix, when troubleshooting mail, dear god how important good logging is.

If you emerge logwatch, included in its summary email will be a list of hosts that get rejected by the RBL.

Of course, I don't like having to review that, so I've chosen conservative but effective RBL's. I could, for example, go with zen.spamhaus.org - Spamhaus Zen includes blacklist data from many sources, including the CBL list that I use, and many others. It's a very acceptably accurate list on the whole, but it isn't one I trust to reject my e-mail at the perimeter (for example, it includes the 'PBL', which is a list of dynamic/residential IP addresses - a lot of ADSL hosts in South Africa end up stuck on the PBL for some reason, and they do a poor job of keeping those .za hosts from being wrongly blacklisted)

CBL on the other hand, check out its listing policy - you basically have to be identified for absolute certain as an infected host, and one that's generating spam. It also has a no-nonsense delisting policy, so it's quick and easy for any false positives to get themselves delisted.

More's the point, since you're rejecting these messages rather than accepting and quarantining, the sender *knows* their message didn't get through, whereas quarantine leaves them in silence.

Now, giving the user the ability to *see* any of their incoming email that's been quarantined would be great, but to date I haven't found any decent freebie anti-spam system that actually has a decent and functional quarantine management interface. Quarantine is fine if you have it, but somewhat of a quiet black hole if you don't. And unless your quarantine management system includes something like quarantine summary digest e-mails, you'll have a difficult time training your users to actually make use of their ability to retrieve messages from quarantine.

If you do at some point go the route of adding extra functionality (e.g. amavis/spamassassin/clamd), instead of doing quarantine for spam (viruses obv you want to delete, flat-out), even though this is something I'd never recommend in the "commercial" filtering world, I'd say you'd be better off simply tagging messages identified as spam, instead of quarantine, then showing your users how to set up rules on their mail client to automatically move messages tagged as spam to a junk folder.

The risk there, is that the lines between spam and malware are pretty damn grey nowadays. Pretty much everything you see is a so-called "blended threat".

Anyway, I'm already off on a tangent. I guess some of the point, is that there's a different strategy when doing "best practices" for freebie solutions, VS commercial solutions.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Mon Apr 25, 2011 11:57 pm    Post subject: Reply with quote

darkphader wrote:

I must admit that my life is so much simpler now that I've moved myself and my clients to Google Apps.


It's a solid enough service, but it has some fundamental problems that IMHO make it completely impractical for commercial use (something a handful of my clients have found out the hard way)

-support: you have pretty well zero chance of getting ahold of a live body. This simply does not fly with a paid service
-routing: every single message is treated by Google's mail systems as an outbound message. So even if you're sending something intra-office, it goes out, then comes back in.

darkphader wrote:

Keeping up with anti-spam tweaks can get to be a full-time job. One could spend hours for weeks (maybe months) on end and still not come close to what Google/Postini provides out-of-the-box (unbelievably low false positives or false negatives).


Depends. That actually *was* part of my full-time job for a commercial vendor for a good while, and because SpamAssassin had a nice large chunk of the same functionality available to me in terms of rules that could be written and checks that could be done, I could pretty closely mirror rules between the two (though, obviously, I couldn't really share the custom SA rules I'd written). Header tokenization - surprisingly effective, and I shall say no more.

If you have the resources to do the research, and have a large volume of spam from which to craft your rules, you can certainly get the same rate of accuracy with a freebie filter. Where the commercial vendors win out, more than anything, the management tools, and the amount that's just flat turnkey.

If I ever get rich, I plan on writing heaps of shit, and contracting out some of the guys I've worked with to help writing heaps of shit, to just flat hand over to the open source community. Of course, first I have to get rich.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Tue Apr 26, 2011 12:34 am    Post subject: Reply with quote

cach0rr0 wrote:

you can forcibly redirect to https via mod_rewrite, but this is among the reasons i think it's easier to just have a secondary vhost with its own alias (e.g. mail.domain.com)


OK. I'll work on setting it up that way.

Here's to hoping you get rich! 8)
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Tue Apr 26, 2011 2:11 am    Post subject: Reply with quote

cach0rr0 wrote:
It's a solid enough service, but it has some fundamental problems that IMHO make it completely impractical for commercial use (something a handful of my clients have found out the hard way)

-support: you have pretty well zero chance of getting ahold of a live body. This simply does not fly with a paid service
-routing: every single message is treated by Google's mail systems as an outbound message. So even if you're sending something intra-office, it goes out, then comes back in.

Don't know about the "live body" issue, so far email support has been fine, haven't needed to get to the phone support level.
As for the routing, not seen as an issue by any one of my clients, even those still using Exchange for an in house app, the benefit of having all messages (outbound and inbound) available from any place far outweighs the issue. Even in this case here, are we not looking at a non-locally hosted IMAP store? Not so different from "the cloud". Except we're missing the calendaring, the Google Docs equivalent, and all of the other Google services.

cach0rr0 wrote:
If you have the resources to do the research, and have a large volume of spam from which to craft your rules, you can certainly get the same rate of accuracy with a freebie filter. Where the commercial vendors win out, more than anything, the management tools, and the amount that's just flat turnkey.

Oh yes, the "resources", my point exactly. It is, at the least, a time consuming job trying to find the balance, tune the whitelist, tune the blacklist, etc. Not to mention that many screw up the basics: making sure the smtp helo name matches the A record and the PTR is in agreement, a proper SPF record and then adding DKIM to help insure delivery.

As to the the commercial vendors it's a bit more than that, it's the economies of scale. When x% of a gazillion gmail/postini/google apps users flag messages as spam you get the benefit when you're part of the system. Sometimes size does matter.

My summation is:
Running email servers is quite an experience, if you do it proper you will learn a lot but spend a lot of time doing so. When you're ready to get a life let the big boys take over and move on to learning something new.

Chris
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Tue Apr 26, 2011 8:09 am    Post subject: Reply with quote

darkphader wrote:

As for the routing, not seen as an issue by any one of my clients, even those still using Exchange for an in house app, the benefit of having all messages (outbound and inbound) available from any place far outweighs the issue.


Hosted Exchange gives that same benefit. And as far as the routing goes, we helped a ~3000 user shop migrate over to Google Apps. The migration was painless and easy once we plucked the data we needed out of their directory server. All looked great. Until 3 or 4 days go by, and users start getting 30+ minute delays for internal e-mail - major problem for your average joe who's just sending an e-mail 10 feet away to someone, "hey, let's grab lunch". Ended up being a major PITA, and worse still, while this was going on there was no live body at google to get in touch with. Nevermind for a moment, when you're having e-mail delivery issues, it's not particularly great to have your sole means of communication with support be, well, e-mail.

That much worse if you're using a different SaaS provider, and not Postini. Now, in order to e-mail someone who's 10 feet away, it goes out through ~3 google systems, hits your SaaS provider - who realistically, because of the very real danger of routing loops shouldn't ever be handling internal e-mail - loops back to google, goes through 3 more different servers, and finally makes it to its destination.

We had to pull a German client off of Google Apps for that very reason - they were using a German data security company, and had special compliance requirements as many operations in Germany and elsewhere in Europe do, this just became an untenable compliance/DLP nightmare.

darkphader wrote:

Oh yes, the "resources", my point exactly. It is, at the least, a time consuming job trying to find the balance, tune the whitelist, tune the blacklist, etc.


Interactive whitelisting/blacklisting is realistically a waste of time. A good commercial content filter is going to have some permutation of what we called "adaptive whitelisting", and well, blacklisting in and of itself has zero merit.

darkphader wrote:

Not to mention that many screw up the basics: making sure the smtp helo name matches the A record and the PTR is in agreement, a proper SPF record and then adding DKIM to help insure delivery.


Never considered improper A/PTR/HELO a particularly good spam indicator for that very reason. I don't know of many vendors that do, most filters (both commercial ones *and* the likes of SpamAssassin actually) use that at most a contributor to overall spam evaluation, and score it in the 5-10% range (of $triggerlevel)

darkphader wrote:

As to the the commercial vendors it's a bit more than that, it's the economies of scale. When x% of a gazillion gmail/postini/google apps users flag messages as spam you get the benefit when you're part of the system. Sometimes size does matter.


True enough having a larger sample set is useful, but what you'll find is:
-users often consider opt-in marketing emails to be spam, when they aren't. Users can't really be trusted to be able to identify genuine spam from solicited junk
-reporting a message that's genuine spam is meaningless if the information a vendor extracts from the message isn't up to snuff
-very few commercial vendors still use a classical bayesian system. We spent more time than anything reverse-engineering spam bots/trojans, watching their templates get downloaded realtime, and updating signatures accordingly. After a while we were able to create processes that would extract tokens from a message header in an existing/known piece of spam, look for those same tokens in an inbound mail system, and automagically update filters on the fly. You find more of this, and behavioural analysis (works beautifully for blended threats), in filters nowadays. Any other magic they purport to use is marketing tripe - makes me laugh seeing some of the shit vendors use as selling points, I'm thinking to myself "all you did was this and this and this, you coded it in a day, and it's something that's been around since the last century"


Is it useful? yes. absolutely. but the utility shouldn't be overstated. Volume itself critical? Heavens yes. But not so much on user submissions. I'd say easily 85% of the user submissions I've seen between the last company and this one, are ones we simply discard.

At any rate, like I said, not to discount the importance of scale, but pretty well every major filter out there - free or otherwise - has the same physical tools available, and pretty well any modern content filter out there is going to fetch you a 95-99% hit rate.

What sells, normally, is an intuitive policy engine, intuitive management interface, then whatever other miscellaneous IT buzzword the manager buys into, or where someone resides in the damn Gartner "magic quadrant".


darkphader wrote:

Running email servers is quite an experience, if you do it proper you will learn a lot but spend a lot of time doing so. When you're ready to get a life let the big boys take over and move on to learning something new.


There are other big boys besides Google, and frankly their development on Postini by Google has been horribly lackluster since the acquisition. It was sad to see really - though, granted, if you're using Google Apps, it pretty much makes using anyone BUT Postini horribly impractical.

Postini is fine as a relatively basic spam filter. As a content filter on the whole, though, that's about all it does.
No real attachment type detection (extensions are meaningless), no recursive unpacking of attachments, no enforcement of S/MIME or PGP, no real DLP scanning, no configurable certificate validation on TLS-enabled connections, no ability to require TLSv1 Method, no ability to require TLS for specific domains, no ability to differentiate between outbound/inbound/internal scanning policy, no ability to do custom policy-based routing. All of which are hugely significant problems in compliance audits, depending on the locale. Functionality available from pretty much any major non-SaaS vendor, available from a handful of other SaaS vendors, but not Postini. Google has pretty much acquired Postini, and not bothered advancing it any further beyond a "spam filter", leaving it pretty well stuck back in ~2004. Its sole real benefit? "It's in the cloud!"

...and of course, in my case, I've been one of those other "big boys" (vendors) for the last decade and change. Not that this little Postfix/Cyrus setup is "big boy" material, but with the problems I've seen from Postini I'm not quite ready to concede victory to Google as anything at all resembling a de facto standard. They let Postini die a slow painful death, couldn't even be bothered buying up a web filtering company (they have an OEM deal with..zscaler is it?), but people will still buy it up in droves because of the promised availability (at the expense of control and security), and because it has the Google name on it. It has its own Applesque cult following.

Having said that, I'm generally a fan of Google. But they did to Postini what the product black hole Symantec seems to do with every company *it* acquires, just shoving it onto a shelf somewhere.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Tue Apr 26, 2011 1:21 pm    Post subject: Reply with quote

It's a real shame that capitalistic behaviour kills off techno-intellectual advancement that way. I'm not saying here that "capitalism sucks" (although that's my personal opinion) - I'm actually looking at this objectively and thinking that if more of these companies were more interested in effectiveness and pragmatism and cared more about quality than quantity (not to mention obsession with The Almighty Dollar At All Costs), I might actually be a Google fan myself. They have some good ideas, but they keep doing things here and there that make me say "uh... no, thanks".

In other words, I just wish that quality rather than making money by whatever - sometimes apparently arbitrary/random/qualitatively meaningless - means were more important to more people. It's a good business approach. If your shit works well and your products are well-liked by people who truly understand the technology, the money will follow (if you want it to). I also mean to say that I would prefer to have the general population a little more educated about computers so that companies like Microsoft don't need to try to make "all things for all people" software in order to make a buck. Goodness, Windows fails so hard and fast sometimes that I'm frequently reminded of why my Windows partition is 1. not used for anything serious (just games) and 2. not allowed to connect to the internet. In fact, I haven't even installed the driver for my NIC.

Well, I figured I'd join the conversational melee. :P
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Tue Apr 26, 2011 3:34 pm    Post subject: Reply with quote

cach0rr0 wrote:
it's easier to just have a secondary vhost with its own alias (e.g. mail.domain.com)



I thought I knew how to do this, but I'm afraid I need to swallow my pride and ask.

What I've done so far:

Put squirrelmail in /var/www/mail/htdocs (htdocs contains is the main squirrelmail dir, there's no subdir in htdocs except for squirrelmail's various subdirs).

Played with settings in /etc/apache2/vhosts.d/00_default_vhost.conf.

Create mail.conf in /etc/apache2/vhosts.d like so:

Code:

<VirtualHost server_name:80>
ServerAdmin my_email
DocumentRoot "/where/it/is"

<Directory "/where/it/is">
    SSLRequireSSL
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

</VirtualHost>


Went to godaddy and set up forwarding for mail.audiodef.com to https://audiodef.com/(subdir).

I'm really not working from knowledge, clearly. What should I do?
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN


Last edited by audiodef on Tue Apr 26, 2011 8:03 pm; edited 1 time in total
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Tue Apr 26, 2011 8:03 pm    Post subject: Reply with quote

I figured it out.

mail.conf:
Code:

<VirtualHost full_server_name:443>

ServerAdmin myemail
DocumentRoot "/where/it/is"
ServerName myservername
SSLEngine on
SSLCertificateFile /etc/ssl/apache2/server.crt
SSLCertificateKeyFile /etc/ssl/apache2/server.key

<Directory "/where/it/is">
    SSLRequireSSL
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

</VirtualHost>


I just needed to add

Code:

ServerName myservername
SSLEngine on
SSLCertificateFile /etc/ssl/apache2/server.crt
SSLCertificateKeyFile /etc/ssl/apache2/server.key

_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Wed Apr 27, 2011 1:32 am    Post subject: Reply with quote

Now I have a problem trying to set up a second virtual host with SSL. Copying my first virtual host file and changing the params, I get:

Code:

[Tue Apr 26 18:30:39 2011] [warn] VirtualHost (subdomain2).audiodef.com:443 overlaps with VirtualHost (subdomain1).audiodef.com:443, the first has precedence, perhaps you need a NameVirtualHost directive


What do I need to do?
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 5 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum