Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
spambayes spam filter replacement?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
jagdpanther
Guru
Guru


Joined: 22 Nov 2003
Posts: 593

PostPosted: Tue Mar 17, 2020 2:50 pm    Post subject: spambayes spam filter replacement? Reply with quote

I have used spambayes for years and after training it works very well. Unfortunately it is being pulled from Gentoo:

Code:
!!! The following installed packages are masked:
- mail-filter/spambayes-1.1_beta2::gentoo (masked by: package.mask)
/usr/portage/profiles/package.mask:
# Michał Górny <mgorny@gentoo.org> (2020-03-17)
# Unmaintained.  Current Gentoo version is from 2015, last upstream
# release in 2017.  Python 2 only.
# Removal in 30 days.  Bug #712950.


What are some good e-mail anti-spam replacements for spambayes that are "trainable" and don't depend on external white or black lists?
(I currently use ~/.forward to send all of my e-mail to procmail which sends the messages through clamav and spambayes.)
Back to top
View user's profile Send private message
jagdpanther
Guru
Guru


Joined: 22 Nov 2003
Posts: 593

PostPosted: Tue Mar 17, 2020 10:00 pm    Post subject: Reply with quote

I found the following:
https://www.linux-magazine.com/Online/Features/Choosing-a-Spam-Filter

I think I will try bogofilter to replace spambayes as I already have a really large spam and ham folder to train bogofilter with.
Back to top
View user's profile Send private message
jagdpanther
Guru
Guru


Joined: 22 Nov 2003
Posts: 593

PostPosted: Fri Mar 20, 2020 9:34 pm    Post subject: Reply with quote

bogofilter does an excellent job of filtering out spam, when you have your own definition of spam ASSUMING you have a large collection of both example spam messages and not-spam (ham) messages.
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 899

PostPosted: Sat Mar 21, 2020 1:03 pm    Post subject: Reply with quote

For what it's worth, I use a stack of four Bayesian analyzers. If one of them finds SPAM, the mail is sent to PURGATORY. If two of them find spam, the mail is classified as SPAM. Three of the four are in portage.

bogofilter
spamprobe
bmf

The fourth is QSF

Of the four, bogofilter has the ability to be unsure, that is, to have a middle result. Not spam, not ham, unsure. The way I have bogofilter tuned here, which I think is its default, it is the most sensitive filter of the four.

I'm not recommending running a stack of Bayesian filters. I did it to learn what was common and different among them. The log below is Feb 2020, a typical month. These guys have been in training since June 2008.

Code:
   1  ${FORMAIL} -A "X-Bayes: BMF/Bogo Concur" >> ${SPAM}
  39  ${FORMAIL} -A "X-Bayes: BMF/Probe/Bogo Concur" >> ${SPAM}
   3  ${FORMAIL} -A "X-Bayes: BMF/Probe+Bogo Unsure" >> ${SPAM}
   1  ${FORMAIL} -A "X-Bayes: BMF/Probe Concur" >> ${SPAM}
  25  ${FORMAIL} -A "X-Bayes: Bogo Only" >> ${PURGATORY}
  36  ${FORMAIL} -A "X-Bayes: Bogo/Probe Concur" >> ${SPAM}
  43  ${FORMAIL} -A "X-Bayes: Bogo Unsure" >> ${PURGATORY}
  19  ${FORMAIL} -A "X-Bayes: Probe+Bogo Unsure" >> ${SPAM}
   3  ${FORMAIL} -A "X-Bayes: Probe Only" >> ${PURGATORY}
   7  ${FORMAIL} -A "X-Bayes: QSF/BMF/Bogo Concur" >> ${SPAM}
   2  ${FORMAIL} -A "X-Bayes: QSF/BMF+Bogo Unsure" >> ${SPAM}
 450  ${FORMAIL} -A "X-Bayes: QSF/BMF/Probe/Bogo Concur" >> ${SPA
  20  ${FORMAIL} -A "X-Bayes: QSF/BMF/Probe+Bogo Unsure" >> ${SPA
  57  ${FORMAIL} -A "X-Bayes: QSF/Bogo Concur" >> ${SPAM}
   7  ${FORMAIL} -A "X-Bayes: QSF+Bogo Unsure" >> ${SPAM}
   4  ${FORMAIL} -A "X-Bayes: QSF Only" >> ${PURGATORY}
 203  ${FORMAIL} -A "X-Bayes: QSF/Probe/Bogo Concur" >> ${SPAM}
  10  ${FORMAIL} -A "X-Bayes: QSF/Probe+Bogo Unsure" >> ${SPAM}
Back to top
View user's profile Send private message
jagdpanther
Guru
Guru


Joined: 22 Nov 2003
Posts: 593

PostPosted: Sat Mar 21, 2020 4:51 pm    Post subject: Reply with quote

cboldt:

Thanks for the post.

I have bogofilter configured to enable the middle result: unsure. Anything that shows up in my Unsure folder will be manually moved to either trainHam or trainSpam folders. (I also will move any Ham or Spam that is incorrectly classified: rare) Then I have a little cron job that runs once a night, determine if the modify time stamp of either the trainHam mail file or trainSpam mail file is newer than ~/.bogofilter/wordlist.db. If so, I delete ~/.bogofilter/wordlist.db and have bogofilter recreate it from the updated trainHam and trainSpam mail folders.

I currently have
ham_cutoff = 0.15
spam_cutoff= 0.65
in by ~/.bogofilter.cf file.

I would try bogotune but I don't have enough non-spam words in wordlist.db.
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 899

PostPosted: Sat Mar 21, 2020 9:19 pm    Post subject: Reply with quote

I do similar for training. I manually review PURGATORY, and false positives are run through a "not spam" script that corrects which ever of the filters thought the mail was spam. The PURGATORY mail that is spam, I just leave it. Once a week cron sends the whole load of them, half a dozen, ten maybe, through a "these are spam" routine.

The cronjob saves eight weeks of SPAM and PURGATORY just in case something important triggered a false positive. That inventory is also useful to train from scratch if I ever decide to do that.
Back to top
View user's profile Send private message
UberLord
Retired Dev
Retired Dev


Joined: 18 Sep 2003
Posts: 6799
Location: Blighty

PostPosted: Sun Mar 22, 2020 10:51 am    Post subject: Reply with quote

I use rspamd

It's pretty good and as it's written in C with LUA scripting it's very light weight and pretty fast compared to the Perl and Python heavy weights that other systems favour.
Light weight is the key as my main server only has 8G ram.
_________________
Use dhcpcd for all your automated network configuration needs
Use dhcpcd-ui (GTK+/Qt) as your System Tray Network tool
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum