Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
watch-logs : IP ban function using syslog-ng and iptables
View unanswered posts
View posts from last 24 hours

Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message

Joined: 24 Aug 2005
Posts: 914

PostPosted: Tue May 26, 2015 12:12 pm    Post subject: watch-logs : IP ban function using syslog-ng and iptables Reply with quote

After running sshguard and fail2ban for some time, I decided to reinvent the wheel and compose an alternative. sshguard and fail2ban work fine, so it's hard for me to ascribe any particular motivation to this effort, beyond a desire to learn some new bash techniques and to have a smaller (but not necessarily more efficient), tailored solution to my desire for automating firewall maintenance.

Most of the intrusion attempts here have been against sshd, and knockd (properly setup) is more secure than reacting to a failed intrusion attempt. At the same time, the knockd method is not as informative as to which IPs are attempting intrusion. So, there was a bit of curiosity factor involved too. The script started out as just watching for sshd and firewall activity, before I figured out generic coding that allows easily expanding the scope of monitoring application logging activity.

syslog-ng facilitates monitoring a variety of applications by providing a way to direct selected logging activity to a program, thereby eliminating a need to watch separate log files. syslog-ng.conf is configured to filter interesting messages, and send them to the "watch-logs" script.

Independent of the script, my firewall has a few chains that reduce the number of hits against sshd and smtpd. These might be sufficient for some systems, but I wanted a tool that imposed time-limited bans that would survive flushing and re-establishing the firewall.

# using iptables to regulate hits against $SSHD_PORT
# on third attempt in four minutes, placed into 15 minute timeout
# timeout is restarted for attempts made during timeout

iptables -N sshd-drop
iptables -A sshd-drop -j LOG --log-prefix "$LOG_SSHD_DENY "
iptables -A sshd-drop -m recent --name SSHD-deny --set -j DROP

iptables -N sshd-persist
iptables -A sshd-persist -m limit --limit 1/hour --limit-burst 1 -j LOG --log-prefix "$LOG_SSHD_PERSIST "
iptables -A sshd-persist -j DROP

iptables -N sshd-scan
iptables -A sshd-scan -m recent --name SSHD-deny --update --reap --seconds 900 -j sshd-persist
iptables -A sshd-scan -m recent --name SSHD      --update --reap --seconds 240 --hitcount 3 -j sshd-drop
iptables -A sshd-scan -m recent --name SSHD --set -j LOG --log-prefix "$LOG_SSHD_ATTEMPT "
iptables -A sshd-scan -j ACCEPT

iptables -A INPUT -p tcp --dport $SSHD_PORT -m conntrack --ctstate NEW -j sshd-scan

# using iptables to regulate hits against ports 25 and 465
# on third attempt in an hour, placed into 24 hour timeout
# timeout is restarted for attempts made during timeout

iptables -N smtp-drop
iptables -A smtp-drop -j LOG --log-prefix "$LOG_SMTP_DENY "
iptables -A smtp-drop -m recent --name SMTP-deny --set -j DROP

iptables -N smtp-scan
iptables -A smtp-scan -m recent --name SMTP-deny --update --reap --seconds 86400 -j DROP
iptables -A smtp-scan -m recent --name SMTP      --update --reap --seconds  3600 --hitcount 3 -j smtp-drop
iptables -A smtp-scan -m recent --name SMTP --set -j LOG --log-prefix "$LOG_SMTP_ATTEMPT "
iptables -A smtp-scan -j ACCEPT

iptables -A INPUT -p tcp --dport 25  -m conntrack --ctstate NEW -j smtp-scan
iptables -A INPUT -p tcp --dport 465 -m conntrack --ctstate NEW -j smtp-scan

That code isn't necessary for the watch-logs script to function, but it does provide a firewall layer that reduces the rate of intrusion attempts reaching the applications, and being logged.

Now the watch-logs script. I don't think any intrusion detection and reaction system is particularly easy to setup, and this is no exception. The user has to configure at least syslog-ng and the firewall to take advantage of the script; and may have to configure watch-logs. I tried to make that easy for both initial setup and for expanding the scope of view. I don't believe this approach scales up well, the script probably becomes overloaded somewhere around 50-200 hits per second. Another issue is that the script is not immune to being spoofed by locally-crafted logging.

#! /bin/bash
# /usr/local/sbin/watch-logs

version=0.1                                             # 25 May 2015

# This script is designed to be invoked by syslog-ng
# Running the script from a shell merely gives brief instructions

# This script uses input from iptables, smtpd, imap, and sshd logging, all
# delivered directly to this script by syslog-ng.  This script tests that input.
# Too many "Failed " or "authentication failed" reports result in banning
# the offending IP for a selectable period of time

# Expired bans can be removed by sending a USR1 or USR2 signal to this script
# It is suggested to `pkill -SIGUSR1 watch-logs` from /etc/cron.daily/logrotate
# and when rebuilding firewall, e.g. from /etc/iptables/establish-iptables

# Default BAN rule threshold is 10 hits in 1day, ban duration 3weeks
#         BAN rule thresholds established by this script are ...
#         ("smtpd" and "imap" activity is grouped under type "mail")
# -----------   -----   --------------  ----------------
# Type of hit   Limit      Hit-life       Ban Duration
# -----------   -----   --------------  ----------------
# firewall       10     1day (default)  3weeks (default)
# sshd login      3     30minutes       5days
# mail login      4     12hours         3weeks (default)

# Firewall --recent thresholds set in /etc/iptables/establish-iptables
# -----------   -----   --------------  ----------------
#    Port       Limit      Hit-life       Ban Duration
# -----------   -----   --------------  ----------------
# sshd port       3     4minutes        15minutes
# smtp ports      3     1hour           1day

# Use `iptables-save | less` to confirm firewall function.
# Banned ip addresses should appear in "watch-banned" rule chain

# Copyright (C) 2015 Chuck Seyboldt <>
# This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License

# ------------   GET CONFIGURATION  -------------
# N.B. configuration is part of main routine, because associative arrays
# created in a subroutine aren't directly available outside that subroutine

typeset -A max_hits hit_life ban_time hit_history
shopt -s extglob

cfg=${cfg:=/etc/conf.d/iptables}                        # All settings, including adding new patterns,
[ -r $cfg ] && . $cfg                                   # can be modified from optional config file

iptables=${IPTABLES:=/sbin/iptables}                    # To adapt to iptables6, someday
ipregex=${WATCH_LOGS_IPREGEX:='([0-9]{1,3}\.){3}[0-9]{1,3}'}            # IPv4 pseudo-regex
cutter=$(which cutter)                                  # Use `cutter` program, if it exists
verbose=${WATCH_LOGS_VERBOSE:=bans,xt_recent}           # Select info to log when processing ban_log
                                                        # Options are "bans" "xt_recent" "yes" and "all"
log=${WATCH_LOGS_LOGFILEW:=/var/log/iptables/watch-logs.log}            # Realtime output
ban_log=${WATCH_LOGS_BANLOG:=/var/log/iptables/watch-logs-ban.log}      # Permanant record
ban_file=${WATCH_LOGS_BANFILE:=/etc/iptables/watch-logs.banned}         # List of unexpired bans
hit_limit=${WATCH_LOGS_MAXHITS:=10}                     # This default number of hits ...
hit_expire=${WATCH_LOGS_HITLIFE:=1day}                  # in this default amount of time ...
ban_expire=${WATCH_LOGS_BANTIME:=3weeks}                # results in a default ban this long
whitelist=${WATCH_LOGS_WHITELIST:=}                     # Better to whitelist by firewall, than here
drop_chain=${WATCH_LOGS_BANNED_IP_CHAIN:=watch-banned}  # New iptables rule chain to hold bans
from_chain=${BLOCKED_IP_CHAIN:=blocked-ip}              # Preexisting banned-ip iptables rule chain

# Incoming activity is categorized by "type" (e.g., firewall, sshd, mail)
# $watch_log_* variable names establish "types"
# $watch_log_* variable contents establish the search/watch patterns
# $watch_log_* variable contents must be bash extended pattern matching expressions

# The $LOG_* variable names are shared with /etc/iptables/establish-iptables script
# in order to coordinate iptables --log-prefix settings with pattern matching in this script


# Default ban threshold variables can be over-ridden from the config file
# Default thresholds set above - 10 hits per 1day results in 3week ban
# Re-start script to apply ban threshold changes
# (pkill watch-logs, syslog-ng will restart watch-logs)



# Adding a new category is done by defining at least one $watch_log_* variable
# The following would add a category of "type" newtype with two matching hit strings
# Ban threshold and duration can also be configured.  The example "newtype"
# is configured with 3 hits in 4hours resulting in a ban of 1month duration

# watch_log_newtype1='failed.*user.*tries'
# watch_log_newtype2='IP.address.*does.not.match'
# max_hits[newtype]=3
# hit_life[newtype]=4hours
# ban_time[newtype]=1month

# syslog-ng must send matching log_action to this script in order for the script to be useful
# A typical addition to syslog-ng.conf follows this form:
# filter f_newtype_warn { program(application)
#                         and message("failed.*user.*tries");
#                         or  message("IP.address.*does.not.match"); };
# log { source(src); filter(f_newtype_warn); destination(watch_logs); };

# --------------------  NOW SOME SUBROUTNES  ---------------------
# -----------------  EXIT AND SIGNAL HANDLING  -------------------

exit_trap() {
printf "%(%s)T $HOSTNAME %s[%5s]: Exiting\n" -1 ${0##*/} $$ >> $log

print_instructions() {
# Test to insure this script is invoked by syslog-ng
# If not, exit with installation message

if [ "$PARENT_NAME" != "syslog-ng" ]; then
  echo "
Hello!  ${0##*/} is designed to be invoked from and by syslog-ng.
Put the following in your syslog-ng.conf file ...

  destination watch_logs   { program(\"$0\" ts_format(unix)); };
  filter f_iptables        { facility(kern) and message(\"IN=.*OUT=.*SRC=\"); };
  filter f_imap_warn       { program(dovecot) and message(\"no auth attempts \"); };
  filter f_mail_warn       { facility(mail) and level(warn); };
  filter f_sshd_warn       { program(sshd)  and message(\"Failed \"); };
  log { source(src); filter(f_iptables);  destination(watch_logs); };
  log { source(src); filter(f_imap_warn); destination(watch-logs); };
  log { source(src); filter(f_mail_warn); destination(watch_logs); };
  log { source(src); filter(f_sshd_warn); destination(watch_logs); };

Put the following rules near the top of your iptables firewall ...

  iptables -N $from_chain
  iptables -A INPUT -j $from_chain

... then restart (or SIGHUP) syslog-ng.
Exiting ${0##*/} version $version now.  Bye!
  exit 1

# -----------------  INITIALIZE FIREWALL  -------------------
build_iptables_rules() {

# Create a new chain ($drop_chain) [${WATCH_LOGS_BANNED_IP_CHAIN:=watch-banned}]
# for holding iptables rules that will drop banned IPs
# $drop_chain is intended to be unique to this script - there is no need
# to create it or refer to it when building the firewall

$iptables -w -N $drop_chain > /dev/null 2>&1
$iptables -w -F $drop_chain
$iptables -w -A $drop_chain -j RETURN

# The first rule in $from_chain [${BLOCKED_IP_CHAIN:=blocked-ip}] is a jump to
# the $drop_chain list of banned IPs.  The "blocked-ip" chain is tested to
# make sure it RETURNs, because INPUT should jump to "blocked-ip" early
# If there is no jump from INPUT to $from_chain (which is possible if there is
# no firewall to begin with, or firewall has no "blocked-ip" chain) then make one,
# insert as 5th rule after priority-packets, bad-flags, whitelist, and portknock

$iptables -w -N $from_chain > /dev/null 2>&1
$iptables -C $from_chain -j $drop_chain > /dev/null 2>&1 || \
  $iptables -w -I $from_chain -j $drop_chain
$iptables -C $from_chain -j RETURN > /dev/null 2>&1 || \
  $iptables -w -A $from_chain -j RETURN
$iptables -C INPUT -j $from_chain > /dev/null 2>&1 || \
  $iptables -w -I INPUT 5 -j $from_chain

# -----------------  PROCESS BAN FILE  -------------------

# This routine has multiple functions:
#  - create variable $BANNED_LIST in memory to avoid repeating bans
#  - trim the banlist $ban_file of expired bans
#  - log unexpired bans and expiration dates to the watch-logs $log file

if [ -r $ban_file ]; then
  mapfile <$ban_file ban_file_array
  printf "%(%c)T $HOSTNAME : $ban_file trimmed by %s[%5s]\n" -1 ${0##*/} $$ > $ban_file
  printf "%(%s)T $HOSTNAME %s[%5s]: Processing banlist $ban_file\n" -1 ${0##*/} $$ >> $log
  nowtime=$(printf "%(%s)T" -1)
  for (( i = 0 ; i < ${#ban_file_array[@]} ; i++ ))
    line=( ${ban_file_array[$i]} )
    if [[ "${line[@]}" =~ " : $ban_file trimmed by" ]]; then
    elif [[ ! ${line[0]} =~ $ipregex ]]; then
      printf "${ban_file_array[$i]}" >> $ban_file
    elif [ -z ${line[1]} ]; then
      printf "${line[0]} "
      printf "${line[0]}\n" >> $ban_file
      [[ "$verbose" =~ bans|yes|all ]] &&
      printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : BAN-LISTED : No expiration\n" -1 ${0##*/} $$ ${line[0]} >> $log
      $iptables -w -I $drop_chain -s ${line[0]} -j $ban_policy
    elif [ $nowtime -lt ${line[1]} ]; then
      printf "${line[0]} "
      printf "%-15s ${line[1]}\n" ${line[0]} >> $ban_file
      [[ "$verbose" =~ bans|yes|all ]] &&
      printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : BAN-LISTED : Expires $(date --date @${line[1]})\n" -1 ${0##*/} $$ ${line[0]} >> $log
      $iptables -w -I $drop_chain -s ${line[0]} -j $ban_policy
  printf "%(%s)T $HOSTNAME %s[%5s]: No banlist at $ban_file\n" -1 ${0##*/} $$ >> $log

if [[ "$verbose" =~ xt_recent|yes|all ]]; then
  for i in /proc/net/xt_recent/*
  do for j in $(cat $i)
    do  [[ $j =~ src= ]] &&
      printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : $i\n" -1 ${0##*/} $$ ${j:4} >> $log

# ---------------------   PROCESS MATCHING HITS   ----------------------

act_on_hit() {
# Build associative arrays ${hit_history[$type]} of "IP +date" pairs, one array for each type of hit
# Associative array hit_history[$type] is transposed into indexed array $flusher[] to trim
# aged entries, then transposed back to associative array until the next hit
# Default $hit_expire value is 1day, default $max_hits value is 10

nowtime=$(printf "%(%s)T" -1)
hit_history[$type]="${hit_history[$type]} $src_ip `date --date=+${hit_life[$type]:=$hit_expire} +%s`"
flusher=( ${hit_history[$type]} )
while [ ${flusher[1]} -lt $nowtime ]
  flusher=( ${flusher[@]:2} )
for i in ${flusher[@]}; do [[ $i == $src_ip ]] && ((hit_count++)); done
printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : %-21s %2s vs. %-2s\n" \
        -1 ${0##*/} $$ $src_ip "$hit_string" $hit_count ${max_hits[$type]:=$hit_limit} >> $log

# Stop if offending IP is already in $BANNED_LIST
# If `cutter` exists, run it to sever connection with that IP

if [ $hit_count -ge  ${max_hits[$type]:=$hit_limit} ]; then
  if [[ "$BANNED_LIST" =~ "$src_ip" ]]; then
    [ -x "$cutter" ] && $cutter $src_ip

# Increase ban_time if offending src_ip already appears in $ban_log (permanent record)
# ban_time set at 1 month for each appearance.  Already in ban_log twice?  2month ban.

    if [ -r $ban_log ]; then
      mapfile <$ban_log ban_log_array
      for (( i = 0 ; i < ${#ban_log_array[@]} ; i++ ))
        line=( ${ban_log_array[$i]} )
        [[ "${line[@]}" =~ $src_ip.*BANNED ]] && ((ban_count++))
      [[ $ban_count -ge 2 ]] && ban_time=${ban_count}months

# Add offending IP to $BANNED_LIST
# Add offending IP with ban expiration to $ban_file
# Add Notice of ban and triggering $log_action line to permanent record ($ban_log)
    expire_date="$(date --date=+$ban_time)"
    printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : BANNED     : Expires $expire_date\n" -1 ${0##*/} $$ $src_ip >> $log
    printf "%-15s $(date --date=+$ban_time +%s)\n" $src_ip >> $ban_file
    printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : BANNED : Banned from %(%c)T to $expire_date\n" -1 ${0##*/} $$ $src_ip -1 >> $ban_log
    printf "$log_action\n\n" >> $ban_log

# Restore $drop_chain and its contents but only if firewall has been modified
    $iptables -C $from_chain -j $drop_chain 2> /dev/null
    if [ "$?" -gt "0" ]; then
      printf "%(%s)T $HOSTNAME %s[%5s]: Firewall was modified, rebuilding banlist from $ban_file ...\n" -1 ${0##*/} $$ >> $log

# Insert a firewall rule to REJECT/DROP the offending IP address
    $iptables -w -I $drop_chain -s $src_ip -j $ban_policy
  fi                            # End condition of being currently banned
fi                              # End condition of reaching $max_hits[$type]

# -----------------  DETECT MATCHING HITS  --------------------

# Handle activity that appears on stdin, via syslog-ng
# A variety of log messages appear, a mix of iptables.log, auth.log, mail.log and mail.warn

# Traps facilitiate interaction with firewall builder and routine flushing of expired bans
# e.g. /etc/cron.daily/logrotate sends a SIGUSR1 to flush expired bans
#      /etc/iptables/establish-iptables sends a SIGUSR1 to rebuild banned IP firewall chain

read_from_stdin() {
trap "verbose=bans build_iptables_rules" SIGUSR1
trap "verbose=all  build_iptables_rules" SIGUSR2

while :
if read log_action; then
  [[ "$log_action" =~ $ipregex ]] && src_ip=$BASH_REMATCH
  if [ "$src_ip" == "" ]; then
  elif [[ "$whitelist" =~ "$src_ip" ]]; then
    unset hit_string
    for i in "${!watch_log_@}"
      if [[ "$log_action" =~ ${!i} ]]; then
        type=${i:10}; type=${type//[0-9]/}
    done        # end loop through $watch_log_* search-for-match strings
  fi            # end nested conditional - $log_action contains an IP, not in whitelist
# May 28, 2015 EDIT/ERROR FIX - added quote around variable "$hit_string"
  [ "$hit_string" == "" ] &&
  [[ "$verbose" =~ bans|yes|all ]] &&
  printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : no hit_string : $log_action\n" -1 ${0##*/} $$ $src_ip >> $log
fi              # end nested conditional - receipt of $log_action line from stdin
done            # end wait forever loop

# ----------------   END OF SUBROUTNES  -------------------
# -----------------  ANNOUNCE  STARTUP  -------------------

trap exit_trap EXIT

PARENT_PID=`ps --no-headers -o ppid --pid $$`
PARENT_NAME=`ps --no-headers -o comm $PARENT_PID`
printf "%(%s)T $HOSTNAME %s[%5s]: Started by ${PARENT_NAME}[%5s]\n" -1 ${0##*/} $$ $PARENT_PID >> $log
if [[ "$verbose" =~ bans|yes|all ]]; then
  for i in "${!watch_log_@}"
    i=${i:10} ; i=${i//[0-9]/} ; [[ "$types" =~ $i ]] || types="$types $i"
  for type in $types
    printf "%(%s)T $HOSTNAME %s[%5s]: %-15s : %2s hits per %-10s -> ${ban_time[$type]:=$ban_expire} ban\n" \
        -1 ${0##*/} $$ $type ${max_hits[$type]:=$hit_limit} ${hit_life[$type]:=$hit_expire} >> $log
  if [ -x "$cutter" ]; then
    printf "%(%s)T $HOSTNAME %s[%5s]: $cutter available\n" -1 ${0##*/} $$ >> $log

# -- FINI

# -------------------------------------------------------------------------------------
# Similar functionality can be obtained with only syslog-ng and iptables.
# Persistence of banned entries during restart of firewall rules can be obtained by writing
# banned IP to a file, then reading that file when rebuilding the firewall.
# The below syslog-ng.conf and iptables routines don't distinguish
# between port or log violation, but could.
# Below routines use syslog-ng parser function to drive iptables blocking, without
# resorting to any interposing script.
# When an entry is added to /proc/net/xt_recent/syslog-scan, by syslog-ng detecting
# failed sshd login or similar, the firewall begins tracking rate of hits.
# Adapted from

# ---- In syslog-ng.conf ----
# destination d_syslogblock
# { pseudofile("/proc/net/xt_recent/syslog-scan" template("+${usracct.device}\n"));
#   file("/var/log/syslog-block"); };
# parser pattern_db { db_parser( file("/var/lib/syslog-ng/patterndb.xml")); };
# filter f_syslogblock { tags("secevt") and match("REJECT" value("secevt.verdict")); };
# log { source(src); parser(pattern_db); filter(f_syslogblock); destination(d_syslogblock); };

# ---- In firewall ----
#iptables -N syslog-block
#iptables -A syslog-block -m recent --name syslog-block --set -j DROP
#iptables -A INPUT -m conntrack --ctstate NEW \
#                 -m recent --name syslog-block --update --reap --seconds 3600 -j DROP
#iptables -A INPUT -m conntrack --ctstate NEW \
#                 -m recent --name syslog-scan  --update --reap --seconds  900 --hitcount 15 -j syslog-block

The script has been running here on a couple of machines, for a couple of weeks. I like the persistent bans that survive restarting the firewall. What hasn't happened yet is a repeat offender triggering an extended ban, but that section of code has been tested, and appears to work.

I learned a few bash tricks while composing this, for example bash equivalents to `grep -o -E`, `wc`, and `uniq`. That saved a few subshell instances, with the aim being to make the script about as efficient as a bash script can be. No doubt the script can be improved, and even though not many (if any!) people will use it, the ideas it contains may help readers better understand iptables and syslog-ng.

Edited the script to add quotes around a variable involved in a test. The function of logging non-matching log_action strings otherwise does not work. The other functions of the script aren't affected by the bug. Change is pretty well marked in the script, noted here too. Sorry about that.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum