View previous topic :: View next topic |
Author |
Message |
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Wed May 04, 2022 7:38 pm Post subject: Multiplayer fail2ban (crowdsec) |
|
|
Hi guys,
I came across a nice project called 'crowdsec' which is a cool implementation to block out some noise on our boxes. If you'd ask me to describe it, it's like a multiplayer version of fail2ban - just more advanced.
More advanced also means, it detects slow bruteforces on the SSHd and other shenanigans, Log4J exploiting and a ton of other things. Thought it might be something worth looking into.
On my home network I managed to get my CPU load down by 10% by blocking out a ton of bad HTTP probing and hammering my wordpress site and managed to lower my CPU usage by around 10%. Not sure if that's something for you, but I generally appreciate the decreased load on my hosts by blocking out the bad guys on my main firewall instead of managing every host by itself.
Cheers
Rei _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
lyallp Veteran
Joined: 15 Jul 2004 Posts: 1557 Location: Adelaide/Australia
|
Posted: Thu Jul 14, 2022 9:28 am Post subject: |
|
|
Whilst crowdsec is unavailable as an ebuild, at this time, I use Fail2ban.
Whilst I expose ssh and apache to the web, my server is regularly probed, resulting in temporary and permanent bans of IPs.
I watch my fail2ban.log using 'tail -f' in an xterm.
Probably takes a bit more to configure and expertise to use, but I have 150+ IPs which are banned, permanently. _________________ ...Lyall |
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Thu Jul 14, 2022 9:45 am Post subject: |
|
|
I just did a manual install from the tarball and I let it do a list of IPs that I import into my firewall and block it at the entrance instead of watching them pick their targets.
What I dislike about fail2ban is that you cannot really detect some attacks like slower bruteforcing or attacks across multiple machines.
I mean, seriously: Fail2ban was introduced back in 2004 - but the regex patterns just drive me nuts. Crowdsec is a tad newer and uses grok patterns and I had a new services monitored in less than 30 minutes. _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
lyallp Veteran
Joined: 15 Jul 2004 Posts: 1557 Location: Adelaide/Australia
|
Posted: Thu Jul 14, 2022 10:56 am Post subject: |
|
|
Interesting.
I agree, the regexps are a bit complicated.
Slow attacks, depends on how you configure your jail. (5 attempts in 60 minutes?)
Multi-machine attacks, how do you differentiate between one machine controlling multiple machines and multiple machines independently operating? _________________ ...Lyall |
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Thu Jul 14, 2022 11:02 am Post subject: |
|
|
I do run parsers on every machine in my network reporting to the LAPI (local API) which then is the source of the 'bouncer' which is what does the blocking - or in my case spits out the list loaded by pfsense.
To be fair, I also get a lot of IPs back from the cloud, blocking them even before they attack. At the point of writing this, I block about 22,933 hosts and I managed to reduce the background noise AND cpu usage on my boxes significantly. Due to less abuse my hypervisor servers load went down from a base load of 20% to around 10% which is huge for me. _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Tue May 30, 2023 8:10 pm Post subject: |
|
|
I had my share of updating things manually. I made an overlay over at https://github.com/ToeiRei/crowdsec-overlay that should do the trick for keeping things up to date.
Bug reports, pull requests, etc are welcome. _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3334 Location: Rasi, Finland
|
Posted: Wed May 31, 2023 8:06 am Post subject: |
|
|
Hi.
Does crowdsec work with nftables?
Also does it automatically add thousands of ip blocks in the firewall rules at the start? Or does it use its database as a first filter which then, if matched upon failed login etc., adds a firewall rule? _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Wed May 31, 2023 8:31 am Post subject: |
|
|
Hi Zucca,
Crowdsec works with 2 components: the security engine (net-analyzer/crowdsec) and the bouncer - in your case net-analyzer/cs-firewall-bouncer. If you want to use nftables, you are required to have USE=json set on nftables;
As for how it works, the security engine runs a little database (can be mysql, pg or sqlite with WAL enabled) that does the decision making and communications with the crowd. Your firewall gets those (in my case currently around 14k) blocks added as their own chain while the bouncer takes care of adding and removing them.
If you want some statistics about blocks - in my case I have around 2/3rds of my blocks 'in advance' - in other words, those hosts haven't had to actually hit my server beforehand to be blocked. _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3334 Location: Rasi, Finland
|
Posted: Wed May 31, 2023 9:00 am Post subject: |
|
|
Thanks for the info.
Crowdsec seems reasonable indeed.
I'll try to remember this topic when I have set up my new server. I hope this summer... Things happen slowly here. New home in the middle of a forest and way too many boring things to do other than the interesting nerdy computery stuff. _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Wed May 31, 2023 10:03 am Post subject: |
|
|
The big plus I see is that it can also detect things like slow bruteforce on SSH, credit card stuffing, HTTP attacks - you name it. Anything you can find in a logfile. And if you're a nerd like me, I run the security engine basically everywhere, feeding to my central instance which spits out an IP list that I slap onto my big firewall in front.
I once did a bit of testing and noticed how much additional power those darn script kiddies and bots cost us (and I wrote an article at their webpage: https://www.crowdsec.net/blog/saving-ressources-with-crowdsec about it)
It's just impressive to see how much "trash load" a server has to handle instead of real work out there - but don't be fooled. Those savings were safely invested into running another gentoo machine XD _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
gr3m1in n00b
Joined: 22 Mar 2010 Posts: 39 Location: Poland
|
Posted: Thu Oct 19, 2023 3:44 pm Post subject: |
|
|
Another thumb-up to CrowdSec is for it's support to feed from centralized syslog server over network.
It is pretty important and even sometimes critical if your services are not placed on the same single host.
For me it was the reason to not use fail2ban.
https://docs.crowdsec.net/docs/data_sources/syslog/ |
|
Back to top |
|
|
ToeiRei Veteran
Joined: 03 Jan 2005 Posts: 1191 Location: Austria
|
Posted: Thu Oct 19, 2023 3:49 pm Post subject: |
|
|
gr3m1in wrote: | Another thumb-up to CrowdSec is for it's support to feed from centralized syslog server over network.
It is pretty important and even sometimes critical if your services are not placed on the same single host.
For me it was the reason to not use fail2ban.
https://docs.crowdsec.net/docs/data_sources/syslog/ |
That's also great if you run switches and stuff where you cannot run an agent. _________________ Please stand by - The mailer daemon is busy burning your messages in hell... |
|
Back to top |
|
|
|