Multiplayer fail2ban (crowdsec)

[ToeiRei]

Hi guys,

I came across a nice project called 'crowdsec' which is a cool implementation to block out some noise on our boxes. If you'd ask me to describe it, it's like a multiplayer version of fail2ban - just more advanced.
More advanced also means, it detects slow bruteforces on the SSHd and other shenanigans, Log4J exploiting and a ton of other things. Thought it might be something worth looking into.

On my home network I managed to get my CPU load down by 10% by blocking out a ton of bad HTTP probing and hammering my wordpress site and managed to lower my CPU usage by around 10%. Not sure if that's something for you, but I generally appreciate the decreased load on my hosts by blocking out the bad guys on my main firewall instead of managing every host by itself.

Cheers
Rei
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[lyallp]

Whilst crowdsec is unavailable as an ebuild, at this time, I use Fail2ban.

Whilst I expose ssh and apache to the web, my server is regularly probed, resulting in temporary and permanent bans of IPs.

I watch my fail2ban.log using 'tail -f' in an xterm.

Probably takes a bit more to configure and expertise to use, but I have 150+ IPs which are banned, permanently.
_________________
...Lyall

[ToeiRei]

I just did a manual install from the tarball and I let it do a list of IPs that I import into my firewall and block it at the entrance instead of watching them pick their targets.

What I dislike about fail2ban is that you cannot really detect some attacks like slower bruteforcing or attacks across multiple machines.
I mean, seriously: Fail2ban was introduced back in 2004 - but the regex patterns just drive me nuts. Crowdsec is a tad newer and uses grok patterns and I had a new services monitored in less than 30 minutes.
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[lyallp]

Interesting.
I agree, the regexps are a bit complicated.
Slow attacks, depends on how you configure your jail. (5 attempts in 60 minutes?)
Multi-machine attacks, how do you differentiate between one machine controlling multiple machines and multiple machines independently operating?
_________________
...Lyall

[ToeiRei]

I do run parsers on every machine in my network reporting to the LAPI (local API) which then is the source of the 'bouncer' which is what does the blocking - or in my case spits out the list loaded by pfsense.

To be fair, I also get a lot of IPs back from the cloud, blocking them even before they attack. At the point of writing this, I block about 22,933 hosts and I managed to reduce the background noise AND cpu usage on my boxes significantly. Due to less abuse my hypervisor servers load went down from a base load of 20% to around 10% which is huge for me.
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[ToeiRei]

I had my share of updating things manually. I made an overlay over at https://github.com/ToeiRei/crowdsec-overlay that should do the trick for keeping things up to date.

Bug reports, pull requests, etc are welcome.
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[Zucca]

Hi.

Does crowdsec work with nftables?
Also does it automatically add thousands of ip blocks in the firewall rules at the start? Or does it use its database as a first filter which then, if matched upon failed login etc., adds a firewall rule?
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

[ToeiRei]

Hi Zucca,

Crowdsec works with 2 components: the security engine (net-analyzer/crowdsec) and the bouncer - in your case net-analyzer/cs-firewall-bouncer. If you want to use nftables, you are required to have USE=json set on nftables;

As for how it works, the security engine runs a little database (can be mysql, pg or sqlite with WAL enabled) that does the decision making and communications with the crowd. Your firewall gets those (in my case currently around 14k) blocks added as their own chain while the bouncer takes care of adding and removing them.

If you want some statistics about blocks - in my case I have around 2/3rds of my blocks 'in advance' - in other words, those hosts haven't had to actually hit my server beforehand to be blocked.
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[Zucca]

Thanks for the info.
Crowdsec seems reasonable indeed.

I'll try to remember this topic when I have set up my new server. I hope this summer... Things happen slowly here. New home in the middle of a forest and way too many boring things to do other than the interesting nerdy computery stuff.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

[ToeiRei]

The big plus I see is that it can also detect things like slow bruteforce on SSH, credit card stuffing, HTTP attacks - you name it. Anything you can find in a logfile. And if you're a nerd like me, I run the security engine basically everywhere, feeding to my central instance which spits out an IP list that I slap onto my big firewall in front.

I once did a bit of testing and noticed how much additional power those darn script kiddies and bots cost us (and I wrote an article at their webpage: https://www.crowdsec.net/blog/saving-ressources-with-crowdsec about it)
It's just impressive to see how much "trash load" a server has to handle instead of real work out there - but don't be fooled. Those savings were safely invested into running another gentoo machine XD
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...

[gr3m1in]

Another thumb-up to CrowdSec is for it's support to feed from centralized syslog server over network.
It is pretty important and even sometimes critical if your services are not placed on the same single host.
For me it was the reason to not use fail2ban.

https://docs.crowdsec.net/docs/data_sources/syslog/

[ToeiRei]