ssh distributed dictionary attack... how many requests/sec?

[eccerr0r]

curious of what other people are getting...

I'm getting almost 1 per 1.5 seconds on average for the past few months it seems. I started port blocking most of them as they tend to be the same hosts for large number of attempts, but there are the small fish that are equally as annoying. Almost 4000 unique hosts so far this year (2 weeks of attacks through Jan 14th).

Anyway with my current fail banning I got it down to about 15-20 attempts per hour that slip past the bans. This should at least slow down the dictionary about two orders of magnitude per hour. So at least I don't send responses but sh*t is still coming in. My IP link is not all that fast, I didn't calculate out how many bytes/sec this comes out to be...

I had forgotten I set up a secondary nonstandard port for ssh in case I locked myself out, but it seems they portscanned me and found it -- and started attacking that too. So I was forced to disable that, seems even if I moved my main port elsewhere, it wouldn't help for long.

*sigh*
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?

[NeddySeagoon]

eccerr0r,

Is key based ssh login an option?
The attack will be guessing user names and passwords.
That does not stop the wasted bandwidth though.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[eccerr0r]

Oh I'm not worried about them breaking into an account, password or kex. It's the wasted bandwidth I'm more worried about in general. Blocking the incoming packet at least saves my machine from transmitting outgoing packets, which is a more limited resource. But it's still wasting downlink.

At this point I just wonder if I'm being targeted more than other people or not for some reason or another. Curious if there's something that makes people want my machine more than others. or not.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?

[xahodo]

How about using port knocking? Leave ssh off and redirect all traffic coming in through the port to a tarpit, unless the correct knock comes in. If that happens, ssh is enabled and you can login.

[eccerr0r]

Again this does not help against the bandwidth waste. If I was really concerned about temporarily unlocking sshd, I think I could easily hack up something silly for apache to unblock sshd, but this is besides the point...

... why me, or is everyone getting these things at 1-2 seconds apart?
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?

[NeddySeagoon]

eccerr0r,

My logs for a bare metal box in a data centre are ..

[Ant P.]

Mine's somewhat quieter.

[eccerr0r]

Nice. My log files are like 16MB compressed per month now. This is current so far this month:

[cboldt]

I picked up using ipset just because I imagined one day I'd have (or somebody other than me might claim) a need to ban thousands. One iptables rule to check the ipset. Never in my wildest dreams would I be banning thousands.

[eccerr0r]

neat, didn't know about ipset, I may have to transition to that, this is starting to get slow.
I'm surprised nobody has ended up having to ban thousands. Right now I've actually not added to the ban list if the host only had one attempt, and I still have a net banlist with 1400 lines. The script tries to choose a /24 or /16 (!) ban depending on how many uniques try...

---

Well, I'm now transitioned to ipset -- and also changed my script from bash to perl, I can fully reprocess my banlists in just a few seconds where it was taking several minutes before... So much better... thanks.

I also wonder why people aren't dealing with thousands of bans, I suspect most people are seeing distributed dictionary attacks...
In fact the list of hosts that I ban probably would be of value to some people to populate their ban lists.

I'm now up over 4000 unique hosts, which crunches down to about 3500 /24's, and 2000 /16's if there are more than one /24 in the /16 in the botnet. The scary thing is that I'm not sure if this is all one botnet or multiple botnets, quite possibly there is more than one bad actor here.

[figueroa]

I can't advise to help more with the bandwidth, but, in addition to using a non-standard ssh port, also set:
a low login grace time
a low MaxAuthTries, i.e. 2 or 3
definitely don't allow root user with a password (default)
consider using the AllowUsers to only allow one or the minimum actual users
consider using unusual or not common usernames, i.e. not fred, mike, sam, andy etc.
of course, use a hideous, long, impossible to guess password

Those are the main tricks I have up my sleeve.

I stopped reading firewall logs and I don't watch the stock market. I don't have the time or the disposition for it.

After reading this thread, I'm paranoid again. But, I must be doing somethings right.

[erm67]

The solution is endlessh an ssh tarpit to slowdown attackers:
https://github.com/skeeto/endlessh
https://nullprogram.com/blog/2019/03/22/
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net

[cboldt]

I'm banning thousands now - but when I took up the task of learning how to use ipset, I didn't imagine that my system would ever have a practical need for it.

I ban by the /24. Some attackers (not many) attack from multiple IP, so my "ban counter" -- which times out attempts after minutes -- converts each individual IP (a /32) into the associated /24, and banning covers that chunk. Of the banned IP's, some of the /24's come from the same /16. I'm pretty sure if I banned by the /16 I'd still have thousands of entries. One machine is now hovering around 1500 bans (that's the one with ssh on port 2223), the other around 3000 (ssh port is 2222) but has been as high as 7000.

[cboldt]

Thanks for that tarpit suggestion. Great idea.

[erm67]

The scanners evolved and now drop the connection if they don't receive an answer quickly, however it will slow them down:

[eccerr0r]

How does the tarpit affect legitimate users also using password authentication?

I suppose if there's early on in the authentication that one is planning to use PKA that maybe this would work if one had the keys ready, but it seems like this would cause legitimate password users to fall into the pit as well.

Anyway after a few days of instituting simple /24 or /16 packet drop, the ssh attempts dropped a lot - still getting a few as they have a lot more machines than I have in my list, but at least the number of attempts have gone down significantly. The number of ICMPs has increased somewhat, but my network is a lot quieter than it was before. Before my machine was constantly sending and receiving packets, now I occasionally see 10 second breaks where there's silence.

Undesirable packets now received, roughly...
- fixed ssh port, few times per minute. Most are blocked but guessing passwords at 1 per 5 minutes is going to take a LONG time.
- fixed, alternate ssh port that I was using but now closed, once every 5 minutes or so. Looks like a handful of machines.
- Random port scans. People are distributedly trying to scan random ports on my machine. About a few a minute.
- Random pings - usually people send one ping from multiple hosts, also a few a minute
- Traceroute. Some people seem to be tracerouting me for whatever reason. once every 5 minutes or so.

People are occasionally attacking my IMAP server, but those don't happen frequently.

I am dismayed I had to go to these measures, breaking TCP/IP standards, just to reduce the amount of traffic I send out that does not benefit myself.

("fun" staring at tcpdump / wireshark output ... blahh...)
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?

[erm67]

[axl]

not sure, we're all in the same spirit?

we talking about exposing our network to the internet. right?

well. postfix is a shit box. so much punishment from the internet. I'm glad I use postfix.

Like today was one of the worst days. over 10.000 different smtpd requests. from non fqdn and hosts that didn't have a reverse, or were listed on spamhaus.

Most days nobody knocks. today it was like a freaking explosion of attacks. from random ips. I just... started early... make a note, redirect to honeypot... come back later.


I can't ever get my honeypot right. I made a server. Whatever mail u want to send.. sure.. send it. whatever user and password u wanna try. sure... login. i made each line of code. i know my honeypot well. it does imap and pop and smtp but... even when you blindly like give access to someone, they wont do much with it.

like for the longest time I was bothered by access on cyrus imap. the pop3, the imap, the pop3s and imaps. and could figure out what was going on.

so instead of just iptables -j REJECT | DROP I decided to DNAT their ass to a honey pot. where the password would always be accepted. they would always get to send their fucking smtpd mail. and u know what they do? nothing. disconnect and move on. they are simply fishing.

which... bugged the living hell out of me. I'm currently looking for correlations with dns. like... one thing I have going for me... if they want to spam me, they gonna look at dns. and I can change that every hour.

u can't mail me if you dont know my mx. and changing the mx every hour... that ... gives some info. most people would not know the difference. but switching mx every hour and looking into dns logging.


now... you would expect... if I was a hacker I would just use google dns. but they dont. they use their own dns. so prior to every attack, ... there are some dns queries. like one second apart.

[erm67]

botnet
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net

[axl]

I have one user. poor old me.

i hope they dont read that and leave me alone. I would get bored.

[Ant P.]

[axl]

that's like cyrus port 993. imaps. yeah we all have that.

what do you do about legacy 25 port. )

[axl]

or who was the last guy checking out the dns logs. seriously.

[eccerr0r]

Yes the idea is to have a box exposed to the full internet, with personal "cloud" services. This is my piece of the cloud. Having everything blocked off of course would stop all "unwanted interest" but then I wouldn't have remote access yet still have full control of my hardware and data.

I would suspect hackers would rather want an ssh interactive account over imap/smtp or http account (most of these accounts nowadays are run as the same "user" and all are virtual accounts these days). I'd think those that attack these services are more for the mail data or identity theft, instead of the ssh interest for an interactive account that they could run arbitrary commands. But yes these are botnets, more likely botnets seeking to expand their botnet. I do wonder how many unique botnets, tough to tell.

Incidentally I do not get a whole bunch of smtp spam open relay searches. They end up getting dropped due to unwanted relay or spamhaus rejects. By far most of my attacks are ssh.

Question is, though one attack is annoying, how many over how long is worrisome?

And the other thing is that I get an eerily large number of ICMP echo requests -- not so much that it's DoS/DDoS, but there are people out there curious to know if my machine is up for whatever reason.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?

[Anon-E-moose]