Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
SOLVED: denyhosts and python not playing together
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
gordonp
Tux's lil' helper
Tux's lil' helper


Joined: 23 May 2005
Posts: 89

PostPosted: Mon Aug 29, 2016 5:00 pm    Post subject: SOLVED: denyhosts and python not playing together Reply with quote

I recently noticed that my Denyhosts service wasn't running on my workstation; I purged every file and directory I could find with the name 'denyosts' in it, removed my former hosts.deny file, and started all over afresh -

I emerged denyhosts, went through the configuration, but still it gives the same error:
Code:
# systemctl status denyhosts.service
● denyhosts.service - SSH log watcher
   Loaded: loaded (/usr/lib64/systemd/system/denyhosts.service; enabled; vendor preset: disabled)
   Active: failed (Result: resources) since Mon 2016-08-29 09:51:15 PDT; 7s ago
  Process: 25852 ExecStart=/usr/bin/denyhosts.py --daemon --config=/etc/denyhosts.conf (code=exited, status=0/SUCCESS)
  Process: 25848 ExecStartPre=/bin/rm -f /var/run/denyhosts.pid (code=exited, status=0/SUCCESS)

Aug 29 09:51:15 pluto denyhosts.py[25852]: File "/usr/lib64/python3.4/site-packages/DenyHosts/deny_hosts.py", line 424, in process_log
Aug 29 09:51:15 pluto denyhosts.py[25852]: for line in fp:
Aug 29 09:51:15 pluto denyhosts.py[25852]: File "/usr/lib64/python3.4/codecs.py", line 319, in decode
Aug 29 09:51:15 pluto denyhosts.py[25852]: (result, consumed) = self._buffer_decode(data, self.errors, final)
Aug 29 09:51:15 pluto denyhosts.py[25852]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 5896: invalid start byte
Aug 29 09:51:15 pluto denyhosts.py[25852]: DenyHosts exited abnormally
Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: PID file /var/run/denyhosts.pid not readable (yet?) after start: No such file or directory
Aug 29 09:51:15 pluto systemd[1]: Failed to start SSH log watcher.
Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: Unit entered failed state.
Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: Failed with result 'resources'.


I would appreciate any suggestions ... THANKS in advance :-)

Other related info about my desktop/workstation (generally all "stable"):

Code:
# emerge -pv python denyhosts readline

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-libs/readline-6.3_p8-r2::gentoo  USE="-static-libs -utils" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild   R    ] dev-lang/python-3.4.3-r1:3.4::gentoo  USE="gdbm ipv6 ncurses readline sqlite ssl threads xml -build -examples -hardened -tk -wininst" 0 KiB
[ebuild   R    ] app-admin/denyhosts-3.0-r1::gentoo  PYTHON_TARGETS="python2_7 python3_4 -python3_3 (-python3_5)" 0 KiB


Code:
# eselect python list
Available Python interpreters:
  [1]   python2.7
  [2]   python3.3
  [3]   python3.4 *


and

Code:
# eselect profile list
Available profile symlink targets:
  [1]   default/linux/amd64/13.0
  [2]   default/linux/amd64/13.0/selinux
  [3]   default/linux/amd64/13.0/desktop
  [4]   default/linux/amd64/13.0/desktop/gnome
  [5]   default/linux/amd64/13.0/desktop/gnome/systemd *
  [6]   default/linux/amd64/13.0/desktop/kde
  [7]   default/linux/amd64/13.0/desktop/kde/systemd
  [8]   default/linux/amd64/13.0/desktop/plasma
  [9]   default/linux/amd64/13.0/desktop/plasma/systemd
  [10]  default/linux/amd64/13.0/developer
  [11]  default/linux/amd64/13.0/no-multilib
  [12]  default/linux/amd64/13.0/systemd
  [13]  default/linux/amd64/13.0/x32
  [14]  hardened/linux/amd64
  [15]  hardened/linux/amd64/selinux
  [16]  hardened/linux/amd64/no-multilib
  [17]  hardened/linux/amd64/no-multilib/selinux
  [18]  hardened/linux/amd64/x32
  [19]  hardened/linux/musl/amd64
  [20]  hardened/linux/musl/amd64/x32
  [21]  default/linux/uclibc/amd64
  [22]  hardened/linux/uclibc/amd64


Last edited by gordonp on Tue Aug 30, 2016 5:05 pm; edited 1 time in total
Back to top
View user's profile Send private message
ian.au
Guru
Guru


Joined: 07 Apr 2011
Posts: 422
Location: Australia

PostPosted: Mon Aug 29, 2016 11:04 pm    Post subject: Reply with quote

Quote:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 5896: invalid start byte

I am going to guess that this is a result of the stricter encoding checks in python3x there are a few gotchas like this if selecting a 3.x default python

Do you have the same problem is you switch back to python 2.7 as the default interpreter?

Quote:
Other related info about my desktop/workstation (generally all "stable"):

If running generally stable then make the default python interpreter 2.7 and comment out any python directives in /etc/portage/make.conf if you have them - that is the default at the moment.

I misread an eselect news item back when 3.4 was made the stable *python3* implementation and dutifully made 3.4 my default interpreter for a few months, I started seeing these types of errors. They all went away after I re-read the python wiki and reverted to the defaults.
Back to top
View user's profile Send private message
gordonp
Tux's lil' helper
Tux's lil' helper


Joined: 23 May 2005
Posts: 89

PostPosted: Tue Aug 30, 2016 2:26 pm    Post subject: Setting python interpreter BACK to 2.7 fixes denyhosts Reply with quote

Great suggestion, @ian.au

I set my python interpreter to 2.7 (which seems like a large step backwards, but maybe that's only my mis-perception), and IT WORKS!
Back to top
View user's profile Send private message
ian.au
Guru
Guru


Joined: 07 Apr 2011
Posts: 422
Location: Australia

PostPosted: Wed Aug 31, 2016 2:35 am    Post subject: Re: Setting python interpreter BACK to 2.7 fixes denyhosts Reply with quote

gordonp wrote:
Great suggestion, @ian.au

I set my python interpreter to 2.7 (which seems like a large step backwards, but maybe that's only my mis-perception), and IT WORKS!

I don't see it a step backwards. I guess 2.7's the default because large chunks of portage are written with it and not all scripts have been refactored to 3.x. As long as you have a 3.x slotted you have access to a 3.x interpreter anyway when needed, and if you run in the default configuration, the devs sort that all out for you (or you can choose per package by setting some use flags).
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5586
Location: Removed by Neddy

PostPosted: Wed Aug 31, 2016 9:33 am    Post subject: Reply with quote

Portage and associated scripts have been py3 compatible for quite some time, that is why there is a py3 USE flag. You can see how long ago that was added
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
ian.au
Guru
Guru


Joined: 07 Apr 2011
Posts: 422
Location: Australia

PostPosted: Wed Aug 31, 2016 9:46 am    Post subject: Reply with quote

Still get corner cases when running 2.7 as default here, not portage related though probably - I did say that was a guess, thanks for the pointer.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 581

PostPosted: Sun May 28, 2017 12:11 am    Post subject: Reply with quote

I had this same issue and fixed it just now only to discover that openssh has removed tcp wrapper support back in 2014, and somehow this was not propagated to us.

Denyhosts no longer denies anything.

Jon.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13493

PostPosted: Sun May 28, 2017 3:25 pm    Post subject: Reply with quote

While it could have been made a bit more prominent, there was (and still is) an effort to warn people who read their Portage log messages:
grep -A3 wrappers /usr/portage/net-misc/openssh/openssh-7.*:

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156
/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then
/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"
/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."
/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-   fi
/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-}
--
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-   fi
/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-}
--
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-   fi
/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-}
--
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-   fi
/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-}
--
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-   fi
/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-}
If this warning is not triggering for you, I suggest filing a bug to have the ebuild adjusted.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 581

PostPosted: Mon May 29, 2017 4:12 pm    Post subject: Reply with quote

Because it is very security related I would have thought a more prominent warning would be warranted. How am I going to see it in a big list of emerge messages? Secondly, I din't use TCP wrappers, I used denyhosts. I, like a lot of even savvy users, only have the vaguest idea what tcp-wrappers is, just that it's a dependency of some really core functionality.

I haven't had any luck with itables, it is very complex software, and all the successor solutions like fail2ban etc are based on it. There is essentially no way of solving this problem without gong to a fully fledged firewall setup. Maybe that's good, one should not run telnetd or ftpd any more either.

This is the achilles heel of FOSS, old solutions silently fail with no replacement. Or the replacement is poorly documented, because hey linux isn't for noobs. OK.

Why couldn't somebody just fix TCP wrappers? That's not a gentoo problem, it seems like an everybody problem. There's also a world of out of date documentation out there telling you how to set up hosts.* to secure your system. And even googling hosts.deny doesn't get you a page that says:

WARNING PEOPLE: hosts.deny NO LONGER WORKS !!! DENYHOSTS RUNS BUT CAN'T DENY ANYTHING!!!

I guess it does now.

Cheers,

Jon

[edit:] Thanks for your input by the way, it's very helpful.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1691

PostPosted: Mon May 29, 2017 5:51 pm    Post subject: Reply with quote

First off, tcp-wrappers is the hosts.deny/allow; so if you used hosts.deny you were using tcp-wrappers...

As far as stuff stopped working with tcp-wrappers, that decision was done by upstream developers, not Gentoo's devs. I know for openssh, there were quite a few posts/articles about this and openssh devs said they won't support them. Their reasoning is that they already have something internal doing the same thing so to save a few lines of code they stopped supporting tcp-wrappers... There is quite a few people upset about this, but there is nothing we can do. Simply put upstream has done a straight out their way or highway, and you don't have a choice of anything else.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13493

PostPosted: Mon May 29, 2017 11:37 pm    Post subject: Reply with quote

With the right configuration, Portage would mail (or save to a special file) ewarn class messages. Assuming (possibly unjustifiably) that Gentoo developers do not use ewarn for noise messages, these messages are always worth reading. I agree that a more aggressive failure would have been nice. I recall one recent openssh ebuild where a particular unsupported configuration would refuse to build.

Basic iptables is not that hard. If you describe your problems with it, we may be able to help you.

Nobody restored the sshd/TCP wrappers integration because nobody with the skill to do so felt it was worth his/her time to do it. I never used TCP wrappers for sshd because it seemed like a bad design to allow connections from hosts you know you don't want to service.

DenyHosts works fine for programs that still support TCP wrappers. OpenSSH is not one of those programs.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 581

PostPosted: Tue May 30, 2017 11:49 pm    Post subject: Reply with quote

ct85711 wrote:
First off, tcp-wrappers is the hosts.deny/allow; so if you used hosts.deny you were using tcp-wrappers...


Yes I was, but I didn't know it.

Quote:

As far as stuff stopped working with tcp-wrappers, that decision was done by upstream developers, not Gentoo's devs. I know for openssh, there were quite a few posts/articles about this and openssh devs said they won't support them. Their reasoning is that they already have something internal doing the same thing so to save a few lines of code they stopped supporting tcp-wrappers... There is quite a few people upset about this, but there is nothing we can do. Simply put upstream has done a straight out their way or highway, and you don't have a choice of anything else.


Yes, I understand that is indeed upstream's problem. I was exactly frustrated because openssh has no such functionality, and there is a russian server hammering me with a few thousand login requests a day, with no way to block it until I finally got iptables going. When you have a lot of kernel flags to set correctly and a lot lot more that you have to search through to find them it gets confusing. tcp-wrapper was very good about "just working" all these years.

I filed a bug asking for denyhosts to be dropped from portage because it no longer does anything in the gentoo context.

Could tcp-wrapper functionality be replaced entirely by a script that sets iptables rules? I have googled this a zillion ways and can't seem to find anything relevant. Or could denyhosts be modified to issue an iptables drop instead of writing to hosts.deny?

It seems like though, fail2ban is much more powerful and works with iptables out of the box, and there are other solutions as well.

Thanks,

Jon.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13493

PostPosted: Wed May 31, 2017 1:20 am    Post subject: Reply with quote

As I understand it, fail2ban is meant to be that script that uses iptables to enforce the denials. According to its features page, it can use Netfilter or TCP wrappers (among other choices). Not that the TCP wrappers support will do you much good in this case.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 581

PostPosted: Fri Jun 02, 2017 12:09 am    Post subject: Reply with quote

Thats' right. IPtables took quite a bit of work to get going. Fail2ban is also a deep dive, very powerful. I'm just goign to ban attackers by hand when I notice them. It's not worth the work. Now I'm having trouble getting nomachine to play with iptables, but I'll have to start another thread for that.

thanks,

Jon.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum