Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Random Must issue a STARTTLS command first - google - exim
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Wed Sep 28, 2016 7:10 am    Post subject: Random Must issue a STARTTLS command first - google - exim Reply with quote

On my server, most messages go out; sometimes, I receive reject notification "Must issue a STARTTLS command first". What means ... first delivery failed, and then, rejection notice is successfully delivered !!!

All test messages are always fine.

Issue happens by group; maybe 20 messages in a raw, once a week, during 2h. Issues are not isolated; it's not 1 in 20 along the day.

This happens since a few months.

What could possibly cause this ?

Could exim randomly stop using SSL, forget the conf, and behave a random way ?

Could Google servers randomly mess message headers ? (happened to me in the past)

Could Google answer this error message, while in fact the real reason is completely different ? (happened to me in the past with google; reason given often has nothing to do with real cause of bug).

Bug could be related to DNS issue ? Once a week, a resolution points me to a bad server, and bug lasts untill DNS cache expires ?

It's years I have this bug, but it became frequent last week: more than 60 bad messages. Used to be 30 per year.

Once conf is set, and test message goes fine, I hardly see how Exim could go wrong ...
dc_smarthost='smtp.gmail.com::587'

Is there a lib in exim that can present things in a bad order for a few hours, untill some process is killed, and daemon is restarted automaticly ?

Bug occurs with both servers, exim 4.87 and 4.69

Rejections come from these servers:
host gmail-smtp-msa.l.google.com [2a00:1450:400c:c0a::6c]
host gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6d]
host gmail-smtp-msa.l.google.com [2a00:1450:400c:c08::6c]

obfuscation of transit make it impossible to have the identity of servers that allow things to work.

I don't know what to do, because old standards of Internet say I should use the SMTP from my ISP, but it's compeltely broken; new standards say I shall use the one from my email provider, and it's also broken ...
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
chiefbag
Guru
Guru


Joined: 01 Oct 2010
Posts: 542
Location: The Kingdom

PostPosted: Wed Sep 28, 2016 7:18 am    Post subject: Reply with quote

Maybe you need to upgrade to use TLS or use port 465 ( SSL )

https://support.google.com/a/answer/176600?hl=en

Code:
Port 465 (SSL required)
Port 587 (TLS required)
Back to top
View user's profile Send private message
chiefbag
Guru
Guru


Joined: 01 Oct 2010
Posts: 542
Location: The Kingdom

PostPosted: Wed Sep 28, 2016 7:26 am    Post subject: Reply with quote

What about the following USE flages for mail-mta/exim
Code:
sasl ssl


Maybe your need to rebuild mail-mta/exim again after an openssl update or something.
Back to top
View user's profile Send private message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Thu Sep 29, 2016 7:44 am    Post subject: Reply with quote

on port 465:
Code:

2016-09-29 09:12:03 1bpC5c-000581-PU Remote host gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6c] closed connection in response to initial connection
2016-09-29 09:12:13 1bpC5c-000581-PU Remote host gmail-smtp-msa.l.google.com [74.125.206.108] closed connection in response to initial connection

back to 587, works fine.

Now digging old logs:
Code:

2016-09-28 12:23:11 1bpBwG-0000Fu-SV TLS error on connection to gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6d]: gnutls_handshake timed out
2016-09-28 12:23:11 1bpBwG-0000Fu-SV TLS session failure: delivering unencrypted to gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6d] (not i
n hosts_require_tls)
2016-09-28 12:23:15 1bpBwG-0000Fu-SV ** ***@demaine.info R=smarthost T=remote_smtp_smarthost: SMTP error from remote mail server after MAIL
FROM:<******> SIZE=3438: host gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6d]: 530 5.7.0 Must issue a STARTTLS command fir
st. s9sm7586362wjh.16 - gsmtp
2016-09-28 12:23:15 1bpC1D-00031o-8V <= <> R=1bpBwG-0000Fu-SV U=Debian-exim P=local S=3410
2016-09-28 12:23:15 1bpBwG-0000Fu-SV Completed
2016-09-28 12:23:17 1bpC1D-00031o-8V => ******** <*******> R=smarthost T=remote_smtp_smarthost H=gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6c] X=TLS1.0:RSA_AES_128_CBC_SHA1:16 DN="C=US,ST=California,L=Mountain View,O=Google Inc,CN=smtp.gmail.com"
2016-09-28 12:23:17 1bpC1D-00031o-8V Completed
2016-09-28 12:26:40 Start queue run: pid=18599
2016-09-28 12:26:40 End queue run: pid=18599
2016-09-28 12:27:11 1bpC51-0004yA-FS <= *** U=root P=local S=847
2016-09-28 12:27:11 1bpC51-0004yH-IL <= ***blehp.org U=root P=local S=795
2016-09-28 12:27:11 1bpC51-0004yO-LP <= ***hp.org U=root P=local S=849
2016-09-28 12:27:12 1bpC51-0004yR-P1 <= ***hp.org U=root P=local S=816
2016-09-28 12:27:12 1bpC52-0004yV-4Z <= ***ehp.org U=root P=local S=820
2016-09-28 12:27:14 1bpC51-0004yA-FS => ***ine.info R=smarthost T=remote_smtp_smarthost H=gmail-smtp-msa.l.google.com [2a00:1450:400c:c
04::6d] X=TLS1.0:RSA_AES_128_CBC_SHA1:16 DN="C=US,ST=California,L=Mountain View,O=Google Inc,CN=smtp.gmail.com"
2016-09-28 12:27:14 1bpC51-0004yA-FS Completed

So, my machine tries without TLS, because TLS failed in first place.

- how to dig why TLS timed out ?

The ideal setup for me would be:
- keep attemppting google every 4 or 6h for 1 day
- after 1 day of continuous failure, try SMTP2.isp.com

I will try hosts_require_tls ...

[Moderator edit: added [code] tags to preserve output layout. -Hu]
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Thu Sep 29, 2016 11:56 am    Post subject: Reply with quote

hosts_require_tls='smtp.gmail.com'
problem still here
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
chiefbag
Guru
Guru


Joined: 01 Oct 2010
Posts: 542
Location: The Kingdom

PostPosted: Tue Oct 04, 2016 11:09 am    Post subject: Reply with quote

Considering gmail-smtp-msa.l.google.com is a CNAME for smtp.gmail.com you could add that or try a wildcard.

Code:
tls_smtp:
    driver = smtp
    hosts_require_tls = *
Back to top
View user's profile Send private message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Wed Oct 05, 2016 6:11 am    Post subject: Reply with quote

One step forward: this happened just in front of me, while I was reading tail log for an other reason:

Code:
2016-10-05 07:49:27 1bqvbD-0001gJ-2w H=gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6c] TLS error on connection (gnutls_handshake): The TLS connection was non-properly terminated.
2016-10-05 07:49:27 1bqvbD-0001gJ-2w TLS session failure: delivering unencrypted to gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6c] (not in hosts_require_tls)
2016-10-05 07:49:27 1bqvbD-0001gJ-2w ** plop@foo.org R=smarthost T=remote_smtp_smarthost H=gmail-smtp-msa.l.google.com [2a00:1450:400c:c04::6c]: SMTP error from remote mail server after MAIL FROM:<> SIZE=3259: 530 5.7.0 Must issue a STARTTLS command first. gg10sm6967541wjd.4 - gsmtp


Means two things to me:
- the previous fix I did does not work
- my server tries noTLS because Google had a problem with TLS ...

hosts_require_tls = * not stupid. Can't harm, let's try it.

Must be joking me ... updated the conf, rebooted, problem persists !!!
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Wed Oct 05, 2016 11:45 am    Post subject: Reply with quote

For reference, this is an invalid tests to check my issue:

According to
http://bradthemad.org/tech/notes/exim_cheatsheet.php
Quote:
y all of Exim's configuration settings:
root@localhost# exim -bP

is false; from IRC #exim:
Quote:
(08:33:42) rjsalts: exim -bP is only the options in the main config. This is for a router
(08:33:49) rjsalts: sorry, transport


So, checking the setting is setup correctly is tricky.

After 3 days, I think I have fixed it. How to check it:

Before the fix:

Quote:
# exim -d -f '<foo@grrr.com>' bar@grrr.com 2>&1 | grep requi

2a00:1450:400c:c04::6d in hosts_require_ocsp? no (option unset)
GnuTLS global init required.
2a00:1450:400c:c04::6d in hosts_require_auth? no (option unset)
file lookup required for gmail-smtp-msa.l.google.com
header read id:S,subid:0,size:00004,required:11,remaining:123,unfinished:0
header read id:X,subid:1,size:00105,required:112,remaining:112,unfinished:0
header read id:X,subid:2,size:01651,required:1658,remaining:1828,unfinished:0
header read id:X,subid:4,size:00002,required:9,remaining:170,unfinished:0
header read id:C,subid:1,size:00006,required:13,remaining:161,unfinished:0
header read id:D,subid:0,size:00004,required:11,remaining:148,unfinished:0
header read id:A,subid:0,size:00122,required:129,remaining:137,unfinished:0
header read id:Z,subid:0,size:00001,required:8,remaining:8,unfinished:0


And after:

Quote:
# exim -d -f '<foo@grrr.com>' bar@grrr.com 2>&1 | grep requi

header read id:H,subid:0,size:00025,required:32,remaining:159,unfinished:0
header read id:H,subid:0,size:00017,required:24,remaining:127,unfinished:0
header read id:H,subid:0,size:00017,required:24,remaining:103,unfinished:0
header read id:S,subid:0,size:00004,required:11,remaining:79,unfinished:0
header read id:D,subid:0,size:00004,required:11,remaining:68,unfinished:0
header read id:A,subid:0,size:00050,required:57,remaining:57,unfinished:0
header read id:Z,subid:0,size:00001,required:8,remaining:8,unfinished:0
root@leon-03:/var/spool/exim4#


I would prefer a direct positive proov saying that Gmail is listed where I want, but, for now, this change in the behaviour is alreadya proof that I have found the correct place to mess with hosts_require_tls
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
doublehp
Guru
Guru


Joined: 11 Apr 2005
Posts: 472
Location: FRANCE

PostPosted: Fri Jan 20, 2017 2:56 pm    Post subject: Reply with quote

After a few months, the situation seems stable for my server.
_________________
DEMAINE Benoît-Pierre (aka DoubleHP ) http://www.demaine.info/
>o_/ Coin coin coin \_o<
to contact me (MSN,ICQ, JABBER, Skype ... ) http://benoit.demaine.info/contact.png
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum