Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
vixie-cron fails to send mail for long jobs
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Dragonlord
Guru
Guru


Joined: 22 Aug 2004
Posts: 346
Location: Switzerland

PostPosted: Tue Apr 03, 2007 4:03 pm    Post subject: vixie-cron fails to send mail for long jobs Reply with quote

This is a strange problem which recently came up. I've added some more scripts to /etc/cron.daily to do maintenance work. Now for some reason no more mails are send to me. It worked before I added the scripts. The scripts itself are working fine as I use them also on other servers. Checking the logs I ended up with this:

Code:
Apr  3 03:01:01 ### cron[9637]: (root) CMD (rm -f /var/spool/cron/lastrun/cron.daily)
Apr  3 03:10:01 ### cron[9666]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
Apr  3 03:18:56 ### sSMTP[9673]: Connection lost in middle of processing
Apr  3 03:19:46 ### cron[9665]: (root) MAIL (mailed 8532 bytes of output but got status 0x0001 )


It looks like the mail job dies while the cron jobs are running. As you can see the jobs takes roughly 10 minutes to run through all. After 9 minutes the mail fails and drops it altogether. I can only explain this that the mail gets a timeout before the jobs are done. I tested by running the jobs by hand and there are no errors. Any ideas what can cause the mailer to fail? Can it be that really 10 minutes is too long to make the mailer break?
_________________
Leader and Head Programmer: Epsylon, Drag[en]gine and others
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Wed Apr 04, 2007 9:37 am    Post subject: Reply with quote

I doubt this has anything to do with the length of your cron jobs, but it might have something to do with their timing. I might also be your ssmtp configuration.

It may be that you have multiple scripts trying to send mail at the same time. I'm not sure how ssmtp responds to concurrent attempts to use it. I would be surprised if this would cause it to fail, but I suppose it's possible. While run-crons should execute the cron scripts themselves in series, it is possible that one of your cron scripts executes something else that is sending mail after a delay (for example, logrotate, aide, sxid, etc.). This scenario could also develop if your daily jobs are overlapping with your hourly jobs. However, the only mail-related symptoms I have seen in those situations are: some messages missing; messages truncated; or messages merged.

I would first test out the ssmtp setup, since the error you're getting is about losing the connection. Verify your configuration. Try using ssmtp to send some large files. Try sending a large file and another while it's still sending the first. Make sure you're doing it as root, so you test how ssmtp is handling the aliasing.

I would also map out the cron cycles to see if there is any overlap. If it looks like there might be, you can adjust the timing of crontab entries that run cron.hourly, cron.daily, etc. If you have difficulty doing that, you could also remove one or more of your longer-running cron jobs from the cron.daily folder, place it elsewhere, and create a separate crontab entry for it that gets run daily but outside of the normal "cron.daily" cycle.

Also, there are other relay-only MTA's (like ssmtp), including msmtp, nbsmtp, esmtp (what I use), etc. They are all similar with minor variations in functionality.
Back to top
View user's profile Send private message
Dragonlord
Guru
Guru


Joined: 22 Aug 2004
Posts: 346
Location: Switzerland

PostPosted: Wed Apr 04, 2007 11:58 am    Post subject: Reply with quote

I poked around the scripts and disabled once the tripwire daily cron ( by simply commenting out the -check line ). This day I got the mail so it looks like the run of tripwire breaks all. I have though no clue how this could happen especially since the tripwire cron job is created by emerge. It looks like this:

Code:
#!/bin/sh
HOST_NAME=`uname -n`
if [ ! -e /var/lib/tripwire/${HOST_NAME}.twd ] ; then
        echo "****    Error: Tripwire database for ${HOST_NAME} not found.    ****"
        echo "**** Run "/etc/tripwire/twinstall.sh" and/or "tripwire --init". ****"
else
        # i suspect a problem. disabling it for the time being
        #test -f /etc/tripwire/tw.cfg &&  /usr/sbin/tripwire --check
fi


I know from test runs that this command provides a lot of output to the console which is useful but I never got a mail send to me while running it so I don't think it comes in between. Why does it break? I'd like to keep tripwire running on this remote system.
_________________
Leader and Head Programmer: Epsylon, Drag[en]gine and others
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Fri Apr 06, 2007 1:29 am    Post subject: Reply with quote

You could crontab tripwire seperately from the normal cron.daily entry.

Another idea that I would probably try is to switch how it mails (file instead of pipe):

Redirect the output of the tripwire --check command to a tripwire log file (if one isn't already being created that has the same or similar contents). This way you get all the output consolidated into a file and you pass that file to ssmtp (instead of opening a pipe and sending it to ssmtp as it is created, which can be a lengthy process). Here are two alternative ways to get it mailed to you:

a) If you use logrotate, you could name or number the logrotate cron job so it runs last of all your daily cron jobs. Then configure logrotate to mail you the newly-created tripwire log. This option makes more sense if there is already a tripwire log being created and you are using logrotate.

b) Just place an additional sendmail (or ssmtp) command at the end of your tripwire cron job that mails you the newly created log. This option is simpler and avoids the dependency on logrotate for mailing.

I honestly have no idea how ssmtp works, but it it opens the ssmtp connection as soon as it starts receiving input to mail, then a long sequence of input, perhaps with lengthy periods of silence, might cause the behavior.
Back to top
View user's profile Send private message
Dragonlord
Guru
Guru


Joined: 22 Aug 2004
Posts: 346
Location: Switzerland

PostPosted: Fri Apr 06, 2007 10:20 pm    Post subject: Reply with quote

Is there a way to hack vixie-cron ( without messing with the sources ) so it does first pipe all output into a temporary file sending it only to sendmail if it is longer than 0 bytes? I tried snooping around the involved files a bit but I could not find a reference to sendmail without looking at the code.
_________________
Leader and Head Programmer: Epsylon, Drag[en]gine and others
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Sun Apr 08, 2007 5:26 pm    Post subject: Reply with quote

I don't know, but if you can't fix the scripts to stop that from happening, you can filter out mail that's empty.

Some mailers have a "don't send empty messages" option, if that's your issue.

If yours doesn't provide this option, you could use a mail delivery agent (like procmail or maildrop) to filter out any empty messages coming from cron. I don't use maildrop, but with procmail you would simply add a "recipe" to root's procmail configuration file (each user can have one, but if you're sending all cron mail to root, this is a convenient way to apply the filter). The recipe would be written to apply only to mail sent by cron, and to drop the email if the message body is empty. You could also have it fix up any headers that are missing or ugly.

For example, I send all system mail to root. So I have a /root/.procmailrc file that says where root's mail get's delivered (sometimes it's my user's system mailbox, sometimes it's an external smtp address). I was getting email from cron (or something, I can't remember now) with no date. So I added a little recipe that adds a date if it's missing (it calls an external program that comes with procmail called formail that can reformat and make changes to mail and headers). In this config file, I could also have conditional branching that sends different system messages to different addresses or at different priorities depending on the headers or contents.
Back to top
View user's profile Send private message
Dragonlord
Guru
Guru


Joined: 22 Aug 2004
Posts: 346
Location: Switzerland

PostPosted: Sun Apr 08, 2007 7:51 pm    Post subject: Reply with quote

Well the problem started to come around even without tripwire. So I changed to postfix now and with that I got it working. Looks like ssmtp is borked beyond bearable but with the postfix solution working I have it back at where I had been before.
_________________
Leader and Head Programmer: Epsylon, Drag[en]gine and others
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum