View previous topic :: View next topic |
Author |
Message |
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Wed May 30, 2018 6:27 pm Post subject: cronbase scripts executing twice per run |
|
|
Cronbase executed scripts (/etc/cron.hourly/*, /etc/cron.daily/* , etc.) are being run twice each time they should be executed once.
(I don't use cron.weekly or cron.monthly so don't know if they are also affected.)
On my two Gentoo systems I have the following emerged:
virtual/cron
sys-process/cronie
sys-process/cronbase
I do NOT use individual user crontab files, only the system /etc/crontab.
I have verified that I only have one instance of crond running.
The entries in /etc/crontab are run on time and NOT duplicated.
The scripts in /etc/cron.daily/ and /etc/cron.hourly/ are all run then run again.
I put the following test bash script in /etc/cron.hourly:
Code: | #! /bin/bash
date >> /tmp/cron.hourly.out |
And the result is:
Code: | /tmp]$ cat cron.hourly.out
Wed May 30 10:00:01 PDT 2018
Wed May 30 10:01:01 PDT 2018
Wed May 30 11:00:01 PDT 2018
Wed May 30 11:01:01 PDT 2018 |
Any ideas on why chronbase is executing scripts twice per run cycle? |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Thu May 31, 2018 3:55 pm Post subject: |
|
|
I have also tried un-emergeing then emerging cronbase. I still have the double run issue. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Thu May 31, 2018 4:25 pm Post subject: |
|
|
jagdpanther ...
while I don't use sys-process/cronie it does have an IUSE dependency on sys-process/anacron, and the later doesn't follow the same time schedule of crontab. So, I expect what's happening is that cronie is running the job while anacron has it on it's todo list, and so that job is re-run at a later time. I expect there is some configuration required to have cronbase and cronie work together, but I see nothing on the wiki, and there is nothing in pkg_postinst(). However, the ebuild does show an /etc/cron.hourly/0anacron installed, which should provide some clue as to what anacron is doing, or should be doing.
HTH & best ... khay |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Thu May 31, 2018 6:27 pm Post subject: |
|
|
khayyam: Thank you.
From the Gentoo Cron wiki:
Quote: | Anacron is not a cron daemon, it is something that usually works in conjunction with one. It executes commands at intervals specified in days and it does not assume that the system is running continuously; it will run jobs that were missed while the system was down. Anacron usually relies on a cron daemon to run it each day. |
Considering that my Gentoo systems are usually on and I don't need "missed" cron jobs to run if the system is down, I'll try removing the 0anacron entries from /etc/cron.hourly and /etc/cron.daily to see if the issue is resolved. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Thu May 31, 2018 10:02 pm Post subject: |
|
|
The issue is not anacron. I removed the /etc/cron.daily/0anacron file and still am getting the dual runs of the scripts in /etc/cron.daily. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Thu May 31, 2018 10:21 pm Post subject: |
|
|
jagdpanther ...
I noticed there is another thread with the exact same issue, see the proposed solution here.
best ... khay |
|
Back to top |
|
|
szatox Advocate
Joined: 27 Aug 2013 Posts: 3104
|
Posted: Thu May 31, 2018 10:53 pm Post subject: |
|
|
You could add a script with command
ps uxwf
and dump it's output to a file. It will show you a bunch of processes organized in a convenient tree, at the time it's being run. You should be able to find ps there and see what invoked it. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Fri Jun 01, 2018 3:56 pm Post subject: |
|
|
Khayyam:
Quote: | I noticed there is another thread with the exact same issue, see the proposed solution here. |
Thanks. I saw that post and there were two issues, duplicate runs like me and runs not happening when expected. I am only having the former issue. Also I can't use his solution because I need to have some jobs run at specific times, so commenting almost everything out of /etc/crontab won't work in my case. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Fri Jun 01, 2018 4:20 pm Post subject: |
|
|
szatox:
Quote: | You could add a script with command
ps uxwf |
Excellent idea. Thank you. Here is what I did:
Added a script, DEBUG_CRONBASE.bash to /etc/cron.daily:
Code: | #! /bin/bash
# What is calling the extra cronbase run?
date >> /tmp/cron-daily-DEBUG.out 2>&1
ps uxwfa >> /tmp/cron-daily-DEBUG.out 2>&1
echo "=========================" >> /tmp/cron-daily-DEBUG.out 2>&1 |
Here are the relevant parts of /tmp/cron-daily-DEBUG.out:
Code: | 1 Fri Jun 1 03:06:04 PDT 2018
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 2 0.0 0.0 0 0 ? S May30 0:00 [kthreadd]
...
root 6675 0.0 0.0 12692 1964 ? Ss 03:00 0:00 /usr/sbin/anacron -s
root 6766 0.0 0.0 4252 744 ? SN 03:06 0:00 \_ run-parts /etc/cron.daily
root 7138 0.0 0.0 11884 2640 ? SN 03:06 0:00 \_ /bin/bash /etc/cron.daily/DEBUG_CRONBASE.bash
root 7140 0.0 0.0 17896 2584 ? RN 03:06 0:00 \_ ps uxwfa
...
=========================
Fri Jun 1 03:10:04 PDT 2018
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 2 0.0 0.0 0 0 ? S May30 0:00 [kthreadd]
...
root 4268 0.0 0.0 17788 2424 ? Ss May30 0:00 /usr/sbin/crond
root 22558 0.0 0.0 17788 1904 ? S 03:10 0:00 \_ /usr/sbin/CROND
root 22560 0.0 0.0 11984 2724 ? Ss 03:10 0:00 \_ /bin/sh /usr/sbin/run-crons
root 25239 0.0 0.0 11884 2644 ? S 03:10 0:00 \_ /bin/bash /etc/cron.daily/DEBUG_CRONBASE.bash
root 25241 0.0 0.0 17896 2580 ? R 03:10 0:00 \_ ps uxwfa |
So it appears that both crond and anacron are running the files in /etc/cron.daily and /etc/cron.hourly (and probably weekly and monthly too.) Both crond and anacron are part of sys-process/cronie-1.5.1-r1 on my systems. So crond runs the files via the run-crons entry in /etc/crontab. According to the anacron man page I should look at the /etc/anacrontab (which I did not know about until 3 min. ago) and I guess I'll comment out the end of that configuration file so anacron doesn't execute anything unless someone has a better idea.
I wonder what is spawning anacron, it is currently not running. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Fri Jun 01, 2018 7:09 pm Post subject: |
|
|
Quote: | I wonder what is spawning anacron, it is currently not running. |
When I tried re-emergeing cronie, it put /etc/cron.hourly/0anacron back. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Fri Jun 01, 2018 11:12 pm Post subject: |
|
|
The other cause of duplicate cron runs was /etc/cron.d/0hourly. Which appears to be another /etc/crontab type file. After commenting out Code: | 01 * * * * root run-parts /etc/cron.hourly | from /etc/cron.d/0hourly the duplicate runs stopped. |
|
Back to top |
|
|
szatox Advocate
Joined: 27 Aug 2013 Posts: 3104
|
Posted: Sun Jun 03, 2018 11:43 am Post subject: |
|
|
Well... So you solved your problem by breaking cron
Glad it helped, but I think it would be a better idea to either remove your script from crontab or move it outside of cron.hourly instead.
If you want your cron jobs to run at a specific time, you put them in crontab directly. If you want them to run on an interval, that's what /etc/cron.<interval> files are for. They work in similar way to anacron (e.g. if you power off your machine and miss a job, it will run within 10 minnutes after reboot)
If you leave it like that, some time in the future you will drop a new script into that directory and then you will be puzzled why it doesn't run at all. |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 727
|
Posted: Sun Jun 03, 2018 4:29 pm Post subject: |
|
|
szatox:
Quote: | If you want your cron jobs to run at a specific time, you put them in crontab directly. If you want them to run on an interval, that's what /etc/cron.<interval> files are for. |
Yes, and that is the way my two Gentoo systems ran for years without issue. Sometime in the last month or two I started getting duplicate runs of everything in /etc/cron.<interval>. (I wasn't paying attention so don't know when it started. I guess if I was motivated I could look in archived /var/log/messages files to find out.)
Now after the 'fix' I still have jobs in /etc/crontab running precisely when I want them to and the scripts in /etc/cron.daily running once a day. So it appears to me that cron is running as it should.
What I turned off to prevent dual-runs of files in /etc/cron.<interval> (and I don't know why I needed to do this as it worked without this 'fix' for years) was to comment out the "root run-parts" line in /etc/crond.d/0hourly and the last couple of lines of /etc/anacrontab. I don't know if I needed to 'fix' both of those files or just one to prevent the dual-runs. I realize that this 'fix' prevents not run jobs do to system down time from being run after the system is back on-line. Did I break something else?
Thank you again for the 'ps uxwf ' debugging suggestion. |
|
Back to top |
|
|
radio_flyer Guru
Joined: 04 Nov 2004 Posts: 317 Location: Northern California
|
Posted: Thu Sep 20, 2018 5:18 pm Post subject: |
|
|
Did your system upgrade from vixie-cron to cronie?
That just hit my stable systems as vixie-cron is undergoing last rites. vixie-cron had the following lines in /etc/crontab:
Code: |
# check scripts in cron.hourly, cron.daily, cron.weekly and cron.monthly
59 * * * * root rm -f /var/spool/cron/lastrun/cron.hourly
9 3 * * * root rm -f /var/spool/cron/lastrun/cron.daily
19 4 * * 6 root rm -f /var/spool/cron/lastrun/cron.weekly
29 5 1 * * root rm -f /var/spool/cron/lastrun/cron.monthly
*/10 * * * * root test -x /usr/sbin/run-crons && /usr/sbin/run-crons
|
/usr/sbin/run-crons is from sys-process/cronbase that both vixie-cron and cronie depend on.
However, cronie removes the lines above from the crontab file, and adds three files to the system:
/etc/cron.d/0hourly
/etc/cron.hourly/0anacron
/etc/anacrontab
/etc/cron.d/0hourly looks like:
Code: |
# Run the hourly jobs
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
01 * * * * root run-parts /etc/cron.hourly
|
where run-parts appears to be from sys-apps/debianutils.
/etc/cron.hourly/0anacron looks like:
Code: |
#!/bin/sh
# Check whether 0anacron was run today already
if test -r /var/spool/anacron/cron.daily; then
day=`cat /var/spool/anacron/cron.daily`
fi
if [ `date +%Y%m%d` = "$day" ]; then
exit 0;
fi
# Do not run jobs when on battery power
if test -x /usr/bin/on_ac_power; then
/usr/bin/on_ac_power >/dev/null 2>&1
if test $? -eq 1; then
exit 0
fi
fi
/usr/sbin/anacron -s
|
which basically ends up looking at /etc/anacrontab, which looks like:
Code: |
# /etc/anacrontab: configuration file for anacron
# See anacron(8) and anacrontab(5) for details.
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22
#period in days delay in minutes job-identifier command
1 5 cron.daily nice run-parts /etc/cron.daily
7 25 cron.weekly nice run-parts /etc/cron.weekly
@monthly 45 cron.monthly nice run-parts /etc/cron.monthly
|
In other words, the old vixie-cron used /etc/crontab directly to run the hourly/daily/weekly/monthly jobs. It's forked replacement cronie instead appears to begin with /etc/cron.d/0hourly and ends up running anacron to schedule the periodic tasks. If both the old vixie-cron commands in /etc/crontab and the new cronie files are on the system the cron jobs would probably run twice. That could happen if the cronie changes to /etc/crontab are zapped during the upgrade.
BTW, don't forget to stop vixie-cron before it gets replaced by cronie; the ebuild doesn't do it apparently. |
|
Back to top |
|
|
|