Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
sys-process/cronie and /etc/cron.hourly,daily,weekly,monthly
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
apurkrt
Tux's lil' helper
Tux's lil' helper


Joined: 26 Feb 2011
Posts: 116
Location: Czechia, Europe

PostPosted: Sat Feb 08, 2020 2:21 pm    Post subject: sys-process/cronie and /etc/cron.hourly,daily,weekly,monthly Reply with quote

Hi!
I would just like to sum up my recent encounter with cronie and its' default config on gentoo, and raise a few questions.

Thanks to people on irc and irl who helped me with this.

I was interested in system-wide cron jobs (not user crontabs).
vixie-cron is now removed from portage, so let's begin with installing cronie:

Code:
gentoo /etc # emerge -av sys-process/cronie

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] sys-process/cronbase-0.3.7-r6::gentoo  0 KiB
[ebuild  N     ] sys-process/cronie-1.5.5::gentoo  USE="anacron inotify pam (-selinux)" 0 KiB

Total: 2 packages (2 new), Size of downloads: 0 KiB
...


Ok, installed.
note: by default, "anacron" USE flag is enabled, so /usr/sbin/anacron will get installed.

Code:
gentoo ~ # qlist -e cronie | grep bin
/usr/bin/crontab
/usr/bin/cronnext
/usr/sbin/crond
/usr/sbin/anacron
gentoo /etc # qlist -e cronbase | grep bin
/usr/sbin/run-crons


To start cronie, you can now:

Code:
gentoo ~ # /etc/init.d/cronie start


note: the daemon is named "crond".

My main question was: How exactly does the /etc/cron.{hourly,daily,weekly,monthly} work?

Let's find out.

The starting point for me was /etc/crontab (I knew about this file before)

Code:
gentoo ~ # cat /etc/crontab
# Global variables
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# For details see man 5 crontab

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name  command to be executed


Some variables, explanatory comments, but no job set up here! Let's head to man:

Code:
gentoo ~ # man 5 crontab
...
Jobs in /etc/cron.d/
       The jobs in cron.d and /etc/crontab are system  jobs,  which  are  used
       usually  for  more  than  one  user, thus, additionally the username is
       needed.  MAILTO on the first line is optional.
...


Ok, so let's examine /etc/cron.d/ directory as well
Code:
gentoo ~ # ls /etc/cron.d
0hourly
gentoo ~ # cat /etc/cron.d/0hourly
# Run the hourly jobs
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
01 * * * * root run-parts /etc/cron.hourly


Finally something - the last line means: execute "run-parts /etc/cron.hourly" as root on every hour, 01 minute (i.e. "<hour>:01").
That would explain how jobs in /etc/cron.hourly would get run (hourly).
Fine, but what about /etc/cron.{daily,weekly,monthly}? Let's investigate further.

Where does run-parts come from and what does it do:
Code:
gentoo ~ # which run-parts
/bin/run-parts
gentoo ~ # qfile /bin/run-parts
sys-apps/debianutils: /bin/run-parts
gentoo ~ # man run-parts
...
run-parts  runs  all  the executable files named within constraints described
below, found in directory directory . Other files and  directories are silently ignored.
...
Files are run in the lexical sort order
...


Ok so let's examine /etc/cron.hourly/

Code:
gentoo ~ # ls -l /etc/cron.hourly/
total 4
-rwxr-x--- 1 root root 580 Feb  8 11:29 0anacron


What does this 0anacron do?

Code:
gentoo ~ # cat /etc/cron.hourly/0anacron
#!/bin/sh
# Check whether 0anacron was run today already
if test -r /var/spool/anacron/cron.daily; then
    day=`cat /var/spool/anacron/cron.daily`
fi
if [ `date +%Y%m%d` = "$day" ]; then
    exit 0
fi

# Do not run jobs when on battery power
online=1
for psupply in AC ADP{0..9} ; do
    sysfile="/sys/class/power_supply/$psupply/online"

    if [ -f $sysfile ] ; then
        if [ `cat $sysfile 2>/dev/null`x = 1x ]; then
            online=1
            break
        else
            online=0
        fi
    fi
done
if [ $online = 0 ]; then
    exit 0
fi
/usr/sbin/anacron -s


Huh. So once a day, and if not on battery power, it does run "/usr/sbin/anacron -s".

So cron is (by default) running anacron. Seems weird? It did to me, but I later found on gentoo wiki
"Anacron is not a cron daemon, it is something that usually works in conjunction with one.".
Only thing contraintuitive is that there is also a solo package sys-process/anacron in portage (that's different anacron implementation).

Anyway, back to the anacron which came with cronie

Code:
gentoo ~ # man anacron
...
Anacron  is  used  to  execute  commands periodically, with a frequency
       specified in days.
...
Anacron  reads  a  list  of jobs from the /etc/anacrontab configuration
       file (see anacrontab(5))
...


So what is in /etc/anacrontab?
Code:
gentoo ~ # cat /etc/anacrontab
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.

SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22

#period in days   delay in minutes   job-identifier   command
1   5   cron.daily      nice run-parts /etc/cron.daily
7   25   cron.weekly      nice run-parts /etc/cron.weekly
@monthly 45   cron.monthly      nice run-parts /etc/cron.monthly


Finally! That is what runs /etc/cron.{daily,weekly,monthly} in the end.

The whole line is like follows:
"/etc/init.d/cronie start" -> crond started, reads "/etc/cron.d/0hourly"
-> "run-parts /etc/cron.hourly" -> runs "/etc/cron.hourly/0anacron" on every <hour>:01
-> runs "/usr/sbin/anacron -s" once a day -> interprets /etc/anacrontab -> runs /etc/cron.{daily,weekly,monthly}

So /etc/cron.{hourly,daily,weekly,monthly} is a mixed bag, wrt how they are executed: /etc/cron.hourly is run directly by crond, whereas /etc/cron.{daily,weekly,monthly} is run indirectly by anacron, which is run by crond

note: anacron cannot start hourly jobs

Question: starting times So when exactly do the cron jobs start?
For jobs in /etc/cron.hourly, the answer is - they will start once 0anacron is done (since 0anacron is usually alphabetically first).
0anacron has a check for running once a day only, "anacron -s" becomes a sleeping process, so mostly the hourly jobs will be run at <hour>:01

When does anacron run the jobs i.e. when do the /etc/cron.{daily,weekly,monthly} start is (afaict) random - there is delay+random delay,
there is START_HOURS_RANGE. Tbh, I had not investigated this that much.

Question: reentrancy (overlapping hourly jobs)
The most important question for me was: will crond start a new (hourly)
job even if the previous instance is still running?

The answer for /etc/cron.hourly/ is a definite yes, happily (tested it experimentally with job containing sleep 65m)

reason: in my case, there is a shell script in /etc/cron.hourly, that synchronizes files from other server, using rsync, and I would not like two rsynces running at the same time (if the sync takes more than an hour).

note: I was advised, that it is a good thing to use flock inside the sync script, something like (a fully atomic way of preventing the race condition)
Code:
...
exec 9>/var/run/myscript.lock; flock -n 9 || exit
...


For /etc/cron.daily (weekly, monthly) and jobs running for more than one day (week, month), i.e. those that are run by anacron, I do not know.
Perhaps anacron in itself prevents the concurrent running? Really not sure
Code:
gentoo ~ # man anacron
...
-s     Serializes execution of jobs.  Anacron does not start a new  job
              before the previous one finished.
...


But "0anacron" is started hourly by cron.d/0hourly, it checks for date, runs "anacron -s" daily.
So perhaps more than one instance of "anacron -s" could finally be running (if the job takes more than 24h). Again, here I am unsure.



Final note

Previously, with vixie-cron (now removed from the tree), the running of /etc/cron.{hourly,daily,weekly,monthly} was accomplished in a different way - /etc/crontab contained (by default) the following:

Code:
59  *  * * *    root    rm -f /var/spool/cron/lastrun/cron.hourly
9  3  * * *     root    rm -f /var/spool/cron/lastrun/cron.daily
19 4  * * 6     root    rm -f /var/spool/cron/lastrun/cron.weekly
29 5  1 * *     root    rm -f /var/spool/cron/lastrun/cron.monthly
*/10  *  * * *  root    test -x /usr/sbin/run-crons && /usr/sbin/run-crons


The above config can still be used but first you have to comment out the following line in /etc/cron.d/0hourly
Code:
#01 * * * * root run-parts /etc/cron.hourly

and also do the following
Code:
gentoo ~ # chmod -x /etc/cron.hourly/0anacron

effectively disabling anacron (otherwise be prepared for https://bugs.gentoo.org/621706).
EDIT: beware - upon reinstalling or updating sys-process/cronie, the executable bit on /etc/cron.hourly/0anacron gets silently enabled again, switching on anacron. So rather than "chmod -x", it is probably better to edit 0anacron and comment out the line with "anacron -s".

The advantage of using the above config, i.e. relying on run-crons script over anacron is probably better control on when exactly the daily,weekly,monthly scripts will get run (though you can setup random delay to zero and have that control with anacron too). Every ten minutes /usr/sbin/run-crons would be run (if it was executable). It would check the /var/spool/cron/lastrun/cron.* files, and run executables in /etc/cron.hourly/* on every <hour>:00, /etc/cron.daily/* on 3:10am, /etc/cron.weekly/* on every saturday 4:20am, /etc/cron.monthly/* on 5:30am every first day of the month.

I have verified experimentally, that the run-crons script (still provided nowadays with sys-process/cronbase) prevents the reentrant execution of the /etc/cron.hourly/ scripts.
To that end, I have created a script

Code:
gentoo ~ # cat /etc/cron.hourly/test65m
#!/bin/sh
ST=`date +%s`
touch /root/crontest/${ST}start
sleep 65m
touch /root/crontest/${ST}end


let it run overnight with the following result (reformatted)

Code:
gentoo ~ # ls -l /root/crontest/

-rw-r--r-- 1 root root  0 Feb  8 02:00 1581123601start
-rw-r--r-- 1 root root  0 Feb  8 03:05 1581123601end

-rw-r--r-- 1 root root  0 Feb  8 03:10 1581127801start
-rw-r--r-- 1 root root  0 Feb  8 04:15 1581127801end

-rw-r--r-- 1 root root  0 Feb  8 04:20 1581132002start
-rw-r--r-- 1 root root  0 Feb  8 05:25 1581132002end

-rw-r--r-- 1 root root  0 Feb  8 05:30 1581136201start
-rw-r--r-- 1 root root  0 Feb  8 06:35 1581136201end

-rw-r--r-- 1 root root  0 Feb  8 06:40 1581140401start
-rw-r--r-- 1 root root  0 Feb  8 07:45 1581140401end

-rw-r--r-- 1 root root  0 Feb  8 07:50 1581144601start
-rw-r--r-- 1 root root  0 Feb  8 08:55 1581144601end


Here are some related commits for those interested (also you can use "git log -p -- sys-process/cronie/files/cronie-1.3-crontab" from within the checked out gentoo tree)

https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=331b15d5fd648fa6b2a43d07757b365e19d749f8
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=c474dda61c900197ef05e64d1e5af35785cbb7c1
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=f79f8ac5d95fa3f88301f0bad8b03e9d5f497a59


Last edited by apurkrt on Wed Feb 12, 2020 8:32 pm; edited 6 times in total
Back to top
View user's profile Send private message
gtwrek
Tux's lil' helper
Tux's lil' helper


Joined: 10 Mar 2017
Posts: 110
Location: San Jose, CA

PostPosted: Sat Feb 08, 2020 3:44 pm    Post subject: Thanks Reply with quote

I've nothing to add other than expressing thanks. With the recent (deprecation?) of vixie-cron, I was looking closer at the details of cron(ish) solutions. - I never paid too much attention.
The whole cron vs anacron and how things worked (under the covers) was a bit confusing too me. Apparently, I'm not alone.

I started the exact exploration you've done, but only got about a third of the way you did.
Your note should be a sticky of some sort. I know I'm bookmarking it.
Back to top
View user's profile Send private message
apurkrt
Tux's lil' helper
Tux's lil' helper


Joined: 26 Feb 2011
Posts: 116
Location: Czechia, Europe

PostPosted: Sun Feb 09, 2020 8:53 am    Post subject: Reply with quote

Some more info on anacron

I used the default setup, did remove the locks
Code:
gentoo ~ # rm /var/spool/anacron/cron.{daily,weekly,monthly}

to clean up the environment beforehand and then
Code:
gentoo ~ # /etc/init.d/cronie start

on 22:40:47, let it run overnight.

Here is an extract from /var/log/syslog

Code:
Feb  8 22:40:47 gentoo crond[10273]: (CRON) STARTUP (1.5.5)
Feb  8 22:40:47 gentoo crond[10273]: (CRON) INFO (Syslog will be used instead of sendmail.)
Feb  8 22:40:47 gentoo crond[10273]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 15% if used.)
Feb  8 22:40:47 gentoo crond[10273]: (CRON) INFO (running with inotify support)
Feb  8 23:01:01 gentoo CROND[11924]: (root) CMD (run-parts /etc/cron.hourly)
Feb  8 23:01:01 gentoo anacron[11931]: Anacron started on 2020-02-08
Feb  8 23:01:01 gentoo anacron[11931]: Will run job `cron.daily' in 7 min.
Feb  8 23:01:01 gentoo anacron[11931]: Will run job `cron.weekly' in 27 min.
Feb  8 23:01:01 gentoo anacron[11931]: Will run job `cron.monthly' in 47 min.
Feb  8 23:01:01 gentoo anacron[11931]: Jobs will be executed sequentially
Feb  8 23:08:01 gentoo anacron[11931]: Job `cron.daily' started
Feb  8 23:08:01 gentoo anacron[11931]: Job `cron.daily' terminated
Feb  8 23:28:01 gentoo anacron[11931]: Job `cron.weekly' started
Feb  8 23:28:01 gentoo anacron[11931]: Job `cron.weekly' terminated
Feb  8 23:48:01 gentoo anacron[11931]: Job `cron.monthly' started
Feb  8 23:48:01 gentoo anacron[11931]: Job `cron.monthly' terminated
Feb  8 23:48:01 gentoo anacron[11931]: Normal exit (3 jobs run)
Feb  9 00:01:01 gentoo CROND[15730]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 00:01:01 gentoo anacron[15737]: Anacron started on 2020-02-09
Feb  9 00:01:01 gentoo anacron[15737]: Normal exit (0 jobs run)
Feb  9 01:01:01 gentoo CROND[19447]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 01:01:01 gentoo anacron[19454]: Anacron started on 2020-02-09
Feb  9 01:01:01 gentoo anacron[19454]: Normal exit (0 jobs run)
Feb  9 02:01:01 gentoo CROND[23240]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 02:01:01 gentoo anacron[23247]: Anacron started on 2020-02-09
Feb  9 02:01:01 gentoo anacron[23247]: Normal exit (0 jobs run)
Feb  9 03:01:01 gentoo CROND[27158]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 03:01:01 gentoo anacron[27165]: Anacron started on 2020-02-09
Feb  9 03:01:01 gentoo anacron[27165]: Will run job `cron.daily' in 48 min.
Feb  9 03:01:01 gentoo anacron[27165]: Jobs will be executed sequentially
Feb  9 03:49:01 gentoo anacron[27165]: Job `cron.daily' started
Feb  9 03:49:01 gentoo anacron[27165]: Job `cron.daily' terminated
Feb  9 03:49:01 gentoo anacron[27165]: Normal exit (1 job run)
Feb  9 04:01:01 gentoo CROND[31110]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 05:01:01 gentoo CROND[2467]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 06:01:01 gentoo CROND[6387]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 07:01:01 gentoo CROND[10424]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 08:01:01 gentoo CROND[14513]: (root) CMD (run-parts /etc/cron.hourly)
Feb  9 09:01:01 gentoo CROND[19041]: (root) CMD (run-parts /etc/cron.hourly)


State of the lock files now
Code:
gentoo ~ # cat /var/spool/anacron/cron.daily
20200209
gentoo ~ # cat /var/spool/anacron/cron.weekly
20200208
gentoo ~ # cat /var/spool/anacron/cron.monthly
20200208


note: Every "Anacron started on ..." corresponds to "anacron -s" command, started by /etc/cron.hourly/0anacron. So "anacron -s" is not run only once in a day as I have written before. It is instead run every hour (see 00:01:01, 01:01:01, 02:01:01, 03:01:01), until /var/spool/anacron/cron.daily gets updated. After that, the check at the beginning of /etc/cron.hourly/0anacron kicks in and no more "anacron -s" starts for the day.

/var/spool/anacron/cron.daily gets updated only after 3am - that comes from "START_HOURS_RANGE=3-22" in /etc/anacrontab.

note: The randomness of start times is also well seen. On 23:01:01 there is "Will run job `cron.daily' in 7 min." (weekly in 27 min, monthly in 47 min).
7 comes from 5+2 (delay 5 + RANDOM_DELAY 2 (from the 0-45 range)). 27=25+2, 47=45+2. The random delay is the same for all jobs within one instance of "anacron -s".
On 03:01:01, there is "Will run job `cron.daily' in 48 min.". Here it is 5+43 (delay 5 + random delay 43).
Back to top
View user's profile Send private message
jmanko
Tux's lil' helper
Tux's lil' helper


Joined: 11 Sep 2005
Posts: 139
Location: NEPA, USA

PostPosted: Fri Jun 05, 2020 6:21 pm    Post subject: Reply with quote

Can't you just emerge cronie without the anacron USE flag? Shouldn't that remove anacron and leave cronie to just run /etc/crontab as normal?
_________________
"What stands in the way becomes the way." -- Marcus Aurelius
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum