Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] fcron systab tasks stopped being executed recently
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
equaeghe
l33t
l33t


Joined: 22 Feb 2005
Posts: 632

PostPosted: Wed Feb 20, 2013 1:28 pm    Post subject: [solved] fcron systab tasks stopped being executed recently Reply with quote

Hi,


A recent update to something cron-related (sorry, don't know which) resulted in my backup fcron systab tasks not being executed anymore. First, it seemed that there was a problem with my systab fcrontab (crashing fcron), but reloading it solved that. Now, fcrondyn shows that fcron is nicely scheduling the tasks, the cron logs say that the tasks are executed (exit code 127?) and I checked manually that my backup script works. However, the backups are not actually being run...

How can I determine what is going wrong? Do not hesitate to ask me for specific logs,...

Should I perhaps add systab as a user?


TIA,

Erik


Last edited by equaeghe on Thu Feb 21, 2013 9:07 am; edited 1 time in total
Back to top
View user's profile Send private message
christofdeluca
n00b
n00b


Joined: 19 Mar 2005
Posts: 34

PostPosted: Thu Feb 21, 2013 6:37 am    Post subject: Reply with quote

Exit code 127 is bash for "Command not found."

http://tldp.org/LDP/abs/html/exitcodes.html
Back to top
View user's profile Send private message
equaeghe
l33t
l33t


Joined: 22 Feb 2005
Posts: 632

PostPosted: Thu Feb 21, 2013 9:07 am    Post subject: Reply with quote

christofdeluca wrote:
Exit code 127 is bash for "Command not found."


Indeed, that seemed to be the problem, /usr/local/bin seems not to be in the path of the user executing, so using the full path in the crontab fixed this.

Thanks!
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Feb 21, 2013 1:43 pm    Post subject: Reply with quote

equaeghe ... I think the issue here is that /usr/sbin/check_system_crontabs is now under /usr/libexec/ ... and your previously installed /var/spool/fcron/systab has the path to /usr/sbin (see comments in bug #453246).

The solution would be to stop fcron, remove the old systab, and recreate it ('emerge --config fcron'), then restart fcron.

I'm fairly sure thats the source of the '127 error', as I also noticed the same log entries after the recent update, a quick check of /var/spool/fcron/systab should verify this.

best ... khay
Back to top
View user's profile Send private message
equaeghe
l33t
l33t


Joined: 22 Feb 2005
Posts: 632

PostPosted: Thu Feb 21, 2013 2:00 pm    Post subject: Reply with quote

khayyam wrote:
equaeghe ... I think the issue here is that /usr/sbin/check_system_crontabs is now under /usr/libexec/ ... and your previously installed /var/spool/fcron/systab has the path to /usr/sbin (see comments in bug #453246).

The solution would be to stop fcron, remove the old systab, and recreate it ('emerge --config fcron'), then restart fcron.

I'm fairly sure thats the source of the '127 error', as I also noticed the same log entries after the recent update, a quick check of /var/spool/fcron/systab should verify this.


Dear Khayyam,


Thanks for looking at this.

In my case, it was really a path issue... I may have been hit by the other issue as well, but solved it inadvertently earlier by reinstalling my systab crontab.


Erik
Back to top
View user's profile Send private message
knight77
n00b
n00b


Joined: 29 Jun 2009
Posts: 25

PostPosted: Tue May 14, 2013 8:11 pm    Post subject: Reply with quote

Actually i'm not sure the issue has been resolved.

I have a Gentoo server i upgraded from fcron-3.0.6-r1 to latest 3.1.1.

Suddenly one cronjob (a shell script) started exiting 1, AFTER rebooting the server and recreating the root crontab.

For some reason, the PATH inside the shell script was "/usr/bin:/bin" and a command that was called was in /usr/sbin.

The script has been running there normally under fcron 3.0.6-r1, now i had to explicitly override the PATH in the script.

I'm not sure if it's a fcron or server issue, i just noticed the problem and googling possible causes brought up this thread.

Will post back after a few days when i find some spare time to get a bigger shovel and start digging.

In the meantime: if anybody here is running the fcron 3.1.1 and you add a shell script (#!/bin/bash) which just outputs the PATH, are you seeing an expected PATH or a truncated one?

Thx.

PS. I searched for a gentoo bug about it and found none, maybe it's a local server issue, altough the upgrade procedure was the same on all servers.
Also, i did the test above:
create /root/testme.sh
#!/bin/bash

echo $PATH

exit 0

and added it as a root cronjob and ran it.

the output was "/usr/bin:/bin".

On another gentoo server running fcron 3.0.6-r5 the output was "/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.5.4", which is the normal PATH of a bash shell script.

So is this a feature or a bug?
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Tue May 14, 2013 10:54 pm    Post subject: Reply with quote

knight77 wrote:
the output was "/usr/bin:/bin". On another gentoo server running fcron 3.0.6-r5 the output was "/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.5.4", which is the normal PATH of a bash shell script.

knight77 ... that looks more like the path one may get when running an interactive shell and having everything from env, generally you wouldn't want gcc in cron's path. Anyhow, same version of fcon:

Code:
#!/bin/sh
echo $PATH | mail -s "test path" root

This provides me with: /sbin:/bin:/usr/sbin:/usr/bin which is what I would expect given that /etc/crontab had the following before being converted to (binary) systab

Code:
# grep '^PATH' /etc/crontab
PATH=/sbin:/bin:/usr/sbin:/usr/bin

So assuming you have the system-crontab useflag enabled, edit /etc/crontab, and re-run 'emerge --config sys-process/fcron'. The systab created in /var/spool/fcron should then have whatever path you need.

best ... khay
Back to top
View user's profile Send private message
knight77
n00b
n00b


Joined: 29 Jun 2009
Posts: 25

PostPosted: Wed May 15, 2013 9:05 am    Post subject: Reply with quote

khayyam wrote:

knight77 ... that looks more like the path one may get when running an interactive shell and having everything from env, generally you wouldn't want gcc in cron's path.


I agree, gcc does not need to be in the PATH for a cronjob. However, most of the paths there should be, at least /sbin and /usr/sbin for root cronjobs.

khayyam wrote:

Anyhow, same version of fcon:
Code:
#!/bin/sh
echo $PATH | mail -s "test path" root

This provides me with: /sbin:/bin:/usr/sbin:/usr/bin which is what I would expect given that /etc/crontab had the following before being converted to (binary) systab
Code:
# grep '^PATH' /etc/crontab
PATH=/sbin:/bin:/usr/sbin:/usr/bin

So assuming you have the system-crontab useflag enabled, edit /etc/crontab, and re-run 'emerge --config sys-process/fcron'. The systab created in /var/spool/fcron should then have whatever path you need.


Just a few observations:
- i was not talking about systab cronjobs, but root ones.
- yes, system-crontab USE flag is enabled

The file /etc/crontab comes with fcron-3.1.1, equery check says all files are intact. /etc/crontab does indeed contain the same PATH you mentioned above.

pkg_config function in fcron-3.1.1 ebuild only checks if you already have a systab, not root crontab (and does nothing if you do), and if not then runs check_system_crontabs which only rebuilds your "systab", nothing related to the root crontab.

My test cronjob was in the root crontab (fcrontab -e), only did echo $PATH and was set as mail(1),mailto(email.address@server.com).

Afaik running fcrontab -e rebuilds the binary root file in /var/spool/fcron, so i have no idea if it also takes the PATH from /etc/crontab (i don't think so).

So the final question would be: Where do the root cronjobs from the root crontab get the PATH? (again, no systab involved, just "fcrontab -e", so the file /var/spool/fcron/root.orig changes and the binary root crontab in /var/spool/fcron/root)
Back to top
View user's profile Send private message
knight77
n00b
n00b


Joined: 29 Jun 2009
Posts: 25

PostPosted: Wed May 15, 2013 9:35 am    Post subject: Reply with quote

I think this issue is caused by this line in the fcron changelog:

Changes from 3.0.6 to 3.1.0:
don't inherit fcron's environment when running a job but build the environment from scratch (cleaner, and this is what Vixie cron does)

The question now is: PATH is set as a hardcoded value "/usr/bin:/bin" or is configurable anywhere (of course, except putting in root's crontab PATH="something").
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Wed May 15, 2013 12:45 pm    Post subject: Reply with quote

knight77 wrote:
The question now is: PATH is set as a hardcoded value "/usr/bin:/bin" or is configurable anywhere (of course, except putting in root's crontab PATH="something").

knight77 ... its configurable in the (f)crontab, I'm not sure why you don't want to do that. The man page states:

Quote:
[...]root can create a systemwide fcrontab file, say /etc/fcrontab, and run "fcrontab /etc/fcrontab" to install the new version after each change of the file. Or (s)he can create a new fcrontab running a simple "fcrontab", and then maintain it using "fcrontab -e".

So, the systab created via 'emerge --config' is that "systemwide fcrontab" and, /etc/fcrontab or /etc/crontab the file used to generate the systab (with the required $PATH). I'm not sure why you want/need both a systab and a 'root crontab' as they serve the same purpose, also, the method for creating these (f)crontabs are the same, so the path can be set to whatever you want.

best ... khay
Back to top
View user's profile Send private message
knight77
n00b
n00b


Joined: 29 Jun 2009
Posts: 25

PostPosted: Wed May 15, 2013 1:37 pm    Post subject: Reply with quote

khayyam wrote:
knight77 ... its configurable in the (f)crontab, I'm not sure why you don't want to do that.


My point in the previous posts was that upto and including fcron 3.0.6, the environment for a running cronjob was inherited from the parent fcron process. That meant that any shell script could call (for instance) programs from /usr/sbin without full path, since /usr/sbin was in the inherited PATH.
Since the changelog for fcron 3.1.0 states that now the environment is built from scratch for every running cronjob, and since i can't find any explicit PATH="/usr/bin:/bin" in any of fcron config files, I was trying to find out if there is a way to specify a PATH to be used by all crontabs (systab, root, some other user etc).
Bottomline: before fcron-3.1.0 I didn't need to explicitly define PATH in any fcrontab (except the systab which uses the definition in /etc/crontab), now I have to.

Quote:
The man page states:
[...]root can create a systemwide fcrontab file, say /etc/fcrontab, and run "fcrontab /etc/fcrontab" to install the new version after each change of the file. Or (s)he can create a new fcrontab running a simple "fcrontab", and then maintain it using "fcrontab -e".
So, the systab created via 'emerge --config' is that "systemwide fcrontab" and, /etc/fcrontab or /etc/crontab the file used to generate the systab (with the required $PATH). I'm not sure why you want/need both a systab and a 'root crontab' as they serve the same purpose, also, the method for creating these (f)crontabs are the same, so the path can be set to whatever you want.


Yes, as i said in my previous post, the PATH is declared in /etc/crontab, but that file is only used by the systab fcrontab, which is not the issue I am having.

The root fcrontab does not include the /etc/crontab file (which is intended for the systab fcrontab).

Yes, if you only use the systab crontab you don't need any PATH change, but if you use multiple fcrontabs, only the systab one uses the PATH defined in /etc/crontab.

The reason why one might need a system crontab and a root crontab is that for instance you have multiple servers running with the same role and only one of them must do some actions, so every server has a root crontab which contains jobs related to that specific server only, while the "common actions-one server doing them" jobs must be managed separately.

Thx.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum