| View previous topic :: View next topic |
| Author |
Message |
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
 Posted: Sun Oct 12, 2014 8:30 am Post subject: runscript debugging |
 |
|
How do I debug init.d scripts?
I'm getting an error during running my script, but can't figure it out what's the matter.
I even put sleep 5 command after every statement to see at which e-prompt it fails, but all the messages scrolls up in the blink of an eye.
Running the script after login is useless, all seems fine, but doesn't work at boot level. |
|
| Back to top |
|
 |
franzf
Advocate
Joined: 29 Mar 2005
Posts: 4565
|
| Posted: Sun Oct 12, 2014 9:07 am Post subject: |
|
|
| Either enable rc_interactive or rc_logger help there. Both variables can be found in /etc/rc.conf. Config files should always be the first place to look at. |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Sun Oct 12, 2014 10:41 am Post subject: |
|
|
Thanks! I've enabled rc_logger, but with enabled rc_logger init script works properly. Seems I have to have it enabled by default 
Anyway I still cant figure it out, what causes a script fault and which statement is wrong when rc_logger disabled |
|
| Back to top |
|
|
franzf
Advocate
Joined: 29 Mar 2005
Posts: 4565
|
| Posted: Sun Oct 12, 2014 1:58 pm Post subject: |
|
|
| creaker wrote: | Thanks! I've enabled rc_logger, but with enabled rc_logger init script works properly. Seems I have to have it enabled by default
Anyway I still cant figure it out, what causes a script fault and which statement is wrong when rc_logger disabled |
You do not have to enable it. I do not have it enabled and everything works fine.
You may want to try the other alternative, rc_interactive.
And nobody can tell you which statement makes your init script fail as long as you don't show the script to us. |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Sun Oct 12, 2014 3:09 pm Post subject: |
|
|
I didn't try rc_interactive because I do not need to interact with rc
| Quote: | # Set rc_interactive to "YES" and you'll be able to press the I key during
# boot so you can choose to start specific services. |
Here the script that refuses to work if rc_logger is disabled. But it works if logging enabled. Script added to boot runlevel
| Code: | #!/sbin/runscript
# Copyright 1999-2009 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
source /etc/portage/make.conf
depend() {
need localmount
}
start() {
ebegin "Mounting read-only usr squashfs image"
mount -rt squashfs -o loop,ro /squashed/usr/usr.sqfs /usr
mount none -t tmpfs -o size=1200M /mnt/ramdisk
einfo "Moving /usr/lib to RAM"
#rsync -a --progress /usr/lib64 /mnt/ramdisk/
cp -a /usr/lib64/* /mnt/ramdisk
ebegin "Mounting read-write usr image with unionfs (aufs)"
mkdir -p /dev/shm/.usr-rw
mount -t aufs -o udba=reval,br=/dev/shm/.usr-rw=rw:/usr=ro usrAufs /usr
ebegin "Mounting read-only lib64 squashfs image"
mount -rt squashfs -o loop,ro /squashed/lib64/lib64.sqfs /lib64
ebegin "Mounting read-write lib64 image with unionfs (aufs)"
mkdir -p /dev/shm/.lib64-rw
mount -t aufs -o udba=reval,br=/dev/shm/.lib64-rw=rw:/lib64=ro lib64Aufs /lib64
ebegin "Mounting read-only lib32 squashfs image"
mount -rt squashfs -o loop,ro /squashed/lib32/lib32.sqfs /lib32
ebegin "Mounting read-write lib32 image with unionfs (aufs)"
mkdir -p /dev/shm/.lib32-rw
mount -t aufs -o udba=reval,br=/dev/shm/.lib32-rw=rw:/lib32=ro lib32Aufs /lib32
ebegin "Mounting read-only home squashfs image"
mount -rt squashfs -o loop,ro /squashed/home/home.sqfs /home
ebegin "Mounting read-write home image with unionfs (aufs)"
mkdir -p /dev/shm/.home-rw
mount -t aufs -o udba=reval,br=/dev/shm/.home-rw=rw:/home=ro homeAufs /home
einfo "Binding ramdisk to /usr/lib64"
mount -o bind /mnt/ramdisk/ /usr/lib64/
eend $?
} |
I think logging may cause some pause/delay (for writing data to log file) and it somehow changes things. Or may be logging causes cashing, not sure.
Anyway, this statements (when doing it manually after login) works properly regardless rc_logger. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Sun Oct 12, 2014 3:48 pm Post subject: |
|
|
Perhaps you should tell us why you think the script does not work. Does it produce some error message? Does it leave the system in an inconsistent state?
What is the point of making /usr/lib64 writable as a RAM disk, but using a unionfs for other directories? Does this system have any writable persistent storage? If so, is that storage written incrementally or only on shutdown? If it is written incrementally, a crash would revert you to the previous version of /usr/lib64, but preserve any directories that were written incrementally, which could cause serious problems. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sun Oct 12, 2014 3:52 pm Post subject: Re: runscript debugging |
|
|
| creaker wrote: | | How do I debug init.d scripts? |
set -x paired with: set +x around the bit you want to debug is always useful.
You're playing with fs which is always fun.. ;) These functions should come in handy to chase down the dependency interaction. As you'll see from the linked udev-no-initramfs thread, the interaction post-localmount is quite tricky. Really it should be in its own FS runlevel, imo, as several initscripts have long dep strings to try and start relatively early after localmount (which is where the complexity starts, as they often have to account for each other, somewhere in the chain, as well as whatever they're actually there for, and those parts normally have some sort of ordering too.)
Another tip is to save the exact info you need out to a file; I had to do this to debug a problem with lvm (which was later fixed in openrc.) If you can't see the set -x output in the log, for example, you'd redirect stderr before the set -x. This is easiest when done by the call to a function, since it will revert back on return, or some other compound like an 'if'.
HTH,
steveL.
edit: Please answer Hu's points, before you go diving into the dep-chain, as that can take up hours.. ;) |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Sun Oct 12, 2014 5:24 pm Post subject: |
|
|
| Hu wrote: | | Perhaps you should tell us why you think the script does not work. Does it produce some error message? Does it leave the system in an inconsistent state? |
This script should mount (in addition to regular filesystems) this stuffs: {usr, lib32, lib64, home}.sqfs as loop devices. The goal of using squashfs is getting filesystem size as small as possible (to fit usb pen drive size). Once squashed filesystems attached at loop0 - loop3, I have to mount them to /usr, /lib32, /lib64 and /home respectively. Why I decided that script doesn't work?
Because
a) /usr, /lib32, /lib32, /home mount points are missed in df -h output.
b) X doesn't start (as well as any command located under /usr)
c) Only very basic commands (from /bin /sbin) are available. Using them I can log in and run script manually "/etc/init.d/script start".
d) Once I ran script by hand, I getting all the filesystems mounted (df confirms it). I can start X and system runs fine.
e) When starting script manually I don't get any errors. It means nothing was made when script was lanched by rc at boot runlevel. Otherwise I would get an errors ("already mounted" "directory exists" etc).
When script launched at boot runlevel it produces few error messages, but I can't read them because they scrolls out of screen too fast.
This topic aimed to get idea on how can I catch these error messages.
However, if rc_logger enabled, script works just fine. No need to launch it by hand after login. rc.log is fine as well, no any error messages.
| Hu wrote: |
What is the point of making /usr/lib64 writable as a RAM disk, but using a unionfs for other directories? Does this system have any writable persistent storage? If so, is that storage written incrementally or only on shutdown? If it is written incrementally, a crash would revert you to the previous version of /usr/lib64, but preserve any directories that were written incrementally, which could cause serious problems. |
Regarding writable RAM disk - OK, it's an mistake, have to add "ro"
Filesystems changes can be saved (added to squashed images) but only on shutdown (not incrementally) right before unmounting, if "savechanges" option was added to kernel command line. Script has stop() routine that checks for changes and updates squashed filesystem if neccessary:
| Code: | stop() {
einfo "Checking for savechanges"
LINE=$(cat /proc/cmdline)
if [[ $LINE == *savechanges* ]]
then
ebegin "Updating home image"
if [ ! -z `ls -A /dev/shm/.home-rw | grep -v '.wh..wh.' | head -n1` ]
then
einfo " Sync the home directory"
mv -f /squashed/home/home-current.sqfs /squashed/home/home-old.sqfs
mksquashfs /home/ /squashed/home/home-current.sqfs -no-duplicates 2>/dev/null
ln -sf /squashed/home/home-current.sqfs /squashed/home/home.sqfs
else
einfo " Nothing to save"
fi
fi
ebegin "Unmounting home"
umount -t aufs /home
umount -t squashfs /home
rm -rf /dev/shm/.home-rw
}
|
the same snippets of code for other filesystems (/usr, /lib32, /lib64)
This part of code (stop routine) works fine.
steveL, thanks for the tip. Have to read & learn. Seems you have a tips for any possible issue  |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sun Oct 12, 2014 8:33 pm Post subject: |
|
|
| Code: | stop() {
einfo "Checking for savechanges"
LINE=$(cat /proc/cmdline)
if [[ $LINE == *savechanges* ]]
then
ebegin "Updating home image"
if [ ! -z `ls -A /dev/shm/.home-rw | grep -v '.wh..wh.' | head -n1` ]
then
einfo " Sync the home directory"
mv -f /squashed/home/home-current.sqfs /squashed/home/home-old.sqfs
mksquashfs /home/ /squashed/home/home-current.sqfs -no-duplicates 2>/dev/null
ln -sf /squashed/home/home-current.sqfs /squashed/home/home.sqfs
else
einfo " Nothing to save"
fi
fi
ebegin "Unmounting home"
umount -t aufs /home
umount -t squashfs /home
rm -rf /dev/shm/.home-rw
}
|
| creaker wrote: | the same snippets of code for other filesystems (/usr, /lib32, /lib64)
This part of code (stop routine) works fine. |
Missing 'fi' but I'm sure you noticed ;)
The capture of ls is undoubtedly a bad idea; wtf are you trying to test there? (additionally, stick to $(..) for command substitution, don't use backticks.)
You should /autojoin #bash on IRC: chat.freenode.net and lurk there; you'll see lots of script flying by and being corrected, and urls which you should look up to understand. In this instance: /msg greybot ls
They'll teach you sh as well, if you tell them upfront that's what you're there to learn; and you can always come to #friendly-coders for more help. ;)
You can get rid of the if altogether (in bash):
| Code: | | [[ $(cat /proc/cmdline) = *savechanges* ]] || return 0 |
and in fact the cat as well with a bashism:
| Code: | | [[ $(</proc/cmdline) = *savechanges* ]] || return 0 |
However all that leads us to is, that this should be POSIX sh.
Get out of the habit of using == as it's not portable, and a waste of time. In fact when you both code and script, you start to appreciate the different operator. In my head, == is numeric.
The presence of a [ or [[ indicates a string conditional, and there is simply no way to confuse: [ foo = "$bar" ] with: foo=$bar
(or the lexer/parser combi would have a much harder time, and sh would never have been implemented.)
Even if you quote the assignment which is not needed there, as assignment is never field-split, just like the word after case:
| Code: |
stop() {
einfo 'Checking for savechanges'
case $(cat /proc/cmdline) in
*savechanges*) :
;; *) return 0
esac
..
|
Though you appear to have used that in your LINE=$(..); another point is don't use CAPS for anything except env vars, and occasional constants like EOL or TAB. Your line var should also have been local, which is specified for any sh that wants to be a candidate for debian, so most of them have it. Certainly all the ones on Gentoo do, sometimes under an alias for typeset (ksh derivatives.)
From what you wrote above about various file-systems, you probably want to make this into a function.
| Quote: | | steveL, thanks for the tip. Have to read & learn. Seems you have a tips for any possible issue :) |
Hehe; just happened to have done a very similar thing, which isn't that surprising given our common interest in a lean machine.
You're always welcome. :-)
Regards,
igli
edit: use 'cat' in the sh 'case'. |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Wed Oct 15, 2014 10:22 am Post subject: |
|
|
I'm back with some new info.
| Code: | USB Flash Drive Read rate Write rate Boot time
Kingston DT101 G2 18.3 Mb/s 2.6 Mb/s 75 sec
Silicon Power Ultima U02 16.2 Mb/s 5.1 Mb/s 53 sec
Smartby Micro SD 17.2 Mb/s 10.6 Mb/s 45 sec |
Initially I had a system installed at Kingston DT101. Yesterday I bought Silicon Power stick and Smartby microSD card. And simply cloned Kingston to them with dd. No any modifications, just ran dd over new drives.
Both SP and Smartby boots just fine, without any issues. Regardless rc_logger option state. Noteworthy, that these two boots faster than Kingston (with rc_logger), though Kingston has a better read rate.
I think boot process somehow depends on write speed. And definetly it isn't a script itself issue.
I think (in case of Kingston) that other init scripts not yet finished their job (possibly due to low write rate) and my script fails due to something not ready yet.
It's my assumption.
@steveL
Thanks for your tips with ridding cat of. However you still trying to drag me into programming swamp. However I'm too old to start study bash, posix standards etc. Once I abandoned assembler I do not studied any other languages from very beginning. When I need to write some script I just googling for my tasks and using a code snippets that most suitable for me. Yes, they not optimal, especially when I made some corrections on them. Nevertheless, I really appreciate your help. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Wed Oct 15, 2014 1:34 pm Post subject: |
|
|
| creaker wrote: | | However I'm too old to start study bash, posix standards etc. Once I abandoned assembler I do not studied any other languages from very beginning. When I need to write some script I just googling for my tasks and using a code snippets that most suitable for me. Yes, they not optimal, especially when I made some corrections on them. Nevertheless, I really appreciate your help. |
Heh that's fine; I'll correct them as I see them then. Just don't expect me not to ;) as it's the same as being in #bash for me. Maybe it'll help you by osmosis, and if not, then at least the bash or sh corrections/efficiency improvements will be documented, as well as in your scripts.
This is in my own self-interest, afaic. It's better for me if more Gentoo users are writing decent shell, than not. For a start it increases the base of things I have to draw on, as well as the pool of people capable of writing more. The corrections become less over time, especially in relation to a particular script, which eases any integration I might later have to do (since I'm the one in my team who gets to do the bash, and the shell-scripting. ;)
But really, it's harder for me not to correct it, due to the ingrained twitch from #bash (and greycat.;)
wrt ordering issues, it's hard to comment without seeing output from: Usually without -v is enough, and what I normally check. But for someone else's machine, you'd want to see what they're not starting, just in case it matters. |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Wed Oct 15, 2014 2:21 pm Post subject: |
|
|
| Code: | UsbGen cr # rc-update -v show
bootmisc | boot
busybox-ntpd |
busybox-watchdog |
consolefont | default
cups-browsed |
cupsd |
dbus |
devfs | sysinit
device-mapper |
dmcrypt |
dmesg | sysinit
dmeventd |
fsck | boot
fuse |
git-daemon |
gpm |
hdparm |
hostname | boot
hwclock | boot
keymaps | boot
killprocs | shutdown
kmod-static-nodes | sysinit
local | default
localmount | boot
loopback | boot
lvm |
lvm-monitoring |
lvmetad |
metalog |
modules | boot
mount-ro | shutdown
mtab | boot
mysql |
net.eth0 | default
net.lo |
netmount | default
nullmailer |
numlock |
pciparm |
procfs | boot
pwcheck |
pydoc-2.7 |
pydoc-3.3 |
root | boot
rsyncd |
saslauthd |
savecache | shutdown
smartd |
squash-all | boot
squash-all.bak |
sshd |
swap | boot
swapfiles | boot
swclock |
sysctl | boot
sysfs | sysinit
termencoding | boot
tmpfiles.dev | sysinit
tmpfiles.setup | boot
udev | sysinit
udev-mount | sysinit
udev-postmount | default
urandom | boot
xdm | default
xdm-setup | |
I have no idea on about what service my script (squash-all) might stumble. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Oct 16, 2014 3:34 am Post subject: |
|
|
| creaker wrote: | | I have no idea on about what service my script (squash-all) might stumble. |
Well it would help if you had looked at the udev-without-initramfs page I mentioned earlier.
Basically you need to copy the before line from the patched udev initscript. I think I'd add after udev as well.
As you're starting in exactly the same position as udev does in our setup, with similar fs-based constraint, you can just steal the line without doing all the hard work. (It took me quite a while to puzzle out that set, using the functions I showed earlier.) |
|
| Back to top |
|
|
creaker
l33t
Joined: 14 Jul 2012
Posts: 651
|
| Posted: Thu Oct 16, 2014 5:27 am Post subject: |
|
|
For some unknown reason the system became bootable again (at the old Kingston stick). It's weird.
I decided not to put "after udev" string there. If udev is a real problem causer, and problem will go due to this string, I'll do not know it for sure. Instead I added "rc-status" command at the very beginning to see what a services still running when script starts. If a system will stuck at my script again I can compare running services lists for bootable and unbootable states.
So have to wait until it stops boot again. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Oct 16, 2014 4:44 pm Post subject: |
|
|
It's the before line that matters.
After udev isn't relevant for you since it's in a different run-level; it would be if your script were used by others. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
