finit "rescue" linux - unable to shutdown [SOLVED]

[Zucca]

Hi.
I've been learning and creating my own rescue... thing... using chrooted environment (with -Oz -march=x86-64).
Right now I'm running this all by:

• initramfs contains zram module along with zramctl, busybox and mkfs.xfs
• init script creates zram device, formats it to xfs, then unpacks another cpio into the newly created zram device
• switch_root into the new root which is in that zram device
• perform few commands inside the new root, then simply exec /sbin/init, which is finit

... now everything goes fine, but I noticed I cannot power off the system. I dig a little deeper and found out that not a single signal goes trough. kill 1 should make finit reboot the machine. Running /sbin/reboot will make finit change runlevel to 6 and run the associated commands, but it never manages to tell kernel to reboot (and it proceeds to switching to S runlevel).

Any, even if wild, guesses, what's going on? Why all signals are blocked? I cannot even kill the busybox shell.
If I switch to busybox init rebooting works, like does poweroff. I suspect I'm missing something with finit...

EDIT: Changed the topic title slightly to avoid errors when replying.

EDIT2: Changed the title again to more accurately reflect the discussion.

[Hu]

When booted in the bad configuration, what is the output of grep -e '^Sig' /proc/1/status? I wonder if something is leaving signals blocked, and finit does not unblock them (or perhaps finit is blocking them on purpose). Also, note this warning from man 2 kill:

Code: Select all

       The  only  signals  that can be sent to process ID 1, the init process,
       are those for which init  has  explicitly  installed  signal  handlers.
       This is done to assure the system is not brought down accidentally.

Perhaps finit is not setting a handler for the signal you wish to use.

[jburns]

https://finit-project.github.io/finit/signals/ may help.

[Zucca]

When booted in the bad configuration, what is the output of grep -e '^Sig' /proc/1/status?

Code: Select all

~ # grep -e '^Sig' /proc/1/status
SigQ:   0/793
SigPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: fffffffe7ffafeff
SigCgt: 0000000000010000

Ok. Now... Do I interpret this correctly, that SigIgn field seem to indicate that almost all signals are ignored?
Earlier when I set up this system I didn't have properly set config for finit, it was was able to reboot (it actually ended up in boot loop).

I wonder if doing exec from busybox sh to finit somehow triggers this.

[Zucca]

https://finit-project.github.io/finit/signals/ may help.

Been there about 30 times already. ;)

[pingtoo]

Hi.
I've been learning and creating my own rescue... thing... using chrooted environment (with -Oz -march=x86-64).
Right now I'm running this all by:

• initramfs contains zram module along with zramctl, busybox and mkfs.xfs
• init script creates zram device, formats it to xfs, then unpacks another cpio into the newly created zram device
• switch_root into the new root which is in that zram device
• perform few commands inside the new root, then simply exec /sbin/init, which is finit

... now everything goes fine, but I noticed I cannot power off the system. I dig a little deeper and found out that not a single signal goes trough. kill 1 should make finit reboot the machine. Running /sbin/reboot will make finit change runlevel to 6 and run the associated commands, but it never manages to tell kernel to reboot (and it proceeds to switching to S runlevel).

Any, even if wild, guesses, what's going on? Why all signals are blocked? I cannot even kill the busybox shell.
If I switch to busybox init rebooting works, like does poweroff. I suspect I'm missing something with finit...

When signaled to finit, what linux state it is in? does all processes killed? or instead poweroff/reboot is is possible to switch to single user mode?

[Zucca]

When signaled to finit, what linux state it is in? does all processes killed? or instead poweroff/reboot is is possible to switch to single user mode?

• Not sure what you mean by "state" in this context. Runlevel 2?
• No. Processes dont't get killed.
  • This may be a good pointer to where look next.
• Yes. I can do

  Code: Select all

  finit 1

  ... to enter single user mode for example.
  • And I can also initiate reboot by

    Code: Select all

    finit 6

    ... which fails the same way.

[Anon-E-moose]

What does verbose or debug mode show?

[Zucca]

What does verbose or debug mode show?

Trying to figure out how that's supposed to work with finit. ;)

However...

Code: Select all

[   16.458072] XFS (zram0): Ending clean mount
[   29.760903] finit[1]: FINIT - the fast init? v9999, entering runlevel S
[   33.253740] finit[1]: Starting keventd[718]
[   34.345114] finit[1]: FINIT - the fast init? v9999, entering runlevel 2
[   34.403384] finit[1]: Starting getty on /dev/tty1
[   34.440236] finit[1]: Starting tty:1[720]
[   34.482178] finit[1]: Starting tty:2[723]
[   34.515677] finit[1]: Starting tty:S0[725]
[   34.527026] finit[1]: Starting getty on /dev/ttyS0
[   35.375021] fbcon: Taking over console
[   35.389657] Console: switching to colour frame buffer device 160x50
[   96.530063] finit[1]: Starting init_rc.sh[945]
[   96.532010] finit[1]: FINIT - the fast init? v9999, entering runlevel 0
[   96.547850] finit[1]: tty:1[720], stopping, sending SIGHUP ...
[   96.555437] finit[1]: tty:2[723], stopping, sending SIGHUP ...
[   96.559183] finit[1]: tty:S0[725], stopping, sending SIGHUP ...
[   96.562554] finit[1]: keventd[718], stopping, sending SIGTERM ...
[   98.947354] finit[1]: conf_save_service():File /var/lib/run/finit/system/keventd.conf already exists, overwriting.
[   99.098030] finit[1]: Starting keventd[997]
[  100.134957] finit[1]: FINIT - the fast init? v9999, entering runlevel 2
[  100.180350] finit[1]: Starting getty on /dev/tty1
[  100.239289] finit[1]: Starting tty:1[998]
[  100.267374] finit[1]: Starting tty:2[1001]
[  100.295387] finit[1]: Starting tty:S0[1004]

gives something to look at... Nothing new there. :(

[pingtoo]

When signaled to finit, what linux state it is in? does all processes killed? or instead poweroff/reboot is is possible to switch to single user mode?

• Not sure what you mean by "state" in this context. Runlevel 2?
• No. Processes dont't get killed.
  • This may be a good pointer to where look next.
• Yes. I can do

  Code: Select all

  finit 1

  ... to enter single user mode for example.
  • And I can also initiate reboot by

    Code: Select all

    finit 6

    ... which fails the same way.

linux state, as in did file system flushed? did file system got unmounted? rootfs should be switch to read-only, processes killed? etc... But you answered the question since no process killed.

In single user mode, in theory there should only shell process. So when successful finish finit 1, using ps -ef did you notice unusual process(es)? for example when I look at finit:src/sig.c in do_shutdown() it did a vfork() so there could be two finit processes at same time, did that happen?

I guess my idea is to looking for evidence if signal received or block/ignored, is it received but stuck in some code logic? or it simply block/ignored therefor nothing happen.

Also I think console should have some messages indicated something were happening. (i.e. finit received signal and start some action(s))

[pingtoo]

What does verbose or debug mode show?

Trying to figure out how that's supposed to work with finit.

However...

Code: Select all

[   16.458072] XFS (zram0): Ending clean mount
[   29.760903] finit[1]: FINIT - the fast init? v9999, entering runlevel S
[   33.253740] finit[1]: Starting keventd[718]
[   34.345114] finit[1]: FINIT - the fast init? v9999, entering runlevel 2
[   34.403384] finit[1]: Starting getty on /dev/tty1
[   34.440236] finit[1]: Starting tty:1[720]
[   34.482178] finit[1]: Starting tty:2[723]
[   34.515677] finit[1]: Starting tty:S0[725]
[   34.527026] finit[1]: Starting getty on /dev/ttyS0
[   35.375021] fbcon: Taking over console
[   35.389657] Console: switching to colour frame buffer device 160x50
[   96.530063] finit[1]: Starting init_rc.sh[945]
[   96.532010] finit[1]: FINIT - the fast init? v9999, entering runlevel 0
[   96.547850] finit[1]: tty:1[720], stopping, sending SIGHUP ...
[   96.555437] finit[1]: tty:2[723], stopping, sending SIGHUP ...
[   96.559183] finit[1]: tty:S0[725], stopping, sending SIGHUP ...
[   96.562554] finit[1]: keventd[718], stopping, sending SIGTERM ...
[   98.947354] finit[1]: conf_save_service():File /var/lib/run/finit/system/keventd.conf already exists, overwriting.
[   99.098030] finit[1]: Starting keventd[997]
[  100.134957] finit[1]: FINIT - the fast init? v9999, entering runlevel 2
[  100.180350] finit[1]: Starting getty on /dev/tty1
[  100.239289] finit[1]: Starting tty:1[998]
[  100.267374] finit[1]: Starting tty:2[1001]
[  100.295387] finit[1]: Starting tty:S0[1004]

gives something to look at... Nothing new there.

Only saw this after I post my message
this looks like it is system boot (because the left most column for time) so what is dmesg after you did finit 1?

[Zucca]

so what is dmesg after you did finit 1?

That is the dmesg output after I ran finit 0
Running kill 1 should initiate reboot, but nothing happens.

I'll crank up kernel logging...
EDIT: No dice. But I noticed that running "killall init" does initiate a reboot, although at the final step of reboot (runlevel 6) finit enters back to runlevel 2. So it seems that signals get trough after all...

[pingtoo]

so what is dmesg after you did finit 1?

That is the dmesg output after I ran finit 0
Running kill 1 should initiate reboot, but nothing happens.

I'll crank up kernel logging...
EDIT: No dice. But I noticed that running "killall init" does initiate a reboot, although at the final step of reboot (runlevel 6) finit enters back to runlevel 2. So it seems that signals get trough after all...

have you try to use "initctl debug" to switch finit into debug mode?
I don't use finit so I can just guess from looking at source code tree.

I can see from source once debug mode turn on, many information would send to console. one interesting point would be when switch runlevel, the source code finit:src/sm.c sm_step() indicate it will print out state changes to stderr which I presume it is attach to console.

[Anon-E-moose]

initctl debug or initctl -v debug (not sure how much more detail is available with -v)

Judging by the documentation
initctl -v debug
initctl runlevel 1 (or whatever)

https://finit-project.github.io/finit/initctl/

[Zucca]

Now... for some reason finit no longer runs the commands I've set it to run during sysinit/S runlevel.
I feel I need to look at this with fresh eyes at some other day.
As a sanity check I temporarily switched back to busybox init. Reboot and poweroff do work there...

I'll try to enable the debug mode via initctl.
Previously I tried to exec /sbin/init debug...

[Zucca]

So. I should be able to activate debug mode by passing

Code: Select all

-- debug

on the kernel command line... Or

Code: Select all

-- finit.debug

... instructions differ depending where I read them.
Nontheless: either way did not work.

Code: Select all

initctl debug

... however did. This means I cannot get all the debugging messages from the very first steps.

Code: Select all

[  323.787325] finit[1]: Debug mode enabled
[  341.756951] finit[1]: api_cb():Setting new runlevel 0
[  341.757053] finit[1]: sm_runlevel():Flushing pending .conf file events ...
[  341.757086] finit[1]: sm_step():state: running, runlevel: 2, newlevel: 0, teardown: 0, reload: 0
[  341.757120] finit[1]: service_step():               tty:1( 945):  running  enabled/clean   cond:on  
[  341.757447] finit[1]: service_step():               tty:2( 975):  running  enabled/clean   cond:on  
[  341.757747] finit[1]: service_step():              tty:S0( 977):  running  enabled/clean   cond:on  
[  341.758057] finit[1]: service_step():        pre-shutdown(   0):   halted disabled/clean   cond:on  
[  341.758358] finit[1]: service_step():             keventd( 944):  running  enabled/clean   cond:on  
[  341.758656] finit[1]: cond_set_noupdate():service/keventd/ready
[  341.758967] finit[1]: cond_set_path():/run/finit/cond/service/keventd/ready <= 2
[  341.759355] finit[1]: sm_step():state: runlevel/change, runlevel: 2, newlevel: 0, teardown: 0, reload: 0
[  341.759600] finit[1]: cond_set_oneshot():hook/sys/shutdown
[  341.759868] finit[1]: cond_set_oneshot_noupdate():hook/sys/shutdown => /run/finit/cond/hook/sys/shutdown
[  341.760197] finit[1]: service_step():        pre-shutdown(   0):   halted  enabled/clean   cond:on  
[  341.761107] finit[1]: service_step():        pre-shutdown(   0): ->  waiting
[  341.761378] finit[1]: service_step():        pre-shutdown(   0):  waiting  enabled/clean   cond:on  
[  341.761681] finit[1]: service_step():        pre-shutdown(   0): -> starting
[  341.761980] finit[1]: service_step():        pre-shutdown(   0): starting  enabled/clean   cond:on  
[  341.763906] finit[1]: cgroup_leaf_init():group system, name finit, pid 1040, cfg 
[  341.764091] finit[1]: group_init():path /sys/fs/cgroup/system/finit, leaf 1, cfg 
[  341.764534] finit[1]: Calling /bin/sh /usr/local/libexec/init_rc.sh shutdown
[  341.764982] finit[1]: exec_runtask():Calling /bin/sh /usr/local/libexec/init_rc.sh shutdown
[  341.910668] finit[1]: iwatch_add1():adding new watcher for path /sys/fs/cgroup/system/finit/cgroup.events
[  341.910764] finit[1]: service_start():Starting pre-shutdown as PID 1040
[  341.910805] finit[1]: Starting pre-shutdown[1040]
[  341.911195] finit[1]: service_step():        pre-shutdown(1040): ->  running
[  341.911530] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on  
[  341.911909] finit[1]: sm_step():Setting new runlevel --> 0 <-- previous 2
[  341.912079] finit[1]: FINIT - the fast init? v4.14, entering runlevel 0
[  341.912411] finit[1]: sm_step():Stopping services not allowed in new runlevel ...
[  341.912790] finit[1]: service_step():               tty:1( 945):  running disabled/clean   cond:on  
[  341.913234] finit[1]: service_stop():Sending SIGHUP to pid:945 name:tty:1(dialog_login.sh)
[  341.913508] finit[1]: tty:1[945], stopping, sending SIGHUP ...
[  341.913711] finit[1]: service_stop():kill(945, 1) => rc 0, errno 2
[  341.914048] finit[1]: svc_set_state():tty:1 is stopping, wait 3 sec before sending SIGKILL ...
[  341.914418] finit[1]: service_step():               tty:1( 945): -> stopping
[  341.914757] finit[1]: service_step():               tty:1( 945): stopping disabled/clean   cond:on  
[  341.915124] finit[1]: service_step():               tty:2( 975):  running disabled/clean   cond:on  
[  341.915528] finit[1]: service_stop():Sending SIGHUP to pid:975 name:tty:2(getty tty2)
[  341.915834] finit[1]: tty:2[975], stopping, sending SIGHUP ...
[  341.916231] finit[1]: service_stop():kill(975, 1) => rc 0, errno 2
[  341.916259] finit[1]: svc_set_state():tty:2 is stopping, wait 3 sec before sending SIGKILL ...
[  341.916457] finit[1]: service_step():               tty:2( 975): -> stopping
[  341.916793] finit[1]: service_step():               tty:2( 975): stopping disabled/clean   cond:on  
[  341.917151] finit[1]: service_step():              tty:S0( 977):  running disabled/clean   cond:on  
[  341.917527] finit[1]: service_stop():Sending SIGHUP to pid:977 name:tty:S0(sh)
[  341.917834] finit[1]: tty:S0[977], stopping, sending SIGHUP ...
[  341.918195] finit[1]: service_stop():kill(977, 1) => rc 0, errno 2
[  341.918354] finit[1]: svc_set_state():tty:S0 is stopping, wait 3 sec before sending SIGKILL ...
[  341.918703] finit[1]: service_step():              tty:S0( 977): -> stopping
[  341.918864] finit[1]: service_step():              tty:S0( 977): stopping disabled/clean   cond:on  
[  341.919215] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on  
[  341.919563] finit[1]: service_step():             keventd( 944):  running disabled/clean   cond:on  
[  341.919773] finit[1]: service_stop():Sending SIGTERM to pid:944 name:keventd(keventd)
[  341.920078] finit[1]: keventd[944], stopping, sending SIGTERM ...
[  341.920459] finit[1]: service_stop():kill(944, 15) => rc 0, errno 2
[  341.920767] finit[1]: svc_set_state():keventd is stopping, wait 3 sec before sending SIGKILL ...
[  341.921137] finit[1]: cond_clear_noupdate():service/keventd/
[  341.921292] finit[1]: cond_set_path():/run/finit/cond/service/keventd/ <= 0
[  341.921539] finit[1]: cond_clear_noupdate():service/keventd/ready
[  341.921820] finit[1]: cond_set_path():/run/finit/cond/service/keventd/ready <= 0
[  341.922196] finit[1]: cond_clear_noupdate():service/keventd/running
[  341.922511] finit[1]: cond_set_path():/run/finit/cond/service/keventd/running <= 0
[  341.923029] finit[1]: service_step():             keventd( 944): -> stopping
[  341.923188] finit[1]: service_step():             keventd( 944): stopping disabled/clean   cond:on  
[  341.923871] finit[1]: sm_step():state: runlevel/wait, runlevel: 0, newlevel: -1, teardown: 1, reload: 0
[  341.924223] finit[1]: sm_step():Waiting to collect tty:1, cmd /bin/getty(945) ...
[  341.924391] finit[1]: networking():Taking down networking ...
[  341.961976] finit[1]: networking():Calling all network down hooks ...
[  341.962078] finit[1]: cond_set_oneshot():hook/net/down
[  341.962406] finit[1]: cond_set_oneshot_noupdate():hook/net/down => /run/finit/cond/hook/net/down
[  341.962753] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on  
[  341.963092] finit[1]: api_cb():Failed sending ACK/NACK back to client
[  341.963447] finit[1]: sigchld_cb():Collected child PID 944, status: 0
[  341.963754] finit[1]: service_monitor():collected keventd(944), normal exit: 1, signaled: 0, exit code: 0
[  341.964098] finit[1]: service_monitor():Killing lingering children in same process group ...
[  341.964781] finit[1]: service_step():             keventd(   0): stopping disabled/clean   cond:on  
[  341.965112] finit[1]: service_step():keventd: stopped, cleaning up timers and conditions ...
[  341.965440] finit[1]: cond_clear_noupdate():pid/keventd
[  341.965611] finit[1]: cond_set_path():/run/finit/cond/pid/keventd <= 0
[  341.965794] finit[1]: cond_clear_noupdate():service/keventd/
[  341.966160] finit[1]: cond_set_path():/run/finit/cond/service/keventd/ <= 0
[  341.966879] finit[1]: cond_set_oneshot():service/keventd/halted
[  341.967173] finit[1]: cond_set_oneshot_noupdate():service/keventd/halted => /run/finit/cond/service/keventd/halted
[  341.967541] finit[1]: service_step():             keventd(   0): ->   halted
[  341.967836] finit[1]: service_step():             keventd(   0):   halted disabled/clean   cond:on  
[  341.968179] finit[1]: sm_step():state: runlevel/wait, runlevel: 0, newlevel: -1, teardown: 1, reload: 0
[  341.968527] finit[1]: sm_step():Waiting to collect tty:1, cmd /bin/getty(945) ...
[  341.968713] finit[1]: sigchld_cb():Collected child PID 945, status: 1
[  341.968864] finit[1]: service_monitor():collected tty:1(945), normal exit: 0, signaled: 1, exit code: 0
[  341.969199] finit[1]: service_monitor():Killing lingering children in same process group ...
[  341.969590] finit[1]: service_step():               tty:1(   0): stopping disabled/clean   cond:on  
[  341.969866] finit[1]: service_step():tty:1: stopped, cleaning up timers and conditions ...
[  341.970207] finit[1]: cond_clear_noupdate():pid/tty:1
[  341.970374] finit[1]: cond_set_path():/run/finit/cond/pid/tty:1 <= 0
[  341.970929] finit[1]: service_step():               tty:1(   0): ->   halted
[  341.971218] finit[1]: service_step():               tty:1(   0):   halted disabled/clean   cond:on  
[  341.971550] finit[1]: sm_step():state: runlevel/wait, runlevel: 0, newlevel: -1, teardown: 1, reload: 0
[  341.971900] finit[1]: sm_step():Waiting to collect tty:2, cmd /bin/getty(975) ...
[  341.972239] finit[1]: sigchld_cb():Collected child PID 975, status: 1
[  341.972553] finit[1]: service_monitor():collected tty:2(975), normal exit: 0, signaled: 1, exit code: 0
[  341.972882] finit[1]: service_monitor():Killing lingering children in same process group ...
[  341.973268] finit[1]: service_step():               tty:2(   0): stopping disabled/clean   cond:on  
[  341.973552] finit[1]: service_step():tty:2: stopped, cleaning up timers and conditions ...
[  341.973887] finit[1]: cond_clear_noupdate():pid/tty:2
[  341.974072] finit[1]: cond_set_path():/run/finit/cond/pid/tty:2 <= 0
[  341.974249] finit[1]: service_step():               tty:2(   0): ->   halted
[  341.974403] finit[1]: service_step():               tty:2(   0):   halted disabled/clean   cond:on  
[  341.974745] finit[1]: sm_step():state: runlevel/wait, runlevel: 0, newlevel: -1, teardown: 1, reload: 0
[  341.975079] finit[1]: sm_step():Waiting to collect tty:S0, cmd /bin/getty(977) ...
[  341.975421] finit[1]: sigchld_cb():Collected child PID 977, status: 1
[  341.975584] finit[1]: service_monitor():collected tty:S0(977), normal exit: 0, signaled: 1, exit code: 0
[  341.975914] finit[1]: service_monitor():Killing lingering children in same process group ...
[  341.976112] finit[1]: service_step():              tty:S0(   0): stopping disabled/clean   cond:on  
[  341.976443] finit[1]: service_step():tty:S0: stopped, cleaning up timers and conditions ...
[  341.976767] finit[1]: cond_clear_noupdate():pid/tty:S0
[  341.977101] finit[1]: cond_set_path():/run/finit/cond/pid/tty:S0 <= 0
[  341.977276] finit[1]: service_step():              tty:S0(   0): ->   halted
[  341.977434] finit[1]: service_step():              tty:S0(   0):   halted disabled/clean   cond:on  
[  341.977750] finit[1]: sm_step():state: runlevel/wait, runlevel: 0, newlevel: -1, teardown: 1, reload: 0
[  341.978081] finit[1]: sm_step():All services have been stopped, calling runlevel change hooks ...
[  341.978418] finit[1]: cond_set_oneshot():nop
[  341.978578] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on  
[  341.978913] finit[1]: sm_step():Starting services new to this runlevel ...
[  341.979242] finit[1]: service_step():               tty:1(   0):   halted disabled/clean   cond:on  
[  341.979571] finit[1]: service_step():               tty:2(   0):   halted disabled/clean   cond:on  
[  341.979911] finit[1]: service_step():              tty:S0(   0):   halted disabled/clean   cond:on  
[  341.980236] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on  
[  341.980606] finit[1]: service_step():             keventd(   0):   halted disabled/clean   cond:on  
[  341.980911] finit[1]: sm_step():state: runlevel/clean, runlevel: 0, newlevel: -1, teardown: 0, reload: 0
[  342.234524] finit[1]: do_iterate_proc():PID 830 is still alive (/sbin/init)
[  342.485984] finit[1]: do_iterate_proc():PID 830 is still alive (/sbin/init)
[  342.737409] finit[1]: do_iterate_proc():PID 830 is still alive (/sbin/init)
[  342.988845] finit[1]: do_iterate_proc():PID 830 is still alive (/sbin/init)
[  343.038319] finit[1]: conf_save_service():File /var/lib/run/finit/system/keventd.conf already exists, overwriting.
[  343.141123] finit[1]: Starting post-init[1094]
[  343.145440] finit[1]: Starting keventd[1170]
[  344.142330] finit[1]: FINIT - the fast init? v4.14, entering runlevel 2
[  344.144575] finit[1]: Starting getty on /dev/tty1
[  344.190670] finit[1]: Starting tty:1[1171]
[  344.192182] finit[1]: Starting tty:2[1207]
[  344.193917] finit[1]: Starting tty:S0[1208]
[  344.194497] finit[1]: Starting getty on /dev/ttyS0
[  362.610817] clocksource: timekeeping watchdog on CPU4: hpet wd-wd read-back delay of 185260ns
[  362.610827] clocksource: wd-tsc-wd read-back delay of 151950ns, clock-skew test skipped!

I don't see anything unusual.
do_iterate_proc():PID 830 is still alive (/sbin/init) also seem to be just a loop checking if a process has terminated itself.

*sigh*
I so much would like to use finit with this project.

[pingtoo]

Can you tell why you have /sbin/init (pid: 830) running? is it because there is a container? or some kind of light-weighted VM? or is this finit running inside a container?

I mean I would expect /sbin/init will always stay as PID:1 therefor I could not understand what is /sbin/init as PID:830 mean.

do_iterate_proc() is meant to terminate "non-monitored processes", initially it will send SIGTERM and eventually it will send SIGKILL.

do_iterate_proc() was called by do_sutdown([halt|reboot]) and if do_iterate_proc() finish do_shutdown() will fork a child process to call reboot(), I wonder if you can tell after you issue shutdown does rootfs becomd read-only? because that part of the forked child process do, There are little debug messages in the forked child process, mostly messages were print out to console, does console show anything about rebooting...?

[Zucca]

finit spawns more processes as it starts its internal services (like a getty service).
Yes it would be nicer it finit had a separate binary for such tasks. But since it doesn't, all those processes have /sbin/init as their command.
So, no containers here. I test run this inside qemu with and without kvm.

does console show anything about rebooting...?

No. That's all I got.

[pingtoo]

I just notice in the messages you post

Code: Select all

[  341.764534] finit[1]: Calling /bin/sh /usr/local/libexec/init_rc.sh shutdown

And once the runlevel switch back to 2 there also a message

Code: Select all

[  341.980236] finit[1]: service_step():        pre-shutdown(1040):  running  enabled/clean   cond:on 

which mean the "/usr/local/libexec/init_rc.sh" may have something hold on so the process cannot complete can you examine it to see if something unexpected happening there?

[Zucca]

which mean the "/usr/local/libexec/init_rc.sh" may have something hold on so the process cannot complete can you examine it to see if something unexpected happening there?

... then init should kill it.
Anyways... That's just my script for setting up environments etc. I've ran the script manually too. No problems there. Also finit doesn't pause even for a millisecond there to wait anything.

I found something odd:

Code: Select all

pgrep '/sbin/init' | wc -l
42

... that doesn't seem right. finit should have spawned somewhere around 9-10 different processes. init, three gettys, and their supervisors, mdev supervisor, kernel event logger. 42 is way too much.

[pingtoo]

which mean the "/usr/local/libexec/init_rc.sh" may have something hold on so the process cannot complete can you examine it to see if something unexpected happening there?

... then init should kill it.
Anyways... That's just my script for setting up environments etc. I've ran the script manually too. No problems there. Also finit doesn't pause even for a millisecond there to wait anything.

I found something odd:

Code: Select all

pgrep '/sbin/init' | wc -l
42

... that doesn't seem right. finit should have spawned somewhere around 9-10 different processes. init, three gettys, and their supervisors, mdev supervisor, kernel event logger. 42 is way too much.

OK, I am not sure now.
from source code I cannot find any reference for "pre-shutdown" is it something you named in configuration?
The finit:/doc/plugins.md mention

Code: Select all

* `HOOK_NETWORK_DN`, `hook/net/down`: Called right after having changed
  to runlevel 6, or 0, when all services have received their 'stop' signal.

* `HOOK_SHUTDOWN`, `hook/sys/shutdown`: Called at shutdown/reboot, right
  before all services are sent `SIGTERM`

,,,

Does that mean `hook/sys/shutdown` is somehow associated to /usr/local/libexec/init_rc.sh?

BTW, I found no reference code that will perform killing hook scripts.

May be you can examine one of the 42 /sbin/init process information /proc/<pid of 1/42>/* to see what they actually is. for example cwd, cmdline and environ.

And what if pkill /sbin/init, what will happen?

[GDH-gentoo]

finit spawns more processes as it starts its internal services (like a getty service).
Yes it would be nicer it finit had a separate binary for such tasks. But since it doesn't, all those processes have /sbin/init as their command.

Judging by the PIDs in the debug output, process 830 must be something that has been started even before tty:1, tty:2 and tty:S0 (those are some kind of service identifiers, right?). Any idea what that could be? It looks like finit is just restarting after those strange process 830 messages. Seems to reenter runlevel 2.

[Zucca]

pingtoo,

pre-shutdown is the same script.

Code: Select all

runlevel 2
log size=0
hostname planer-finit


run [S] name:post-init /usr/local/libexec/init_rc.sh sysinit -- Post-init actions

# mdev is one-shottable
# Not really needed.
run [S12] name:dev-mngr log:/var/log/mdev.log /sbin/mdev -s -f -- Device manager

# Built-in, format
#tty [LVLS] <COND> DEV [BAUD] [noclear] [nowait] [nologin] [TERM]

# External format
#tty [LVLS] <COND> CMD <ARGS> [noclear] [nowait]

tty [12] /bin/getty -n -i -l /usr/local/libexec/root_login.sh 0 /dev/tty1 noclear nowait linux

# Silly me.
# Waits for user to activate the tty, so doesn't run normally.
tty [2] /bin/getty -n -i -l '/usr/bin/cxxmatrix --scene=banner --message="Access denied"' 0 /dev/tty2 linux

# Serial login
tty [12] /bin/getty -n -i -l /usr/local/libexec/root_login.sh 0 /dev/ttyS0 noclear nowait linux

run [06] name:pre-shutdown /usr/local/libexec/init_rc.sh shutdown -- Pre-shutdown actions

I need to examine more closely on those extra finit processes...

[Zucca]

Ok. I solved the problem. Although I couldn't precisely pinpoint the source of the problem.

Here's the solution in a nutshell:

Code: Select all

pre-init)
    /bin/busybox --install -s
;;

So finit relies on some external binary/binaries (be it from coreutils or from busybox)?
I find it strange, since finit already provides its own /sbin/{poweroff,shutdown,reboot}, which are links to ./initctl.
This also fixed the issue which spawned over 40 extra finit processes.

As to why I hadn't the complete busybox install activated, was because most of the functionality of this project happens inside the busybox shell, where all those utils are shell built-ins. I did link some of the busybox utils as symlinks that I thought were essential. Mainly sh, and env.
I have also plans to have GNU coreutils (among some others) on a separate cpio/tar/whathaveyou ready for "full command line experience", so I had little need to install all the busybox links.
This also partly explains why busybox init worked fine, while finit didn't.

It's strange, the log didn't indicate anything about missing files.

I guess I could determine exactly which files are missing by running

Code: Select all

strace /sbin/finit > /var/log/finit.strace 2>&1

from my pre-init script.
Doing quick search by running

Code: Select all

# strings /sbin/finit | egrep '^/s?bin/'
/bin/sh

... does not reveal much.

Marking as [SOLVED]. (What a journey.)

EDIT: One interesting thing came to my mind: finit can reboot succesfully, without busybox install if it cannot find finit.conf. So the capability is there, but... Oh well. I think I'm not gonna bother myself about it anymore... at least for now. :D