xargs boot error

[cebewee]

Ok, correction: The problem is that there are too many functions declared in the various startup scripts, as all functions are also stored in the environment.

[no usernames left]

A better solution to this problem is the following line:

Code: Select all

( find /var/lock -type f -exec rm -f -- {} \; 1>&2 )

Basically it does the same the new line does (it doesn't change the current directory) but uses the old -exec method for deleting the files.

[jukka]

A better solution to this problem is the following line:

Code: Select all

( find /var/lock -type f -exec rm -f -- {} \; 1>&2 )

That's not a solution, that's a hack which conceals that there is too much exported to the environment. And BTW it's almost exactly what bootmisc 1.24 did.

[CRC]

I'm curious as to why the change was made to use xargs anyway?

It seems like you are just adding yet another process into the mix needlessly when the -exec argument to find does what is required. Isn't this similar to using "cat file | command" instead of "command <file" ??

-- Evan
aka Taro

[jukka]

I'm curious as to why the change was made to use xargs anyway?

'find $dir | xargs $cmd' is generally faster than 'find $dir -exec $cmd {} \;' if 'find' finds lot of files, because $cmd is executed for a set of files (vs. $cmd is executed for every single file).

It seems like you are just adding yet another process into the mix needlessly when the -exec argument to find does what is required. Isn't this similar to using "cat file | command" instead of "command <file" ??

Correct. As long as execution time does not matter (what generally is the case while booting...) one should use finds -exec action. And, BTW, a common /var/lock does not contain thousands of files, so 'find | xargs' won't be much faster...

[quattro]

I know this is a rehash of what has already been said, but I am repeating it for my sanity's sake.

I have been researching this for a different thread and I realized that I had not run etc-update and I had a number of outstanding changes that needed to be merged; one of them was /etc/init.d/bootmisc. I made a backup of /etc to make sure I could recover if I hosed anything and I merged the xargs change to study what was going on.

As it turns out, on my system the environment will grow to about 50K at the point that particular xargs statement is run durning init. The environment drops down to only 2K once init is complete.

From what I can tell, the original code to delete stale locks worked fine, I didn't see anything in the CVS to indicate why the code was changed to use xargs. As it has already been mentioned, the simple fix is to change the line in /etc/init.d/bootmisc from:

Code: Select all

( find /var/lock -type f -print0 | xargs -0 rm -f -- 1>&2 )

to:

Code: Select all

( cd /var/lock && find . -type f -exec rm -f -- {} \; 1>&2 )

[quattro]

From what I understand, xargs uses the exec() system call to run the command. Linux limits the amount of space for exec() to 128K total for the environment and arguments.

What I don't understand is, my experiments show that during init, my environment was approximately 50K. xargs limits the command line to 20K. Add those two up and you get approx 70K. What happened to the other 58K? Is xargs broken? Is the system limited to only 128K for all processes? What don't I understand?

[Eyecannon]

Just found that a backup of /etc/init.d was made in /etc/init_d.old. Replace in your /etc/init.d/bootmisc

Code: Select all

( find /var/lock -type f -print0 | xargs -0 rm -f -- 1>&2 )

by

Code: Select all

( cd /var/lock && find . -type f -exec rm -f -- {} \; 1>&2 )

It worked for me.

Ciao

Worked great, thanks!

[Arkanjo]

worked here too

[jukka]

From what I understand, xargs uses the exec() system call to run the command. Linux limits the amount of space for exec() to 128K total for the environment and arguments.

What I don't understand is, my experiments show that during init, my environment was approximately 50K. xargs limits the command line to 20K. Add those two up and you get approx 70K. What happened to the other 58K? Is xargs broken? Is the system limited to only 128K for all processes? What don't I understand?

GNU xargs first determines the max arg list length (128kB on Linux) and then subtracts 2kB (POSIX requires this). If the result is >20kB it is set to 20Kb. From this the env size is subtracted. If this result is <=0 xargs gives up complaining about "environment is too large for exec".
The Gentoo boot problem (I don't know if it still exists) was caused by this problem: the environment was >=20kB what caused xargs to fail.
For details see xargs.c from your GNU findutils source archive.

HTH, Jukka

[watersb]

Not to call xargs works of course, but the real problem is IMHO that the environment is about 28k during boot, at least on my box.

To get the size of the environment during boot i executed the following program from /etc/init.d/bootmisc:
...snip...
My env size was 28261...

Jukka;

That's a very nice bit of code-spelunking, thanks!

[atescha]

I accidently did a find instead of a cd where the fix of the error message was. Now, nothing is at it used to be . Something´s seems to be deleted, anyone know a fix for this, or is it reinstall?

[jukka]

I accidently did a find instead of a cd where the fix of the error message was. Now, nothing is at it used to be . Something´s seems to be deleted, anyone know a fix for this, or is it reinstall?

If I understand correctly your /etc/init.d/bootmisc contained the line

Code: Select all

( find /var/lock && find . -type f -exec rm -f -- {} \; 1>&2 )

while it was executed...
If you were lucky and /var/lock did not exist nothing was deleted. But I'm sure /var/lock did exist, i.e. you deleted all files from the current working directory while the cwd was / and you were root...

I think the fastest way to fix your problem is to reinstall the OS.

Jukka

[quixoticsycophant]

I have the most recent baselayout yet this problem still appears.

Was there a resolution? It looks to me like the -exec solution is the right thing to do, but this change hasn't been commited for some reason.

Jeff

[ikeaman]

Code: Select all

( cd /var/lock && find . -type f -exec rm -f -- {} \; 1>&2 )

It worked for me.

Ciao

This might work. But what happens, if /var/lock can not be found for any reason? Don't ask me what the reason could be!!! Then rm will kill all files in the current directory, right?

Why not using:

Code: Select all

( find /var/lock -type f -exec rm -f -- {} \; 1>&2 )

This would make sure, that nothing happens at all if the directory does not exist for any reason.

Btw. i haven't tried it yet.

[jukka]

[...]

Code: Select all

( cd /var/lock && find . -type f -exec rm -f -- {} \; 1>&2 )

This might work. But what happens, if /var/lock can not be found for any reason? Don't ask me what the reason could be!!! Then rm will kill all files in the current directory, right?

No.

The control operators && and || denote AND lists and OR lists, respectively. An AND list has the form

command1 && command2

command2 is executed if, and only if, command1 returns an exit status of zero.

If /var/lock doesn't exist, cd returns 1.

[andersbk]

I just did an emerg sync and an emerge -puD world to see what was new (I have installed 1.4rc4.)

Nothing in etc is updated...so the problem still persists.

Has this been logged to the bug database so the developer responsible can fix it?

-anders.

[quixoticsycophant]

I just did an emerg sync and an emerge -puD world to see what was new (I have installed 1.4rc4.)

Nothing in etc is updated...so the problem still persists.

Has this been logged to the bug database so the developer responsible can fix it?

-anders.

It has already been fixed in baselayout-1.8.6.9.ebuild. See the end of http://bugs.gentoo.org/show_bug.cgi?id=21438.

You just aren't seeing it because it's masked out for whatever reason. I have it installed and the problem is gone. If you want to install it,

Code: Select all

emerge /usr/portage/sys-apps/baselayout/baselayout-1.8.6.9.ebuild

The real question is why is it still masked.
-qs

[andersbk]

Code: Select all

emerge /usr/portage/sys-apps/baselayout/baselayout-1.8.6.9.ebuild

The real question is why is it still masked.
-qs

I'm a bit dubious about emerging the whole package, as it affects all init scripts. So, I will just be hacking my "bootmisc" script until the next baselayout ebuild is released.

I also notice that there's a baselayout-1.8.6.10.ebuild.

Maybe they will skip releaseing the .9 ebuild in favor of the .10 ebuild??

/shrug

-anders.