An emerge wrapper for breaking emerges into chunks

[taipan67]

If 'maguire' or anybody else would be interested...

A large part of the collective thinking on 'toolchain purity' has been referenced to the Linux From Scratch methodology, & this in itself, since LFS Book-5.0, has been based (almost word-for-word) on an initial thesis which can be downloaded from here.

I should warn any curious readers, it's a one-chunk text-file, with no hyperlinks for ease of navigation...

...It does, however, explain beautifully the reasoning behind apparently redundant-rebuilds (& toolchain-package dependencies or requirements), plus what the authors call 'GNU-magic' - a concept which Gentoo already appears to incorporate very nicely.

For myself, i tend to lean towards the notion that a clean toolchain rebuilt against just the rebuilt system-packages ought to be sufficient (like Bob P's various methods), with no further risk incurred by not rebuilding it again, against any extra world-packages. However, there are one or two packages in the Beyond Linux From Scratch guide, which suggest that they would be better incorporated into the initial system-build, if you require them (i don't recall which ones, sorry ).

I hope anyone not familiar with the links provided finds the pure-lfs file as educational as i did when i first encountered it.

[arjay]

Thanks for the links Taipan67...this is exactly what I needed!

[sfp-a7x]

Thanks for the info, maguire.

I think I ran into a bug: I ran

Code: Select all

emwrap -DNet

and after it finished gcc (7 of 10), I got this:

Code: Select all

BEFORE selecting new GCC:
[1] i686-pc-linux-gnu-3.3.5-20050130 *
[2] i686-pc-linux-gnu-3.3.5-20050130-hardenednopie
[3] i686-pc-linux-gnu-3.3.5-20050130-hardenednopiessp
[4] i686-pc-linux-gnu-3.3.5-20050130-hardenednossp
[5] i686-pc-linux-gnu-3.3.5-20050130-vanilla

***************************************************************************
I'm going to execute:  # gcc-config i686-pc-linux-gnu-3.3.5.20050130
***************************************************************************
Kill me NOW, or forever hold your peace!
(sleeping for 15 seconds...)
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
(continuing...)
 * /usr/bin/gcc-config: Could not locate 'i686-pc-linux-gnu-3.3.5.20050130' in '/etc/env.d/gcc/' !
ERROR: emwrap: gcc-config i686-pc-linux-gnu-3.3.5.20050130 failed!

I do have i686-pc-linux-gnu-3.3.5-20050130, but not i686-pc-linux-gnu-3.3.5.20050130 (notice the hypen instead of the period).

Thanks!

[taipan67]

......

Code: Select all

BEFORE selecting new GCC:
[1] i686-pc-linux-gnu-3.3.5-20050130 *
[2] i686-pc-linux-gnu-3.3.5-20050130-hardenednopie
[3] i686-pc-linux-gnu-3.3.5-20050130-hardenednopiessp
[4] i686-pc-linux-gnu-3.3.5-20050130-hardenednossp
[5] i686-pc-linux-gnu-3.3.5-20050130-vanilla

***************************************************************************
I'm going to execute:  # gcc-config i686-pc-linux-gnu-3.3.5.20050130
***************************************************************************
Kill me NOW, or forever hold your peace!
(sleeping for 15 seconds...)
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
(continuing...)
 * /usr/bin/gcc-config: Could not locate 'i686-pc-linux-gnu-3.3.5.20050130' in '/etc/env.d/gcc/' !
ERROR: emwrap: gcc-config i686-pc-linux-gnu-3.3.5.20050130 failed!

......

As i understand it, gcc-config is designed to handle 'minor-version bumps' automatically (it appears to have done so here, having updated gcc-3.3.*, & pruned the original before this run). It should only be necessary to run it manually after a 'major-version bump' (gcc-3.3.* ---> gcc-3.4.*).

Would there be any way to incorporate that conditional option into these scripts, or would that just be unnecessary overkill?

[maguire]

sfp-a7x:

Code: Select all

BEFORE selecting new GCC:
[1] i686-pc-linux-gnu-3.3.5-20050130 *
[2] i686-pc-linux-gnu-3.3.5-20050130-hardenednopie
[3] i686-pc-linux-gnu-3.3.5-20050130-hardenednopiessp
[4] i686-pc-linux-gnu-3.3.5-20050130-hardenednossp
[5] i686-pc-linux-gnu-3.3.5-20050130-vanilla

***************************************************************************
I'm going to execute:  # gcc-config i686-pc-linux-gnu-3.3.5.20050130
***************************************************************************
Kill me NOW, or forever hold your peace!
(sleeping for 15 seconds...)
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
(continuing...)
 * /usr/bin/gcc-config: Could not locate 'i686-pc-linux-gnu-3.3.5.20050130' in '/etc/env.d/gcc/' !
ERROR: emwrap: gcc-config i686-pc-linux-gnu-3.3.5.20050130 failed!

I do have i686-pc-linux-gnu-3.3.5-20050130, but not i686-pc-linux-gnu-3.3.5.20050130 (notice the hypen instead of the period).

A-ha! Thanks for being my guinea pug! Obviously, you found a little bug! taipan67 is probably right about the automatic version change, but that shouldn't be a problem. I tested re-selecting the same version of gcc that I currently have installed, and there was no problem. And I actually did foresee that this situation might occur (no new, unselected gcc version existing). That is why I elected to select the gcc via the name, rather than the number---since the number could be 1, or 5, or 6, or...

Let me take a look at the code, and find out why the period is incorrectly replacing the hyphen in the name. Should be straightforward. (famous last words!).

Thanks,
Bruce.

[maguire]

sfp-a7x wrote:

Code: Select all

BEFORE selecting new GCC:
[1] i686-pc-linux-gnu-3.3.5-20050130 *
[2] i686-pc-linux-gnu-3.3.5-20050130-hardenednopie
[3] i686-pc-linux-gnu-3.3.5-20050130-hardenednopiessp
[4] i686-pc-linux-gnu-3.3.5-20050130-hardenednossp
[5] i686-pc-linux-gnu-3.3.5-20050130-vanilla

***************************************************************************
I'm going to execute:  # gcc-config i686-pc-linux-gnu-3.3.5.20050130
***************************************************************************
Kill me NOW, or forever hold your peace!
(sleeping for 15 seconds...)
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
(continuing...)
 * /usr/bin/gcc-config: Could not locate 'i686-pc-linux-gnu-3.3.5.20050130' in '/etc/env.d/gcc/' !
ERROR: emwrap: gcc-config i686-pc-linux-gnu-3.3.5.20050130 failed!

I do have i686-pc-linux-gnu-3.3.5-20050130, but not i686-pc-linux-gnu-3.3.5.20050130 (notice the hypen instead of the period).

OK. I see the problem. The problem is that I build the name of the newest gcc---to pass to gcc-config---using the package name as returned from emerge, and the package name returned is:

Code: Select all

$ emerge -pe system | grep "\/gcc-[0-9]"
[ebuild  N    ] sys-devel/gcc-3.3.5.20050130

So there is a minor change in the gcc name: period versus hyphen.

I'm fixing the problem now, and I should be done soon. Thanks for testing this out for me! Will you have the opportunity to try it again?

Bruce.

[maguire]

OK. I just e-mailed the new emwrap (version 4.2) to dol-sen. When he gets the new version put on his site (thanks again for hosting this, dol-sen!), please give it a try. (I also put the source back on page 9, but please remember that some people seem to have corruption problems when copy-pasting!). The link is on the first page of this thread, and on page 10 in dol-sen's post (the fourth from the top of the page).

There have been several modifications and a few bug-fixes. The most trivial is that emwrap now hands the "verbose" flag to emerge for all of the actual emerges. Much less trivial, was extracting the code that called emerge from the function that handles TC updates and the function that handles SYSTEM and WORLD updates into a single function. Now, all TC emerges and all SYSTEM and WORLD emerges call the exact same function to do the actual emerge. This is really only for proper coding style; it shouldn't affect functionality. Finally, I think I fixed the bug regarding the automatic call to gcc-config whenever gcc is updated. Because the version name for gcc differs between the emerge output and the required name---the one that is a filename in "/etc/env.d/gcc/", I do some Regular Expression magic! I replace every dot (.), dash (-), and underscore (_) in the name returned from emerge with a "[._-]" in a regular expression that is used to filter the listing of the "/etc/env.d/gcc/" directory, in order to come-up with the proper name for the most recent gcc to hand to gcc-config. Whew...

Please let me know if you try the new version, 4.2, and whether it works as expected.

Thanks,
Bruce.

[sam_i_am]

Could you please grab the 4.1 version of emwrap (if that's not the one you used), and could you please post the results of this emerge command:

Sure thing. Here goes:

emerge -pvuD system

Code: Select all

[ebuild     U ] app-arch/bzip2-1.0.3-r5 [1.0.3] -build -static 0 kB
[ebuild     U ] app-shells/bash-3.0-r12 [3.0-r11] -bashlogger -build +nls 0 kB
[ebuild     U ] sys-devel/m4-1.4.3 [1.4.2-r1] +nls 298 kB
[ebuild     U ] sys-devel/gcc-config-1.3.11-r4 [1.3.11-r3] 0 kB
[ebuild     U ] sys-libs/zlib-1.2.3 [1.2.2] -build 415 kB
[ebuild     U ] sys-libs/libstdc++-v3-3.3.6 [3.3.4] -build (-multilib) +nls +nptl 23,410 kB
[ebuild     U ] sys-kernel/linux-headers-2.6.11-r2 [2.6.8.1-r4] 32 kB
[ebuild     U ] sys-libs/glibc-2.3.5-r1 [2.3.4.20041102-r1] -build -erandom -glibc-compat20 -glibc-omitfp -hardened -linuxthreads-tls (-multilib) +nls +nptl +nptlonly -pic -profile (-selinux) +userlocales 15,628 kB
[ebuild     U ] sys-apps/man-1.6-r1 [1.6] +nls 0 kB
[ebuild     U ] sys-devel/flex-2.5.4a-r6 [2.5.4a-r5] -build -static 13 kB
[ebuild     U ] sys-apps/debianutils-2.14.1-r1 [2.13.1-r1] -build -static 123 kB
[ebuild  N    ] sys-apps/sandbox-1.2.12  217 kB
[ebuild     U ] sys-apps/portage-2.0.51.22-r2 [2.0.51.19] -build (-selinux) 251 kB
*** Portage will stop merging at this point and reload itself,
    recalculate dependencies, and complete the merge.
[ebuild     U ] net-misc/rsync-2.6.0-r6 [2.6.0-r5] -acl -build -livecd -static 0 kB
[ebuild     U ] sys-devel/autoconf-wrapper-3-r1 [2-r1] 0 kB
[ebuild     U ] sys-apps/baselayout-1.11.13 [1.11.12-r4] -bootstrap -build -static -unicode 152 kB
[ebuild     U ] sys-apps/file-4.13 [4.12] -build +python 410 kB
[ebuild     U ] sys-apps/grep-2.5.1-r8 [2.5.1-r7] -build +nls -pcre -static 0 kB
[ebuild     U ] sys-apps/man-pages-2.07 [2.02] 1,652 kB
[ebuild     U ] sys-devel/libtool-1.5.18-r1 [1.5.16] 2,715 kB
[ebuild     U ] sys-process/psmisc-21.6 [21.5] +nls (-selinux) 217 kB
[ebuild     U ] sys-libs/cracklib-2.8.3-r1 [2.7-r11] 469 kB
[ebuild     U ] sys-libs/pam-0.78-r2 [0.77-r6] +berkdb -nis -pam_chroot -pam_console -pam_timestamp -pwdb (-selinux) 6,345 kB
[ebuild     U ] sys-apps/shadow-4.0.7-r3 [4.0.5-r3] +nls +pam (-selinux) -skey 995 kB
[ebuild     U ] sys-apps/pam-login-3.17 [3.14] -livecd +nls (-selinux) 154 kB
[ebuild     U ] sys-devel/make-3.80-r2 [3.80-r1] -build -hardened +nls -static 0 kB
[ebuild     U ] sys-libs/com_err-1.38 [1.37] +nls 3,536 kB
[ebuild     U ] sys-libs/ss-1.38 [1.37] +nls 0 kB
[ebuild     U ] sys-fs/e2fsprogs-1.38 [1.37-r1] +nls -static 0 kB

emwrap -pvuD system tool

Code: Select all

--pretend (p)
--verbose (v)
--update (u)
--deep (D)
--system
--toolchain

Building a list of all EMERGE WORLD packages...
Building a list of all EMERGE SYSTEM packages...

No primary Tool Chain updates are available.

NO packages were 'emerge'd.

Done!

/tmp/emwrap/ seems to contain three files all of which are empty.

Code: Select all

# ls -l /tmp/emwrap/
total 0

-rw-r--r--  1 root root 0 Aug 23 16:26 system_list.txt
-rw-r--r--  1 root root 0 Aug 23 16:26 world_and_system_list.txt
-rw-r--r--  1 root root 0 Aug 23 16:26 world_list.txt

[maguire]

sam_i_am:

Code: Select all

# ls -l /tmp/emwrap/
total 0

-rw-r--r--  1 root root 0 Aug 23 16:26 system_list.txt
-rw-r--r--  1 root root 0 Aug 23 16:26 world_and_system_list.txt
-rw-r--r--  1 root root 0 Aug 23 16:26 world_list.txt 

OK. That's definitely a problem!

I'm guessing that this is a permission problem. I'm assuming that you ran the emwrap as non-root; is that right? That, in itself, is fine for any "pretend" run of emwrap. But, you need to be able to write to that "/tmp/emwrap/" directory as the normal user!

But, now that I look at the code, I don't see how there could be a permission problem, since the function that changes to that directory has extensive error checking... Maybe I missed something, or there's a bug in that code... (Does anyone else see any obvious problem?)

Code: Select all

function change_to_tmp_emwrap_directory {
   # Local variables:
   declare -i return_code=0 # Hold return status.

   # (If this is a brand new build, $HOME isn't set!)
   # Does the directory already exist?:
   if [ ! -d /tmp/emwrap ] ; then
      # No; create it:
      mkdir /tmp/emwrap \
      || {
         return_code=$?
         echo "${RD}ERROR: $me: ${Rd}Cannot create \"/tmp/emwrap\" directory!${nc}" >&2
         exit $return_code
      }
      # Optional (these don't *need* to succeed):
      chmod 775 /tmp/emwrap             >/dev/null 2>&1
      chown portage:portage /tmp/emwrap >/dev/null 2>&1
   fi
   # Does the directory have the necessary permissions?:
   if [ ! -r /tmp/emwrap ] \
   || [ ! -w /tmp/emwrap ] \
   || [ ! -x /tmp/emwrap ]; then
      # Bad permissions!
      echo "${RD}ERROR: $me: ${Rd}Don't have r|w|x permissions on \"/tmp/emwrap\" directory!${nc}" >&2
      exit 9
   fi
   # Change pwd to the temporary directory:
   cd /tmp/emwrap \
   || {
      return_code=$?
      echo "${RD}ERROR: $me: ${Rd}Cannot 'cd' to the \"/tmp/emwrap\" directory!${nc}" >&2
      exit $return_code
   }
   # Just to be super-duper safe:
   if [ "$( pwd )" != "/tmp/emwrap" ] ; then
      echo "${RD}ERROR: $me: ${Rd}The 'pwd' is not \"/tmp/emwrap\"?!?  How?!?${nc}" >&2
      exit 19
   fi

   # I *always* want to clear-out the failed-list file:
   cat /dev/null >failed_list.txt
}

What are the permissions for the "/tmp/emwrap/" directory?

Code: Select all

$ ls -la /tmp/emwrap

For me, I have:

Code: Select all

$ /bin/ls -la /tmp/emwrap  
total 29
drwxrwxr-x   2 portage portage  200 Aug 23 12:49 .
drwxrwxrwt  13 root    root    1384 Aug 23 15:20 ..
-rw-rw----   1 maguire maguire  220 Aug 23 12:49 alltools_list.txt
-rw-rw----   1 maguire maguire 2446 Aug 23 12:49 system_list.txt
-rw-rw----   1 maguire maguire 9604 Aug 23 12:49 world_and_system_list.txt
-rw-rw----   1 maguire maguire 7158 Aug 23 12:49 world_list.txt

and "maguire" is a member of the "portage" group. (If the "/tmp/emwrap/" directory doesn't exist, then the code tries to create it with permissions of 0775 and both a user and group of "portage".)

If you don't understand permissions and ownership of files very well (and most of the above sounded like gobbledy-gook), then why don't you try doing the following as root:

Code: Select all

# rm /tmp/emwrap/*
# chown root:root /tmp/emwrap
# chmod 0777 /tmp/emwrap

Then change back to your normal user, and try the emwrap again.

It has to be something pretty basic... If it's not permissions, then let me know, and I'll think about it...

Thanks,
Bruce.

[taipan67]

...I'm guessing that this is a permission problem. I'm assuming that you ran the emwrap as non-root; is that right? That, in itself, is fine for any "pretend" run of emwrap. But, you need to be able to write to that "/tmp/emwrap/" directory as the normal user!

I was about to say that, for normal use of the emerge command, i need root-privileges, or presumably (because i'm not) to be in the portage-group...

...Then i read the rest of your post...

I think you've answered this riddle yourself, mate. Write-permission must be restricted to 'root' & 'portage' group-members, so either run the script as root, or add your username to the portage-group.

EDIT: As soon as 4.2 appears on dol-sen's site, i intend to download it & have a bit of a play - i like the sound of the modifications you've made...

[taipan67]

...I think you've answered this riddle yourself, mate. Write-permission must be restricted to 'root' & 'portage' group-members, so either run the script as root, or add your username to the portage-group...

I just had a thought, which has turned out to be accurate, on my system at least...

I think this script will require root-privileges, because root's the only one that can write to /etc/env.d/05gcc - which is necessary if the run of gcc-config involves a change of profile...

Code: Select all

taipan@gentoo ~ $ gcc-config -l
[1] i686-pc-linux-gnu-3.4.3-20050110 *
[2] i686-pc-linux-gnu-3.4.3-20050110-hardened
[3] i686-pc-linux-gnu-3.4.3-20050110-hardenednopie
[4] i686-pc-linux-gnu-3.4.3-20050110-hardenednopiessp
[5] i686-pc-linux-gnu-3.4.3-20050110-hardenednossp
taipan@gentoo ~ $ gcc-config 2
 * /usr/bin/gcc-config: Must be root.

...Apologies if i'm wrong about that, & it's just my system being screwy...

[maguire]

taipan67:

I just had a thought, which has turned out to be accurate, on my system at least...

I think this script will require root-privileges, because root's the only one that can write to /etc/env.d/05gcc - which is necessary if the run of gcc-config involves a change of profile...

...Apologies if i'm wrong about that, & it's just my system being screwy...

No, you are exactly correct. emwrap allows you to run as non-root, only if you specified the "--pretend" option. If you run emwrap as non-root without a "-p", then you will see the following:

Code: Select all

ERROR: emwrap: Must be run by ROOT if no "pretend" flag is supplied!

Because all the real work is passed to emerge, it behaves just like emerge does as far as required privileges go. If you are in the "portage" group, you can run emerge with the "pretend" option, but it requires root when you are ready to install/remove packages. emwrap is exactly the same. I use sudo for both emerge and emwrap when I'm ready to run without the "pretend" flag.

BTW, I went ahead and posted the code back on page 9 (I edited the original post). So version 4.2 is available for copy-pasting there. Just make sure you run the md5sum and it matches! If it does not match, then run the wc, and see if it looks like extra characters are being added. One user was able to correct his copy-paste version by removing a trailing space from the end of most of the lines (I posted a perl one-liner---on page 10 I think---that would remove those). So, it's there...just be careful!

I hope you do get the chance to try/look at it. I've tried to make the code as easy to read as possible (lots of loooong variable and function names and lots of comments!). (It's actually a pretty good sample of Bash programming, for those just learning.)

Bruce.

[taipan67]

...One user was able to correct his copy-paste version by removing a trailing space from the end of most of the lines (I posted a perl one-liner---on page 10 I think---that would remove those). So, it's there...just be careful!

1189 lines (including the final blank line). Your perl one-liner did a nice job on the trailing white-space - any pointers on how i can adapt it to remove the leading white-space on every line bar the first..?

EDIT: Actually, it's not every line, since the blank ones have already had their single additional space removed by the trailing-run. To my limited understanding, that sounds like it'll probably complicate things...

[dol-sen]

4.2 is uploaded, Sorry for the delay. I was very busy today/tonight.

[lavacano]

4.2 is uploaded, Sorry for the delay. I was very busy today/tonight.

you got teh sexxorz :-p

[taipan67]

...Your perl one-liner did a nice job on the trailing white-space - any pointers on how i can adapt it to remove the leading white-space on every line bar the first..?

I had an attack of insomnia, & bugger-all else to do, so i edited my copy by hand - it was only when i was half-way through that it occurred to me how easy it would be to cock it up, & how hard to find such a cock-up would be...

My numbers now match, so no modification of the one-liner is necessary for me, but in the event that any future updates are posted, & anyone wants to go the 'copy 'n' paste' route, it might be prudent to amend it, anyway...

EDIT: Re: dol-sen's preceding post - BOLLOCKS!!!

[taipan67]

Initial impressions :-

Code: Select all

taipan@gentoo ~ $ software/emwrap -etsp
--emptytree (e)
--toolchain (t)
--system (s)
--pretend (p)

ERROR: emwrap: Don't have r|w|x permissions on "/tmp/emwrap" directory!

My normal username (not a member of the 'portage' group) gets that, in spite of the '--pretend' option. I don't consider that a problem, as i'm so used to running 'emerge' as root, anyway.

The same options when given as root produce the expected package-list on the screen, in the expected order (as best as i can recall). I like the inclusion of the '--verbose' default, as i'm in the habit of using '-ptv' with 'emerge' so as to check USE-flags & download-requirements. So far, i like what i see.

All things being equal, i should be doing a fresh 'Stage1/3' install imminently on my other partition, so i'll let you know about the performance of the toolchain-aspects as soon as i know anything.

[sam_i_am]

It has to be something pretty basic... If it's not permissions, then let me know, and I'll think about it...

Hello Maguire,

I'm running the script as root and /tmp permissions looks OK. Removed all the files in /tmp/emwrap and ran 4.2. Now I'm getting a different error.

Code: Select all

# whoami
root
# ./emwrap -pvuD system tool
--pretend (p)
--verbose (v)
--update (u)
--deep (D)
--system
--toolchain

Building a list of EMERGE WORLD packages...
Building a list of EMERGE SYSTEM packages...
Building a list of (emptytree) TOOL CHAIN packages...

No primary Tool Chain updates are available.

NO TOOL CHAIN packages were 'emerge'd.

1 of 0  SYSTEM packages:
system_list.txt

These are the packages that I would merge, in order:

Calculating dependencies
emerge: there are no ebuilds to satisfy "=system_list.txt".

Warning: emwrap: system_list.txt failed to build.  'emerge' exited code: 0.

NO SYSTEM or WORLD packages were 'emerge'd.


Packages that failed to 'emerge':
system_list.txt

# ls -al /tmp/emwrap/
total 12
drwxrwxrw-   2 portage portage 4096 Aug 24 09:12 .
drwxrwxrwt  16 root    root    4096 Aug 24 03:05 ..
-rw-r--r--   1 root    root       0 Aug 24 09:12 alltools_list.txt
-rw-r--r--   1 root    root      16 Aug 24 09:12 failed_list.txt
-rw-r--r--   1 root    root       0 Aug 24 09:12 system_list.txt
-rw-r--r--   1 root    root       0 Aug 24 09:12 world_and_system_list.txt
-rw-r--r--   1 root    root       0 Aug 24 09:12 world_list.txt

# cat /tmp/emwrap/failed_list.txt
system_list.txt

Do you think the portage update might be screwing things up? I'll upgrade portage separately and try this as well.

Sam

PS: I've been using Linux since 0.92 kernel (on a state of the art 33 MHz 486 with a whopping 16M memory ). So, please feel free to ask me to run any tests. I'll be happy to help you debug this.

Totally OT: I still have that machine. Upgraded it to 133MHZ and 32MB in 1996 and happily running Linux 2.0.24

EDIT: My suspicion proved right. upgrading portage separately did the trick! It now says

Code: Select all

... [stuff snipped]
A Tool Chain config-file or portage update was found.
'emerge'ing these packages will only occur with a TC build:

   gcc-config-1.3.11-r4

A Tool Chain update was found!
'emerge'ing these packages will only occur with a TC build:

   linux-headers-2.6.11-r2
   glibc-2.3.5-r1

+---------------------------+
| Begin Tool Chain updates! |
+---------------------------+

   linux-headers
   glibc
   sys-devel/gcc-config-1.3.11-r4
   binutils
   gcc
   glibc
   binutils
   gcc

1 of 8  TOOL CHAIN packages:
linux-headers
... [snipped]

[maguire]

sam_i_am:

EDIT: My suspicion proved right. upgrading portage separately did the trick!

Whew! I was not seeing the problem! Your experience just emphasizes the fact that 3rd-party scripts like emwrap, that depend on aspects of the output formatting of emerge, are inherently delicate and will probably break when portage/emerge changes!

Thanks for debugging that one for us.

Also, please let me know if the gcc re-build (where gcc-config is automatically run) works. As far as I know, that aspect remains untested.

Thanks,
Bruce.

[maguire]

Working backwards in posting order...

taipan67:

Code: Select all

taipan@gentoo ~ $ software/emwrap -etsp
--emptytree (e)
--toolchain (t)
--system (s)
--pretend (p)

ERROR: emwrap: Don't have r|w|x permissions on "/tmp/emwrap" directory!

My normal username (not a member of the 'portage' group) gets that, in spite of the '--pretend' option. I don't consider that a problem, as i'm so used to running 'emerge' as root, anyway.

Well, that just means that your normal user is missing a read, write, or execute privilege on the "/tmp/emwrap/" directory. If the directory does not exist, then the code creates it---as whatever user is running emwrap. Then, it attempts to change the mode (to 0775) and the user/group owner (to portage/portage), but if either of those operations fails, it is not considered an error. So, if you run emwrap as root---when there is no "/tmp/emwrap/" directory---then that directory will be created, and root will successfully change the user/group ownership to "portage". Then, subsequent runs by a normal user that is not a member of the portage group, will fail because the "/tmp/emwrap/" directory isn't world-writable.

So, long story long, if you want to run emwrap as regular user that is not a member of the portage group, then you must temporarily switch to root and remove the "/tmp/emwrap/" directory. Then you can either manually create the directory with whatever ownership and permissions work for you, or you can just run emwrap as that normal user, in which case the directory will be created with 775 permissions and the user/group for that normal user. Then, what you did above should work.

So far, i like what i see. ... All things being equal, i should be doing a fresh 'Stage1/3' install imminently on my other partition, so i'll let you know about the performance of the toolchain-aspects as soon as i know anything.

Thanks! Good, I have another guinea pig!

Bruce.

[maguire]

taipan67:

1189 lines (including the final blank line). Your perl one-liner did a nice job on the trailing white-space - any pointers on how i can adapt it to remove the leading white-space on every line bar the first..?

EDIT: Actually, it's not every line, since the blank ones have already had their single additional space removed by the trailing-run. To my limited understanding, that sounds like it'll probably complicate things...

Yes, I edited my post that you refer to (the bottom of page 9), and added a one-liner to remove a leading space. That is a little more dangerous, because---unlike the trailing spaces (of which there should be none)---many lines of the code should have a leading space (all the ones that are indented)!

So, the should-be-always-safe (unless I inadvertantly add a trailing space to the source ) remove-trailing-space one-liner is:

Code: Select all

$ perl -pi -e 's/ +$//' emwrap

And the command to remove one leading space from every line (that has one or more leading spaces)---which should only be run if you're sure you need it---is:

Code: Select all

$ perl -pi -e 's/^ //' emwrap

These types of problems are why dol-sen has graciously offered to host a link here: download emwrap source here. Thanks, dol-sen!

Bruce.

[sfp-a7x]

"emwrap -DNet" completed successfully with version 4.2.

Now to try "emwrap -DNes"...

[maguire]

sfp-a7x:

"emwrap -DNet" completed successfully with version 4.2.

Now to try "emwrap -DNes"...

Great! Just out of curiosity, did that involve changing versions of gcc? I'm just wondering if the automatic run of gcc-config successfully chose a new gcc. If you don't know, I believe that since emwrap doesn't run an emerge --prune gcc, you should be able to see if there was an older gcc with:

Code: Select all

$ gcc-config -l

(That's a lowercase letter L, not the number one!)
I figured that the user can manually remove the old gcc, if (s)he wants to, with:

Code: Select all

$ emerge --prune gcc

By the way, I don't think there is any need for the "D" (deep) and "N" (newuse) flags, if you are also specifying "e" (emptytree). The "emptytree" flag means emerge every package in the class (assume the dependency tree is empty).

So the following should do the same thing:

Code: Select all

$ emwrap -et

$ emwrap -es

...unless I'm missing something, in which case, please enlighten me.

Thanks for the feedback!

Bruce.

[sfp-a7x]

Great! Just out of curiosity, did that involve changing versions of gcc?

Nope, I already had the latest stable gcc (sys-devel/gcc-3.3.5.20050130-r1). emwrap just recompiled it.

By the way, I don't think there is any need for the "D" (deep) and "N" (newuse) flags, if you are also specifying "e" (emptytree). The "emptytree" flag means emerge every package in the class (assume the dependency tree is empty).

Yeah, I figured as much. I do it anyway out of habit because I upgrade with "emerge -aDNtuv world" (or "emwrap -DNusw").

Oh, I just thought of something else that would be nice: an --ask (-a) parameter that causes emwrap to act as if it was run with the -p option but then provides a prompt that, if answered "yes", would run as if the -p option wasn't given. What do you think?

Thanks for the feedback!

Thanks for the program!

[taipan67]

...I figured that the user can manually remove the old gcc, if (s)he wants to, with:

Code: Select all

$ emerge --prune gcc

There are certain circumstances under which a superceded version of 'gcc' is still protected against pruning, so unless you can figure out how to cater to every conceivable scenario, it's almost certainly better to leave it as a manual operation.

As a follow-up to my last post, i should be starting a test-install after some sleep - so the results could be available inside 24 hours...

I want to try two different ideas; The first is a Stage1/3 on a 2005.1-athlon-xp tarball (which i've heard might be buggy), upgrading 'gcc' to 3.4.3.20050110, which should definitely need the 'gcc-config' run. As i prefer not to modify my CFLAGS during such installs, i should be able to just use the script, without an initial manual-emerge to allow for tweaking...

The second idea i want to test is as above, but on a Jackass! 2005.0-athlon-xp tarball, to find out if updating switches it over to the 2005.1 profile. This alternative would already have the required gcc-version in situ, but the toolchain will still get rebuilt to modify glibc's locales. This will therefore test the gcc-config part of the script in a situation where no adjustment is needed.

I'll check back before i start (in the morning), in case anyone spots any howling errors in my grand-plan that i've been too blinkered to notice...