View previous topic :: View next topic |
Author |
Message |
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Sun Mar 20, 2005 5:06 am Post subject: |
|
|
One of the nice things about MindEraser's script is that the "-p or --pretend" works with the other switchs. This allows you to see whats going to be done. To understand this script better, play with "emerge world -ep|wc " and then emerge system -ep|wc" the same with -uD. Redirect them into files and look at them.
evoweiss technically portage and gcc-config arent part of the basic toolchain but they are part of gentoos toolchain and toolchain config. I started this with the idea that most people who would be interested are staying on top keeping their system pretty well up to date already. and so including portage and gcc-config werent that important. When I check out what emerge wants do for me I usually will emerge portage by its self to get it out of the list and that is why MindEraser included it and it is part a gentoo system toolchain . I guess you can put it this way " gentoo without portage would be LFS, LInux From Scratch." and thats not a put down of LFS. At the time when Minderaser added gcc-config to tcupdate.sh it seemed to depend on what phase the moon was in as whehter gcc-config would change your profile for you or not. At least once I was half way through a long emerge before I thought to check and it hadnt switched of course. _________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
reteo n00b
Joined: 26 Dec 2004 Posts: 62 Location: Phoenix, Arizona
|
Posted: Sun Mar 20, 2005 11:12 pm Post subject: |
|
|
MindEraser wrote: |
TODO:
...
Can't think of any way of overriding what emerge puts on xterm titles.
|
Doesn't "FEATURES=notitles" work? _________________ Lampros Liontos (aka. Reteo Varala)
The Penguin Producer - Tips, tricks, and techniques for producing various forms of media with Linux tools. |
|
Back to top |
|
|
Bob P Advocate
Joined: 20 Oct 2004 Posts: 3355 Location: Jackass! Development Labs
|
Posted: Mon Mar 21, 2005 9:16 pm Post subject: |
|
|
Maedhros wrote: | Moved from Portage & Programming. |
Unsupported Software!?!
Congrats, Gents. you are now Officially members of the Lunatic Fringe. _________________ .
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
Back to top |
|
|
kimchi_sg Advocate
Joined: 26 Nov 2004 Posts: 2968
|
Posted: Tue Mar 22, 2005 8:26 am Post subject: |
|
|
Bob P wrote: | Congrats, Gents. you are now Officially members of the Lunatic Fringe. |
Come on... which sane user would want to touch (no, remerge) his toolchain with a 5-foot long wooden pole (this script) ? |
|
Back to top |
|
|
mlybarger Guru
Joined: 04 Sep 2002 Posts: 475
|
Posted: Thu Mar 24, 2005 5:21 pm Post subject: |
|
|
could someone please give a 1000ft. overview of what these scripts are intended to do? it's a tad challenging to pick it out of the 4 pages of posts here and other related threads.
my impression from reading these threads is that emerge has some major issues with upgrading "system" type stuff, and sometimes it needs to emerge them twice?
basically my problem is that i'm getting tons of unresolved symbols when trying to start xorg. i've tried various versions of xorg in portage, each build and install ok, but none will start. this just started recently, perhaps after doing an "emerge -uD world". i've been considering just backing up /etc and /home and other areas, then re-installing, but would like to avoid that if possible just to keep as much stuff in tact. this seems like perhaps one solution to rebuild/clean the system w/o impacting other stuff.
any comments would be most appreciated.
Thanks~
~mark |
|
Back to top |
|
|
reteo n00b
Joined: 26 Dec 2004 Posts: 62 Location: Phoenix, Arizona
|
Posted: Thu Mar 24, 2005 7:53 pm Post subject: |
|
|
It would work something like this:
Using tools: old-gcc, old-glibc, old-binutils, old-linux26-headers:
build new-linux26-heders
build new-gcc
build new-glibc
build new-binutils
New tools are now made, but they've been built using the logic of the old tools; this could pose problems if enough changes were introduced to confuse the optimizations, or especially if there is a bug in the old version of one of those tools. So we repeat the process so the new tools are remade, but using themselves rather than their previous versions.
Using tools: new-gcc, new-glibc, new-binutils, new-linux26-headers:
build new-linux26-heders
build new-gcc
build new-glibc
build new-binutils
They are now rebuilt, but using themselves as a base, thus we have a system toolchain in parity with itself. Since they all depend on each other to work properly, it's important all are remade when one of them is updated... and before they are used to build anything else.
I hope I've gotten all the necessary cogent points elaborated. _________________ Lampros Liontos (aka. Reteo Varala)
The Penguin Producer - Tips, tricks, and techniques for producing various forms of media with Linux tools. |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Fri Mar 25, 2005 3:18 am Post subject: |
|
|
the basic progs needed to build a TC and the order built : linux-headers glibc binutils gcc. For an explanation of these particularly linux-headers and glibc and why they should be installed first "emerge app-doc/linuxfromscratch-html " and read chapters 5 and 6.
Currently when portage builds the TC its done willy nilly. I just did an "emerge system -ep>>sys.lst and here is the order listed. Quote: |
10 [ebuild N ] sys-devel/binutils-config-1.8-r1
11 [ebuild N ] sys-devel/binutils-2.15.92.0.2-r7
......
45 [ebuild N ] sys-devel/gcc-config-1.3.10-r1
46[ebuild N ] sys-devel/gcc-3.4.3.20050110-r1
47[ebuild N ] sys-libs/libstdc++-v3-3.3.4
48[ebuild N ] sys-kernel/linux26-headers-2.6.8.1-r2
49[ebuild N ] sys-apps/sysvinit-2.86
50[ebuild N ] sys-apps/baselayout-1.11.10-r4
51[ebuild N ] sys-libs/glibc-2.3.4.20050125-r1 |
If you look in your /var/log/portage/linux-headers you'll see that at the end it even says that its recomended that you re-emerge glibc. Now is this build order super critical no its not. As I said in Bob_Ps Stage 1/3 Installation with GCC 343 on 2005.0-rc5 = Problems
hielvc wrote: | Bob tcupdate.sh uses the order " glibc binutils gcc && glibc binutils gcc". Both work, hell going by the forums 98 to 99% of the time "emerge -uD/or -e " work well other wise there would be tons of post complaining about " me broke after emerge -uD .. " and traceing it back to a broken TC. It happens, but a hugh majority of post are about newbei errors, this package is blocked or broken, I optimized the shit out my CFLAGS so that when I jump out the window with it we go faster but then it breaks , and portage circular dependenceies that crop up and go away, and how to use emerge. If you go by the number of registered users 80747 and figure just a quater of them use gentoo full time and update once or twice a year, then a very small percentage of people have problems. Its mainly us nutters jumping out the window |
The advantage of useing tcupdate.sh is that you have control of when and if you want to update your TC, preventing one set of pesky probs. _________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
eandry n00b
Joined: 15 Nov 2004 Posts: 50
|
Posted: Mon Mar 28, 2005 5:00 am Post subject: |
|
|
Excellent work on this script. I use it everytime there's a change in toolchain components. And for the most part it works great!
After rebuilding a new TC, I rebuild 'system' and then 'world' packages to take advantage of any new TC features.
However when I build 'world', packages in 'system' are also rebuilt. Is there a way to keep this from happening?
Here's the tcupdate.sh commands I used in the order described above:
Code: | tcupdate.sh -ew system |
Code: | tcupdate.sh -ew world --exclude='system' |
On the third use of tcupdate.sh, it just rebuilds the entire system including system and world packages. Unless I specify '-e' isn't there a chance that I won't get a clean rebuild of the entire system? From the help documentation looks like exclude is experimental. Is this still the case? I'm using version 2.2.0. |
|
Back to top |
|
|
mope Apprentice
Joined: 23 Feb 2003 Posts: 206
|
Posted: Mon Mar 28, 2005 1:46 pm Post subject: |
|
|
When I do this:
Quote: |
tcupdate.sh -ew --exclude='system' world
|
it looks like it excludes the system files.
Am I looking at it correctly?
Also, there doesn't appear to be linux26-headers anymore.
Is it time to update the scripts again?
(and which one do we use anyway? are they both essentially the same?) |
|
Back to top |
|
|
agent_jdh Veteran
Joined: 08 Aug 2002 Posts: 1783 Location: Scotland
|
Posted: Mon Mar 28, 2005 2:20 pm Post subject: |
|
|
mope wrote: | When I do this:
Quote: |
tcupdate.sh -ew --exclude='system' world
|
it looks like it excludes the system files.
Am I looking at it correctly?
Also, there doesn't appear to be linux26-headers anymore.
Is it time to update the scripts again?
(and which one do we use anyway? are they both essentially the same?) |
w.r.t. the 2nd point, it looks like with the 2005.0 release, linux26-headers has been consumed by linux-headers, which now uses 2.6 kernel headers by default; also development-sources is no more, vanilla-sources is now a 2.6 kernel. This is on x86 at least. _________________ Jingle Jangle Jewellery |
|
Back to top |
|
|
mope Apprentice
Joined: 23 Feb 2003 Posts: 206
|
Posted: Mon Mar 28, 2005 2:29 pm Post subject: |
|
|
agent_jdh wrote: | also development-sources is no more, vanilla-sources is now a 2.6 kernel. This is on x86 at least. |
sweet!
only problem is that the latest gentoo-sources hangs my computer when I reboot.
don't know what that's about, but I was waiting for dev-sources to figure it out. bummer, now it looks like I just have to refrain from rebooting! |
|
Back to top |
|
|
wiktorw n00b
Joined: 14 Dec 2004 Posts: 39 Location: Lodz, Poland
|
Posted: Mon Mar 28, 2005 10:24 pm Post subject: First try - and success! (on amd64/2005.0 profile) |
|
|
Hello,
Just wanted to say that I used the "tcupdate.sh" script with great success.
My intention was to use it as the preliminary step for a new Gentoo installation.
The method I used:
Code: | # tcupdate.sh -e -t -s |
Of special interest may be the length of this process.
<irony mode="sweet"> Whew, it took only short 10 hours... </irony>
But at least I'm confident that the whole toolchain is correct.
Which bothered me sometime ago, actually.
Now it's time to follow the Stage 1/3 Installation - Gentoo 2005.0 & GCC 3.4.3 by Bob P
Even though I've done most of the steps myself already
(like: partitioning, untarring stage3 & portage snapshot, setting USE flags, fstab, etc.)
And a new 2.6.11-nitro2 awaits too (/me also wonders when 2.6.12 nitro comes out...)
A very fine piece of work! Thanks!
Machine:
939-pin Athlon64 3000+ on ASUS A8V Deluxe with 1GB dual-channel RAM and 120 GB SATA HDD.
Code: | # emerge info
Portage 2.0.51.19 (default-linux/amd64/2005.0, gcc-3.4.3, glibc-2.3.4.20050125-r1, 2.6.11-nitro1 x86_64)
=================================================================
System uname: 2.6.11-nitro1 x86_64 AMD Athlon(tm) 64 Processor 3000+
Gentoo Base System version 1.4.16
Python: dev-lang/python-2.3.4-r1 [2.3.4 (#1, Mar 4 2005, 20:56:19)]
dev-lang/python: 2.3.4-r1
sys-devel/autoconf: 2.13, 2.59-r6
sys-devel/automake: 1.4_p6, 1.7.9-r1, 1.8.5-r3, 1.6.3, 1.9.4, 1.5
sys-devel/binutils: 2.15.92.0.2-r1
sys-devel/libtool: 1.5.10-r4
virtual/os-headers: 2.6.8.1-r4
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CFLAGS="-O3 -march=k8 -ffast-math -fomit-frame-pointer -mfpmath=sse -funit-at-a-time -fpeel-loops -ftracer -funswitch-loops"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown/usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O3 -march=k8 -ffast-math -fomit-frame-pointer -mfpmath=sse -funit-at-a-time -fpeel-loops -ftracer -funswitch-loops"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox"
GENTOO_MIRRORS="http://gentoo.prz.rzeszow.pl http://src.gentoo.pl http://gentoo.netnitco.net http://gentoo.inode.at"
LANG="pl_PL"
LC_ALL="pl_PL"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="amd64 acl acpi alsa bash-completion berkdb bitmap-fonts cdr crypt dio doc dvd dvdr font-server fortran gdbm gif gpm hal inifile ipv6 java jp2 jpeg lzw lzw-tiff mime mozilla mp3 ncurses nls nptl nptlonly opengl oss pam pcre perl pic png python readline ssl tcpd tiff truetype truetype-fonts type1-fonts unicode usb userlocales xml2 xpm xrandr xv zlib linguas_pl"
Unset: ASFLAGS, CBUILD, CTARGET, LDFLAGS
|
_________________ Change is the only constant in the universe.
Heraclitus |
|
Back to top |
|
|
Bob P Advocate
Joined: 20 Oct 2004 Posts: 3355 Location: Jackass! Development Labs
|
Posted: Mon Mar 28, 2005 11:16 pm Post subject: |
|
|
you know, when i was writing the Stage 1/3 Guide for 2005.0, i thought very seriously about just turning it into a Stage 3 install, then after rebooting, having the user do something like this:
Code: | # tcupdate.sh -et --keeplog
# cp make.conf.343 make.conf
# tcupdate.sh -etw --keeplog |
the only downside to doing it this way is that you trade-off by adding some compile time for the sake of command line convenience. you build some world packages twice this way, and you don't do that in the Stage 1/3 guide.
hiel, is this similar to what you did on your new install? _________________ .
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Tue Mar 29, 2005 5:03 am Post subject: |
|
|
Yes But I built the TC with a oneliner.
I just updated my script and you could use the oneliner or 2 liner for changing the CFLAGS. WIth mine you would then "emwrap.sh -se" to do a "emerge system -e" minus the TC. In other words the TC wouldnt get rebuilt but everything else would be updated saveing time. tcupdate.sh Im sure does the same but Id have to look at it again. _________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
eandry n00b
Joined: 15 Nov 2004 Posts: 50
|
Posted: Tue Mar 29, 2005 5:13 am Post subject: |
|
|
mope wrote: | When I do this:
Quote: |
tcupdate.sh -ew --exclude='system' world
|
it looks like it excludes the system files.
Am I looking at it correctly? |
No you're not.
For example xorg-x11.
Code: | # tcupdate.sh -p -ew --exclude='system' world | grep xorg
[ebuild R ] x11-base/xorg-x11-6.8.2-r1 |
Code: | # tcupdate.sh -p -ew system | grep xorg
[ebuild R ] x11-base/xorg-x11-6.8.2-r1
|
It wants to be re-emerged if doing world or just system.
But I think though this is considered "normal" because when you use '-e' the package is removed from the repository. So of course it has to replace the package again because it's no longer there.
Am I right about this...script authors? |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Tue Mar 29, 2005 4:41 pm Post subject: |
|
|
Updated my script "emwrap.sh" changed command line args to more closely follow portage short args. the big difference sicne last bight is added "-d and -r" options. "emwrap.sh -d" does a "emerge system -e" and no TC. After running it if you run "emwrap.sh -r" it builds the wrld list minus the system files from the "emwrap.sh -d" command. That saves you from re-emergeing 100+ files _________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Tue Mar 29, 2005 4:52 pm Post subject: |
|
|
If you were really quick and got the new emwrap.sh before the time of this post then the -r wont work unless you add "r " to this line, 210, in the script
Quote: | while getopts ":bdehoprstz" OPT; do |
_________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
eandry n00b
Joined: 15 Nov 2004 Posts: 50
|
Posted: Tue Mar 29, 2005 5:10 pm Post subject: |
|
|
hielvc wrote: | If you were really quick and got the new emwrap.sh before the time of this... |
Would have had I known you posted a link to the script somewhere!
Did I miss a new thread for "emwrap.sh"? I didn't see an update at the beginning of the thread for this new script.
Is it used in conjunction or replace tcupdate.sh? |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
Posted: Tue Mar 29, 2005 5:27 pm Post subject: |
|
|
MIndEraser wrote tcupdate.sh and thats the link. I finally updated my script which Im calling emwrap.sh now, and it is the one that you have to cut and paste from the listing.
Ive got to get on the road but just though of more error. in lines 155 and 156 change the Quote: |
155 tc_filter>>wrld.tst from to tc_filter>wrld.tst
156 tc_filter>>sys.tst from to tc_filter>sys.tst |
Change " >> " to ">" other wise the files grow and grow or just copy it again. _________________ An A-Z Index of the Linux BASH command line |
|
Back to top |
|
|
Bob P Advocate
Joined: 20 Oct 2004 Posts: 3355 Location: Jackass! Development Labs
|
Posted: Sat Apr 02, 2005 9:27 pm Post subject: |
|
|
have either of you guys considered adding a --fetchonly option to the script?
personally, i'd like to cut down on the total rebuild time by fetching concurrently with rebuilding (running tasks in separate consoles), instead of fetching every ebuild, pausing during the download, and sequentially building it. concurrent fetching instead of sequential fetching could dramatically cut down on the total build time by essentially removing the sum of the download times from the total build time.
even though this would help peeps like me with broadband connections, the fetching concept would really help dial-up users more than anyone else.
just a thought. _________________ .
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
Back to top |
|
|
zendal n00b
Joined: 30 Nov 2002 Posts: 23 Location: Olympia, WA
|
Posted: Fri Apr 08, 2005 4:42 am Post subject: --fetchonly in the script |
|
|
What is to stop them from just running
Code: | emerge -f system
emerge -f world |
to get the packages then to run the script.
It would download all the files and put them in /usr/portage/distfiles |
|
Back to top |
|
|
powerman n00b
Joined: 21 Dec 2003 Posts: 19 Location: Kharkov
|
Posted: Sat Apr 09, 2005 9:37 pm Post subject: |
|
|
Information in this thread help me a lot, thanks!!
You try to solve two different problems: compile toolchain in right order and minimize compilation time. I just don't understand why not use internal portage feature (buildpkg) to speedup compilation without complex scripts? Please explain why this example is wrong or not-so-fast as your scripts:
Code: | rm -r $(portageq pkgdir)/All/ # just a dirty example of cleanup
emerge linux-headers glibc binutils gcc-config gcc
gcc-config YOUR_SELECTION && source /etc/profile # ONLY if major gcc upgrade
emerge -b glibc binutils gcc portage
emerge -bke system
emerge -bke world |
For me this is sort of resume for all 4 pages in this thread (excluding some crazy ideas like emerging linux-headers twice). _________________ WBR, Alex |
|
Back to top |
|
|
rhill Retired Dev
Joined: 22 Oct 2004 Posts: 1629 Location: sk.ca
|
Posted: Sat Apr 09, 2005 10:13 pm Post subject: |
|
|
that's exactly what the script does, except you're building packages on the second pass. throw "buildpkg" in to FEATURES in make.conf and it does the same thing.
but how does that speed up compile times?
edit: this would give the same result as yours:
Code: | # tcupdate --nodeps -etw system && tcupdate -ew --exclude=system world |
and in my .bashrc i alias that to "overhaul". _________________ by design, by neglect
for a fact or just for effect |
|
Back to top |
|
|
powerman n00b
Joined: 21 Dec 2003 Posts: 19 Location: Kharkov
|
Posted: Sat Apr 09, 2005 10:23 pm Post subject: |
|
|
dirtyepic wrote: | but how does that speed up compile times? |
Speedup mean don't recompile packages when this isn't needed. My `emerge -bke system` will not recompile glibc, binutils and gcc - it will just unpack them (unpacking will take some additional time, but I think we don't notice it because compilation of other packages take much more time). _________________ WBR, Alex |
|
Back to top |
|
|
rhill Retired Dev
Joined: 22 Oct 2004 Posts: 1629 Location: sk.ca
|
Posted: Sat Apr 09, 2005 10:30 pm Post subject: |
|
|
oh. no, the script actually takes that into account. if you use the -t flag it builds only the toolchain, if you use -w system, it builds the system packages MINUS the toolchain. _________________ by design, by neglect
for a fact or just for effect |
|
Back to top |
|
|
|