| View previous topic :: View next topic |
| Author |
Message |
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
 Posted: Mon Mar 28, 2005 9:07 am Post subject: The Jackass! Project for Intel ~x86 |
 |
|
.
.
The Jackass Penguin Spheniscus demersus is a flightless seabird that is indigenous to the coast of southern Africa. It has been named after a donkey because of the loud, braying noise that it makes.
>> Click Here for Image <<
The Jackass! Project has been conceived to create an Unauthorized Minimal Installation CD and processor-specific Stage 3 Tarballs for Gentoo 2005.0 that contain an optimized GCC 3.4.3 toolkit built upon an insane set of CFLAGS that only a Jackass would consider using. 
This project has evolved from the Stage 1/3 NPTL with Optimized GCC 3.4.3 Toolkit installation methods for Gentoo 2004.3 and 2005.0. Its main contributors are the usual suspects that you would associate with a jackass project like this one. The project was named by a jackass with a propensity to pontificate in a boldfaced dark blue font.
| Code: | _ _ _
(_) __ _ ___| | ____ _ ___ ___| |
| |/ _` |/ __| |/ / _` / __/ __| |
| | (_| | (__| < (_| \__ \__ \_|
_/ |\__,_|\___|_|\_\__,_|___/___(_)
|__/
/\ /\
( \\ // )
\ \\ // /
\_\\||||//_/
\/ _ _ \
\/|(O)(O)|
\/ | |
___________________\/ \ /
// // |____|
// || / \
//| \| \ 0 0 /
// \ ) V / \____/
// \ / ( /
"" \ /_________| |_/
/ /\ / | ||
/ / / / \ ||
| | | | | ||
| | | | | ||
|_| |_| |_||
\_\ \_\ \_\\
____ ___ ___ ____ ___
|___ \ / _ \ / _ \| ___| / _ \
__) | | | | | | |___ \| | | |
/ __/| |_| | |_| |___) | |_| |
|_____|\___/ \___/|____(_)___/
|
Status Report: The following table reflects the current status on each segment of the project
| Code: |
No Task Assigned Progress Testing Completed
1. Temporary File Host [ ok ] [ ok ] [ ok ] [ ok ]
2. Permanent File Host [ ok ] [ ok ] [ !! ] [ !! ]
3. Create P-1 Tarball [ ok ] [ ok ] [ ok ] [ ok ]
4. Create P-1-MMX Tarball [ ok ] [ ok ] [ ok ] [ ok ]
5. Create P-Pro Tarbal [ ok ] [ ok ] [ ok ] [ ok ]
6. Create P-2 Tarball [ ok ] [ ok ] [ ok ] [ ok ]
7. Create P-3 Tarball [ ok ] [ ok ] [ ok ] [ ok ]
8. Create P-4 Tarball [ ok ] [ ok ] [ ok ] [ ok ]
9. Test Intel Tarballs [ ok ] [ ok ] [ ok ] [ !! ]
10. Create AMD Tarballs [ xx ] [ xx ] [ xx ] [ xx ]
11. Test AMD Tarballs [ xx ] [ xx ] [ xx ] [ xx ]
12. Jackass Framebuffer [ ok ] [ ok ] [ ok ] [ ok ]
13. Emerge Info/Branding [ ok ] [ ok ] [ ok ] [ ok ]
14. Bootable CD [ ok ] [ ok ] [ ok ] [ !! ]
15. Documentation [ ok ] [ ok ] [ ok ] [ ok ]
|
| Code: | ...._ ..,,-======-.
`''<<:: o. `\
`-." .cdbc`.
; ?$$$$$b.
;... .d$$$$$$!>
.$$$$$$P"??$$??!!!!>.
,$$$$$$P .?!!!!!!!!!>
$$$$$$$k !!!!!!!!!!!!!!>
d$$$$$$$$ !!!!!!!!!!!!!!:
d$$$$$$$$$F '!!!!!!!!!!!!!!!:
d$$$$$$$$$$ '!!!!!!!!!!!!!!!!>
$$$$$$$$$$$L !!!!!!!!!!!!!!!!!!
d$$$$$$$$$$$$ '!!!!!!!!!!!!!!!!!!
d$$$$$$$$$$$$ !!!!!XX!!!!!!!!!!!!!
d$$$$$$$$$$$$x!!!!!!#X!!!!!!!!!!!!>
3$$$$$$$$$$$$!!!!!!!!$!!!!!!!!!!!!!
?$$$$$$$$$$$!!!!!!!!!$!!!!!!!!!!!!!>
?$$$$$$$$$$?!!!!!!!Xd!!!!!!!!!!!!!!>
$$$$$$$$$?!!!!!!!WT!!!!!!!!!!!!!!!!
?$$$$$$$F!!!!!!!td!!!!!!!!!!!!!!!!!
$$$$$$$!!!!!!!Ud!!!!!!!!!!!!!!!!!!
?$$$$$$!!!!!!W?!!!!!!!!!!!!!!!!!!!>
$$$$$C!!!!!!E!!!!!!!!!!!!!!!!!!!!>
?$$$$$!!!!!!E!!!!!!!!!!!!!!!!!!!!>
$$$$$X!!!!!!!!!!!!!!!!!!!!!!!!!!>
$$$$b!!!!9!!!!!!!!!!!!!!!!!!!!!>
`$$$$$C!!9!!!!!!!!!!!!!!!!!!!!!
?$$$$$$bUi!!!!!!!!!!!!!!!!!!!!
`$$$$$$$$$$$$$b!!!!!!!!!!!!!!!
?$$$$$$$$$$$$$f!!!!!!!!!!!!!!
$$$$$$$$$$$$$)!!!!!!!!!!!!!>
?$$$$$$$$$$F!!!!!!!!!!!!!!!
"$$$$$$$$$!!!!!!!!!!!!!!!!
"$$$$$$$!!!!!!!!!!!!!!!!!;
?$$$$$%!!!!!!!!!!!!!!!!!!;
$$$$$!!!!!!!!!!!!!!!!!!!!!
`$$$$P?(`-, `'(-(-`<>.\'-
,;<(?$$$$-";); `\\ \.<\.'
=<;<<;(<;'?$P"```` `` ` '`
|
here's how the project got started:
| nightmorph wrote: | | IIRC, the entire point of this 2004.3 Stage 1/Stage 3 guide was making use of a better toolchain, and it's a shame that the new (delayed) 2005 CD doesn't include the better toolchain packages, even though they're 'unstable'. I'd sure like to try a stage 1 install without having to use a Stage 3 tarball; wouldn't it be nice if bootstrap.sh ran perfectly on an 'unstable' toolchain? Mmm, gcc 3.4.3.x.....needs to be on an optional ~x86 CD! |
hey, i think you're on to something! 
i like the idea of having an ~x86 install CD that has a ~x86 toolkit built in, and ~x86 Stage tarballs so that nobody would have to rebuild the toolkit to get to GCC 3.4.3. you know, its a real waste of time to have all of the end users rebuilding the toolkit, when it would make alot more sense to build the toolkit once, and stamp it onto an unauthorized CD. hmmm.... 
anybody game for this? 
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks
Last edited by Bob P on Fri Apr 29, 2005 6:07 am; edited 17 times in total |
|
| Back to top |
|
 |
kimchi_sg
Advocate
Joined: 26 Nov 2004
Posts: 2968
|
| Posted: Mon Mar 28, 2005 11:22 am Post subject: |
|
|
I could do it. But most likely it will be done using catalyst, if it is ever done.  |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 3:24 pm Post subject: |
|
|
Catalyst is done, I use it all the time to make portage snapshots. I've been looking into creating my own live-cd's with it for a while. Between the project documentation and the incomplete gentoo wiki on creating a live-cd using Catalyst, I could probably do it.
EDIT: In fact I'm going to ask somewhere if somebody could get a hold of the catalyst specfiles for the 2005.0 stages and live-cd's and post them on the forums. The specfiles were posted along with the 2004.3 release, but not so it seems with the 2005.0 release.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
96140
Retired Dev
Joined: 23 Jan 2005
Posts: 1324
|
| Posted: Mon Mar 28, 2005 7:54 pm Post subject: |
|
|
| Bob P wrote: | hey, i think you're on to something!
i like the idea of having an ~x86 install CD that has a ~x86 toolkit built in, and ~x86 Stage tarballs so that nobody would have to rebuild the toolkit to get to GCC 3.4.3. you know, its a real waste of time to have all of the end users rebuilding the toolkit, when it would make alot more sense to build the toolkit once, and stamp it onto an unauthorized CD. hmmm....
anybody game for this? |
Actually, when I wrote that post I was already thinking about the possibilities Catalyst could offer. That, and there's another app that I remember reading about in the last month that (I think) has something to do with tarball creation. I was thinking of giving Catalyst a shot, at least, though that'd be plunging into some deep waters for me.
The entire contents of the CD wouldn't need to be ~x86, though, just gcc, glibc, gcc-config, and libstdc++ (which depclean keeps wanting to remove, saying absolutely nothing uses it. Let's see, I remember that the Stage 1/3 guide called for placing those packages in /etc/portage/package.keywords, though only three of them seem to be necessary....help me out, someone-- Are there any other ~x86 packages that are definitely needed in this Guide? Or that would be necessary to include in a specialized tarball? I think the fewer ~x86 packages there are, the better, as we still want to have a sturdy x86 system from which the end user can optionally migrate to ~x86 after getting a working system.
So to sum up, I think the ~x86 CD would consist of, what, a rebuilt Stage 3 tarball with an ~x86 toolchain containing a better gcc in addition to the small environment found on the Minimal CDs, right? It'd be like a very stripped-down Universal LiveCD, with only one (or two, at most: maybe a tarball for AMD64 users) tarball. Hmm, maybe this needs its own thread to avoid cluttering up this one? |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 8:02 pm Post subject: |
|
|
| nightmorph wrote: | | Bob P wrote: | hey, i think you're on to something!
i like the idea of having an ~x86 install CD that has a ~x86 toolkit built in, and ~x86 Stage tarballs so that nobody would have to rebuild the toolkit to get to GCC 3.4.3. you know, its a real waste of time to have all of the end users rebuilding the toolkit, when it would make alot more sense to build the toolkit once, and stamp it onto an unauthorized CD. hmmm....
anybody game for this? |
Actually, when I wrote that post I was already thinking about the possibilities Catalyst could offer. That, and there's another app that I remember reading about in the last month that (I think) has something to do with tarball creation. I was thinking of giving Catalyst a shot, at least, though that'd be plunging into some deep waters for me.
The entire contents of the CD wouldn't need to be ~x86, though, just gcc, glibc, gcc-config, and libstdc++ (which depclean keeps wanting to remove, saying absolutely nothing uses it. Let's see, I remember that the Stage 1/3 guide called for placing those packages in /etc/portage/package.keywords, though only three of them seem to be necessary....help me out, someone-- Are there any other ~x86 packages that are definitely needed in this Guide? Or that would be necessary to include in a specialized tarball? I think the fewer ~x86 packages there are, the better, as we still want to have a sturdy x86 system from which the end user can optionally migrate to ~x86 after getting a working system.
So to sum up, I think the ~x86 CD would consist of, what, a rebuilt Stage 3 tarball with an ~x86 toolchain containing a better gcc in addition to the small environment found on the Minimal CDs, right? It'd be like a very stripped-down Universal LiveCD, with only one (or two, at most: maybe a tarball for AMD64 users) tarball. Hmm, maybe this needs its own thread to avoid cluttering up this one? |
Your correct on the tarball, in fact you don't even really need to use catalystt to make the tarball. Just utar a stage 3 tarball, untar a portage snapshot, chroot, edit the make.conf, edit /etc/portage/packages.keywords, rebuild the toolchain, and package the whole thing up again (after removing /usr/portage of course). Ba-da-bing ba-da-boom, stage-1/3-i-686-2005.0.tar.bz2. In fact I'm building one right now. The only problem with this would be the picking settling on some well balanced cflags for the second rebuild. BTW does anybody have an ftp server we could host these things on? We wouldn't really need to build our own live-cd, since, like I said, once you do hardware detection, get your network up, and chroot, it's job is done. The only problem is, without catalyst I can't build stages for any other arch aside from x86.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 8:16 pm Post subject: |
|
|
Allright, I'm building an 2005.0 i-686 stage 1/3 tarball, this is what I've done so far:
1) Untarred a 2005.0 stage 3 tarball
2) Edited the make.conf as follows (I simply edited the useflags): | Code: | # These settings were set by the catalyst build script that automatically built this stage
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-O2 -march=i686 -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
USE="nptl"
|
3) Edited /etc/portage/package.keywords as follows: | Code: | sys-libs/glibc ~x86
sys-devel/gcc ~x86
sys-devel/gcc-config ~x86
sys-libs/libstdc++-v3 ~x86 |
At the moment I'm doing the first rebuild: | Code: | | emerge gcc-config glibc binutils gcc |
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
96140
Retired Dev
Joined: 23 Jan 2005
Posts: 1324
|
| Posted: Mon Mar 28, 2005 8:19 pm Post subject: |
|
|
| Sith_Happens wrote: | | You're correct on the tarball, in fact you don't even really need to use catalyst to make the tarball. Just untar a stage 3 tarball, untar a portage snapshot, chroot, edit the make.conf, edit /etc/portage/packages.keywords, rebuild the toolchain, and package the whole thing up again (after removing /usr/portage of course). Ba-da-bing ba-da-boom, stage-1/3-i-686-2005.0.tar.bz2. In fact I'm building one right now. The only problem with this would be the picking settling on some well balanced cflags for the second rebuild. BTW does anybody have an ftp server we could host these things on? We wouldn't really need to build our own live-cd, since, like I said, once you do hardware detection, get your network up, and chroot, it's job is done. The only problem is, without catalyst I can't build stages for any other arch aside from x86. |
Waaiiiiitaminute. I would have thought one would have to use catalyst to build the Stage 3 tarball, since we want the user to have a 3.4 compiler installed in it; wouldn't it be easier to just swap out gcc 3.3 for 3.4 in the tarball (and the other ~x86 packages) instead? And then you'd package that tarball back up.
And about the CFLAGS...is it really necessary to try to guess an optimized set for all users? I think that even with a customized LiveCD, users are still going to have to do some amount of compiling. It's the price to pay for moving from a generic x86 environment to one optimized for their hardware. I really don't think that anything more than -O2 -pipe would work if you tried to do any toolchain rebuilding ahead of time on the stage. This guide explicitly calls for some compiling and recompiling. It's up to the user to decide CFLAGS for the second compile, as it says in the Guide--and just makes sense!
And I still think it's a good idea to include the tarball on the LiveCD, that way users would be spared some download time. Maybe even include a portage snapshot, as well? Sure, users might grab the latest one anyway, but at the moment, most--if not all--the recent difficulties with the ~x86 toolchain components have been resolved (no more glibc problems!). And right now, the portage snapshot would have everything a user needs. With the current stage of packages, I wouldn't think there would be a need to update the portage snapshot until after rebooting and then doing the very first emerge sync. Oh wait. Never mind. It just occured to me that if some new & improved version of gcc came along, then using 20050110 would be silly, if the new compiler is stable and better.
So...I guess we need some mirrors set up for the tarball(s)? |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 8:23 pm Post subject: |
|
|
You don't really need to use catalyst for this, the main reason for not using catalyst is that the documentation isn't that great and it would take me longer to figure out how to use it correctly than it would to simply modify a stage 3 tarball, then tar it back up. As far as CFLAGS go, I think I'm going to use -O2 -march=i686 -pipe -fomit-frame-pointer (pretty much the same flags in the stock stage 3 tarball). If people want to use other flags, they can always go back and recompile. However, I'll leave the final decision on what cflags to do the second rebuild on the tarball with to Bob P. After I make this one, I'm going to make a generic i586 tarball. BTW we do need some mirrors to host these things. I've got a static IP through the university, and If need be I'll go through the trouble of setting up an ftp server, but I'd rather not.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 8:36 pm Post subject: |
|
|
| nightmorph wrote: | | The entire contents of the CD wouldn't need to be ~x86, though, just gcc, glibc, gcc-config, and libstdc++ (which depclean keeps wanting to remove, saying absolutely nothing uses it. Let's see, I remember that the Stage 1/3 guide called for placing those packages in /etc/portage/package.keywords, though only three of them seem to be necessary....help me out, someone-- Are there any other ~x86 packages that are definitely needed in this Guide? Or that would be necessary to include in a specialized tarball? I think the fewer ~x86 packages there are, the better, as we still want to have a sturdy x86 system from which the end user can optionally migrate to ~x86 after getting a working system. |
libstdc++--v3 is emerged as a dependency of gcc. i think we've discussed that in the later pages of the Guide.
if depclean is buggering-up your system by trying to get rid of libstdc++-v3, you have a problem on your system, as this behavior is abnormal.
are you sure that you've never performed --oneshot or --nodeps emerges when building the toolkit? a number of people have recommended doing things like that, and i do NOT agree with that approach. it could cause the problem you're having.
i run decplean and revdep-rebuild on my PCs after every toolkit and world update, and i have never had problems with libstdc++-v3 being molested by depclean. you might want to assure that your world file is accurate, rebuilding it if necessary.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 8:37 pm Post subject: |
|
|
Hey Bob, I'm building 2005.0 i686 and i586 stage 1/3 tarballs, what cflags/useflags do you want me to use for the final build?
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 8:42 pm Post subject: GCC 3.4.3 Unauthorized Install CD for x86 |
|
|
i'm starting this thread to catch all of the discussions we've been having about creating an unauthorized install CD based on a GCC 3.4.3 toolkit for the x86 platform.
at this time the CD is nothing more than an idea.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 8:46 pm Post subject: |
|
|
| Sith_Happens wrote: | | Ba-da-bing ba-da-boom, stage-1/3-i-686-2005.0.tar.bz2. In fact I'm building one right now. The only problem with this would be the picking settling on some well balanced cflags for the second rebuild. BTW does anybody have an ftp server we could host these things on? We wouldn't really need to build our own live-cd, since, like I said, once you do hardware detection, get your network up, and chroot, it's job is done. The only problem is, without catalyst I can't build stages for any other arch aside from x86. |
i have a couple of things to say about this:
1. nightmorph is right. if we're going to do an unauthorized live CD, we need to split that topic off from this thread. we should probably start a new thread in Unsupported Software. i've started a thread here, and i'm going to ask the mods to move the relevant posts out of this thread to the new one:
https://forums.gentoo.org/viewtopic-p-2242902.html#2242902
2. i don't think that 686 is a good idea. that is to say, its not retro-compatible enough. to do this, i think that we need to make the default archtitecture pentium. otherwise we'll be creating a 686 CD instead of an x86 CD. to be strictly x86 we should probably support 486, but i would be willing to sacrifice compatibility for pre-pentium x86. imho any pentium class PC with a PCI interface could still be useful as a server, so i wouldn't cut off everyone below 686.
Fwiw, in creating the Stage 1/3 Guide, i have a couple of machines built on "pentium" that are always kept up to date. if we decide to create an alterntative live CD with the new toolkit, i can supply the pentium-compatible files for the project. this would make the CD compatible with any x86 platform that is 586 or later. personally, i think that this is better than doing it as 686, as it will support a wider variety of platforms.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 8:57 pm Post subject: |
|
|
Like I said before, we don't really need to make any edits to the live cd itself, so If you guys want to take a 2005.0 minimal live-cd and throw some new stages on there I think that's all we will need. As I said, I'm building some new stages right now. The thing is I'm building them with nptl support (which If I remember correctly only works on >=i586 processors), which explains why I'm building an i586, and i686 stage tarballs. I could also build an i486 tarball without nptl support, and we could include all of them on the cd. What I'm asking of you Bob is to post the make.conf's you want the i486, i586, and i686 tarballs built with so I can make these stages to your specifications. Then we could just distribute these tarballs, write a new guide outlining their use in an install, and we've got a great install meathod. 
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 9:28 pm Post subject: |
|
|
i am open to discussion about the contents of make.conf, specifically related to USE flags and CFLAGS. because my experience is limited to intel architectures and stops at P3, and because i have no experience with modern AMD platforms, i'm reluctant to proclaim any "official" settings for make.conf. i'd prefer to get feedback from people who are using other architectures who may be aware of things that i don't know. with that in mind, here are my thoguhts:
USE FLAGS
there are some USE flags in the Guides that just aren't necessary for building the toolkit -- some of them were added becasue i had intended to build a server or a router out of the boxes that were being used as testbeds when i was writing the install guides. for example, we may not need to have USE statements samba, readline, ssl, and many others. i guess we should just look at what's on the official install CD and tarballs, and add things that are needed for NPTL and perl/libperl. personally, i hate the subject of USE flags, so I'd like to delegate responsibility for this. (this is another term for passing the buck!)
CFLAGS
i think that it is safe to use my more advanced set of CFLAGS for intel architecture when building the toolkit that will go into the tarballs, but i think that we should probably revert to the most stable "recommended" CFLAGS for the default make.conf that goes on the tarballs. my reason for saying this is because i do not have a good handle on the problems that the AMD guys are having with my more advanced set of CFLAGS. i cannot tell if the problem is that AMD doesn't like the flags, or we've just got alot of people who are posting with complaints, when the real problem is that they cannot tell when a glibc ebuild is b0rken.
i am working on a Windows PC right now, so I don't have access to my make.conf.343 file. when i get a chance i'll post that, and you can look it over and tell me what you think. its by no means the final solution, but i think its as good a place to start as any.
RETRO-COMPATABILITY
my idea about backward compatability was only intended to make sure that nobody who has hardware that meets the gentoo minimal installation requirements ends up being left out in the cold. that could make it somewhat cumbersome to make (and host) a wide array of stageballs.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 9:59 pm Post subject: |
|
|
We'll, here are my ideas for the make.conf's: | Code: | CFLAGS="-march=i686 -O3 -fforce-addr -momit-leaf-frame-pointer -fomit-frame-pointer -ftracer -pipe"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"#
CHOST="i686-pc-linux-gnu"
MAKEOPTS="-j2"
USE="nptl" |
In my experience these flags are stable on i586 and up, although I haven't tried them on i386 or i486. We might want to tone down the flags for the lower two x86 arches, perhaps reduce -O3 to -O2 or something like that. As far as global useflags that should be set, the only useflag we really need to set should be nptl for >=i586, and probably none for the lower arches. We have to remember that we are building a stage tarball, not doing a complete install. We should try to build the stages to have just enough configuration to achieve our objective, but still preserve flexibility. Any other useflags can be added by users as in a stage 3 install. I think that if we are going to be creating stage tarballs, we should follow the same format as the Gentoo stages, i.e we build i386 (generic x86), pentium3, pentium4, and Athlon-XP stages. Although it is more cumbersome than simply creating a single x86 tarball, it allows us to better optimize the tarballs for each architecture. Also, creating at least two tarballs is neccessary in this case, since NTPL requires a CHOST of i586-pc-linux-gnu or above.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Deathwing00
Bodhisattva
Joined: 13 Jun 2003
Posts: 4087
Location: Dresden, Germany
|
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 10:25 pm Post subject: |
|
|
Aren't these mods the best.  Although locking the 2004.3 support thread wasn't really what I had in mind at least. However, since 2005.0 is the latest release, maybe we should just use 2005.0 stage 1/3 thread for all support requests?
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall
Last edited by Sith_Happens on Mon Mar 28, 2005 10:27 pm; edited 1 time in total |
|
| Back to top |
|
|
nmcsween
Guru
Joined: 12 Nov 2003
Posts: 381
|
| Posted: Mon Mar 28, 2005 10:26 pm Post subject: |
|
|
One thing straight off that is bad with those flags is that gcc inlines functions way to much to gain any performance with C. C++ can have template heavy code inlined which is a good thing. -ftracer is a shot in the dark since it only works with profiled feedback from gcc. Then theres -mregparm=3 which passes integers through registers but needs all binaries to be compiled with it. I say this live cd have two different branches custom and current.
_________________
Great Resources |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 10:32 pm Post subject: |
|
|
| nmcsween wrote: | | One thing straight off that is bad with those flags is that gcc inlines functions way to much to gain any performance with C. C++ can have template heavy code inlined which is a good thing. -ftracer is a shot in the dark since it only works with profiled feedback from gcc. Then theres -mregparm=3 which passes integers through registers but needs all binaries to be compiled with it. I say this live cd have two different branches custom and current. |
We'll I've tried to use the cflags described in the original 2004.3 stage 1/3 install, however I would be against adding any cflags which would make the packages compiled with it binary incompatable to those compiled without (this is of course excluding -march). As far as having two branches, our overall concern is creating stages that are flexible but also simple to maintain. I think two branches is probably going too far, and would create a whole host of support problems.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 10:32 pm Post subject: |
|
|
| Sith_Happens wrote: | | Although it is more cumbersome than simply creating a single x86 tarball, it allows us to better optimize the tarballs for each architecture. Also, creating at least two tarballs is neccessary in this case, since NTPL requires a CHOST of i586-pc-linux-gnu or above. |
i am in agreement that the only USE flag that we actually need is NPTL, but i would argue that we should put "ithreads" in there for perl/libperl. if necessary, we can put them in package.use, as i think that perl/libperl are the only packages that use the ithreads flag, but i'm not absolutely postitive on that.
personally, i think its okay to deviate a tad from the typical "official gentoo style" tarballs. even though the gentoo minimal hardware requirement specifies 486, i think its safe to say that because we're going to build an ~x86 tookit, we have the freedom to specify that our project must support NPTL. NPTL means that we can abandon 486 and specify that the oldest arch & chost supported will be pentium/i586.
so i wouls say that we should make tarballs for:
intel:
pentium/i586-pc-linux-gnu
pentium2/i686-pc-linux-gnu
pentium3/i686-pc-linux-gnu
pentium4/i686-pc-linux-gnu
amd:
whatever
here are my thoughts regarding the 4 tarballs for intel:
pentium is the common denominator platform that anybody can use to install NPTL on any intel or amd platform. its an absolute requirement.
pentium2 should be available, as its the basis for the i686 chost.
pentium 3 and 4 should be available because those cpus are so prevalent.
what do you think?
i'll dig up my make.conf.343 when i get a chance and look it over.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks
Last edited by Bob P on Mon Mar 28, 2005 10:42 pm; edited 1 time in total |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 10:36 pm Post subject: |
|
|
one branch, definintely.
if CFLAGS are going to be a potential problem, i think that it would be good enough to supply a GCC 3.4.3 toolkit with an absolute minimal set of CFLAGS like "-O2 -fomit-frame-pointer -pipe" and then let the user make changes if need be.
imho, we're doing a great service even if we only provide a minimally flagged 3.4.3 toolkit. if adding flags is going to create support problems for us and create more headaches, maybe we should follow the developers' lead and keep the clfags simple.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Sith_Happens
Veteran
Joined: 15 Dec 2004
Posts: 1807
Location: The University of Maryland at College Park
|
| Posted: Mon Mar 28, 2005 10:40 pm Post subject: |
|
|
We'll as far as chosts go, we should definantly create two sets of tarballs, on set configured with CHOST set to i586-pc-linux-gnu and one set to i686-pc-linux-gnu. I think within those two catagories, creating tarballs compiled with the CFLAGS -march=pentium4, -march=pentium3, -march=pentium2, and -march=pentium. I think if we create one set of tarballs for intel and one set for AMD we should be ok. Like I said, I've got a P4, so I can make stages for all the intel processors fast and easy. As far as AMD, we'll have to delegate that to somebody else who's got a powerful AMD processor. I'm certainly not opposed to minimal cflags, they are overrated anyway IMHO. NPTL is a much bigger deal performance wise in my experience, and if we include that I think we'll be ok.
_________________
"That question was less stupid; though you asked it in a profoundly stupid way."
I'm the brains behind Jackass! | Tutorials: Shorewall |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 10:53 pm Post subject: |
|
|
that sounds good to me. four tarballs for intel would be more than anyone could have hoped for. i would be content to make this an intel-only project, but i think that somebody will probably volunteer to take-on the AMD branch of the project. i have somebody in mind, but i'd prefer that he volunteer rather than me volunteering him!
are you near a gentoo box now? could you grep the ebuilds and see which ones respond to ithreads and pthreads? those are the only two USE flags that i would like to look into to see if they're needed by any packages. (we already know that ithreads worlks for the perls).
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
96140
Retired Dev
Joined: 23 Jan 2005
Posts: 1324
|
| Posted: Mon Mar 28, 2005 11:16 pm Post subject: |
|
|
| Bob P wrote: | there are some USE flags in the Guides that just aren't necessary for building the toolkit -- some of them were added becasue i had intended to build a server or a router out of the boxes that were being used as testbeds when i was writing the install guides. for example, we may not need to have USE statements samba, readline, ssl, and many others. i guess we should just look at what's on the official install CD and tarballs, and add things that are needed for NPTL and perl/libperl. personally, i hate the subject of USE flags, so I'd like to delegate responsibility for this. (this is another term for passing the buck!)
CFLAGS
i think that it is safe to use my more advanced set of CFLAGS for intel architecture when building the toolkit that will go into the tarballs, but i think that we should probably revert to the most stable "recommended" CFLAGS for the default make.conf that goes on the tarballs. my reason for saying this is because i do not have a good handle on the problems that the AMD guys are having with my more advanced set of CFLAGS. i cannot tell if the problem is that AMD doesn't like the flags, or we've just got alot of people who are posting with complaints, when the real problem is that they cannot tell when a glibc ebuild is b0rken. |
On USE flags: nptl is probably the only thing that will need to be used to build the toolchain to be put in the tarball. This is because there are some (many?) enterprising users who will not be using ithreads or pthreads. Since this is an advanced install, it is possible to go so many different ways with it. Obviously, significant deviations like nptlonly should not be posted in the support thread.
BUT, as everyone wants nptl, it'd be a good idea to have that one USE flag set for the pre-compile. From there, users can further recompile their system with any mix of nptlonly, ithreads, samba, perl, usb, posix & so on flags. I think it would be a mistake to put in ithreads and pthreads, as that would only mean taking away optimizations from many users who want an nptlonly system, like me!
On CFLAGS: there's a sort of problem with your suggestion: as you say, we have to have a skeletal make.conf or the user to modify later on, as per the Guide, and a make.conf for any initial compiling that goes on once the tarball is downloaded. BUT, what's the point of having a sparse set of CFLAGS if the toolchian that goes into the compiler has already been compiled with a set of advanced, potentially unstable set of flags? If, say, we target the tarball to users of 586/686 and up, and use flags like -fweb -frename-registers -ftracer, and so on (many of these are redundant and specified by -O2 or -O3), but then only have -pipe -fomit-frame-pointer in the make.conf included in the tarball, that's somewhat wrongfully forward-thinking, isn't it?
After all, we want the stage to at least somewhat adhere to the general Gentoo stage guidelines, which means it needs to support older architectures. And by including flags that seem to be aimed more toward Pentium owners, the AMD crowd is disadvantaged. And vice-versa; some group will end up with a nonworking, noncompilable system. Or at least, that's what I'd think would be the logical result of extreme optimization ahead of time. Perhaps it would be a better idea to, at most compile the toolchain for the tarball with the optimzations specified by -O2, as those flags work on both Intel and AMD. IIRC some of the CFLAGs used in your make.conf, Bob, are part of -O3 or not from either optimization level, and thus could be more problematic. |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Mar 28, 2005 11:24 pm Post subject: |
|
|
| nightmorph wrote: | | BUT, as everyone wants nptl, it'd be a good idea to have that one USE flag set for the pre-compile. From there, users can further recompile their system with any mix of nptlonly, ithreads, samba, perl, usb, posix & so on flags. I think it would be a mistake to put in ithreads and pthreads, as that would only mean taking away optimizations from many users who want an nptlonly system, like me! |
my two cents say that interpreter threading in perl is desirable and it does not undermine the performance of NPTL in any way. as far as "nptlonly" goes, that will definitely not be considered, as it will break things.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
