View previous topic :: View next topic |
Author |
Message |
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Sat May 24, 2003 4:30 pm Post subject: |
|
|
Ok .. it looks like the forum does forget posts .. mine was lost (and one other I think). Ok .. here we go again:
1.)
Got another Testingsystem (with dual 1,6 Ghz Opteron). Will have a look on this to check performance differneces between 1,4 and 1,6.
2.)
My pure 64 bit is getting purer . My only problem is .. how can I get glibc to compile with -m64. Glibc seems to strip all the Options provided by make.conf. I had a look in the ebuild file but didn't found anything .. so .. anyone out there who knows?
3.)
Someone posted if I'm sure about the -fexpensive-optimizations flag (and maybe the other stuff too).
In fact, I'm not. -fexpensive-optimizations had an 2% performance increase in some benchmarks on an XEON System and didn't make any other benchmark slower. I will check the performance difference on an opteron between -march=k8 -O2 and the flags I used later.
Ok .. guess tomorrow I will have some more informations regarding those stuff! |
|
Back to top |
|
|
daewerk n00b
Joined: 23 Apr 2003 Posts: 1
|
Posted: Tue May 27, 2003 5:37 pm Post subject: |
|
|
Any more progress? Tips? I'm hoping to have hardware to start a test build soon. |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Thu May 29, 2003 2:30 pm Post subject: |
|
|
Ok .. here is the update (later than promised .. I know):
1.) Opteron 1,6 Ghz versus Opteron 1,4 Ghz (32-bit Pure):
Hm .. this is a little bit strange. No performanceboost at all (around 2% in unixbenchmark as an example) .. it looks like more Mhz brings no performance at all (in 32 bit mode..) .. which is quite confusing.
2.) Test of different Compilerflags is still running (at the moment the benchmarks run .. I will have those results tomorrow)
3.) Still having problems to build a 64bit only System ... I'm working on it, but it is a little bit toughly. |
|
Back to top |
|
|
cchapman Guru
Joined: 16 Jan 2003 Posts: 440 Location: Fremont, NE
|
Posted: Thu May 29, 2003 4:01 pm Post subject: |
|
|
Post results please......
I am anxiuos to see them
Also post current CCFLAGS and such |
|
Back to top |
|
|
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Sat May 31, 2003 3:20 am Post subject: |
|
|
yes, more details would be great |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Sun Jun 01, 2003 3:24 pm Post subject: |
|
|
Ok .. so here I am with further details (tried to post this several times now, but the forum wasn't reachable. Unsure if that depends on my connection or on the forum ...
Ok here the (quite interesting) stuff:
First of all, a short resumè:
I got two testmachines, one dual Opteron 240 (e.g. 1,4 Ghz) with 2 GB DDRRAM on an MSI K8Master-FT (http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=441&MODEL=MS-9131) and another dual Opteron 242 (e.g. 1,6 Ghz) with 6 GB DDRRAM on an MSI K8Master-F (same URL as above).
Some time ago I benched a dual P4 XEON 2,667 and 2,8 using Unixbench so those results are for comparisson.
In this configuration I got the following results for pure 32bit (using Unixbench):
Code: |
Average of 20 Runs with Opteron 240 (1,4 Ghz): 415
Average of 20 Runs with Opteron 242 /1,6 Ghz): 414
Average of 20 Runs with DUAL XEON 2,667 Ghz: 411
Average of 20 Runs with DUAL XEON 2,800 Ghz: 423
|
Ok .. what we see here is that the Opteron 240 is in Unixbench quite as fast as an P4 XEON 2,667 Ghz. But suprise suprise: Opteron 242 as fast as Opteron 240? After some tests and several changes I found out what was the problem. When I use only 4 GB of the 6 GB in the Opteron 242 (so if I physicaly remove 2 GB) the speed is much higher.
Ok, as I understand the Opteron, he uses NUMA to access Prozessorspecific Memory (if you have a look on the MSI K8, you will learn that there are 6 banks for RAM, 4 on a block an 2 on another). If I remove the RAM-Modules in the second block (e.g. with only 2 banks) the speed is as expected. If I put the modules into, and only use 4GB (e.g. compile the Linuxkernel with 4GB Memorysupport) the numbers drop to Opteron 240 level. If I compile the Kernel with max 64 GB Memorysupport and NUMA Accessmode the numbers doesn't change at all.
So .. maybe the NUMA Support of the Kernel is not optimal for the Opteron, or, something different is buggy (maybe I? )
So here are the different numbers for Opteron 242 with different RAM and Kernel:
Code: |
Average of 20 Runs (6 GB RAM, 2.5.69, 64GB, NUMA) : 414
Average of 20 Runs (6 GB RAM, 2.5.69, 4GB, NoNUMA): 413
Average of 20 Runs (4 GB RAM, 2.5.69, 4GB, NoNUMA): 466
|
This puzzles me a bit, I will have a talk to an AMD developer next week.
If you compare the Operton 240 and 242 with equal RAM (e.g. both with 2 GB), you get an 12,3% faster Opteron 242, while the Opteron 242 ist in reality 14,3% faster (1,6 Ghz versus 1,4 Ghz). I think this scales quite ok.
Unfortunaly I have to return the Opteron 242 tommorow. Eventualy I can have the Opteron 240 for another week, cause my 64bits Benchmark aren't finished yet. I made other benchmarks then Unixbench, but those results are nearby equal to those of Unixbench. If someone would like to have the benchresults in detail (e.g. not only the totalscore), I would post them.
So which CFLAGS did I use? Those I posted already above (for 32bit). I tested a little bit without those and only -O2 -fomit-frame-pointer -pipe. The last was a little bit slower in most cases (around 2-8%).
So I hope to finish my 64bit Benchmarks, but am unsure at the moment how representative those might be.
I keep you updated! |
|
Back to top |
|
|
TheMatt n00b
Joined: 22 May 2003 Posts: 35 Location: Boulder, CO
|
Posted: Fri Jun 06, 2003 7:24 pm Post subject: |
|
|
As we wait for nimrod to update (no good way to say that...), I thought I'd share this with interested parties, in case you haven't seen it.
To wit, the first Opteron workstations (yes, AGP) I know of are being offered by BOXX Tech: link!.
Frankly, it isn't as expensive as I thought. I also assume it uses the VIA workstation chipset over the nForce3 since it has two Opterons instead of one. My only wish would be for a couple PCI-X slots, but the AGP is what I've been waiting for.
This is really tempting me.
The Matt _________________ The Matt
I am a theoretical chemist. Fear me! Please. |
|
Back to top |
|
|
iplayfast l33t
Joined: 08 Jul 2002 Posts: 642 Location: Cambridge On,CA
|
Posted: Fri Jun 06, 2003 7:56 pm Post subject: |
|
|
TheMatt wrote: | To wit, the first Opteron workstations (yes, AGP) I know of are being offered by BOXX Tech: link! |
To bad you have to buy Xp with it.... |
|
Back to top |
|
|
TheMatt n00b
Joined: 22 May 2003 Posts: 35 Location: Boulder, CO
|
Posted: Fri Jun 06, 2003 8:08 pm Post subject: |
|
|
Quote: | To bad you have to buy Xp with it.... |
Yeah, that seems odd since they offer Red Hat (ver 8!) with their dual Xeons, &c. My guess is they need it on there to test with.
I am surprised they didn't go to SuSE and get a 64 licence to offer. Maybe they are planning it, but it isn't ready yet?
But, their main business is graphics, HD editing, &c. Maybe they think most people will use Photoshop with this workstation, while the Houdini work is reserved for bigger boxen? Photoshop 32-bit does smoke on Opterons, and most think it'll fly on a 64-bit platform. _________________ The Matt
I am a theoretical chemist. Fear me! Please. |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Sat Jun 07, 2003 9:47 am Post subject: |
|
|
Quote: | As we wait for nimrod to update (no good way to say that...) |
Don't get to angry with me, have some real work to do too.
Ok, had no time to contact an AMD developer, will send an email this weekend.
The pure 64bit system is stoped at the moment, because of too much problems with gcc 3.3 and the packages. Have to test a little bit more which flags work and which not.
Another thing which troubles me: I asked this before .. If I want to run 64bit Code which is linked against 32bit glibc ... does this work? I personaly would say: No! But if not, does running 32bit code against an 64bit Glibc? And if that also doesn't works, do I need to have a 32bit and a 64bit Version of libraries available?
Any news from the Opteron optimized gentoo? I searched a little bit but found nothing new at all. Maybe someone knows more?
Hm .. Suse Enterprise Server 8 is available for Opteronsystems. But I guess they don't bundle it with there hardware cause of the licences. At the moment you have to pay around 550$ for an one CPU Licence and around 300$ for the second CPU. So you would have to pay around 850$ for an dual Opteronsystem for the SUSE Enterprise Server 8 (and this is the only Opteronoptimized SUSE Product I know). |
|
Back to top |
|
|
dweigert Guru
Joined: 04 Oct 2002 Posts: 369 Location: Somerset, NJ USA
|
Posted: Sun Jun 08, 2003 1:40 pm Post subject: |
|
|
Other OS's need to have a duplicate set of libraries. Mostly for binary compatibility issues where packages are distributed (including kernel patches) One for 32 bit applications, and one for 64 bit applications. I'm not sure Gentoo NEEDS to do this, as MOST things are compiled from source. However, for corner cases, I suspect that it would be a good thing to do, same as SGI, Sun, et al. If one wanted to run the latest version of Maya for instance or use other non opensource binary packages, it will be needed. _________________ "Always remember to mount a scratch monkey..." |
|
Back to top |
|
|
DaJoker n00b
Joined: 09 Jun 2003 Posts: 6 Location: USA
|
Posted: Mon Jun 09, 2003 7:21 pm Post subject: My Dual Opteron System |
|
|
nimrod...I have been following/enjoying your reviews. Eventually (as funds permit) I should have a Dual Opteron 1400 (MSI K8D Master-F, 1GB) system up and running. When that comes I want to try out Gentoo on her as well. At some point when you have time (I know, I know) could you leave some extended detailed instructions detailing how you have done what you have done? I'm not a huge benchmarker, but I just want to learn as much as I can about 64-bit stuff as I can quickly...installing, programming for it... I haven't tried out the 2.5 kernel yet, so I guess that will be a good starting point. Anyway, keep up the great posts, and as many details as gentoo.org will allow without assigning you your own quota.
DJ |
|
Back to top |
|
|
DaJoker n00b
Joined: 09 Jun 2003 Posts: 6 Location: USA
|
Posted: Wed Jun 11, 2003 2:58 pm Post subject: Dual Opteron Chassis |
|
|
Just out of curiosity,what kind of chassis/case (brand, model #, etc) are you using? Where did you get it from, and for how much? |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Wed Jun 11, 2003 4:57 pm Post subject: |
|
|
Hm I got 2 Systems ... one was directly from AMD .. a Tower (Aluminium, Collermaster) the second was a 1U Serverchasis (Genesis-chasi). The second was around 2700$ (without tax) for dual Opteron 240, 2GB DDR RAM, 80 GB IDE Disk! |
|
Back to top |
|
|
MarkK n00b
Joined: 18 Nov 2002 Posts: 4
|
Posted: Thu Jun 12, 2003 3:31 pm Post subject: |
|
|
Since I am also working on a 64bit linux distribution right now on a different architexture that also has 32bit compatibility (PPC64). Maybe I can answer your glibc query. Yeah, you need 2 copies (if you want compatibility with binary packages)... what we do is put a whole seperate branch of 64bit libraries for glibc, and the 64bit compilers into /opt/ppc64, and also compile a 64-bit kernel.
I hope to get a gentoo branch for PPC64 made eventually (I posted about this back in November), but right now I'm getting paid to make a debian version. It'll probably be trivial to make a gentoo version actually... We'll see...
If you still need help in a month or so, we should have a couple dual opterons and a quad or two available for me to mess with.
-Mark |
|
Back to top |
|
|
skeptic n00b
Joined: 12 Jun 2003 Posts: 2
|
Posted: Thu Jun 12, 2003 8:48 pm Post subject: |
|
|
I'll be building an opteron system next week, and my goal is to make
gentoo the primary distro on the system. I'll summarize my progress
on a regular basis, and I look forward to your suggestions!
The last server system I built like this from scratch was about 5 1/2
years ago. It's amazing that in that time the total system cost is
50% less, while the processing capability is about 6-8x faster, 4x
the RAM, 15x the storage. |
|
Back to top |
|
|
kermitjunior Apprentice
Joined: 04 Aug 2002 Posts: 167
|
Posted: Sat Jun 14, 2003 12:15 am Post subject: |
|
|
nimrod wrote: | For those who care:
the dual AMD Opteron (240, 1,4ghz) is as fast as an dual Intel P4 XEON with 2,667 ghz.
|
So my question is this: is it worth spending the extra money to buy Opteron, or would a Dual MP 2800 be better? The extra money could be spent on RAM and RAID. Just curious as to your opinion.
I have about 2000 to spend. Would you recommend the Opteron or MP. I'm looking for speed of compiling for Gentoo.
Thanks,
KJ _________________ -----
Toshiba Satellite A15-S157, 2.2 Celery, 40GB, 512MB
AMD Athlon XP 1900+, 640MB PC2100, ABIT KG-7R
IBM 120GB (Linux), WD 30GB (WinDoze), ATI All-In-Wonder 128 Pro PCI |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Sat Jun 14, 2003 4:12 pm Post subject: |
|
|
Hm .. compared to the XEON I would buy the Opteron (it is just cheaper for the same / respective more 32bit Performance and has an upgrade to 64bit onboard .
Compared to the Athlon MP .. I guess the Athlon MP isn't much faster than the Opteron (in fact I would think that the Opteron is nearly as fast as it, if not faster). Again, Opteron has also the 64bit advantage!
An Option is to buy a Single Opteron (dual Board, but with only one cpu) and buy the second cpu some time later.. |
|
Back to top |
|
|
kermitjunior Apprentice
Joined: 04 Aug 2002 Posts: 167
|
Posted: Sun Jun 15, 2003 12:14 am Post subject: mismatch possible |
|
|
Can you mismatch the speeds on the opteron, by chance? _________________ -----
Toshiba Satellite A15-S157, 2.2 Celery, 40GB, 512MB
AMD Athlon XP 1900+, 640MB PC2100, ABIT KG-7R
IBM 120GB (Linux), WD 30GB (WinDoze), ATI All-In-Wonder 128 Pro PCI |
|
Back to top |
|
|
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Mon Jun 16, 2003 12:57 am Post subject: |
|
|
2.5.71 kernel is out, should be relevant to getting it all up and running.
kev |
|
Back to top |
|
|
DaJoker n00b
Joined: 09 Jun 2003 Posts: 6 Location: USA
|
Posted: Mon Jun 16, 2003 3:21 pm Post subject: Different Opteron Clock Speeds |
|
|
Although it seems to me that last week I read something about somebody developing/testing the different clock speeds for Opterons on a single multi-proc mobo, I do not believe it is out. Even if it is developed, I would be hesitant to try it out anytime in the first year because it's just an odd thing to do. There are probably a lot of reasons it has not been done (to my knowledge, or at least mainstream) to date.
As for the other guys, I would go Opteron over Athlon MP just because of the 64-bit. IF YOU DO...be forewarned that there are costs beyond the motherboard and processors to keep your eyes on. If your motherboard is SSI form factor it will probably not work in your ATX case unless it is an eATX case. Power supplies are also incompatible (as far as I've seen). RAM may have a higher requirement on it (ECC Registered vs. NON-ECC Unregistered). Still, the benefit is that you are running SOTA for a while and nobody on your block will have even a single 64-bit machine, much less a 64-bit SMP machine (unless you live somewhere very cool). |
|
Back to top |
|
|
nimrod n00b
Joined: 16 May 2003 Posts: 29
|
Posted: Tue Jun 17, 2003 9:36 pm Post subject: |
|
|
Hm .. Kernel 2.5.72 is out .. here are some relevant things done for Opteron ... some extracts from the Changelog (http://www.kernel.org/pub/linux/kernel/v2.5/ChangeLog-2.5.72):
Quote: |
<ak@suse.de>
[PATCH] x86-64 merge
Lots of small fixes and I merged the PCI subsystem with i386 again
because it was mostly identical. This makes the patch quite big,
but it only removes files.
- Merge PCI subsystem with i386.
This changes the initialization order of PCI and IOMMU slightly. It
didn't see any problems yet, but it could cause some in theory. It
re-adds some code that used to be removed, but it's only small stuff
and it should hopefully cause less mainteance overhead longer term.
- Fix warnings
- Merge cpuid.c with i386.c
- Sync msr.c with i386
- Consolidate externs in asm/proto.h
- sysfs/sysdev fixes for apic/nmi (Bryan O'Sullivan)
- Fix /proc/kcore access
- Add real kern_addr_valid (used for above).
- Support consistent dma_mask in IOMMU
- Fix double print of AMD for CPU model.
- Remove unused wakeup.S file.
- Remove obsolete CONFIG_SIMNOW ifdef.
- Support ptrace access for 32bit vsyscalls.
- Fix warnings in 32bit boot code compilation.
<ak@suse.de>
[PATCH] Fix typo in timing changes + support for x86-64
The us->ns conversion must be in the "tv" if branch of settimeofday,
not in the "tz" if.
Also make it compile for x86-64 again. The 32bit emulation was forgotten
in the conversion.
<ak@suse.de>
[PATCH] Fix compilation of 32bit ioctl emulation on x86-64
The NCP ioctls are declared unconditionally in compat_ioctl now, which needs this
include to still compile on x86-64.
<ak@colin.muc.de>
[PATCH] Fix over-alignment problem on x86-64
Thanks to Jan Hubicka who suggested this fix.
The problem seems to be that gcc generates a 32byte alignment for static
objects > 32bytes. This causes gas to set a high alignment on the
section, which causes the uneven (not multiple of sizeof(struct
kernel_param)) section size. The pointer division with a base not being
a multiple of sizeof(*ptr) then causes the invalid result.
This just forces a small alignment, which makes the section end come out
with the correct alignment.
The only mystery left is why ld chose a 16 byte padding instead of
32byte.
|
Ok .. this stuff with the alignment is quite important in my opinion .. there are several more detailed fixes concerning x86_64, but I think those were the most important one! |
|
Back to top |
|
|
cchapman Guru
Joined: 16 Jan 2003 Posts: 440 Location: Fremont, NE
|
Posted: Wed Jun 25, 2003 4:50 pm Post subject: |
|
|
UPDATE:
Try these flags when compiling:
Code: |
-m64 -03 -msse -msse2 -m3dnow
|
NOTE: MUST USE GCC 3.3 |
|
Back to top |
|
|
0ctane Tux's lil' helper
Joined: 21 Jun 2003 Posts: 115 Location: Cincinnati, OH
|
Posted: Thu Jul 03, 2003 4:31 pm Post subject: Re: mismatch possible |
|
|
kermitjunior wrote: | Can you mismatch the speeds on the opteron, by chance? |
i did read that you can have two different speed opteron CPUs in the same computer (I think it was a tom's hardware opteron vs xeon comparison). I am not sure why one would want to do this however. |
|
Back to top |
|
|
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Sat Jul 05, 2003 6:35 am Post subject: |
|
|
cchapman wrote: | UPDATE:
Try these flags when compiling:
Code: |
-m64 -03 -msse -msse2 -m3dnow
|
NOTE: MUST USE GCC 3.3 |
does that mean you have a working 64bit compiled gentoo?
kev |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|