Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo on the Opteron (x86-64)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo on Alternative Architectures
View previous topic :: View next topic  
Author Message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Sat May 24, 2003 4:30 pm    Post subject: Reply with quote

Ok .. it looks like the forum does forget posts .. mine was lost (and one other I think). Ok .. here we go again:

1.)
Got another Testingsystem (with dual 1,6 Ghz Opteron). Will have a look on this to check performance differneces between 1,4 and 1,6.

2.)
My pure 64 bit is getting purer :). My only problem is .. how can I get glibc to compile with -m64. Glibc seems to strip all the Options provided by make.conf. I had a look in the ebuild file but didn't found anything .. so .. anyone out there who knows?

3.)
Someone posted if I'm sure about the -fexpensive-optimizations flag (and maybe the other stuff too).

In fact, I'm not. -fexpensive-optimizations had an 2% performance increase in some benchmarks on an XEON System and didn't make any other benchmark slower. I will check the performance difference on an opteron between -march=k8 -O2 and the flags I used later.


Ok .. guess tomorrow I will have some more informations regarding those stuff!
Back to top
View user's profile Send private message
daewerk
n00b
n00b


Joined: 23 Apr 2003
Posts: 1

PostPosted: Tue May 27, 2003 5:37 pm    Post subject: Reply with quote

Any more progress? Tips? I'm hoping to have hardware to start a test build soon.
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Thu May 29, 2003 2:30 pm    Post subject: Reply with quote

Ok .. here is the update (later than promised .. I know):

1.) Opteron 1,6 Ghz versus Opteron 1,4 Ghz (32-bit Pure):
Hm .. this is a little bit strange. No performanceboost at all (around 2% in unixbenchmark as an example) .. it looks like more Mhz brings no performance at all (in 32 bit mode..) .. which is quite confusing. :?

2.) Test of different Compilerflags is still running (at the moment the benchmarks run .. I will have those results tomorrow)

3.) Still having problems to build a 64bit only System ... I'm working on it, but it is a little bit toughly.
Back to top
View user's profile Send private message
cchapman
Guru
Guru


Joined: 16 Jan 2003
Posts: 440
Location: Fremont, NE

PostPosted: Thu May 29, 2003 4:01 pm    Post subject: Reply with quote

Post results please......
I am anxiuos to see them

Also post current CCFLAGS and such
Back to top
View user's profile Send private message
gladbach
n00b
n00b


Joined: 15 Jun 2002
Posts: 50

PostPosted: Sat May 31, 2003 3:20 am    Post subject: Reply with quote

yes, more details would be great
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Sun Jun 01, 2003 3:24 pm    Post subject: Reply with quote

Ok .. so here I am with further details (tried to post this several times now, but the forum wasn't reachable. Unsure if that depends on my connection or on the forum ...

Ok here the (quite interesting) stuff:

First of all, a short resumè:

I got two testmachines, one dual Opteron 240 (e.g. 1,4 Ghz) with 2 GB DDRRAM on an MSI K8Master-FT (http://www.msi.com.tw/program/products/server/svr/pro_svr_detail.php?UID=441&MODEL=MS-9131) and another dual Opteron 242 (e.g. 1,6 Ghz) with 6 GB DDRRAM on an MSI K8Master-F (same URL as above).

Some time ago I benched a dual P4 XEON 2,667 and 2,8 using Unixbench so those results are for comparisson.

In this configuration I got the following results for pure 32bit (using Unixbench):

Code:

Average of 20 Runs with Opteron 240 (1,4 Ghz): 415
Average of 20 Runs with Opteron 242 /1,6 Ghz): 414
Average of 20 Runs with DUAL XEON 2,667 Ghz: 411
Average of 20 Runs with DUAL XEON 2,800 Ghz: 423


Ok .. what we see here is that the Opteron 240 is in Unixbench quite as fast as an P4 XEON 2,667 Ghz. But suprise suprise: Opteron 242 as fast as Opteron 240? After some tests and several changes I found out what was the problem. When I use only 4 GB of the 6 GB in the Opteron 242 (so if I physicaly remove 2 GB) the speed is much higher.

Ok, as I understand the Opteron, he uses NUMA to access Prozessorspecific Memory (if you have a look on the MSI K8, you will learn that there are 6 banks for RAM, 4 on a block an 2 on another). If I remove the RAM-Modules in the second block (e.g. with only 2 banks) the speed is as expected. If I put the modules into, and only use 4GB (e.g. compile the Linuxkernel with 4GB Memorysupport) the numbers drop to Opteron 240 level. If I compile the Kernel with max 64 GB Memorysupport and NUMA Accessmode the numbers doesn't change at all.

So .. maybe the NUMA Support of the Kernel is not optimal for the Opteron, or, something different is buggy (maybe I? ;))

So here are the different numbers for Opteron 242 with different RAM and Kernel:

Code:

Average of 20 Runs (6 GB RAM, 2.5.69, 64GB, NUMA) : 414
Average of 20 Runs (6 GB RAM, 2.5.69, 4GB, NoNUMA): 413
Average of 20 Runs (4 GB RAM, 2.5.69, 4GB, NoNUMA): 466


This puzzles me a bit, I will have a talk to an AMD developer next week.

If you compare the Operton 240 and 242 with equal RAM (e.g. both with 2 GB), you get an 12,3% faster Opteron 242, while the Opteron 242 ist in reality 14,3% faster (1,6 Ghz versus 1,4 Ghz). I think this scales quite ok.

Unfortunaly I have to return the Opteron 242 tommorow. Eventualy I can have the Opteron 240 for another week, cause my 64bits Benchmark aren't finished yet. I made other benchmarks then Unixbench, but those results are nearby equal to those of Unixbench. If someone would like to have the benchresults in detail (e.g. not only the totalscore), I would post them.

So which CFLAGS did I use? Those I posted already above (for 32bit). I tested a little bit without those and only -O2 -fomit-frame-pointer -pipe. The last was a little bit slower in most cases (around 2-8%).

So I hope to finish my 64bit Benchmarks, but am unsure at the moment how representative those might be.

I keep you updated!
Back to top
View user's profile Send private message
TheMatt
n00b
n00b


Joined: 22 May 2003
Posts: 35
Location: Boulder, CO

PostPosted: Fri Jun 06, 2003 7:24 pm    Post subject: Reply with quote

As we wait for nimrod to update (no good way to say that...), I thought I'd share this with interested parties, in case you haven't seen it.

To wit, the first Opteron workstations (yes, AGP) I know of are being offered by BOXX Tech: link!.

Frankly, it isn't as expensive as I thought. I also assume it uses the VIA workstation chipset over the nForce3 since it has two Opterons instead of one. My only wish would be for a couple PCI-X slots, but the AGP is what I've been waiting for.

This is really tempting me.

The Matt
_________________
The Matt
I am a theoretical chemist. Fear me! Please.
Back to top
View user's profile Send private message
iplayfast
l33t
l33t


Joined: 08 Jul 2002
Posts: 642
Location: Cambridge On,CA

PostPosted: Fri Jun 06, 2003 7:56 pm    Post subject: Reply with quote

TheMatt wrote:
To wit, the first Opteron workstations (yes, AGP) I know of are being offered by BOXX Tech: link!


To bad you have to buy Xp with it....
Back to top
View user's profile Send private message
TheMatt
n00b
n00b


Joined: 22 May 2003
Posts: 35
Location: Boulder, CO

PostPosted: Fri Jun 06, 2003 8:08 pm    Post subject: Reply with quote

Quote:
To bad you have to buy Xp with it....


Yeah, that seems odd since they offer Red Hat (ver 8!) with their dual Xeons, &c. My guess is they need it on there to test with.

I am surprised they didn't go to SuSE and get a 64 licence to offer. Maybe they are planning it, but it isn't ready yet?

But, their main business is graphics, HD editing, &c. Maybe they think most people will use Photoshop with this workstation, while the Houdini work is reserved for bigger boxen? Photoshop 32-bit does smoke on Opterons, and most think it'll fly on a 64-bit platform.
_________________
The Matt
I am a theoretical chemist. Fear me! Please.
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Sat Jun 07, 2003 9:47 am    Post subject: Reply with quote

Quote:
As we wait for nimrod to update (no good way to say that...)


Don't get to angry with me, have some real work to do too. :wink:

Ok, had no time to contact an AMD developer, will send an email this weekend.

The pure 64bit system is stoped at the moment, because of too much problems with gcc 3.3 and the packages. Have to test a little bit more which flags work and which not.

Another thing which troubles me: I asked this before .. If I want to run 64bit Code which is linked against 32bit glibc ... does this work? I personaly would say: No! But if not, does running 32bit code against an 64bit Glibc? And if that also doesn't works, do I need to have a 32bit and a 64bit Version of libraries available? :?:

Any news from the Opteron optimized gentoo? I searched a little bit but found nothing new at all. Maybe someone knows more?

Hm .. Suse Enterprise Server 8 is available for Opteronsystems. But I guess they don't bundle it with there hardware cause of the licences. At the moment you have to pay around 550$ for an one CPU Licence and around 300$ for the second CPU. So you would have to pay around 850$ for an dual Opteronsystem for the SUSE Enterprise Server 8 (and this is the only Opteronoptimized SUSE Product I know). :(
Back to top
View user's profile Send private message
dweigert
Guru
Guru


Joined: 04 Oct 2002
Posts: 369
Location: Somerset, NJ USA

PostPosted: Sun Jun 08, 2003 1:40 pm    Post subject: Reply with quote

Other OS's need to have a duplicate set of libraries. Mostly for binary compatibility issues where packages are distributed (including kernel patches) One for 32 bit applications, and one for 64 bit applications. I'm not sure Gentoo NEEDS to do this, as MOST things are compiled from source. However, for corner cases, I suspect that it would be a good thing to do, same as SGI, Sun, et al. If one wanted to run the latest version of Maya for instance or use other non opensource binary packages, it will be needed.
_________________
"Always remember to mount a scratch monkey..."
Back to top
View user's profile Send private message
DaJoker
n00b
n00b


Joined: 09 Jun 2003
Posts: 6
Location: USA

PostPosted: Mon Jun 09, 2003 7:21 pm    Post subject: My Dual Opteron System Reply with quote

nimrod...I have been following/enjoying your reviews. Eventually (as funds permit) I should have a Dual Opteron 1400 (MSI K8D Master-F, 1GB) system up and running. When that comes I want to try out Gentoo on her as well. At some point when you have time (I know, I know) could you leave some extended detailed instructions detailing how you have done what you have done? I'm not a huge benchmarker, but I just want to learn as much as I can about 64-bit stuff as I can quickly...installing, programming for it... I haven't tried out the 2.5 kernel yet, so I guess that will be a good starting point. Anyway, keep up the great posts, and as many details as gentoo.org will allow without assigning you your own quota. :wink:

DJ
Back to top
View user's profile Send private message
DaJoker
n00b
n00b


Joined: 09 Jun 2003
Posts: 6
Location: USA

PostPosted: Wed Jun 11, 2003 2:58 pm    Post subject: Dual Opteron Chassis Reply with quote

Just out of curiosity,what kind of chassis/case (brand, model #, etc) are you using? Where did you get it from, and for how much?
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Wed Jun 11, 2003 4:57 pm    Post subject: Reply with quote

Hm I got 2 Systems ... one was directly from AMD .. a Tower (Aluminium, Collermaster) the second was a 1U Serverchasis (Genesis-chasi). The second was around 2700$ (without tax) for dual Opteron 240, 2GB DDR RAM, 80 GB IDE Disk!
Back to top
View user's profile Send private message
MarkK
n00b
n00b


Joined: 18 Nov 2002
Posts: 4

PostPosted: Thu Jun 12, 2003 3:31 pm    Post subject: Reply with quote

Since I am also working on a 64bit linux distribution right now on a different architexture that also has 32bit compatibility (PPC64). Maybe I can answer your glibc query. Yeah, you need 2 copies (if you want compatibility with binary packages)... what we do is put a whole seperate branch of 64bit libraries for glibc, and the 64bit compilers into /opt/ppc64, and also compile a 64-bit kernel.

I hope to get a gentoo branch for PPC64 made eventually (I posted about this back in November), but right now I'm getting paid to make a debian version. It'll probably be trivial to make a gentoo version actually... We'll see...

If you still need help in a month or so, we should have a couple dual opterons and a quad or two available for me to mess with.
-Mark
Back to top
View user's profile Send private message
skeptic
n00b
n00b


Joined: 12 Jun 2003
Posts: 2

PostPosted: Thu Jun 12, 2003 8:48 pm    Post subject: Reply with quote

I'll be building an opteron system next week, and my goal is to make
gentoo the primary distro on the system. I'll summarize my progress
on a regular basis, and I look forward to your suggestions!

The last server system I built like this from scratch was about 5 1/2
years ago. It's amazing that in that time the total system cost is
50% less, while the processing capability is about 6-8x faster, 4x
the RAM, 15x the storage.
Back to top
View user's profile Send private message
kermitjunior
Apprentice
Apprentice


Joined: 04 Aug 2002
Posts: 167

PostPosted: Sat Jun 14, 2003 12:15 am    Post subject: Reply with quote

nimrod wrote:
For those who care:
the dual AMD Opteron (240, 1,4ghz) is as fast as an dual Intel P4 XEON with 2,667 ghz.


So my question is this: is it worth spending the extra money to buy Opteron, or would a Dual MP 2800 be better? The extra money could be spent on RAM and RAID. Just curious as to your opinion.

I have about 2000 to spend. Would you recommend the Opteron or MP. I'm looking for speed of compiling for Gentoo.

Thanks,
KJ
_________________
-----
Toshiba Satellite A15-S157, 2.2 Celery, 40GB, 512MB
AMD Athlon XP 1900+, 640MB PC2100, ABIT KG-7R
IBM 120GB (Linux), WD 30GB (WinDoze), ATI All-In-Wonder 128 Pro PCI
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Sat Jun 14, 2003 4:12 pm    Post subject: Reply with quote

Hm .. compared to the XEON I would buy the Opteron (it is just cheaper for the same / respective more 32bit Performance and has an upgrade to 64bit onboard :).

Compared to the Athlon MP .. I guess the Athlon MP isn't much faster than the Opteron (in fact I would think that the Opteron is nearly as fast as it, if not faster). Again, Opteron has also the 64bit advantage!

An Option is to buy a Single Opteron (dual Board, but with only one cpu) and buy the second cpu some time later..
Back to top
View user's profile Send private message
kermitjunior
Apprentice
Apprentice


Joined: 04 Aug 2002
Posts: 167

PostPosted: Sun Jun 15, 2003 12:14 am    Post subject: mismatch possible Reply with quote

Can you mismatch the speeds on the opteron, by chance?
_________________
-----
Toshiba Satellite A15-S157, 2.2 Celery, 40GB, 512MB
AMD Athlon XP 1900+, 640MB PC2100, ABIT KG-7R
IBM 120GB (Linux), WD 30GB (WinDoze), ATI All-In-Wonder 128 Pro PCI
Back to top
View user's profile Send private message
gladbach
n00b
n00b


Joined: 15 Jun 2002
Posts: 50

PostPosted: Mon Jun 16, 2003 12:57 am    Post subject: Reply with quote

2.5.71 kernel is out, should be relevant to getting it all up and running.

kev
Back to top
View user's profile Send private message
DaJoker
n00b
n00b


Joined: 09 Jun 2003
Posts: 6
Location: USA

PostPosted: Mon Jun 16, 2003 3:21 pm    Post subject: Different Opteron Clock Speeds Reply with quote

Although it seems to me that last week I read something about somebody developing/testing the different clock speeds for Opterons on a single multi-proc mobo, I do not believe it is out. Even if it is developed, I would be hesitant to try it out anytime in the first year because it's just an odd thing to do. There are probably a lot of reasons it has not been done (to my knowledge, or at least mainstream) to date.

As for the other guys, I would go Opteron over Athlon MP just because of the 64-bit. IF YOU DO...be forewarned that there are costs beyond the motherboard and processors to keep your eyes on. If your motherboard is SSI form factor it will probably not work in your ATX case unless it is an eATX case. Power supplies are also incompatible (as far as I've seen). RAM may have a higher requirement on it (ECC Registered vs. NON-ECC Unregistered). Still, the benefit is that you are running SOTA for a while and nobody on your block will have even a single 64-bit machine, much less a 64-bit SMP machine (unless you live somewhere very cool).
Back to top
View user's profile Send private message
nimrod
n00b
n00b


Joined: 16 May 2003
Posts: 29

PostPosted: Tue Jun 17, 2003 9:36 pm    Post subject: Reply with quote

Hm .. Kernel 2.5.72 is out .. here are some relevant things done for Opteron ... some extracts from the Changelog (http://www.kernel.org/pub/linux/kernel/v2.5/ChangeLog-2.5.72):


Quote:

<ak@suse.de>
[PATCH] x86-64 merge

Lots of small fixes and I merged the PCI subsystem with i386 again
because it was mostly identical. This makes the patch quite big,
but it only removes files.

- Merge PCI subsystem with i386.

This changes the initialization order of PCI and IOMMU slightly. It
didn't see any problems yet, but it could cause some in theory. It
re-adds some code that used to be removed, but it's only small stuff
and it should hopefully cause less mainteance overhead longer term.

- Fix warnings
- Merge cpuid.c with i386.c
- Sync msr.c with i386
- Consolidate externs in asm/proto.h
- sysfs/sysdev fixes for apic/nmi (Bryan O'Sullivan)
- Fix /proc/kcore access
- Add real kern_addr_valid (used for above).
- Support consistent dma_mask in IOMMU
- Fix double print of AMD for CPU model.
- Remove unused wakeup.S file.
- Remove obsolete CONFIG_SIMNOW ifdef.
- Support ptrace access for 32bit vsyscalls.
- Fix warnings in 32bit boot code compilation.


<ak@suse.de>
[PATCH] Fix typo in timing changes + support for x86-64

The us->ns conversion must be in the "tv" if branch of settimeofday,
not in the "tz" if.

Also make it compile for x86-64 again. The 32bit emulation was forgotten
in the conversion.

<ak@suse.de>
[PATCH] Fix compilation of 32bit ioctl emulation on x86-64

The NCP ioctls are declared unconditionally in compat_ioctl now, which needs this
include to still compile on x86-64.

<ak@colin.muc.de>
[PATCH] Fix over-alignment problem on x86-64

Thanks to Jan Hubicka who suggested this fix.

The problem seems to be that gcc generates a 32byte alignment for static
objects > 32bytes. This causes gas to set a high alignment on the
section, which causes the uneven (not multiple of sizeof(struct
kernel_param)) section size. The pointer division with a base not being
a multiple of sizeof(*ptr) then causes the invalid result.

This just forces a small alignment, which makes the section end come out
with the correct alignment.

The only mystery left is why ld chose a 16 byte padding instead of
32byte.



Ok .. this stuff with the alignment is quite important in my opinion .. there are several more detailed fixes concerning x86_64, but I think those were the most important one!
Back to top
View user's profile Send private message
cchapman
Guru
Guru


Joined: 16 Jan 2003
Posts: 440
Location: Fremont, NE

PostPosted: Wed Jun 25, 2003 4:50 pm    Post subject: Reply with quote

UPDATE:
Try these flags when compiling:

Code:

-m64 -03 -msse -msse2 -m3dnow

NOTE: MUST USE GCC 3.3
Back to top
View user's profile Send private message
0ctane
Tux's lil' helper
Tux's lil' helper


Joined: 21 Jun 2003
Posts: 115
Location: Cincinnati, OH

PostPosted: Thu Jul 03, 2003 4:31 pm    Post subject: Re: mismatch possible Reply with quote

kermitjunior wrote:
Can you mismatch the speeds on the opteron, by chance?

i did read that you can have two different speed opteron CPUs in the same computer (I think it was a tom's hardware opteron vs xeon comparison). I am not sure why one would want to do this however.
Back to top
View user's profile Send private message
gladbach
n00b
n00b


Joined: 15 Jun 2002
Posts: 50

PostPosted: Sat Jul 05, 2003 6:35 am    Post subject: Reply with quote

cchapman wrote:
UPDATE:
Try these flags when compiling:

Code:

-m64 -03 -msse -msse2 -m3dnow

NOTE: MUST USE GCC 3.3


does that mean you have a working 64bit compiled gentoo?

kev
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on Alternative Architectures All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 2 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum