| View previous topic :: View next topic |
| Author |
Message |
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
 Posted: Sat Nov 08, 2003 7:33 pm Post subject: What Settings are good for this HIGH POWER server? |
 |
|
Hi,
we are getting a nice powerfull server delivered next week that should be used as Terminal Server (via XDMCP). Since Gentoo will be much more optimized for the Hardware I decided to use this nice Distro.
Some questions are to be clarified. What special compile settings are recommended for the following hardware:
IBM e-Server 445 (2 together via SMP Cable)
16 x 2.0 GHz P4 XEON CPU
64 GB RAM (yes... GigaByte...) 
IBM ServeRaid 4Mx Controller
Does this server needs SWAP ???????? 
The server should handle 70 - 150 X Sessions at a time !!
You can probably imagine that my fingers are burning for this server and I will probably post something how the install performs...
Execute |
|
| Back to top |
|
 |
Lovechild
Advocate
Joined: 17 May 2002
Posts: 2858
Location: Århus, Denmark
|
| Posted: Sat Nov 08, 2003 7:51 pm Post subject: |
|
|
Yes it needs swapping... it needs to be swapped with my Athlon XP...
I would allocate some swap just for safety. |
|
| Back to top |
|
|
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
| Posted: Sat Nov 08, 2003 8:06 pm Post subject: |
|
|
mhh.. I think I give it 1 or 2 Gig SWAP.
I can't waste 2 x RAM for SWAP on this server.... 
ciao
Execute |
|
| Back to top |
|
|
madchaz
l33t
Joined: 01 Jul 2003
Posts: 993
Location: Quebec, Canada
|
| Posted: Sat Nov 08, 2003 8:34 pm Post subject: |
|
|
I'd also recomend swap, if only because of the high number of X sessions.
make sure you compile SMB into teh kernel, otherwise all those nice CPUs will do exactly the same thing
Because it's an IMB server, it might be a good idea to consult with your representative on what distros are supported by them however. IBM is good with linux but I doupt they support gentoo yet. Also, some of the hardware might not be compatible with gentoo. the IBM serverRaid 4Mx comes to mind. you also need some very special configurations to support so many cpus with so much ram. have to remember that teh 32bit architecture of those XEON can only support 4Gigs by itself, so chances are you need to run one "OS" per CPU and then they work togeter as a cluster. So geting in touch with the IBM guy is probably the best idea.
_________________
Someone asked me once if I suffered from mental illness. I told him I enjoyed every second of it.
www.madchaz.com A small candle of a website. As my lab specs on it. |
|
| Back to top |
|
|
delta407
Bodhisattva
Joined: 23 Apr 2002
Posts: 2876
Location: Chicago, IL
|
| Posted: Sat Nov 08, 2003 8:34 pm Post subject: Re: What Settings are good for this HIGH POWER server? |
|
|
| Execute wrote: | IBM e-Server 445 (2 together via SMP Cable)
16 x 2.0 GHz P4 XEON CPU
64 GB RAM (yes... GigaByte...)
IBM ServeRaid 4Mx Controller |
| Execute wrote: | | Does this server needs SWAP ???????? |
Probably not, but give it swap anyway. In that manner, memory pages that aren't often used can be swapped to disk, and RAM can be used as a block device cache. Moving unused things out of RAM improves performance, which is why Linux does it by default. So, yes, give it swap.
| Execute wrote: | | The server should handle 70 - 150 X Sessions at a time !! |
You should be able to handle way more than that.
_________________
I don't believe in witty sigs. |
|
| Back to top |
|
|
delta407
Bodhisattva
Joined: 23 Apr 2002
Posts: 2876
Location: Chicago, IL
|
| Posted: Sat Nov 08, 2003 8:44 pm Post subject: |
|
|
| madchaz wrote: | | I'd also recomend swap, if only because of the high number of X sessions. |
You're not going to allocate all 64 GB of RAM with 150 X sessions. Swap isn't needed because RAM is tight, swap is useful because it allows RAM to be used for other purposes (speeding up disk access).
| madchaz wrote: | | make sure you compile SMB into teh kernel, otherwise all those nice CPUs will do exactly the same thing |
SMP, you mean.
This reminds me: do some tests. Your Xeons should have Hyperthreading as an option. Sometimes enabling it increases performance, sometimes it decreases it. So, figure out which is best for you.
| madchaz wrote: | | Also, some of the hardware might not be compatible with gentoo. the IBM serverRaid 4Mx comes to mind. |
What? The ServeRAID is well-supported under Linux. Besides, if it runs under IBM's favorite distros (Red Hat and SuSE), it'll run under Gentoo. Same kernel, same drivers.
| madchaz wrote: | | you also need some very special configurations to support so many cpus with so much ram. have to remember that teh 32bit architecture of those XEON can only support 4Gigs by itself |
It can directly address 4 GB, but PAE lets the system address up to 64 GB. Enabled by "High Memory Support" in the kernel's "General Setup" page.
| madchaz wrote: | | so chances are you need to run one "OS" per CPU and then they work togeter as a cluster |
Where on earth did you get that idea?
_________________
I don't believe in witty sigs. |
|
| Back to top |
|
|
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
| Posted: Sat Nov 08, 2003 9:11 pm Post subject: |
|
|
| madchaz wrote: |
Also, some of the hardware might not be compatible with gentoo. the IBM serverRaid 4Mx comes to mind. |
I've found that out already. The ServeRAID is a special thing under Linux and Wind0ws. IBM always matches the BIOS version with the Driver Version. All other configurations are not supported and technical support will blame everything in you. 
BUT: IBM has a Kenel patch to support the latest BIOS of the ServeRAID. The native kernel driver should be for bios 5.10 AFAIK (BIOS 6.10.24 Firmware 6.10.70 is the latest) and the kernel gives warnings during the boot until you update. 
I saw that on RedHat.
ciao
Execute |
|
| Back to top |
|
|
Ari Rahikkala
Guru
Joined: 02 Oct 2002
Posts: 370
Location: Finland
|
| Posted: Sat Nov 08, 2003 9:19 pm Post subject: |
|
|
| Quote: | | Does this server needs SWAP ???????? |
I'm just reiterating what delta407 said, but anyway... the question isn't whether it needs swap or not, the question is how much it benefits from swap. Cache is good, pages that sit untouched for days in RAM are not.
| Quote: | | What special compile settings are recommended for the following hardware: |
I'm not sure if you should even be interested in this. You mean CFLAGS, right? -march=pentium4 -pipe -O2 and that's it. For anything higher than that you shouldn't trust the word of any single user on the Gentoo forums - you should benchmark and stresstest your stuff to death and apply special compile flags only for specific packages, keeping the system in general stable. As delta407 implied, that system is quite beefy and definitely doesn't need optimization trickery to manage serving ~100 X terminals.
There's one aspect where I would actually be very torn between stability and performance when considering such a system: the kernel. Sure, 2.4 is kinda tried and true, although it's also known to be among the more unstable "stable" Linux kernel series. It would just be such a shame to see this baby's resources wasted on the... lesser scalability... of 2.4. 2.6 has several features like the O(1) scheduler and the anticipatory scheduler (the earlier is for determining which process gets to run and when, the latter for determining when HD read and write requests are sent to the disk - and they're both a lot better at it than anything the 2.4 series can offer without patching) that you simply do not want to lack on this hardware. And I'm no expert on 2.6 - if you need to be convinced, I'm sure that there are 2^(pi^(e^(random))) users here who can tell you more about the greatness of Linux 2.6...
_________________
<laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny |
|
| Back to top |
|
|
geoffwa
n00b
Joined: 05 Nov 2003
Posts: 6
Location: AU
|
| Posted: Wed Nov 12, 2003 3:30 am Post subject: |
|
|
64GB of RAM!?!? (aiee)
There are two SUN servers at my university have only 4GB and 12GB of RAM respectively and yet the later has gotten up to 400 X sessions running on it. The main problem is getting CPU time, particularly when there are lots of people compiling stuff (viz. assignment due). With 64GB of RAM I would say disable swap, and trade in 32GB for more processors.
Go for the 2.6 kernel definitely, as it scales on to multiprocessor machines *much* better than the 2.4 kernel (see kerneltrap benchmarks/discussions). |
|
| Back to top |
|
|
ewan.paton
Veteran
Joined: 29 Jul 2003
Posts: 1219
Location: glasgow, scotland
|
| Posted: Wed Nov 12, 2003 6:19 am Post subject: |
|
|
just a thought but why not have a ramdisk for all the progs users need fast access to, should save disk io probs when 150 users log in at 9am. i dont know how well it would work with such a machines ram-cpu layout will they be HT enabled[1], also would you post how long a build takes just so we can drool
[1]i know its had its problems but as long as you hav the 2nd version ones its suposed to be ok and given the number of users its unlikely they will be useing the same part of the cpu
_________________
Giay tay nam | Giay nam cao cap | Giay luoi |
|
| Back to top |
|
|
delta407
Bodhisattva
Joined: 23 Apr 2002
Posts: 2876
Location: Chicago, IL
|
| Posted: Wed Nov 12, 2003 6:45 am Post subject: |
|
|
| ewan.paton wrote: | | just a thought but why not have a ramdisk for all the progs users need fast access to, should save disk io probs when 150 users log in at 9am. |
Ramdisks aren't really a good idea -- you would have a copy of everything on disk anyway, and a ramdisk means you have another copy to maintain. Plus, this adds even more administrative overhead on top of that: moving stuff to a ramdisk assumes that all of it is accessed often, and therefore all of it should be kept in RAM. This simply isn't true.
Recall that Linux caches disk access automatically. (This is why lots of RAM is good.) This lets the operating system choose how to balance the needs of the kernel, the applications, and the I/O. Beyond this, creating a swap partition allows inactive portions of RAM to be paged to disk. The operating system will take the swap into consideration, and will make choices (like swapping out highly unused pages) that positively impact performance. Swap is a good thing because it gives the kernel more options.
_________________
I don't believe in witty sigs. |
|
| Back to top |
|
|
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
| Posted: Wed Nov 12, 2003 11:16 am Post subject: |
|
|
| ewan.paton wrote: | i dont know how well it would work with such a machines ram-cpu layout will they be HT enabled[1], also would you post how long a build takes just so we can drool
[1]i know its had its problems but as long as you hav the 2nd version ones its suposed to be ok and given the number of users its unlikely they will be useing the same part of the cpu |
We already got some smaller IBM servers in and they run fine with HT enabled. There is currently RedHat installed, but pretty soon we will migrate them to gentoo. The big server will be the first one.
I will post some nice numbers, i.e: Build time, nice TOP with HT enabled... whatever looks cool |
|
| Back to top |
|
|
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
| Posted: Wed Nov 12, 2003 11:23 am Post subject: |
|
|
| geoffwa wrote: | 64GB of RAM!?!? (aiee)
With 64GB of RAM I would say disable swap, and trade in 32GB for more processors.
|
That is a problem with the IBM 445. The option to have more than 16 CPU's requires 2 more 445 servers with the nice expensive SMP cables. Even if it is affordable, IBM does not offer it yet. (AFAIK) They announced that feature for end of this year.
It would be nice to see a server with 32 CPU's. Than with HT 64.....
The 445 then even has 512mb third level cache instad of 256.
Maybee if we run into a performance problem with this server..... so probably never 
ciao
Execute |
|
| Back to top |
|
|
Ari Rahikkala
Guru
Joined: 02 Oct 2002
Posts: 370
Location: Finland
|
| Posted: Wed Nov 12, 2003 12:03 pm Post subject: |
|
|
| geoffwa wrote: | 64GB of RAM!?!? (aiee)
There are two SUN servers at my university have only 4GB and 12GB of RAM respectively and yet the later has gotten up to 400 X sessions running on it. The main problem is getting CPU time, particularly when there are lots of people compiling stuff (viz. assignment due). With 64GB of RAM I would say disable swap, and trade in 32GB for more processors. |
Well, in that case... hm... could disabling swapping (it's called something like "enable paging of anonymous memory" or something in the kernel config, at least in 2.6) possibly decrease the CPU time used by the kernel? I mean, if 64 gigabytes or RAM is so much more than what this guy needs, even a small decrease in CPU time use might be more beneficial than swap...
Of course, few people probably have ran 2.6 kernels with swapping disabled, so I wouldn't be surprised if there were instabilities that haven't been found by any tester yet in the VM with swapping disabled...
_________________
<laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny |
|
| Back to top |
|
|
Ralphus Maximus
Tux's lil' helper
Joined: 22 Jul 2002
Posts: 148
Location: At Large
|
| Posted: Wed Nov 12, 2003 8:36 pm Post subject: Re: What Settings are good for this HIGH POWER server? |
|
|
| Execute wrote: | IBM e-Server 445 (2 together via SMP Cable)
16 x 2.0 GHz P4 XEON CPU
64 GB RAM (yes... GigaByte...)
IBM ServeRaid 4Mx Controller
|
Man! What kind of FPS will you get with Quake?
Cheers,
RM
_________________
Hero for hire. Dragons rescued, Virgins slain.
You say "Cheesy" like it's a bad thing!
In a society that is drug infested, violence-wracked and polluted by chemical greed, no one has ever died from an overdose of pornography. |
|
| Back to top |
|
|
Execute
n00b
Joined: 08 Nov 2003
Posts: 14
Location: Germany
|
| Posted: Sat Nov 15, 2003 10:18 am Post subject: Re: What Settings are good for this HIGH POWER server? |
|
|
| Ralphus Maximus wrote: |
Man! What kind of FPS will you get with Quake?
|
A real bad one. It's only a ATI Rage XL w 8meg or so. And since it's a server, no AGP.
Of course you could hook it up via Gigabit Ethernet to a powerfull SGI X-Server and just do the Computing on the IBM |
|
| Back to top |
|
|
|
Gentoo Chat
