What's the best hardware for a Gentoo OpenMosix cluster?

_Max_

Hi,

I am thinking of setting up a small cluster (3-4) machines to do some highly parallel calculations (Monte Carlo simulations), and got the impression that OpenMosix is the easiest way to do this (any thoughts on that?).

What's the best hardware out there in terms of bang for the buck, but also in terms of being well tested under Linux (Gentoo)? I would not need fast disk access, and the individual threads don't need a lot of memory.

What do you run?

Max

joeldg · n00b Joined: 15 May 2003 Posts: 30 Location: New York

it is the best by far.
I run an openmosix cluster on my machine here (see .sig)

I currently have three machines 2.1Ghz, 1.9Ghz(running redhat) and a 533Hmz combines ram is 2G and combines diskspace is 160GB. All are running athalons right now and the cluster is running over a 100MB network. (no gigabit cards right now).

This was a test case which I use here locally (mostly crunching seti and whatnot) as I am going to be setting up a ten-node cluster for actual usage on a project.

For my personal cluster I am planning on purchasing a rack of mini-itx boxen (around $300-$400) each and setting them up running over a local gigabit network with my primary box acting as the head node. If you consider it, ten mini servers running at 1.2Ghz is quite a resonable number crunching machine for the price.

That is my suggestion. Also, make sure you enable OMFS support when compiling or you won't be able to use the cluster filesystem.

-cheers
_________________
---------------
coder

_Max_ · Posted: Mon May 19, 2003 8:00 pm Post subject:

I had a look at mini-itx.com, and it has to be said, these things look nice!

But in terms of bang for the buck, suppose you want to spend around 3000 British pounds.

At present, that would buy you around 8 mini-itx with 800 Ghz each (mini-itx.com)
or
8 Athlon XP2600 systems
(www.savastore.com - Watford Electronics)

So to me that sounds as if the big ugly machines win on processing power / price ratio.

Or maybe I am just not looking in the right place?

mpsii · l33t Joined: 11 Mar 2003 Posts: 658 Location: Jackson, TN

Mini-ITX boxes are not for power computing!

Even if you want to spend only $300-400, or pounds or whatever, per box, nothing can equal Athlons for bang-for-buck. You can find Athlon/Duron 1GHz+ systems with at least 128MB RAM apiece for around $250 on a regular basis. Though these would not be rackmount or SFF like Mini-ITX boxes typically are, they would pound the ever-loving smoosh out of a C3 processor in the mini-itx format.

FYI... there is now a P4 mini-itx mobo out, but I have not seen prices for it yet. If you are looking for bang-for-buck and SFF to reduce space, this might be the route to go... even with Celerons instead of P4s.
_________________
-----------------------------------------
Michael

joeldg · n00b Joined: 15 May 2003 Posts: 30 Location: New York

hrmm

well, if you guys have any linkage to sites that are selling cheaper mini-itx boxes please post them... I have been looking around for what might be best. Obviously we are going for quantity over quality here, but quantity+quality would obviously be preferred, but not over cost.

timfreeman · Posted: Fri May 23, 2003 6:24 pm Post subject:

I've never run a cluster, but from researching SMP and noiseless systems for my new box, I think a lot of people choose PIII because of this power consumption vs. performance issue. It has a larger L2 cache than most CPUs in its class (the Piii Tualatin has a 512k cache and is based on .13 micron which makes it use a lot less voltage).

Also, at a linux clustering site, they had a poll for their users' preferred clustering platform:
http://www.linuxhpc.org/pollbooth.php

The Via C3 chip I think is just not very powerful, but very low voltage and very low cooling requirement (so less power for fans etc.). It seems to be used by those looking for extremely silent computing (which would be awesome, but I decided to go with the AthlonMP still for the bang-buck issue).

Here are two great tables that rank CPUs by heat dissapation (you have to scroll down to get to the table).

silentPCreview

anandtech
_________________
|||

neysx · Retired Dev Joined: 27 Jan 2003 Posts: 795

As mentioned above if you consider only the ratio $/power then AMD is the choice.
On the other hand you might want to consider heat dissipation, noise levels, power consumption,... in which case a box of via C3's might be a good trade off.
I built one such beast for the fun of it.
It has 4 C3-800, 1HD, 1CDRom, 1 PSU, 1 hub, 3 fans, all in one home made box.
More recent ITX boards from VIA will give more power obviously. Consider the latest Nehemiah 1Gz e.g.

[img:15ce15e915]http://www.xs4all.nl/~neysx/cluster/closedfront.jpg[/img:15ce15e915]
[img:15ce15e915]http://www.xs4all.nl/~neysx/cluster/openback.jpg[/img:15ce15e915]

timfreeman · Posted: Sun May 25, 2003 4:36 pm Post subject:

sweet, are the green buttons to switch the HD and CDROM around to each motherboard?

dol-sen · Posted: Sun May 25, 2003 5:02 pm Post subject:

I saw an article on the web somewhere that talked about supercomputer cluster bang for the buck. When they considered all factors, the C3 based cluster gave them more useable cpu cycles for the $ spent. By all factors they included the cost of the equipment, electricity to run it,the building & air conditioning requirements, it's electricity & maintainance, and even the fact that the cpu doesn't use a fan therefore increasing the reliablility factor by not having another mechanical device that needed maintainance and replacement.

Just another observation. I have several computers running zetagrid, my new Athalon-XP 2000+ completes 3 work units a day, an old pentium I 200MMx completes 1 work unit in slightly more than 1 day. My new one, the processor is nearly 10 times faster than the old one but only accomplishes 3 times more work.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

neysx · Retired Dev Joined: 27 Jan 2003 Posts: 795

timfreeman · Posted: Sun May 25, 2003 7:38 pm Post subject:

vibidoo · Guru Joined: 27 Nov 2002 Posts: 409

dol-sen · Posted: Thu Jul 03, 2003 6:35 am Post subject:

I don't know how he did it, but you can get a DC to DC adapter (some have cable, some direct connect) that only need a 12VDC supply power. Anyone have a cluster in their car? Van? running on battery?
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

neysx · Retired Dev Joined: 27 Jan 2003 Posts: 795

I have had some private requests about this, so I thought I'd let everyone know a bit more about my experiment.
Here are a few pics of the building process:
www.xs4all.nl/~neysx/cluster/cluster_01.jpg
www.xs4all.nl/~neysx/cluster/cluster_02.jpg
www.xs4all.nl/~neysx/cluster/cluster_03.jpg
www.xs4all.nl/~neysx/cluster/cluster_04.jpg
www.xs4all.nl/~neysx/cluster/cluster_05.jpg
www.xs4all.nl/~neysx/cluster/cluster_06.jpg
www.xs4all.nl/~neysx/cluster/cluster_07.jpg
www.xs4all.nl/~neysx/cluster/cluster_08.jpg
www.xs4all.nl/~neysx/cluster/cluster_09.jpg
www.xs4all.nl/~neysx/cluster/cluster_10.jpg

Yellow stuff is 3mm PVC and black stuff is painted aluminium.
One node has 512Mb and CD+HD, three other nodes are 256Mb diskless.
Hub is an old cheap "target" powered with 5V which is handy to get from ATX PSU.
ATTENTION: This kind of cheap hub is not suitable for a serious cluster project. After a few days the hub freezes the network layer of one of the nodes randomly. This is a known problem. A switch is better suited for the job, e.g. Netgear FS105. Besides one should split "cluster" traffic from normal traffic, specially with diskless stations using two network adapters per node. Epia mobos have a free slot to make this possible.
The recurring question is how to power 4 mobos with one PSU. Simple. I made a 1in-4out cable with a simple electrical connector. One one side, 4 ATX connectors salvaged from defect PSU's, on the other side, the female connector from an ATX extender (I did not want to cut PSU cable). The green wire should only be connected to the first mobo. When it is powered on, the other mobos will receive power and initialize themselves.
Fans are powered with 7V insted of 12V (+5/+12 instead of GND/+12) to lower noise. The lateral air-in fan comes from shown PSU and the two air-out fans at rear come from defect PSU's.
Soft is obviously Gentoo with common root instead of one root per node. It means more work to administer, e.g when baselayout is emerged but was an interesting experimentation. ClusterNFS is used to help differentiate files and dirs between nodes when required.

AvEnGeRs · n00b Joined: 23 Jul 2003 Posts: 13

On psilentreview says somthing like that the PIII-S dissipate less power.... but on the other side it's difficult to cool this cpu (why?????)

charlieg · Posted: Wed Jul 23, 2003 1:22 pm Post subject:

1 PSU != good idea

What happens if it fails? Any commercially available server has redundant PSUs, and with good reason.
_________________
Want Free games?
Free Gamer - open source games list & commentary

Open source web-enabled rich UI platform: Vexi

neysx · Retired Dev Joined: 27 Jan 2003 Posts: 795

AvEnGeRs · n00b Joined: 23 Jul 2003 Posts: 13

Hello!

If i want to go for a cluster (the best i thinks is to use amd for processor....prest/cost it's ok

)

what kinda of ram do you advise to use?? (cause in this case need a good relation performance/cost )

and for motherboard?? (need performant/stable/configurable micro-atx.. so don't waste of space

)

...and at last...very fast processor are used at top with the limit of 100mb lan??

and what heatasinks?? (good...but not much expansive cause it's needed more than one...)

Sorry for the much question

but it's very hard to choose the hw for a cluster cause it's very much problematic choose the hw when need the best for a price... (if is for a single pc it's more more easy cause if we don't know, or are sure, about the quality/performance of components...we can buy one that cost a little more..without spent so much more 8but for cluster one price for.....xx machine...)

thx much in advance!

Greeting!