Poorman's GPU cluster with crap cards and OpenCL?

[eccerr0r]

I have a motherboard with three PCIe x16 slots and a few x1 slots.

Can I run two RadeonHD 6450 x16's, a V3900 x16, and a Radeon 5450 x1 and have all of the under the control of one OpenCL app?

This seems like this is a typical setup except with much higher performing cards for bitcoin mining, but I wonder if it's possible with dissimilar low end cards too? I don't think it's even worth it to mine with such low cards but wonder if it would even work?

Also is there a general benchmark that will test all these cards at the same time,proving there's some benefit to having all the cards in one box?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Ralphred]

XMrig is a monero miner that has ocl integration. Just comparing hashrates from that will give you a "benchmark", and you can run multiple instances if you can't "consolidate" cards, but my experience with this is that you need new and efficient hardware before you even start breaking even with "western energy costs". We even worked out that running it on an RPi4 was a net loss unless you had it running off a solar panel.
We had a bunch of 5-8yr old hardware that used to be running a video wall, so there were quite a few high end "professional" grade GPU's floating around. That said consolidation to lower the "host system" overhead might yield better results, and if your end goal isn't mining then it should work out.

[szatox]

It was a long time ago so this could be outdated, but I red some guides on using multiple GPUs in one machine and basically the requirement was to have 1 model per 1 kernel driver in use.
So you can have a bunch of the same card running in parallel, or a bunch of very different cards (like 1 amd and 1 nvidia), but you can't have a few similar cards in one rig.

[Goverp]

This may be totally irrelevant, but my ASUS X570 motherboard has 2 slots, but only one will let the video card work with the OpenCL stack. Before investing too much effort, I recommend moving one card from slot to slot and checking it still works.
_________________
Greybeard

[gorg86]

I actually did this for the giggles with an HD 5450 and an R9 280X back in the day and it worked. But when I put in a third card things became icky. It turned out that the motherboard was the problem, the app worked but the card in the third slot became unresponsive after maybe 1h.

[eccerr0r]

Hmm...interesting, so it might work. Now seeing if I can get the infrastructure going... or are these old cards even supported anymore? Fortunately I think all these old cards (V3900, 6450, 5450) are all Evergreen cards so they have some similarities.

There indeed is another problem with a lot of motherboards, hoping it can supply power to all these low end cards - since they don't have auxiliary power connectors on the board, they depend on the motherboard supplying power. Fortunately this board does have the 2x4 power connector to supply four +12 lines to the motherboard, for both cpu and pcie... I suspect some of these pcie slots are actually electrically x8s but nevertheless not intended for GPUs but for RAID or network cards...

As an aside before dumping all those cards into a single machine, I'm trying to get opencl to work on a single RadeonHD 6520M and it's not working, installed mesa [opencl,video_cards_radeon,video_cards_r600] and clinfo, and it reports no devices...ugh

btw other than just playing around with it, I'm hoping ffmpeg will take advantage of the cards...all at once?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[gorg86]

I did that around 2015, a lot of things have changed since then. You will run into trouble, some low end cards don't work with OpenCL anymore and that won't be fixed unfortunately. There is no documentation about which cards are affected, you'll have to try it out. On top of that you got an old PRO card thrown in the mix, I have no experience with those. If I was on a budget I would get an RX 5500 with 8GB on ebay (60-100 bucks) and just work with that card.
Some other user on here with an R7 240 (I think) wanted to get that thing to work with OpenCL, we tried everything even the Mesa OpenCL implementation as a last resort, it didn't work.
It was this thread: https://forums.gentoo.org/viewtopic-t-1164936-highlight-.html

[eccerr0r]

Yeah I just found that out sorta, the deprecation of clover and going to rusticl. Seems R600 cards...though partially implemented...may not get fully finished :(

Ugh, I have so many R600 / Evergreen cards (Caicos, Turks, Sumo) -- those cards listed above are all of this type.

Seems even my R7 250 is too old (GCN1?)?

I don't know about rocm which yet another system and I don't have cards for anything that new, so that's yet another barrier...

BTW I might have to remove the USE=opencl ,,, not worth holding breath for it to start working again? https://gitlab.freedesktop.org/mesa/mesa/-/issues/7420
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[gorg86]

If those cards are still worth something, then I would sell them and buy a more recent one with that money
One budget RDNA1 GPU (except the 6500 XT) would probably outperform a cluster of R600s.

If you already compiled mesa with OpenCL try to get your GPU running with that, if it doesn't work use the proprietary OpenCL libs.
But I would not try that with a notebook GPU though, from my experience they are more problematic then their desktop counterparts (especially the low end models, which were never intended to do any computing anyway)

[eccerr0r]

I just added a link above, and says rusticl is not ready for the cards I have.

Cards aren't worth anything either, so no funds to get a newer card, not to mention the laptop/mobile gpu. I still want it to help ffmpeg if possible, seems reasonable?

The proprietary libs cause clinfo to segfault on the laptop, not sure what to make of that... haven't tried the desktop machine yet.

More links: b.g.o link failure on rusticl

--- EDIT ---

Hmm.... it seems that to actually do something with rusticl due to it's pre-release status, running things like clinfo requires

[wanne32]

Just a few words about sense

About performance:
These are not only old but also pretty low end GPUs.
Even if you are getting it running:
All Cards together have 0.9TFLOP FP32-Speed and absolute zero FP64 capability. [3]
Because of the later most scientific applications won't work at all. For the first even a GTX 2060 (which is also low end and 2 generations outdated) that is now available for ~90€ on ebay will outrun them by over 10 times (11TFLOP FP32). If you are not living in the Middle East or the United States you can calculate how fast your electricity bill will outrun the price.

Even worse: More most applications will distribute the work over all cores evenly and then wait until everything is finished. So for every task the slowest GPU will determine your performance for all cards. resulting in even worse results than 3 times the slowest GPU because in some special cases it will outperform your 2nd lowest. You can prevent this by running separate applications on every card.

PCIe:
Even the most modern Desktop CPUs have only 20 PCIe-lanes. You would need 33.
So what the expensive boards are doing for the "non colored" slot is something like splitting up 4 PCIev4 lanes into 8 PCIev3 lanes and than connecting only the first 4 lanes of the x16 slot. The cheaper ones just connect the first slot and that is it. Since your GPU use PCIev2 anyway, you do no have a problem with the lower speed. But very much about the x4 instead of x16 which will result in a quarter bandwidth. And this is assuming you are running your cards on a modern day CPU with a high end mainboard. – This can be a problem if your application is very bandwidth heavy (like image-editing) but has not to be.
This is why Gamers bundle their GPUs together with an external Datalink (SLI HB) and everybody else is using Server CPUs if you are having more than one GPU.

Mining.
For mining neither float nor bandwidth are relevant. INT32 and maybe INT8 is the interesting value. You wont find any values for these low end cards. But I assume would will be at a little bit less than 2TOPs INT32 which has to be used for INT8 too. for comparison a modern H100<200b> is at 2000TOPs INT32 and 4000 INT8 TOPs. [4] So when thy need 3 months to mine a monero it would take you 250 Years.

Bitcoin mining:
No. Noone is using GPUs for bitcoin mining any more since asics are magnitudes faster.
Here your odds: An RTX 4090 is doing ~50GH/s [2] so your 3 cards will be at 2GH/s or 2·10⁹ if you are running them in an optimal manner. together. A the moment you need in average around 2.5·10²¹ Hashes to get a bitcoin [1]. So you would need ~10¹²s or over 30 thousand years to mine one Bitcoin.

Drivers with multiple Cards:
While IO usually has a little bit of overhead in a VM PCI-Passthough has not. If you do not get them working together: Just do PCI-Passthrough in different VMs.

[1] https://www.blockchain.com/explorer/charts/hash-rate
[2] https://gist.github.com/Chick3nman/32e662a5bb63bc4f51b847bb422222fd
[3] https://www.techpowerup.com/gpu-specs/firepro-v3900.c579 https://www.techpowerup.com/gpu-specs/radeon-hd-5450.c503 https://www.techpowerup.com/gpu-specs/radeon-hd-6450.c402
[4] https://en.wikipedia.org/wiki/Ampere_(microarchitecture)

[eccerr0r]

^^^ We never liked you, please stop making new accounts and go away *sigh*
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[wanne32]

[Hu]