Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
OpenCL: New ebuild and eselect module.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Tue Aug 02, 2011 5:15 pm    Post subject: OpenCL: New ebuild and eselect module. Reply with quote

Hi all,

I have written a couple of ebuilds and programs that I think could be interesting for others.

First, I have written a library that abstract the initialization of OpenCL paltforms and devices. You choose which platform to use, and it will sort the available devices on it according to number of compute units (cores). It will try to create a context on the best device, and will fallback to the next one in the list if it does not work.
You can find oclutils here:
https://github.com/nbigaouette/oclutils/

Second, I have written an ebuild for the Intel OpenCL SDK 1.1. You'll find it here:
https://github.com/nbigaouette/ebuilds/tree/master/dev-util/intel-opencl-sdk

Third, I have updated the AMD Accelerated Parallel Processing (APP, formaly AMD Stream) ebuild to 2.5:
https://github.com/nbigaouette/ebuilds/tree/master/dev-util/amdstream

Fourth, I have forked eselect-opencl 0.0.2 from pentoo (http://dev.pentoo.ch/~grimmlin/distfiles/eselect-opencl-0.0.2.tar.bz2). I've updated it and cleaned it up a bit. The only issue I see is with the nvidia opencl file: x11-drivers/nvidia-drivers install its files to /usr/lib*/libOpenCL.so*. I have tried it on my box with intel, amd and nvidia's opencl and it works, but there might be some corner case I've haven't seen. There might be a better way to handle nvidia's case too. I'm open to suggestions.
You'll find the eselect module here:
https://github.com/nbigaouette/eselect-opencl/
and the ebuild here:
https://github.com/nbigaouette/ebuilds/tree/master/app-admin/eselect-opencl

I hope it will be useful to somebody! ;)

Cheers

Edit: Bumped amdstream from 2.4 to 2.5


Last edited by big_gie on Wed Oct 12, 2011 2:24 pm; edited 1 time in total
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 9753
Location: Córdoba (Spain)

PostPosted: Fri Aug 05, 2011 9:42 am    Post subject: Reply with quote

Moved from Gentoo Chat to Unsupported Software.

It fits better here.
_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
metapsyborg
n00b
n00b


Joined: 28 Sep 2011
Posts: 11

PostPosted: Thu Sep 29, 2011 7:35 pm    Post subject: Re: OpenCL: New ebuild and eselect module. Reply with quote

big_gie wrote:
Hi all,

I have written a couple of ebuilds and programs that I think could be interesting for others.

First, I have written a library that abstract the initialization of OpenCL paltforms and devices. You choose which platform to use, and it will sort the available devices on it according to number of compute units (cores). It will try to create a context on the best device, and will fallback to the next one in the list if it does not work.
You can find oclutils here:
https://github.com/nbigaouette/oclutils/

Second, I have written an ebuild for the Intel OpenCL SDK 1.1. You'll find it here:
https://github.com/nbigaouette/ebuilds/tree/master/dev-util/intel-opencl-sdk

Third, I have updated the AMD Accelerated Parallel Processing (APP, formaly AMD Stream) ebuild to 2.4:
https://github.com/nbigaouette/ebuilds/tree/master/dev-util/amdstream

Fourth, I have forked eselect-opencl 0.0.2 from pentoo (http://dev.pentoo.ch/~grimmlin/distfiles/eselect-opencl-0.0.2.tar.bz2). I've updated it and cleaned it up a bit. The only issue I see is with the nvidia opencl file: x11-drivers/nvidia-drivers install its files to /usr/lib*/libOpenCL.so*. I have tried it on my box with intel, amd and nvidia's opencl and it works, but there might be some corner case I've haven't seen. There might be a better way to handle nvidia's case too. I'm open to suggestions.
You'll find the eselect module here:
https://github.com/nbigaouette/eselect-opencl/
and the ebuild here:
https://github.com/nbigaouette/ebuilds/tree/master/app-admin/eselect-opencl

I hope it will be useful to somebody! ;)

Cheers


Thanks big_gie! Yesterday I installed your overlay. Before I was just using AMD APP SDK from their website built in my home directory, but your overlay and ebuild is much easier. My Master's project is focusing on parallel document clustering and I hope to use openCL.

Could you update the AMD APP ebuild for 2.5? It was released a while ago.
Back to top
View user's profile Send private message
yellowhat
Guru
Guru


Joined: 10 Sep 2008
Posts: 482

PostPosted: Thu Sep 29, 2011 9:27 pm    Post subject: Reply with quote

Some question for you:
What can opencl speed using GPU?
Can opencl help in compiling during emerge?

I am using a amd HD6870.

Thnaks
_________________
Gentoo Linux ~x86, Kernel 3.11.2, Enermax Revolutions85+ 850W, AMD Athlon 64 X2 5000+ BE, Corsair XMS2 PC6400 3GB, ASUS M2R32, Sapphire HD6870, Creative X-Fi Platinum, WD Raptor 150GB, WD 2.5 320GB, Asus DRW-24B1ST
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Thu Sep 29, 2011 9:45 pm    Post subject: Reply with quote

@metapsyborg,
I'm glad you find it useful! I had to do this myself and I realized it could be useful to others. I will update to 2.5 when I'll found the time. You can always fork, modify and then send a pull request ;)

@yellowhat,
OpenCL can speedup many different things. I use it to accelerate a molecular dynamics code and an quantum-fdtd code that I wrote for my thesis. To give you an example, one instance of the molecular dynamics' OpenMP implementation has been running since September 2nd on 24 cores. It's 31% done. I have launched the OpenCL implementation (on an nvidia GTX 590) on September 21st and it's at 69%. The time evolution is not linear so it's hard to evaluate how much faster it is, but a rough estimate is 40 times faster. And because on the 24 cores machine there is two video cards, the gpu version is basically 80 times faster :D

Apple developped OpenCL to accelerate it's own applications on a GPU, a completely different usage then mine. But because not every mac have a discrete video card, they needed a fallback to CPU. They inspired themselves from nvidia's cuda and created an open standard. Now AMD and Intel both have an OpenCL driver to expose the CPU as an accelerator. This is counter-intuitive, but can ease the development and even speed things up! (I do see a speedup on my physics large scale simulations when running with OpenCL as opposed to OpenMP, on the same machine).

OpenCL is not a just a useflag that, when enabled, speeds everything else. It's a programming language, similar to C. Instead of developing your calculations intensive programs in C, you move the heavy part to OpenCL (or cuda). To speed compilation, gcc would need to have some parts re-written in OpenCL. There is no way this is going to happen. And if I remember correctly, long compilations are often disk-bound. This is why you can launch more gcc instances then cores available (using make's -j flag). So it wouldn't be a good optimization anyway.

OpenCL (or any other GPU programming language) is relatively hard to learn. This is because a GPU is a completely different architecture then a CPU which we are used to. When learning it, I remember having the same feeling I had when learning MPI: you need to think differently.
Back to top
View user's profile Send private message
Trog Dog
Apprentice
Apprentice


Joined: 04 Aug 2007
Posts: 282

PostPosted: Sun Oct 02, 2011 4:52 am    Post subject: Reply with quote

G'day big_gie

Thanks for your amdstream and eselect-opencl ebuilds I was previously using the ebuilds from pentoo overlay, just a couple of questions why does the amdstream ebuild want to pull in nvidia-cuda-toolkit and why doesn't it pull in eselect-opencl?
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Mon Oct 03, 2011 4:23 pm    Post subject: Reply with quote

Hi Trog Dog,

I just checked the ebuild. It does pull either nvidia-cuda-toolkit or dev-util/opencl-headers. Both of these packages are in conflict. dev-util/opencl-headers will provide only the opencl headers, necessary to compile opencl code. You'll have to unmask it first and install it if you don't want the nvidia stuff. (the cuda toolkit is probably stable, so this would be why emerge pulls it first before dev-util/opencl-headers).

As for the eselect-opencl, I could add a dependency. I'll have a look when bumping to 2.5.
Back to top
View user's profile Send private message
Trog Dog
Apprentice
Apprentice


Joined: 04 Aug 2007
Posts: 282

PostPosted: Mon Oct 03, 2011 10:33 pm    Post subject: Reply with quote

Thanks for the reply big_gie, I did unmask opencl-headers - I was just confused as to why I would need nvidia code with an ATI/AMD gpu, thanks for clearing that up. WRT eselect-opencl after installing your amdstream package the only way to get my existing binaries to see the new libopencl.so was to upgrade to your eselect-opencl package, the one from pentoo (which is pulled in by the pentoo amd-stream-sdk-bin) just wouldn't cut the mustard, even after env-update source /etc/profile.
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Mon Oct 03, 2011 11:03 pm    Post subject: Reply with quote

If I remember correctly, eselect-opencl looks in a specify place for the .so files. Pentoo's are probably installed somewhere else. If you look at the eselect module's source (https://github.com/nbigaouette/eselect-opencl/blob/master/opencl.eselect) you'll see neer the top:
Code:
AMD_APP="/usr/lib/amd"
# NVIDIA_CUDA="/usr"
INTEL_OCL_SDK="/opt/intel/opencl-sdk"

Pentoo might not install its files under /usr/lib/amd, while my package do.
Back to top
View user's profile Send private message
Trog Dog
Apprentice
Apprentice


Joined: 04 Aug 2007
Posts: 282

PostPosted: Tue Oct 04, 2011 11:15 am    Post subject: Reply with quote

Once again, thank you big_gie.
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Wed Oct 12, 2011 2:26 pm    Post subject: Re: OpenCL: New ebuild and eselect module. Reply with quote

metapsyborg wrote:
Could you update the AMD APP ebuild for 2.5? It was released a while ago.

I just bumped AMD APP (dev-util/amdstream) from 2.4 to 2.5. I haven't tried it though, so please report any success/problem. Thanks!
Back to top
View user's profile Send private message
alaaibrahim
n00b
n00b


Joined: 09 Nov 2011
Posts: 2

PostPosted: Wed Nov 09, 2011 8:55 pm    Post subject: Reply with quote

Everytime I try to install dev-util/amdstream I get with the error
Code:
undefined reference to `gluPerspective'


I tried version 2.5 2.5-r1 and 2.4-r2
I'm not sure what should I do ..
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Wed Nov 09, 2011 9:01 pm    Post subject: Reply with quote

You mean when you try to compile/run one of your opencl program or when emerging amdstream?

On my machine gluPerspective is defined in /usr/include/GL/glu_mangle.h which is part of media-libs/mesa-7.11. For the linking, it is defined in /usr/lib/debug/usr/lib64/libGLU.so.1.3.071100.debug (actually /usr/lib64/libGLU.so.1.3.071100) which is also part of mesa...

Maybe it's just a linking problem?
Back to top
View user's profile Send private message
alaaibrahim
n00b
n00b


Joined: 09 Nov 2011
Posts: 2

PostPosted: Thu Nov 10, 2011 8:46 pm    Post subject: Reply with quote

When I emerge amdstream

I tried to install it from amd SDK right away, it worked fine
then
Code:

ln -s /opt/AMDAPP/lib/x86_64/libOpenCL.so /usr/lib64/libOpenCL.so

compiled my app, and it worked fine.

I'm not sure what is causing the issue in emerge (anyway, for now I wont need it). thank you
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Thu Nov 10, 2011 8:51 pm    Post subject: Reply with quote

Please open an issue here: https://github.com/nbigaouette/ebuilds/issues so I can track the problem and fix it without spamming the thread... Thanks!
Back to top
View user's profile Send private message
Pj0tr
n00b
n00b


Joined: 01 Mar 2012
Posts: 2

PostPosted: Thu Mar 01, 2012 4:32 pm    Post subject: Re: OpenCL: New ebuild and eselect module. Reply with quote

big_gie wrote:
Third, I have updated the AMD Accelerated Parallel Processing (APP, formaly AMD Stream) ebuild to 2.5:
https://github.com/nbigaouette/ebuilds/tree/master/dev-util/amdstream
[...]
I hope it will be useful to somebody! ;)
It would certainly be useful to me, but I fail to install it. I keep getting
Code:
emerge: there are no ebuilds to satisfy "dev-util/amdapp".
when typing "emerge dev-util/amdapp", do I need to activate an unsupported repository somehow?

Also, does this package include the OpenCL drivers too? My machine has 4x AMD Opteron 8-core processors (no GPU), on which I want to run OpenCL.
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Thu Mar 01, 2012 5:20 pm    Post subject: Reply with quote

I renamed the package since AMD had renamed it. So the valid one should be "dev-util/amdapp". I was trying to ease the transition by making =dev-util/amdstream-2.6 an empty ebuild that just depends on =dev-util/amdapp-2.6.

I think everything is in order, but won't be able to check right now, sorry, stomach flu... :S
Back to top
View user's profile Send private message
Pj0tr
n00b
n00b


Joined: 01 Mar 2012
Posts: 2

PostPosted: Thu Mar 01, 2012 5:54 pm    Post subject: Reply with quote

big_gie wrote:
So the valid one should be "dev-util/amdapp".
That's what I wrote, no? Doesn't seem to work though.

What does seem to work though is "emerge x11-drivers/ati-drivers", now I have OpenCL up and running. But perhaps this is not the preferred method for some reason?

big_gie wrote:
but won't be able to check right now, sorry, stomach flu... :S
Np, no hurry. :)
Back to top
View user's profile Send private message
big_gie
Apprentice
Apprentice


Joined: 31 Aug 2004
Posts: 158

PostPosted: Wed Apr 25, 2012 3:26 pm    Post subject: Reply with quote

I've updated dev-util/amdapp-2.6-r3. I've included a fix to the "../../../thread/semaphore.cpp:87: sem_wait() failed" issue faced when nvidia is eselect-ed as opengl provider (use the 'mesa' useflag to include the fix).

To use, make sure to remove "dev-util/amdstream" before (I renamed the package to dev-util/amdapp).

Of course, you will need the ebuild. Either enable my overlay (layman --add nbigaouette) or download the ebuild from either the bug report (https://bugs.gentoo.org/show_bug.cgi?id=257626) or my github (https://github.com/nbigaouette/ebuilds/tree/master/dev-util/amdapp).

Note that I did not test it with an ATI/AMD video card. I use it to run OpenCL on the CPU.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum