Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
help in threading a piece of code, plz
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Sat Jan 26, 2013 6:12 pm    Post subject: help in threading a piece of code, plz Reply with quote

I caught inkscape using ONE thread and choking when tracing a bitmap. Dunno what to suspect, mplayer -lavdopts threads=8 evently distributes processing over cores. Gimp implements something about sectioning the image and process in pieces... Scratch this.

Gimp also exhibits odd behavior, GEGL operations take one core 100% cpu usage (shouldn't be none as it's using the graphics card??). And although it does process the image in sections it does so sequentially (one core via htop) Regular filters DO seem to use multiple cores but no visible processes are spawned in htop like mplayer.

How is it possible for a prog to implement multiple threading but only one process is visible in htop?

Now gimp choked with an apparently processing intensive GEGL operation (the entire GUI ain't re-drawing). DOES GEGL work with open source gfx drivers?

Sorry if what I am saying is all mixed up, just doing some testing how threading works.

Anyways, I want to add threading in inkscape's bitmap tracing. On them webz I saw that to implement seems fairly trivial:

single thread:
Code:
var a[b+c]

multiple thread:
Code:
var a[b] + a[c]


(can't recall the exact example but the idea is there)

From this example it appears that there isn't any kind of havy code modification as I can retain the entire code block for each pic section.

This is the idea:

1 before passing bitmap tracing cut up the image into several sections. Code for this must be stupid simple, even shell's cut could be used here.
2 Create several tracers workers, using all cores, for each pic section (looping involved here to do all the sections?)

Is this possible? A way to fool inkscape to do this?

I think the relevant file is trace/trace.cpp

That file has something about threading:

Code:
/**
 *  Threaded method that does single bitmap--->path conversion
 */
void Tracer::traceThread()
{


Code:
#if HAVE_THREADS
    //Ensure that thread support is running
    if (!Glib::thread_supported())
        Glib::thread_init();

    //Create our thread and run it
    Glib::Thread::create(
        sigc::mem_fun(*this, &Tracer::traceThread), false);
#else
    traceThread();
#endif

}


I am not sure whether inkscape is using threading. A small file does the tracing and there's slight bump in activity accross the cores. Yet with a 4000x3000 image makes ONE core spikes to 100% usage and inkscape stops re-drawing itself :/

Is my idea doable?

Another aidea is to take trace.cpp and make it standalon app and use it from command line.

What are ppl's opinion on a solution to this? Are there other routes to convert a bitmap to svg the threaded way or using the GPU?

thanks
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3964
Location: USA

PostPosted: Sat Jan 26, 2013 8:39 pm    Post subject: Reply with quote

Uh...what?

a[b+c] (array reference, I'm assuming) VERY rarely equals a[b]+a[c]...

a*(b+c) = a*b+a*c however, since this the mathematical distributive property.

This is why threading is difficult. You just can't do some things in parallel.

Converting a flat bitmap to a vector map is something that I'm not sure about, this seems non trivial. Cutting out fragments doesn't quite work (imagine tracing a circle that spans multiple pieces), separating the initial image out by color may help. This is not a trivial task.

Also, though using the video card seems like it should not use a core, but since Linux/the main OS does not schedule processes over the video card, it has to still make one CPU supervise the code running on the video card...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Sun Jan 27, 2013 8:00 pm    Post subject: Reply with quote

mm... do you mean it would be problematic to calculate the vector curves/lines for adjecent pixels in the cut out sections?

What about start from different parts of the photo, each corner for instance, towards the center?

tbh, dunno how converting works, but there's gotta be a better way.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 32027
Location: 56N 3W

PostPosted: Sun Jan 27, 2013 8:17 pm    Post subject: Reply with quote

_______0,

Maybe there is but if it were easy, it would have been done by now. Multi CPU hardware has been around for a long time.

Just like intersteller space travel. Either we are special, (first or only planet with intelligent life) or interstellar space travel is hard.
If it were easy, ET would already be here.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3964
Location: USA

PostPosted: Sun Jan 27, 2013 10:45 pm    Post subject: Reply with quote

Here's an example why multithreading is hard by proof of intimidation:

Suppose we had a 512x512 picture. Let's split it up to four 256x256 chunks. No, 256 is too big. Let's split those up to 128x128 chunks... NO..128 is still way too big, let's split those to 64x64.

Nah, even 64x64 is too big, let's use more processors. Fine 32x32. MORE ... 16x16... 8x8... 4x4... 2x2 ... ah.. 1x1 we can do something with these 1x1 chunks...

uh...what can we really do with these 1x1 chunks?

If you can find a solution to do the initial computation with these 1x1 chunks without data from the other pieces, then you've solved our multithreading problem.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Tue Jan 29, 2013 8:53 pm    Post subject: Reply with quote

eccerr0r wrote:
uh...what can we really do with these 1x1 chunks?


I am not sure if you are being sarcastic here or not. 1x1 chunks would be ONE pixel.

Actually there's no shortage of math to address this problem from multiple methods and any of which would be a thousand times better than the current one. From some ppl experimenting it appears that vectorizing large images does work however takes damn long time making inkscape look it hung (problem with re-drawing itself).

On a picture, how to exctract single pixels? Could this be done from command line? Something to traverse the binary jpeg format and separate it into single pixels files.

thanks
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 32027
Location: 56N 3W

PostPosted: Tue Jan 29, 2013 9:11 pm    Post subject: Reply with quote

_______0,

You miss the point. A vector is a line joining two points.
The problem with vectorising tiles in a large image is what to do about the joins?
Sure, you can treat each tile as its own image and parallel process them with the same algorithm but then you have extra processing to paper over the joins.

Its non trivial as a succession of pixels that lie along a straight line are reduced to a single vector and tiling will break up many straight lines.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Tue Jan 29, 2013 9:30 pm    Post subject: Reply with quote

is there a format that has no header info about the pic? Because if I want to brake up a photo into tiles each section is gonna have to have a corresponding jpeg header.

damn, there don't seem to be cli tools to manipulate pics.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3964
Location: USA

PostPosted: Tue Jan 29, 2013 10:03 pm    Post subject: Reply with quote

"proof of intimidation" was meant to be sarcastic :P The whole point was to indicate you cannot keep on splitting the job up into threads, it will get to a point where it doesn't make sense and thus not possible.

Netpbm (in portage: media-libs/netpbm) is a neat set of tools to manipulate images from the command line.

pnm is an uncompressed format you can use that has minimal header information.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum