| View previous topic :: View next topic |
| Author |
Message |
Dralnu
Veteran
Joined: 24 May 2006
Posts: 1919
|
 Posted: Thu Apr 05, 2007 7:41 am Post subject: Multithreading scripts |
 |
|
Ok, so I've been piddling around with an idea or two for awhile about a set of scripts to help with Gentoo managment (you know, to help remedy some of the common annoyances that come with admining the system, like a quick fix for the rev dep issue), and was wondering if anyone here had any experiance with threading a script into multiple parts. Example:
I want to sync my system, but while doing this I also want to find out how many packages I have installed, and the total size of these packages. This would be alot faster if it could be done simutaneously, and help save some time. After this, though, you could go through, and have the system spit out what needs updating, while also gathering other data about these packages (how much is going to be downloaded (this is easy), the size of the original package to help estimate install time, ect), then once this is done, take all this info and email it to the user.
Maybe a bad example, but I'm wondering if anyone had messed with this before.
Edit:
I'm looking for help mostly with bash (think that has been covered already), and Ruby (what I do most of my scripting in)
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Last edited by Dralnu on Thu Apr 05, 2007 4:33 pm; edited 1 time in total |
|
| Back to top |
|
 |
JeliJami
Veteran
Joined: 17 Jan 2006
Posts: 1086
Location: Belgium
|
| Posted: Thu Apr 05, 2007 8:51 am Post subject: |
|
|
in general you could do something like:
| Code: | #!/bin/bash
(myfirstcommand > fistcommand.out 2>&1 )&
firstcommand_pid=$!
(mysecondcommand > secondcommand.out 2>&1 )&
secondcommand_pid=$!
# some more commands; you get the picture
wait ${firstcommand_pid}
wait ${secondcommand_pid}
# process .out files for email
|
_________________
Unanswered Post Initiative | Search | FAQ
Former username: davjel |
|
| Back to top |
|
|
d_m
Guru
Joined: 12 Jun 2003
Posts: 570
Location: Philadelphia, PA, USA
|
| Posted: Thu Apr 05, 2007 1:25 pm Post subject: |
|
|
Technically the previous example isn't threaded, but instead uses job control.
I'm not sure if it's possible to write a threaded script in bash. In python it's pretty easy, something like this:
| Code: |
import threading
# we can use this communicate between threads
lock = threading.Lock()
# first acquire the lock so the (future) child will wait
lock.acquire()
# then run the child thread
def child():
lock.acquire()
print 'child does some stuff'
lock.release()
thread = threading.Thread(target=child)
thread.setDaemon(True)
thread.start()
# now do some work, while the child waits
print 'parent does some stuff'
# now release the lock so the child can work, and wait until the child is done
lock.release()
thread.join()
# finally do more work and exit
print "parent does some more stuff, and we're done"
|
_________________
The name that can be named is not the eternal name. |
|
| Back to top |
|
|
Dralnu
Veteran
Joined: 24 May 2006
Posts: 1919
|
| Posted: Thu Apr 05, 2007 4:32 pm Post subject: |
|
|
| davjel wrote: | in general you could do something like:
| Code: | #!/bin/bash
(myfirstcommand > fistcommand.out 2>&1 )&
firstcommand_pid=$!
(mysecondcommand > secondcommand.out 2>&1 )&
secondcommand_pid=$!
# some more commands; you get the picture
wait ${firstcommand_pid}
wait ${secondcommand_pid}
# process .out files for email
|
|
Actually, I think this is close to what I was looking for. I appreciate the python example, but I use Ruby for most (if not all) of my scripting. My fault for not specifying that in the original post.
Edit:
Sweet. My little update script now runs in 6 seconds instead of 14 (minus the sync and update-eix). That helps ALOT (and from all I can tell, it delivers the right info.).
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner. |
|
| Back to top |
|
|
johnnymac
Tux's lil' helper
Joined: 27 Oct 2004
Posts: 105
Location: Hot A$$ Texas
|
| Posted: Fri Apr 06, 2007 1:09 am Post subject: fork and waitpid in perl |
|
|
Another alternative is using fork, exec, and waitpid in perl (I like perl).
#Just a simple example - not real code...
| Code: |
my $pid = fork();
die "Cannot fork $!" if(!defined $pid);
if(!$pid)
{
exec("some system-call or other script here");
}
#do some other stuff here, but once done say you want to wait for the above to finish
waitpid($pid,0);
|
_________________
I've never seen a purple cow...
I never hope to see one...
but I can tell you this...
I'd rather see one than be one. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
