Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
problem with variable scope in bash
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
javeree
Apprentice
Apprentice


Joined: 29 Jan 2006
Posts: 285

PostPosted: Wed Jan 16, 2013 9:16 am    Post subject: problem with variable scope in bash Reply with quote

I have a problem with the following code in bash:
Code:

#!/bin.sh
find . -name '*'| while read FILE ; do
        OUTPUTFILELIST="$OUTPUTFILELIST $FILE"
        echo $OUTPUTFILELIST
done
# the next line outputs nothing ???
echo "End result= $OUTPUTFILELIST"

As long as I am in the loop, I see the variable OUTPUTFILELIST being filled. As soon as I leave the loop, teh variable is empty again ? How is this possible, as I thought bash variables had global scope ?
How shoudl I rewrite this to retain the result of the loop ?
Back to top
View user's profile Send private message
Naib
Advocate
Advocate


Joined: 21 May 2004
Posts: 4104
Location: Removed by Neddy

PostPosted: Wed Jan 16, 2013 9:53 am    Post subject: Reply with quote

09:50 Naib | !faq 24
09:50 greybot | http://mywiki.wooledge.org/BashFAQ/024 -- I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?

Quote:

In most shells, each command of a pipeline is executed in a separate SubShell. Non-working example:


# Works only in ksh88/ksh93, or bash 4.2 with lastpipe enabled
# In other shells, this will print 0
linecnt=0
printf '%s\n' foo bar | while read -r line
do
linecnt=$((linecnt+1))
done
echo "total number of lines: $linecnt"

The reason for this potentially surprising behaviour, as described above, is that each SubShell introduces a new variable context and environment. The while loop above is executed in a new subshell with its own copy of the variable linecnt created with the initial value of '0' taken from the parent shell. This copy then is used for counting. When the while loop is finished, the subshell copy is discarded, and the original variable linecnt of the parent (whose value hasn't changed) is used in the echo command.

Different shells exhibit different behaviors in this situation:

BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>').
BASH creates a new process only if the loop is part of a pipeline.
KornShell creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88 and ksh93! (but not mksh)
POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).



Quote:

If the input is a file, a simple redirect will suffice:
# POSIX
while read -r line; do linecnt=$(($linecnt+1)); done < file
echo $linecnt

Unfortunately, this doesn't work with a Bourne shell; see sh(1) from the Heirloom Bourne Shell for a workaround.
Use command grouping and do everything in the subshell:
# POSIX
linecnt=0
cat /etc/passwd | {
while read -r line ; do
linecnt=$((linecnt+1))
done
echo "total number of lines: $linecnt"
}

This doesn't really change the subshell situation, but if nothing from the subshell is needed in the rest of your code then destroying the local environment after you're through with it could be just what you want anyway.
Use ProcessSubstitution (Bash only):
# Bash
while read -r line; do
((linecnt++))
done < <(grep PATH /etc/profile)
echo "total number of lines: $linecnt"

This is essentially identical to the first workaround above. We still redirect a file, only this time the file happens to be a named pipe temporarily created by our process substitution to transport the output of grep.
Use a named pipe:
# POSIX
mkfifo mypipe
grep PATH /etc/profile > mypipe &
while read -r line;do
linecnt=$(($linecnt+1))
done < mypipe
echo "total number of lines: $linecnt"

Use a coprocess (ksh, even pdksh, bash 4, oksh, mksh..):
# ksh
grep PATH /etc/profile |&
while read -r -p line; do
linecnt=$((linecnt+1))
done
echo "total number of lines: $linecnt"
Use a HereString (Bash only):
read -ra words <<< 'hi ho hum'
printf 'total number of words: %d' "${#words[@]}"

The <<< operator is specific to bash (2.05b and later), however it is a very clean and handy way to specify a small string of literal input to a command.
With a POSIX shell, or for longer multi-line data, you can use a here document instead:
# Bash
declare -i linecnt
while read -r; do
((linecnt++))
done <<EOF
hi
ho
hum
EOF
printf 'total number of lines: %d' "$linecnt"

Use lastpipe (Bash 4.2)
# Bash 4.2
set +m
shopt -s lastpipe

printf '%s\n' hi{,,,,,} | while read -r "lines[x++]"; do :; done
printf 'total number of lines: %d' "${#lines[@]}"

Bash 4.2 introduces the aforementioned ksh-like behavior to Bash. The one caveat is that job control must not be enabled, thereby limiting its usefulness in an interactive shell.
Back to top
View user's profile Send private message
salahx
Guru
Guru


Joined: 12 Mar 2005
Posts: 368

PostPosted: Wed Jan 16, 2013 10:24 am    Post subject: Reply with quote

Essentially (and what the above post goes into), is that stuff run in a pipe runs in subshell - in this case the entire loop runs in a subshell.
Many years ago, I ran into this problem with a script i wrote (the ffmpeg concat fitter did not exist back then). The data from mplayer was not being saved into the variables. that when I found out about this. That's why there's the rather strange syntax at the end of the "while read MPLAYERIDENTIFY" loop - so the loop doesn't run in a subshell.
Back to top
View user's profile Send private message
javeree
Apprentice
Apprentice


Joined: 29 Jan 2006
Posts: 285

PostPosted: Thu Jan 17, 2013 4:19 pm    Post subject: Reply with quote

Thanks for that very informative answer !
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum