| View previous topic :: View next topic |
| Author |
Message |
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
 Posted: Sat Jan 04, 2014 11:54 pm Post subject: stupid bash tricks: multi-emerge-tail |
 |
|
Here is a very ugly script I've whipped up to watch the good ol' meaningless spew scroll by when doing parallel emerges:
| Code: |
#!/bin/bash
[[ $( id -u ) -eq 0 ]] || {
[[ -z "$I_WANT_TO_BE_ME" ]] && {
echo 'becoming root via sudo and reinvoking myself...' >&2
echo '(or perhaps you [[ -z "$I_WANT_TO_BE_ME" ]] ?)' >&2
echo >&2
SCRIPT=`realpath -s $0`
echo "running: ${SCRIPT} $*" >&2
echo >&2
sleep 0.5
exec sudo "${SCRIPT}" "$@"
exit 0
}
}
delay=2.5
vartmpportage="$( portageq envvar PORTAGE_TMPDIR )/portage"
notfirst=
bam=poo
# turns out it surprisingly difficult to figure out what to test here...
# while [[ $( ps -eo args | grep '^/usr/bin/python.*/emerge' ) ]] ; do
while [[ 1 ]] ; do
oldbam=bam
bam=
if [[ $notfirst ]] ; then
sleep ${delay}
else
notfirst=exactly
fi
active_emerges=$( qlop -cC | grep '^[[:space:]]\*' | sed 's/^[[:space:]]*\*[[:space:]]*//' | sed 's/[[:space:]][[:space:]]*/\n/g' | sort )
lines=5
if [[ ${active_emerges} ]] ; then
lines=$(( ( $( tput lines ) / $( echo ${active_emerges} | wc -w ) ) - 2 ))
fi
(( lines < 2 )) && lines=2
for merge in ${active_emerges} ; do
hdr=$( echo -e "\033[1m[[ ${merge} ]]\033[0m" )
cols=$(tput cols)
thelines="$( tail -n ${lines} "${vartmpportage}"/${merge}/temp/build.log | fold -w ${cols} | tail -n ${lines} )"
# if bam already nonempty, add a newline for aesthetic purposes
if [[ ${bam} ]] ; then
bam=$( echo "${bam}" ; echo ; echo "${hdr}" )
else
bam="${hdr}"
fi
bam=$( echo "${bam}" ; echo "${thelines}" )
done
if [[ ${oldbam} != ${bam} ]] ; then
clear
echo "${bam:-"nuthin'"}"
fi
done
echo "looks like emerge finished."
|
_________________
-gmt |
|
| Back to top |
|
 |
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Jan 07, 2014 2:40 am Post subject: |
|
|
You might like to check out update which does a similar thing when using --jobs, and also filters emerge output when not.
It's an awful lot bigger than yours, but started out along the same lines, only there was no --jobs back then, and indeed no --keep-going, which was the original problem to fix.
Your grep | sed | sed can all be done with one sed afaict:
| Code: | | sed -n '/^[[:blank:]]*\*[[:blank:]]*/{s///;s/[[:blank:]][[:blank:]]*//;p;}' |
should give you the basis, though I'm not sure what's going on with \n in there. sed is line-based, so you might as well use [[:blank:]] which is slightly quicker. The only way you could have newlines is if you used the n command. I have a feeling the last substitution may be better as s/[[:blank:]].*/;
| Code: | | lines=$(( ( $( tput lines ) / $( echo ${active_emerges} | wc -w ) ) - 2 )) |
makes little sense to me: you'd be better off using an array for active imo, and then ${#active[@]} is the number of elements in the array, or build it with active[n++]=$pkg to get rid of the echo | wc -w subshell.
| Code: | | bam=$( echo "${bam}" ; echo ; echo "${hdr}" ) |
is much easier as or I use an $EOL readonly so: (shells do not field-split assignment, same as case.)
subshells are the thing which slows stuff down the most, ime, especially with bash which is quite a heavy program in comparison to smaller shells. #bash and #sed on IRC: chat.freenode.net are your friends.
HTH,
steveL |
|
| Back to top |
|
|
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
| Posted: Tue Jan 07, 2014 9:22 am Post subject: |
|
|
| steveL wrote: | You might like to check out update which does a similar thing when using --jobs, and also filters emerge output when not.
It's an awful lot bigger than yours, but started out along the same lines, only there was no --jobs back then, and indeed no --keep-going, which was the original problem to fix.
That looks pretty cool. Many's time time I considered building such a beast myself but, well there's only room in my brain for so many half-finished projects 
Your grep | sed | sed can all be done with one sed afaict:
| Code: | | sed -n '/^[[:blank:]]*\*[[:blank:]]*/{s///;s/[[:blank:]][[:blank:]]*//;p;}' |
should give you the basis, though I'm not sure what's going on with \n in there. sed is line-based, so you might as well use [[:blank:]] which is slightly quicker. The only way you could have newlines is if you used the n command. I have a feeling the last substitution may be better as s/[[:blank:]].*/;
|
Since I wrote this code (if you can even call it that hehe) I finally broke down and learned a few basics of how sed works... I'm a way better sed scripter now but I still think I pretty much suck at it
To be clear, I'm well aware that this is not anything even vaguely resembling decently written.
I occasionally throw together a crappy little script like this as a one-off but end up addicted to it... this was one of those.
Definitely one from the Jackson Pollack school of software design... Don't think I've used qlop or tput before or since -- today I have no idea what they are (of course I could ask Google but then I'd probably have to forget something to make room for it -- with my luck, it'd be the knowledge of where my genitals are, or how to put on socks, so I'm not gonna risk it  )
| steveL wrote: |
subshells are the thing which slows stuff down the most, ime, especially with bash which is quite a heavy program in comparison to smaller shells. #bash and #sed on IRC: chat.freenode.net are your friends.
|
Many ACKs above. I fucking /love/ #bash. They are shockingly nice there, and as often as not, happy to explore whatever impossibly obscure corner case I've decided I need to obsess over on any given 3am coding bender run amok. And subshells are clearly from the devil, I'm fully on that page with you.
Dunno about other shells. Bash seems pretty damn fast to me ... but I don't have meaningful amounts of experience with anything else and anyhow, my computer has rack ears instead of a megapixel camera on both ends -- I should probably bear in mind that most of theirs will go way slower than mine does when I'm expecting people to consume code I've written...
_________________
-gmt |
|
| Back to top |
|
|
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
| Posted: Tue Jan 07, 2014 9:25 am Post subject: |
|
|
Looks kinda cool, btw! Or if not cool, useful. I'll check it out.
_________________
-gmt |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Wed Jan 08, 2014 6:01 am Post subject: |
|
|
| gmt wrote: | Since I wrote this code (if you can even call it that hehe) I finally broke down and learned a few basics of how sed works... I'm a way better sed scripter now but I still think I pretty much suck at it :)
To be clear, I'm well aware that this is not anything even vaguely resembling decently written.
I occasionally throw together a crappy little script like this as a one-off but end up addicted to it... this was one of those.
Definitely one from the Jackson Pollack school of software design... Don't think I've used qlop or tput before or since -- today I have no idea what they are (of course I could ask Google) |
Use the manpages silly ;) With portage-utils (q*) they're pretty minimal, but they don't change much, and are designed for use in scripts. They're all the same binary q, it's multi-call, built for use in rescue situations. Though algorithmically it had a lot of problems last time I looked (over 3 years ago) it's always been very quick. So, you end up just playing with it each time you're writing something that uses it, to check the output. That's a good habit to get into in any case, since a lot of scripting is wrapping other commands; basically building a bit of knowledge in around them, to make routine tasks routine so complex issues are easier to handle.
Hmm tput and terminal handling is a big one: but you're already using #bash so you're in the right place. There's ##tty as well, though it tends to be a bit more on the code side. You'll find some stuff in the libIgli file that's part of update that deals with tput and colours. $CUP is used to move up the terminal for example, and the --jobs filtering uses a lot of that.
As for stuff you've thrown together, don't worry about it: I know how you feel and that's exactly how I started out. update was simply an exercise to learn bash with, and the issue at the time was no --keep-going, but for me it was the pages of spew from every compile output meaning I couldn't see what was happening. So filtering emerges (build-logs essentially) became the major focus, til --jobs came along. It's still used when -j1 or we're installing eg toolchain stuff, where we do one at a time before the main build.
You can try that out using --test for a package you've already installed: it'll just work from the build-log to show you a compacted display as if it had run that compile now (only a lot quicker ofc.:)
| Quote: | | Dunno about other shells. Bash seems pretty damn fast to me ... but I don't have meaningful amounts of experience with anything else and anyhow, my computer has rack ears instead of a megapixel camera on both ends -- I should probably bear in mind that most of theirs will go way slower than mine does when I'm expecting people to consume code I've written... |
I was amazed how far I could push my old 32-bit 1G machine with update; I kept waiting for it to blow up on the script, but it never did. But yeah, most of the "optimisation" in shellscript comes from not shelling out, paradoxically, for minor things that can be done by the runtime. Bash is definitely a step up if you need to write larger glue apps; arrays for instance are essential.
| Quote: | | Looks kinda cool, btw! Or if not cool, useful. I'll check it out. |
Heh thanks :) That's another reason i haven't reworked it and split it into different files: it's had to stay in use, and I don't have the time. Also at some point we'll reimplement it in C or something, so it's easier to keep it as is and save the rewrite for another language, preferably by parsing the bash (which is a much more fun use of time;)
Feel free to patch it and play with it: you can't hurt anyone else's machine ;) and in any case any mistakes you make are unlikely to lead to damage to your machine. For a start it'll likely crash on you immediately, and if not you just hit CTRL-C at any point and it'll bail. Most errors occur in the logic around emerge, not during the actual emerge itself. It's got various safety checks; criticalPkg comes to mind, but there are lots of places where it will abort and give you a stack trace if you pass it dodgy data. And it won't downgrade your toolchain unless you force it to. ;) Having said that, you can also use it for ROOT and PORTAGE_CONFIGROOT builds (an area that has been expanded but not fully-tested to my knowledge) or in a chroot (I did that for 6 months when we were working on ABI upgrades in /etc/warning for expat; from binhosts as well which was kinda spooky: just like a binary distro ;)
Looking at your script, you're outputting from the build-logs: that would be nice to incorporate into the current display which is simply a list of packages and where they are in the process. Something I'd like, so if you fancy it, play with runInstall() which is the multi-job, terminal-aware function. Don't worry too much about keeping it in sync with runInstallPipe() just get something displaying how you'd like it (if you want) and i'll sync anything that needs it (unlikely since the other function doesn't display much by design, and there's no terminal.) If not, no bother, but don't hold back from patching if you feel like it (or asking me to, assuming you have an idea that we can reasonably implement, and you're willing to test.)
People sometimes get put off by how large it is, but each piece is not complex, and that's more an indication of how ad-hoc it is, which should make you feel more relaxed about changing it. Most of the tricky code is in support functions to make handling the tree reasonable, and then higher-layers build on top of that. Oh and some faking of associative arrays, from bash-3 days ;) Tricky stuff is more likely to be found in the lib file, unless it's to do with the portage tree, and most of that is only complex if you're trying to rework version comparison, or something fundamental.
The rest is just dealing with output from various programs and deciding what to do about it. The interesting things from our end are usually making it do something specific for a package, when a hook is not sufficient. The two main mechanisms for that are the warning file, and something being in toolchain (an area I've been working on for last year on and off, and wanted for several more: update -a --toolchain; I'm still working on the last bits for that, around perl, but it tests the version of the compiler used to build stuff now, for example.)
Hmm that went on a bit longer than I intended; blame old age ;)
Regards,
steveL. |
|
| Back to top |
|
|
Maitreya
Guru
Joined: 11 Jan 2006
Posts: 441
|
| Posted: Wed Jan 08, 2014 1:27 pm Post subject: |
|
|
That is some nice Fu there. Here I am just using a one-liner :
| Code: |
watch -n 3 -c "ccache -s && genlop -c && tail /var/tmp/portage/*/*/temp/build.log"
|
|
|
| Back to top |
|
|
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
| Posted: Thu Jan 30, 2014 10:26 am Post subject: Re: stupid bash tricks: multi-emerge-tail |
|
|
| gmt wrote: | | Here is a very ugly script |
This updated version uses lsof to implement a considerably better qlop than qlop is. Oh, since my script stopped working I was forced to figure out what qlop is btw It's part of portage-utils. The problem is, it assumes that portage will present its $0 to /proc/${PID} in a certain format that it sometimes doesn't. I should probably fix qlop while I'm at it but I'm pretty lazy...
| Code: |
#!/bin/bash
[[ $( id -u ) -eq 0 ]] || {
[[ -z "$I_WANT_TO_BE_ME" ]] && {
echo 'becoming root via sudo and reinvoking myself...' >&2
echo '(or perhaps you [[ -z "$I_WANT_TO_BE_ME" ]] ?)' >&2
echo >&2
SCRIPT=`realpath -s $0`
echo "running: ${SCRIPT} $*" >&2
echo >&2
sleep 0.5
exec sudo "${SCRIPT}" "$@"
exit 0
}
}
delay=2.5
vartmpportage="$( portageq envvar PORTAGE_TMPDIR )/portage"
notfirst=
bam=poo
# turns out it surprisingly difficult to figure out what to test here...
# while [[ $( ps -eo args | grep '^/usr/bin/python.*/emerge' ) ]] ; do
while [[ 1 ]] ; do
oldbam=bam
bam=
if [[ $notfirst ]] ; then
sleep ${delay}
else
notfirst=exactly
fi
# active_emerges=$( qlop -cC | grep '^[[:space:]]\*' | sed 's/^[[:space:]]*\*[[:space:]]*//' | sed 's/[[:space:]][[:space:]]*/\n/g' | sort )
active_emerges=$( lsof "${vartmpportage}"/*/*/temp/build.log 2>/dev/null | sed -e 's|^.*/\([^/]*/[^/]*\)/temp/build.log|\1|' | tail -n +2 | sort -u )
lines=5
if [[ ${active_emerges} ]] ; then
lines=$(( ( $( tput lines ) / $( echo ${active_emerges} | wc -w ) ) - 2 ))
fi
(( lines < 2 )) && lines=2
for merge in ${active_emerges} ; do
hdr=$( echo -e "\033[1m[[ ${merge} ]]\033[0m" )
cols=$(tput cols)
thelines="$( tail -n ${lines} "${vartmpportage}"/${merge}/temp/build.log | fold -w ${cols} | tail -n ${lines} )"
# if bam already nonempty, add a newline for aesthetic purposes
if [[ ${bam} ]] ; then
bam=$( echo "${bam}" ; echo ; echo "${hdr}" )
else
bam="${hdr}"
fi
bam=$( echo "${bam}" ; echo "${thelines}" )
done
if [[ ${oldbam} != ${bam} ]] ; then
clear
echo "${bam:-"nuthin'"}"
fi
done
echo "looks like emerge finished."
|
_________________
-gmt |
|
| Back to top |
|
|
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
| Posted: Thu Jan 30, 2014 11:00 pm Post subject: Re: stupid bash tricks: multi-emerge-tail |
|
|
| gmt wrote: | | Here is a very ugly scrip |
Discovered there is vastly superior implementation of this at: https://github.com/mgorny/portage-jobsmon
Unfortunately, it does suffer, somewhat, (as does mine, still, but not nearly so much after I put in the lsof hack) from false negatives when finding the running emerges.
Meanwhile, I figured out a way to split lines without getting confused by color escape codes. Unfortunately it requires some perl modules for which there are no ebuilds:
| Code: |
#!/bin/bash
[[ $( id -u ) -eq 0 ]] || {
[[ -z "$I_WANT_TO_BE_ME" ]] && {
echo 'becoming root via sudo and reinvoking myself...' >&2
echo '(or perhaps you [[ -z "$I_WANT_TO_BE_ME" ]] ?)' >&2
echo >&2
SCRIPT=`realpath -s $0`
echo "running: ${SCRIPT} $*" >&2
echo >&2
sleep 0.5
exec sudo "${SCRIPT}" "$@"
exit 0
}
}
delay=2.5
vartmpportage="$( portageq envvar PORTAGE_TMPDIR )/portage"
notfirst=
bam=poo
# turns out it surprisingly difficult to figure out what to test here...
# while [[ $( ps -eo args | grep '^/usr/bin/python.*/emerge' ) ]] ; do
while [[ 1 ]] ; do
oldbam=bam
bam=
if [[ $notfirst ]] ; then
sleep ${delay}
else
notfirst=exactly
fi
active_emerges=()
while read mrg; do
active_emerges+=("${mrg}")
done < <( lsof "${vartmpportage}"/*/*/temp/build.log 2>/dev/null | sed -e 's|^.*/\([^/]*/[^/]*\)/temp/build\.log|\1|' | tail -n +2 | sort -u )
lines=5
active_logs=()
for merge in "${active_emerges[@]}"; do
active_logs+=("${vartmpportage}"/${merge}/temp/build.log)
done
if (( ${#active_logs[@]} )); then
lines=$(( ( $( tput lines ) / ${#active_emerges[@]} ) - 2 ))
if (( lines > 8 )) ; then
poss_active_logs=()
for amerge in "${active_emerges[@]}" ; do
poss_active_logs+=("${vartmpportage}"/${amerge}/temp/build.log)
sublogs=()
while read nulog; do
sublogs+=("${nulog}")
done < <( find "${vartmpportage}"/${amerge}/temp -name 'build-*.log' 2>/dev/null )
poss_active_logs+=("${sublogs[@]}")
done
poss_lines=$(( ( $( tput lines ) / ${#poss_active_logs[@]} ) - 2 ))
if (( poss_lines >= 3 )) ; then
lines=${poss_lines}
active_logs=("${poss_active_logs[@]}")
fi
fi
fi
(( lines < 2 )) && lines=2
for log in "${active_logs[@]}" ; do
merge="${log#${vartmpportage}/}"
merge="${merge%.log}"
merge="${merge/\/temp\/build/}"
hdr=$( echo -e "\033[1m[[ ${merge} ]]\033[0m" )
cols=$(tput cols)
thelines="$( tail -n ${lines} "${log}" | perl -e "use Text::ANSI::Util; binmode(STDOUT, \":utf8\"); while(<STDIN>) { print Text::ANSI::Util::ta_mbwrap(\$_, ${cols}); }" | tail -n ${lines} )"
# if bam already nonempty, add a newline for aesthetic purposes
if [[ ${bam} ]] ; then
bam=$( echo "${bam}" ; echo ; echo "${hdr}" )
else
bam="${hdr}"
fi
bam=$( echo "${bam}" ; echo "${thelines}" )
done
if [[ ${oldbam} != ${bam} ]] ; then
clear
echo "${bam:-"nuthin'"}"
fi
done
echo "looks like emerge finished."
|
This requires perl modules:
dev-perl/Text-ANSI-Util/Text-ANSI-Util-0.12.ebuild:
| Code: |
# $Header: $
EAPI=4
MODULE_AUTHOR="SHARYANTO"
MODULE_VERSION=${PV}
inherit perl-module
DESCRIPTION="Terminal control using ANSI escape sequences"
LICENSE="GPL-1"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""
SRC_TEST="do"
RDEPEND="dev-perl/Data-Dump
dev-perl/Text-WideChar-Util"
|
and dev-perl/Text-WideChar-Util/Text-WideChar-Util-0.08.ebuild
| Code: |
# Copyright 1999-2013 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
EAPI=4
MODULE_AUTHOR="SHARYANTO"
MODULE_VERSION=${PV}
inherit perl-module
DESCRIPTION="Terminal control using ANSI escape sequences"
LICENSE="GPL-1"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""
SRC_TEST="do"
RDEPEND="dev-perl/Unicode-LineBreak"
|
Anyhow, I doubt I'll work on it any more now that I've discovered mgorny's script which has a far more "practical" (iow: not shit) implementation based on python and a real curses library. Going forward it seems more sensible to contribute to that than to continue to dink around with this.
_________________
-gmt |
|
| Back to top |
|
|
gmt
n00b
Joined: 15 Feb 2013
Posts: 11
Location: meatspace
|
| Posted: Thu May 09, 2019 12:59 am Post subject: Re: stupid bash tricks: multi-emerge-tail |
|
|
| gmt wrote: | | ...doubt I'll work on it any more now that I've discovered mgorny's script |
So much for that. I've continued to maintain and improve this thing... I'd might as well post the latest, since the old post probably doesn't work, these days, and what I do have works nicely. I find it to be a super-convenient fix for parallel portage's attempts to deprive me of the Gentoo experience of gobs of gibberish scrolling past faster than I can read it:
Here's a link to a "screen shot"
It is still a pretty poor example of bash scripting (a bit less so than it used to be) but the performance/results are much better:
| /usr/bin/multi-ebuild-tail: | #!/bin/bash
: ${PORTAGEY_ROOTISH_UID:=$( id -u ${I_JUST_WANT_TO_BE:-$(id -un root)} )}
: ${PORTAGEY_ROOTISH_UID:=0}
#: ${MULTI_ABI:=hellno}
: ${MULTI_ABI:=fuckyeah}
case ${MULTI_ABI} in
no|No|NO|false|False|FALSE|nope|Nope|NOPE|0) MULTI_ABI=hellno ;;
esac
: ${DEFDELAY:=2}
: ${DELAY:=${DEFDELAY:-2}}
: ${FORCED:=0}
pqvars() {
[[ -n ${VARTMPPORTAGE} ]] && return 0
: ${VARTMPPORTAGE:=$(portageq envvar PORTAGE_TMPDIR)/portage}
: ${VARTMPPORTAGE:=/var/tmp/portage}
: ${VARTMPPORTAGE2:=${VARTMPPORTAGE%/tmp/portage}/slow/portage}
}
active_logs=()
old_active_logs=()
older_active_logs=()
notfirst=0
bam=poo
# of the given filenames, pick the one most recently modified
# if several files modified within the same most-recent second,
# pick the one second-to-last one provided
most_recently_modified_file() {
local reord=("${@:$(($#)):1}" "${@:1:$(($#-1))}")
local best_guess=""
local best_guess_modified_unixtime=0
local logfile logfile_modified_unixtime
for logfile in "${reord[@]}"; do
logfile_modified_unixtime=$(stat -c %Y "${logfile}")
if [[ ${best_guess_modified_unixtime} -le ${logfile_modified_unixtime} ]]; then
best_guess=${logfile}
best_guess_modified_unixtime=${logfile_modified_unixtime}
fi
done
echo "${best_guess}"
}
usage() {
local thisprog="${0##*/}"
pqvars
echo
echo "${thisprog}: sort-of tail multiple simultaneous emerges at the console"
echo
echo " ${thisprog} [-f|--force] [-dN.N|--delay=N.N]"
echo
echo " -f: Force all files matching the shell globs:"
echo
echo " \"${VARTMPPORTAGE}\"/*/*/temp/build.log"
echo
echo " or"
echo
echo " \"${VARTMPPORTAGE2}\"/*/*/temp/build.log"
echo
echo " to be treated as active logs. Otherwise ${thisprog} will"
echo " only treat the subset of these files which lsof reports as"
echo " currently active as active logs. The lsof tests are quite"
echo " costly so this will improve performance and provide feedback"
echo " for merges that failed, or which portage has put on ice for"
echo " the time being, which may be preferable in some circumstances."
echo
echo " -dN.N: Any reasonable positive decimal value may be provided attached"
echo " to this argument; when provided this will replace the default"
echo " \"delay\" period of ${DEFDELAY}. Note that the delay is not"
echo " a period; ${thisprog} simply sleeps for the \"delay\" period"
echo " between each processing cycle. Since the processing time will"
echo " depend on a number of factors, there is no way to provide true"
echo " periodic screen updates in ${thisprog}."
echo
}
processargs() {
local thisarg
for thisarg in "$@"; do
case ${thisarg} in
-h|--help)
usage
exit 0
;;
-f|--force)
FORCED=1
;;
-d?*|--delay=?*)
thisarg2=${thisarg#-}
thisarg3=${thisarg2#-}
thisarg4=${thisarg3#d}
thisarg5=${thisarg4#elay=}
DELAY=${thisarg5}
;;
*)
echo "Error: argument \"${thisarg}\" not grokked." >&2
echo >&2
usage >&2
exit 1
;;
esac
done
}
processargs "$@"
if [[ ! $(id -u) -eq ${PORTAGEY_ROOTISH_UID} ]]; then
PORTAGEY_ROOTISH_UN=$(id -un ${PORTAGEY_ROOTISH_UID})
echo "be-sudo-coming $(id -un ${PORTAGEY_ROOTISH_UN}) and reinvoking ${0} ..." >&2
echo '(or... perhaps you want to be someone else? Then:' >&2
echo >&2
echo " export I_JUST_WANT_TO_BE=\"[ other_guy's_username | other_gal's_user_id ]\"" >&2
echo >&2
echo "(note: users of all POSIX-compliant genders are supported by either syntax)" >&2
echo >&2
LE_SCRIPT=$(realpath -s ${0})
case ${LE_SCRIPT} in
*\ *) LE_SCRIPT_SH_REP="\"${LE_SCRIPT}\"" ;;
*) LE_SCRIPT_SH_REP=${LE_SCRIPT} ;;
esac
echo -n "Ctrl-C to abort..."
sleep 0.75
echo
exec sudo -u ${PORTAGEY_ROOTISH_UN} "${LE_SCRIPT}" "$@"
exit $?
fi
pqvars
# turns out it surprisingly difficult to figure out what to test here...
while [[ 1 ]] ; do
oldbam="${bam}"
bam=""
if [[ $notfirst -eq 1 ]] ; then
sleep ${DELAY}
else
notfirst=1
fi
active_logs=()
for f in "${VARTMPPORTAGE}"/*/*/temp/build.log "${VARTMPPORTAGE2}"/*/*/temp/build.log; do
if [[ -f ${f} ]]; then
if [[ ${FORCED} -eq 1 ]]; then
active_logs+=("${f}")
elif lsof -- "${f}" &>/dev/null; then
active_logs+=("${f}")
fi
fi
done
lines=5
logs_to_display=()
if [[ ${#active_logs[*]} -gt 0 || ${#old_active_logs[*]} -gt 0 || ${#older_active_logs[*]} -gt 0 ]]; then
candidate_raw_candidate_logs=("${active_logs[@]}" "${old_active_logs[@]}" "${older_active_logs[@]}")
raw_candidate_logs=()
candidate_sublogs=()
for candidate in "${candidate_raw_candidate_logs[@]}"; do
for athing in "${raw_candidate_logs[@]}"; do
[[ ${candidate} == ${athing} ]] && break 2
done
raw_candidate_logs+=("${candidate}")
done
if [[ ${MULTI_ABI} != hellno ]]; then
for candidate in "${raw_candidate_logs[@]}"; do
for candidate_sublog in "${candidate%/build.log}"/build-*.log; do
# candidate sublogs still use lsof for now :(
if lsof -- "${candidate_sublog}" &>/dev/null; then
case ${candidate_sublog} in
*/build.log)
:;
;;
*)
candidate_sublogs+=("${candidate_sublog}")
active_logs+=("${candidate_sublog}")
;;
esac
fi
done
done
#orig_candidate_sublogs=("${candidate_sublogs[@]}")
fi
candidate_logs=()
# make sure these things actually exist as they could have finished
# (they still could, but, whatchagonnado? the point is to not fuck
# up the lines calculation due to useless entries.
for raw_candidate in "${raw_candidate_logs[@]}"; do
[[ -f "${raw_candidate}" ]] && candidate_logs+=("${raw_candidate}")
done
#orig_candidate_logs=("${candidate_logs[@]}")
# sort the logs alphabetically by filename
merged_candidate_logs_pre=()
readarray -t merged_candidate_logs_pre < <( for log in "${candidate_logs[@]}" "${candidate_sublogs[@]}"; do echo "${log}"; done | sort -d )
# push substrings to the bottom
pending=""
merged_candidate_logs=()
for nextpending in "${merged_candidate_logs_pre[@]}"; do
nextpending=${nextpending%.log}
if [[ -n ${pending} ]]; then
if [[ "${nextpending}" == "${pending}"-* ]] ; then
merged_candidate_logs+=( "${nextpending}.log" )
else
merged_candidate_logs+=( "${pending}.log" )
pending=${nextpending}
fi
else
pending=${nextpending}
fi
done
[[ -n ${pending} ]] && merged_candidate_logs+=( "${pending}.log" )
# for each master, filter "slave" logs from the list, gathering into a "slavebundle".
# for each "slavebundle", preserve the most recently modified logfile only. collect
# all selected logs into filtered_merged_candidate_logs (including non-bundled logs).
filtered_merged_candidate_logs=()
slavebundle=()
bundle_master=""
for this_log in "${merged_candidate_logs[@]}"; do
if [[ -n ${bundle_master} ]]; then
slavebundle+=("${this_log}")
if [[ ${this_log} == ${bundle_master} ]]; then
filtered_merged_candidate_logs+=("$(most_recently_modified_file "${slavebundle[@]}")")
bundle_master=""
fi
else
this_file="${this_log##*/}"
this_dir="${this_log%/*}"
if [[ ${this_file} == build-?* ]]; then
bundle_master="${this_dir}/build.log"
slavebundle=("${this_log}")
else
filtered_merged_candidate_logs+=("${this_log}")
fi
fi
done
# calculate lines-per-log (aka ${lines}). crop if we have more logs than fit comfortably on the screen
for this_candidate_log in "${filtered_merged_candidate_logs[@]}"; do
possible_lines=$(( ( $( tput lines ) + 1 ) / ( ${#logs_to_display[@]} + 1 ) ))
[[ ${possible_lines} -le 3 ]] && break
lines=${possible_lines}
logs_to_display+=("${this_candidate_log}")
done
fi
# two lines are wasted per. build -- except for the first log tailed,
# for which only one line is wasted, because we don't need to make extra space
# to separate the logs; but we already accounted for that one extra line, when
# we added one to the numerator in the division test, above.
lines=$(( lines - 2 ))
# echo
# for var in active_logs old_active_logs older_active_logs candidate_raw_candidate_logs raw_candidate_logs candidate_logs candidate_sublogs merged_candidate_logs_pre merged_candidate_logs filtered_merged_candidate_logs logs_to_display; do
# declare -p "${var}" | sed 's/\[[[:digit:]]*\]=//g'
# done
# echo
# echo "log_to_display -> merge mappings"
# echo "================================"
# for log in "${logs_to_display[@]}" ; do
# merge="${log#${VARTMPPORTAGE2}/}"
# merge="${merge#${VARTMPPORTAGE}/}"
# merge="${merge%.log}"
# merge="${merge/\/temp\/build/}"
# echo "log=\"${log}\" -> merge=\"${merge}\""
# done
# sleep 20
# buffer lines for printing
nextcr=""
for log in "${logs_to_display[@]}"; do
merge="${log#${VARTMPPORTAGE2}/}"
merge="${merge#${VARTMPPORTAGE}/}"
merge="${merge%.log}"
if [[ ${merge} != *build ]]; then
merge="${merge/\/temp\/build-/ <}>"
else
merge="${merge/\/temp\/build/}"
fi
bam+="${nextcr}"
nextcr=$'\n\n'
hdr=$( echo -e "\033[1m[[ $(
[[ "${merge}" ]] && echo "${merge}" || echo "${log}" # failsafe for if (when, ime) crappy path->package-ish mapping hacks break
) ]]\033[0m" )
cols=$(tput cols)
thelines="$( tail -n ${lines} "${log}" | perl -e "use Text::ANSI::WideUtil; binmode(STDOUT, \":utf8\"); while(<STDIN>) { print Text::ANSI::WideUtil::ta_mbwrap(\$_, ${cols}); }" 2>>/tmp/meterr.log | tail -n ${lines} )"
bam+="${hdr}"$'\n'"${thelines}"
done
# prepend the magic to clear the screen; if no text to display, replace with no activity message.
bam=$(clear ; echo -n "${bam:-"no recent build.log activity..."}")
# bam! clear screen and dump the whole next screen image to console as fast as bash can print it
if [[ ${oldbam} != ${bam} ]] ; then
echo -n "${bam}"
fi
older_active_logs=("${old_active_logs[@]}")
old_active_logs=("${active_logs[@]}")
done |
It now requires three out-of-tree ebuilds (and, therefore, some kind of overlay), unfortunately:
| dev-perl/Text-ANSI-Util/Text-ANSI-Util-0.22.ebuild: | # Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
EAPI=6
DIST_AUTHOR=PERLANCAR
DIST_VERSION=0.22
inherit perl-module
DESCRIPTION="Used to justify strings to various alignment styles"
LICENSE="ISC"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""
RDEPEND="
virtual/perl-Exporter
>=virtual/perl-Scalar-List-Utils-1.460.0-r1"
DEPEND="${RDEPEND}
>=dev-perl/Module-Build-0.280.0
" |
| dev-perl/Text-ANSI-WideUtil/Text-ANSI-WideUtil-0.22.ebuild: | # Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
EAPI=6
DIST_AUTHOR=PERLANCAR
DIST_VERSION=0.22
inherit perl-module
DESCRIPTION="Used to justify strings to various alignment styles"
LICENSE="ISC"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""
RDEPEND="
virtual/perl-Exporter
>=dev-perl/Text-ANSI-Util-${PV}
>=dev-perl/Text-WideChar-Util-0.16"
DEPEND="${RDEPEND}
>=dev-perl/Module-Build-0.280.0
" |
| dev-perl/Text-WideChar-Util/Text-WideChar-Util-0.16.ebuild: | # Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
EAPI=6
DIST_AUTHOR=PERLANCAR
DIST_VERSION=0.16
inherit perl-module
DESCRIPTION="Used to justify strings to various alignment styles"
LICENSE="ISC"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""
RDEPEND="
virtual/perl-Exporter
dev-perl/Unicode-LineBreak"
DEPEND="${RDEPEND}
>=dev-perl/Module-Build-0.280.0
" |
_________________
-gmt |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
