Fast and flexible curses ebuild browser

[optilude]

Updated 13th June: See changelog

Hello,

Inspired by my recent love affair with dialog (http://forums.gentoo.org/viewtopic.php?t=45827, which I now use almost every day), I decided to make another tool I've been missing: pbrowser is an ebuild browser that by default looks in your portage and portage overlay directories, and lets you browse through all the packages there. You can select and deselect packages until you're happy with them, and the selected packages will be printed to the screen when you press "Done". Here are some places where this may be useful:

• To simply see what packaes are in portage, browsing through portage directories and viewing the description of each package, run:

  Code: Select all

  $ pbrowser

• To do the same thing, but only view installed packages, and their versions, run:

  Code: Select all

  $ pbrowser -i

• To select some packages to emerge and then emerge then, run:

  Code: Select all

  # emerge `pbrowser`

  (note the use of backticks!)
• To have more control, combine this with my femerge script (http://forums.gentoo.org/viewtopic.php?t=45827), which will let you deselect packages you don't wish to emerge after all:

  Code: Select all

  # femerge `pbrowser`

  (note the use of backticks!)
• If you don't want to use femerge, but want the chance to cancel before installing, use the bemerge script below:

  Code: Select all

  # bemerge --upgradeonly

  (any options to bemerge are passed straight through to emerge, with the exception of -i and -s which is passed to pbrowser, so you'll have to use --inject instead of -i and search instead of -s with bemerge)
• To select some installed packages and unmerge them, run:

  Code: Select all

  # emerge unmerge `pbrowser -i`

  (note the use of backticks!)
• Or, more concicely:

  Code: Select all

  # bemerge -i unmerge 

• To create a custom world file containing only packages you wish to keep track of when doing an emerge -u world, run:

  Code: Select all

  # pbrowser -s > new_world

  Make sure the file new_world is what you want, and replace /var/cache/edb/world with it (it would be a good idea to do a backup of /var/cache/edb/world first!)
• To do the same thing, but only consider currently installed packages, run:

  Code: Select all

  # pbrowser -is > new_world

  (the -i option normally lists packages as they exist in /var/db/pkg, that is with version numbers and revisions appended - the -s option strips the version and revision information from the output).

I'm sure you could come up with other uses for it as well!


Changelog:
2003-04-21: Changed name from ebb to pbrowser to avoid possible naming conflict.
2003-04-29: pbrowser now displays installed versions of packages when not using the -i option. Pass -d to pbrowser or bemerge to override. New "Try emerge" option: Select a package and see the output of "emerge -p <package>".
2003-05-27: Er.... I tried to use pbrowser today and for some obscure reason it'd stopped working. I traced the problem to dialog not outputting "HELP" when you press the Done button (which is really just a disguised help button) - and I SWEAR it used to do that! Anyway, it now uses the return error code from dialog, which should be safer anyhow... Sorry about that!
2003-06-13: Fixed minor bug whereby output would be muddled if a directory was empty.

Known bugs:
* If you enter a directory, select or deselect some packages and then use using the "Try emerge" option, your changes are lost. As far as I can tell, this is a bug in dialog (there is no way to find out what the user had selected in a dialog --checklist if HELP is pressed).
* Some ebuilds have DESCRIPTION variables that reference other variables (e.g. $PV for current version). These will not be displayed correctly by pbrowser.


Place the two scripts below in /usr/local/bin/pbrowser and /usr/local/bin/bemerge respectively, and make sure you have dialog installed (emerge dialog).

pbrowser 1.2:

Code: Select all

#!/bin/bash

# pbrowser (portage browser) 1.2 by Martin Aspeli <optilude@gmx.net>

# Selector for Gentoo Linux (www.gentoo.org)'s portage system. Iterates
# through all packages in your portage directory (/usr/portage) and your portage
# overlay directory (/usr/local/portage) as defined in /etc/make.globals and
# /etc/make.conf, if applicable, and allows you to select and de-select packages
# from all directories in the portage hierarchy. When you finally select Done,
# the selected packages are printed to the standard output, one per line.

# Suggested use: # pbrowser | xargs emerge
#            or: # emerge `pbrowser`

# This will let you select packages to emerge interactively, and then emerge
# them. pbrowser can also be used to generate a new world file to replace the one
# in /var/cache/edb/world:

# Suggested use: # pbrowser > new_world

# Then examine new_world and replace /var/cache/edb/world with it if you wish.
# Please be careful with replacing a world file you don't want to lose!

# You can also use pbrowser to browse installed packages only instead of all
# packages in portage, by supplying the -i option. This will let you browse all
# versions of all installed packages. By default, version information is printed
# for selected packages. Supply -s option to suppress version numbers.

# Normally, pbrowser will show the installed versions of any packages when
# browsing the portage tree (without -i). To suppress this, pass the -d option.

source /sbin/functions.sh

make_globals="/etc/make.globals" # The make.globals file
make_conf="/etc/make.conf"       # The make.conf file
pkg_db="/var/db/pkg"          # Hierarchy of installed packages

################################################################################

view_installed_versions=1       # Whether to view installed versions
tmpfile=/tmp/${$}             # Temporary file

################################################################################

# buld input suitable for a dialog --menu of the directories in portage
build_directories_menu () {
  if test -n "${overlay_dir}" ; then
   find ${portage_dir} ${overlay_dir} \
     -type d -name "*-*" -maxdepth 1 -mindepth 1 2>/dev/null | \
      sed -e "s#${portage_dir}\(.\+\)\$#\1 \"-->\"#" \
         -e "s#${overlay_dir}\(.\+\)\$#\1 \"-->\"#" | sort | uniq
  else
   find ${portage_dir} -type d -name "*-*" -maxdepth 1 -mindepth 1 | \
      sed -e "s#${portage_dir}\(.\+\)\$#\1 \"-->\"#" | sort | uniq
  fi
}

# find the description for the package given as an argument, use the newest
# .ebuild if several exist
find_description () {
  local ebuild_glob="${portage_dir}${1}/*.ebuild"
  test -n "${overlay_dir}" && \
   ebuild_glob="${ebuild_glob} ${overlay_dir}${1}/*.ebuild"
   
  local ebuild=$(ls ${ebuild_glob} --sort=time 2>/dev/null | head -n 1)
  local desc=""
 
  if test -z "${ebuild}" ; then
   echo "(No ebuilds found)"
  elif test -r "${ebuild}" ; then
   desc=$(grep "DESCRIPTION=" "${ebuild}" | sed 's/DESCRIPTION="\(.\+\)"/\1/')
   test -n "${desc}" && echo "${desc}" || echo "(Description not found)"
  else
   echo "(Description not found)"
  fi
}

# Find all installed versions of package $1/$2 ($1 = directory, $2 = package,
# without version).
find_installed_versions () {
  directory="${pkg_db%/}/${1}"
  find ${directory} -mindepth 1 -maxdepth 1 \
                -regex "${directory}/${2}-[0-9].*" | \
   grep -oe '-[[:digit:]].*$' | sed 's/^-//'
}

# build input suitable for a dialog --checklist of the packages in the
# directory specified as first argument; second argument is string of
# currently selected packages.
build_packages_checklist () {
  local package=""
  for package in $(find ${portage_dir}${1}/ ${overlay_dir}${1}/ \
        -type d -maxdepth 1 -mindepth 1 2>/dev/null | sort | uniq) ; do
   package=${package##*/}
   local description=$(find_description "${1}/${package}")
   description=${description//\'/}

   local version_str=""
   
   # If we're not viewing installed packages, get string of installed
   # versions.
   if test "${view_installed_versions}" = "1" ; then
     local version=""
     for version in $(find_installed_versions ${1} ${package}) ; do
      version_str="${version_str}${version}, "
     done

     test "${version_str}" != "" &&
      version_str="[${version_str%, }] " ||
      version_str="[n/i] "
   fi
   
   # find out if we should mark this package as selected
   local onoff="off"
   echo "${2}" | grep -qs "${1}/${package}" && onoff="on"

   echo "${package} '${version_str}${description}' ${onoff} '${package}: ${description}'"
  done
 
}

# display select dialog --menu with directories specified as arguments; print
# DONE if user selected Done, nothing if user selected cancel, or the selected
# directory, if user selected OK; first argument is directory to select by
# default
display_directories_select () {
  default_item=${1}
  shift

  # We're cheating - calling the help item for done and fixing the output...
  eval "dialog --title 'pbrowser: ebuild Browser' \
            --help-button --help-label 'Done' \
            --default-item \"${default_item}\" \
            --menu 'Directories:' 0 0 0 ${@} 2>&1"
  return ${?}
}

# display package-selecting dialog --checklist; first argument is name of
# directory, subsequent options are --checklist items.
display_packages_select () {
  local directory=${1}
  shift

  if test -z "${1}" ; then
	dialog --title 'pbrowser: Portage Browser' \
		   --msgbox "No packages found." 0 0
  else 
	eval "dialog --title 'pbrowser: Portage Browser'  \
            --help-button --help-label 'Try emerge' \
            --separate-output \
            --item-help \
            --checklist \"Packages in ${directory}:\" 0 0 0 ${@} 2>&1"
	return ${?}
  fi
}

update_selection () {
  local all="${1}"
  local directory="${2}"
  # prepend $directory to each part of $3 and call in $new
  local new=""

  for package in ${3} ; do
   test -n "${new}" &&
     new="${new} ${directory}/${package}" ||
     new="${directory}/${package}"
    
  done
 
  # remove all packages in same directory that were not selected
  for package in $(echo "${all}" | \
                    grep -o "${directory}/[^[:space:]]\+"); do
   echo "${new}" | grep -qs "${package}" || all=${all//${package}/}
  done

  # add each newly selected package to $all if necessary
  for package in ${new} ; do
   echo "${all}" | grep -qs "${package}" || all="${all} ${package}"
  done

  echo "${all}"
}

# Try emerge -p for a package ($1 = directory, $2 = package name)
try_emerge () {
  local directory="${1}"
  local package="${2}"
  local emerge=""
 
  # strip version number
  package=$(echo "${package}" | sed -e 's/-r[0-9]\+$//' -e 's/-[0-9][^-]*$//')

  emerge -p ${directory}/${package} > ${tmpfile}
   
  dialog --backtitle "Running emerge -p ${directory}/${package}..." \
       --textbox "${tmpfile}" 15 76

  rm -rf ${tmpfile} 2>/dev/null
 
}

# print all the selected packages (arguments), one on each line
print_all_selected () {
  local package=""
  
  for package in ${@} ; do
   if test "${strip_versions}" = "1" ; then
     echo "${package}" | sed -e 's/-r[0-9]\+$//' -e 's/-[0-9][^-]*$//'
   else
     echo ${package}
   fi
  done
}

print_help () {
  echo "Usage: pbrowser [options]"
  echo
  echo "Options:"
  echo "  -i       Browse installed packages"
  echo "  -s       Strip version numbers when printing installed packages"
  echo "  -d      Don't view installed versions of packages"
  echo "  -h       Display this message"
  echo
}

init () {

  # Get configuration files
  source ${make_globals}
  source ${make_conf}

  view_installed_versions="1"
 
  # Parse command line options
  while getopts "hisd" opt ; do
   case "${opt}" in
     h)
      print_help
      exit 0
      ;;
     i)
      portage_dir="${pkg_db%/}/"
      overlay_dir=""
      view_installed_versions="0"
      ;;
     s)
      strip_versions=1
      ;;
     d)
      view_installed_versions="0"
      ;;
     *)
      print_help && exit 1
      ;;
   esac
  done
 

  # default to PORTDIR
  if test -z "${portage_dir}" ; then
   portage_dir="${PORTDIR%/}/"
   test -d "${PORTDIR_OVERLAY}" && overlay_dir="${PORTDIR_OVERLAY%/}/"
  fi

  # make sure it exists
  if ! test -d "${portage_dir}" ; then
   echo "Portage directory ${portage_dir} not found!"
   exit 1
  fi
}

portage_dir=""
overlay_dir=""
all_selected=""
strip_versions=0

init ${@}

directories_menu=$(build_directories_menu)
directory=""
package=""
value=""

while true ; do
 
  directory=$(display_directories_select "${directory}" ${directories_menu})
  value="${?}"
  
  test "${value}" = "1" && exit 0
  
  if test "${value}" = "2" ; then
	print_all_selected ${all_selected} | sort | uniq
	exit 0
  fi
   
  packages=$(build_packages_checklist "${directory}" "${all_selected}")
  while true ; do
   selected=$(display_packages_select "${directory}" ${packages})
   value="${?}"
  
   test "${value}" = "1" && break
   
   if echo "${selected}" | grep -qs "^HELP" ; then
     package="${selected#HELP }"
     package="${package%%:*}"
     try_emerge "${directory}" "${package}"
   else 
     all_selected=$(update_selection "${all_selected}" \
                 "${directory}" "${selected}")
     break
   fi
  done
done

bemerge 1.1

Code: Select all

#!/bin/bash

# bemerge 1.1 by Martin Aspeli <optilude@gmx.net>

# Simple wrapper for emerge that lets you chooce packages using pbrowser.
# -i or -s options are passed to pbrowser if supplied, all other options are 
# passed straight to emerge. The output of emerge -p will be displayed and you 
# may cancel before running emerge for real. 

# Suggested use: # bemerge -U 
#                # bemerge -i unmerge

# The first will let you choose packages to emerge using pbrowser, and then 
# upgrade those packages if possible. The second will let you choose among 
# installed packages and unmerge those. 

# Please ensure dialog and pbrowser are both installed and in your $PATH

source /sbin/functions.sh

pbrowser_options=""
emerge_options=""

# Get options: Let -i and -d pass to pbrowser, all other to emerge
while true ; do
  case "${1}" in
	-d|-i|-di|-id)
	  pbrowser_options="${pbrowser_options} ${1}"
	  ;;
	-p|--pretend) # ignore - we pass -p to emerge when necessary
	  ;;
	*)
	  emerge_options="${emerge_options} ${1}"
	  ;;
  esac
  
  shift || break;
done

builds=$(pbrowser ${pbrowser_options})
test -n "${builds}" || exit 0 # cancelled
emerge -p ${emerge_options} ${builds} || exit 1

choice=""
until test "${choice}" = "e" -o "${choice}" = "E" -o \
		   "${choice}" = "a" -o "${choice}" = "A" ; do

  einfon "Press (e) to emerge, (a) to abort: "
  read choice

  case "${choice}" in
	a|A)
	  ;;
	e|E)
	  emerge ${emerge_options} ${builds}
	  exit ${?}
	  ;;
	*)
	  eerror "Invalid input."
	  ;;
  esac
done

Enjoy!
Martin

[boglin]

I love ebb, great program
,but

Code: Select all

#femerge `ebb`

does not work, you can select something in ebb, then <Done>, and you get:

Code: Select all

Usage: femerge [options] < packages >
    femerge [options] < system | world >

See emerge --help for more information.

I have the latest femerge and pfilter from the link above, 0.9 and 0.9, respectively.
Edit: Umm, sorry I am noob I just read Lars' post : http://forums.gentoo.org/viewtopic.php?t=46977
ebb is great !

[l0rdt]

Hello,

Inspired by my recent love affair with dialog (http://forums.gentoo.org/viewtopic.php?t=45827, which I now use almost every day), I decided to make another tool I've been missing: ebb is an ebuild browser

really useful!
but consider renaming it, because an executable named "ebb" already exists from tetex.can i suggest "eb"

[optilude]

really useful!
but consider renaming it, because an executable named "ebb" already exists from tetex.can i suggest "eb"

Er... didn't realise. Well, you can call it whatever you want, of course, although you'll have to edit the bemerge script if you rename it. If I can find somewhere sensible to host it (my current webspace provider will expire in two months, so not really good idea to put it there), I should make an ebuild for it, at which point I'll change the name, too, but I may to it before then (going away for a while now, so it'll have to wait till I get back, sorry).

[optilude]

Edit: Umm, sorry I am noob I just read Lars' post : http://forums.gentoo.org/viewtopic.php?t=46977
ebb is great !

So that you mean you got it sorted? What browser did you use for copying and pasting the script from the forums? I think there is a genuine problem with copy-and-paste scripts (I had trouble with odd space/tab/something characters in konqueror and had to use phoenix...) here.

When I come back from holiday, I'll look at making some ebuilds and maybe getting femerge/pfilter and bemerge/ebb into portage, if people think it's worth it.

Martin

[metalhedd]

I had that problem with the \'s being followed by whitespaces, i just went through the script and stripped them all... I really like ebb btw. great work... How difficult would it be to have it display whether or not each package is currently installed, and possibly the version as well?

you could basically make it a curses based version of KPortage, similar to Debian's curses package in staller, I can't remember what it's called now

[optilude]

How difficult would it be to have it display whether or not each package is currently installed, and possibly the version as well?

Hmm.... not very difficult, I guess... I'll see what I can do once I get some spare time!

you could basically make it a curses based version of KPortage, similar to Debian's curses package in staller, I can't remember what it's called now

Yes, I'm sortof working towards that... however what I'm doing at the moment is use bash scripts to do the work and use the dialog tool (a wonderful curses program) to do the UI. I'm trying to create a set of small tools that you can put together however you like to do "interactive" emerge's. Dialog is not powerful enough to create an integrated program - that would mean coding it all in C, which I really don't wanna do (way too much text processing!).

Here are my goals - comments welcome:

* Being able to take a set of packages (e.g. from emerge -u world) and set different custom environement USE flags for each one (so I can set USE="-gnome" when it wants to emerge all of gnome 1.4 when I try to emerge xmms, which is in my world file).

* Being able to show which packges are being emerged as dependencies of others (so when I do an emerge -up world and it tells me I have to emerge libXYZ, I want to know what package(s) is including it as a dependency). I'm working on this now.

* Make ebb more powerful by showing installed packages (when browsing all of portage) or newer (unmasked) packages in portage (when browsing installed packages with the -i option).

Anything else you may think of?

[optilude]

Just out of interest... are people actually using these scripts? How about their cusins pfilter and femerge?

I find them quite useful and use them pretty much every week - if other people do too, I'd appreciate some feedback on possible improvements etc. Once my exams are over, I plan to do a larger project (probably in python instead of shell - what better excuse to learn python) for a similar tool that should be a lot more powerful, including:

* Set USE flags individually in a ufed-style interface for packages to emerge.
* Set ACCEPT_KEYWORDS individually for packages to emerge.
* Select/deselect "trees" of packages, so if you deselect a package, all its dependencies go too.
* Better browsing facilities with more information on what's installed, what's in the system profile, etc.
* World-file editing.

These are features I've been missing in gentoo. Comments?

[e-tigger]

Just out of interest... are people actually using these scripts? How about their cusins pfilter and femerge?

I find them quite useful and use them pretty much every week - if other people do too, I'd appreciate some feedback on possible improvements etc. Once my exams are over, I plan to do a larger project (probably in python instead of shell - what better excuse to learn python) for a similar tool that should be a lot more powerful, including:

* Set USE flags individually in a ufed-style interface for packages to emerge.
* Set ACCEPT_KEYWORDS individually for packages to emerge.
* Select/deselect "trees" of packages, so if you deselect a package, all its dependencies go too.
* Better browsing facilities with more information on what's installed, what's in the system profile, etc.
* World-file editing.

These are features I've been missing in gentoo. Comments?

I've been using these very handy scripts for 6 months or more. I'd forgotten that they weren't a part of protage toolkit.

Thanks again.

[optilude]

I've been using these very handy scripts for 6 months or more. I'd forgotten that they weren't a part of protage toolkit.

Thanks again.

Thanks for the support! I've been wanting to update pbrowser and pfilter for a while, but I'm just so busy these days it doesn't happen! :-/

[e-tigger]

First: Great script
But it doesn't seem to work on one of my rigs.

Gentoo 2004.1 AMD64
Dialog 0.9_beta20031207
Bash 2.05b-r9

The problem is that I get no initial dialog display untill I hit enter. Then the terminal session freaks out for a bit. I hit <Ctrl-C>, and then, by necessity, reset my terminal.

A little dialog testing shows that my system does not like the following type calls:

Code: Select all

value=$(dialog [options])

The fix is fortunately quite simple:

Code: Select all

 value=$(dialog --stdout [options])

voila, everybody's happy. The --stdout option is normally implied, so I'm not sure why explicitly stating it makes any difference.

Here's a patch:

Code: Select all

#######################################
--- pbrowser.orig       2004-06-01 20:37:56.000000000 -0500
+++ pbrowser    2004-06-02 20:11:17.166577832 -0500
@@ -128,7 +128,7 @@
   shift

   # We're cheating - calling the help item for done and fixing the output...
-  eval "dialog --title 'pbrowser: ebuild Browser' \
+  eval "dialog --stdout --title 'pbrowser: ebuild Browser' \
             --help-button --help-label 'Done' \
             --default-item \"${default_item}\" \
             --menu 'Directories:' 0 0 0 ${@} 2>&1"
@@ -142,10 +142,10 @@
   shift

   if test -z "${1}" ; then
-   dialog --title 'pbrowser: Portage Browser' \
+   dialog --stdout --title 'pbrowser: Portage Browser' \
          --msgbox "No packages found." 0 0
   else
-   eval "dialog --title 'pbrowser: Portage Browser'  \
+   eval "dialog --stdout --title 'pbrowser: Portage Browser'  \
             --help-button --help-label 'Try emerge' \
             --separate-output \
             --item-help \
@@ -192,7 +192,7 @@

   emerge -p ${directory}/${package} > ${tmpfile}

-  dialog --backtitle "Running emerge -p ${directory}/${package}..." \
+  dialog --stdout --backtitle "Running emerge -p ${directory}/${package}..." \
        --textbox "${tmpfile}" 15 76

   rm -rf ${tmpfile} 2>/dev/null
@@ -277,7 +277,7 @@
 init ${@}

 directories_menu=$(build_directories_menu)
-directory=""
+directory=""
 package=""
 value=""
############################################

[theDreamer]

superb scripts...

thanks