Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[unmasker] yet another auto-unmask script
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
david_e
n00b
n00b


Joined: 27 Jul 2007
Posts: 51
Location: Berlin, Germany

PostPosted: Fri Jul 11, 2008 10:57 pm    Post subject: [unmasker] yet another auto-unmask script Reply with quote

1. Background

One of the best things of Gentoo is that one can easily mix "~arch" and "arch" software, most of the times is just a matter of adding one or two entries in /etc/portage/package.keywords, but sometimes one needs to pull in many dependencies at once and it can be a very tedious job. So there is a need for tools which can handle this automatically. Obviously there are many scripts around for this: on the wiki or in portage (app-portage/autounmask), but I was not satisfied by these tools so I made mine.

2. Main Ideas

What this script does do:

1. Reads the package name, the wanted KEYWORD and a coupple of other things from the command line.
2. Prints to the stdout the needed package.keywords entries.
3. Prints to stderr a couple of other things just to let you know that he is alive.

What this script does not do:

1. Touch in any way any of your files, apart from creating some temp files in /tmp
2. Tries to merge anything or mess in other ways with the system.
3. Need to be run as root.

What this script can't do:

1. Unmask hard-masked or p-masked packages
2. Work with anything else then emerge (i.e. with paludis)

3. The Script

Here it is:
Code:

#!/bin/bash

# This script will show how to unmask one or more package and their
# dependencies if keyword masked. Output can be redirected directly to
# /etc/portage/package.keywords as every extra output will be sent to
# stderr. See help for more informations.
#
# This script needs `lndir' to work!
#
# Author  : david.e.pi.3.14[at]gmail.com
#
# License : 'GPLv2 or later'
#           
#           Unmasker: shows how to unmask a package
#           Copyright (C) 2008, Radice David <david.e.pi.3.14[at]gmail.com>
#
#           This program is free software; you can redistribute it and/or
#           modify it under the terms of the GNU General Public License
#           as published by the Free Software Foundation; either version 2
#           of the License, or (at your option) any later version.
#
#           This program is distributed in the hope that it will be useful,
#           but WITHOUT ANY WARRANTY; without even the implied warranty of
#           MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

PORTAGE_CONFIGROOT=/

function cant_unmask() {
    echo "failed!" 1>&2
    echo "It seems that it is impossible to unmask $TARGET. Here is emerge output:" 1>&2
    cat "$1" 1>&2
    exit 1
}

function clone_portage_config() {
    local string="Portage config cloning failed!"

    FAKE_PORTAGE_CONFIGROOT=$(mktemp -d) || die "$string"
    mkdir -p $FAKE_PORTAGE_CONFIGROOT/etc/portage

    ln -s $PORTAGE_CONFIGROOT/etc/make.conf \
          $FAKE_PORTAGE_CONFIGROOT/etc/make.conf

    lndir $PORTAGE_CONFIGROOT/etc/portage \
          $FAKE_PORTAGE_CONFIGROOT/etc/portage \
          &> /dev/null || die "$string"

    local oldpwd="$PWD"
    cd $PORTAGE_CONFIGROOT/etc
    cd `readlink make.profile`
    local profile=$PWD
    cd "$oldpwd"
    ln -s $profile $FAKE_PORTAGE_CONFIGROOT/etc/make.profile

    if [ -f $FAKE_PORTAGE_CONFIGROOT/etc/portage/package.keywords ]; then
        rm $FAKE_PORTAGE_CONFIGROOT/etc/portage/package.keywords
        mkdir $FAKE_PORTAGE_CONFIGROOT/etc/portage/package.keywords

        cat $PORTAGE_CONFIGROOT/etc/portage/package.keywords \
            > $FAKE_PORTAGE_CONFIGROOT/etc/portage/package.keywords/base
    fi

    UNMASK_FILE=$(mktemp -p $FAKE_PORTAGE_CONFIGROOT/etc/portage/package.keywords) || die "$string"
}

function die() {
    echo "" 1>&2
    echo "$1" 1>&2
    exit 1
}

function explode_atom() {
    local full_name="$1"
    CATEGORY=${full_name%/*}
    PVR=${full_name#*/}
    P=${PVR%-r*}
    PR=${PVR#$P}
    PR=${PR#-}
    PN=${P%-*}
    PV=${P#$PN-}
}

function help() {
    echo "unmasker: shows how to ~arch unmask a package and its dependencies,"
    echo "output can be redirected to /etc/portage/package.keywords as extra"
    echo "output will be redirected to stderr."
    echo ""
    echo "Usage: unmasker [options] [target]"
    echo ""
    echo " -d, --debug            : keep temporary files for debugging purpose"
    echo " -h, --help             : shows this help"
    echo " -f, --file [file]      : append output to [file]"
    echo " -k, --keywords [keys]  : set accepted keywords (defaults to '**')"
    echo " -o, --only-one-version : unmask only the needed version of each package"
    echo "     --vim              : insert vim syntax autocmd"
    echo ""
    echo " [target]               : the package or the set of packages to unmask"
    echo ""
    echo "Examples:"
    echo ""
    echo "  single package        : unmasker pidgin"
    echo "  specific version      : unmasker '=www-browser/mozilla-firefox-3.0-r1'"
    echo "  two packages          : unmasker pidgin octave"
}

function hit_bug() {
    echo "Something went bad. This is probably a bug in the unmask script." 1>&2
    echo "Here is the emerge output (for debugging purposes):" 1>&2
    cat $1 1>&2
    echo "" 1>&2
    echo "FAKE_PORTAGE_CONFIGROOT : " $FAKE_PORTAGE_CONFIGROOT 1>&2
    echo "outfile                 : " $outfile 1>&2
    echo "list                    : " $list 1>&2
    exit 1
}

function main() {
    KEYS='**'
    DEBUG=false
    ONLY_ONE_VERSION=false
    SAVE_TO_FILE=false
    SHOW_HELP=false
    USE_VIM=false

    unset TARGET
    unset V_TARGET

    local atom
    local proceed=true
    local old_atom
    local list=$(mktemp) || die "Unable to make a temporary file!"
    local outfile=$(mktemp) || die "Unable to make a temporary file!"
    local target
    local pn_target

    read_line "$@"
   
    if [ "$SHOW_HELP" == "true" ]; then
        help 1>&2
        exit 1
    fi

    if [ -z "$TARGET" ]; then
        echo "You must set a [target] at least" 1>&2
        echo "Try $0 --help for more information." 1>&2
        exit 1
    fi

    clone_portage_config

    echo -n "Checking for consistency..." 1>&2
    ACCEPT_KEYWORDS="$KEYS" \
    emerge --pretend $TARGET \
        > $outfile \
        2> /dev/null \
        || cant_unmask $outfile
    echo "done!" 1>&2
    echo "" 1>&2

    # This is used to select the best version of a package
    # if no version is given
    cat $outfile \
                 | grep -v -E '\[blocks B     \]' \
                 | grep -E '\[ebuild' \
                 | cut -d ']' -f 2 \
                 | cut -d ' ' -f 2 \
                 > $list

    for target in $TARGET; do
        if [ -z $(echo $target | grep '/') ]; then
            pn_target=$target
        else
            explode_atom $target
            pn_target=$PN
        fi

        atom=$( \
                cat $list \
                | grep $pn_target-[[:digit:]] \
              )

        explode_atom $atom
        V_TARGET="$V_TARGET "="$CATEGORY/$PVR"
    done

    atom=""

    if [ "$USE_VIM" == "true" ]; then
        echo '# vim: set ft=gentoo-package-keywords :' | tee -a $UNMASK_FILE
        echo "" | tee -a $UNMASK_FILE
    fi

    while [ "$proceed" == true ]; do
        emerge --pretend \
               --config-root=$FAKE_PORTAGE_CONFIGROOT \
               $V_TARGET \
               &> $outfile \
               && proceed=false

        if [ "$proceed" == true ]; then
            old_atom=$atom
            atom=$( \
                    cat $outfile \
                    | grep "masked by" \
                    | head -n 1 \
                    | cut -d ' ' -f 2 \
                  )

            # Make sure we don't go in an endless loop for some reason
            if [ "$atom" == "$old_atom" ]; then
                hit_bug $outfile
            fi

            explode_atom $atom

            if [[ -z "$CATEGORY"  || -z "$PVR" || -z "$PN" ]]; then
                hit_bug $outfile
            fi

            if [ "$ONLY_ONE_VERSION" == "true" ]; then
                echo '='$CATEGORY/$PVR "$KEYS" | tee -a $UNMASK_FILE
            else
                echo $CATEGORY/$PN "$KEYS" | tee -a $UNMASK_FILE
            fi
       fi
    done
   
    if [ "$SAVE_TO_FILE" == "true" ]; then
        if [ "$FILE_EXISTS" == "true" ]; then
            echo "###################################################" >> "$FILENAME"
            echo "#   Unmasking: $TARGET" >> "$FILENAME"
            echo "###################################################" >> "$FILENAME"
            cat $UNMASK_FILE >> "$FILENAME"
            echo "###################################################" >> "$FILENAME"
        else
            cat $UNMASK_FILE >> "$FILENAME"
        fi
    fi

    if [ "$DEBUG" == true ]; then
        echo "" 1>&2
        echo "FAKE_PORTAGE_CONFIGROOT : " $FAKE_PORTAGE_CONFIGROOT 1>&2
        echo "outfile                 : " $outfile 1>&2
        echo "list                    : " $list 1>&2
    else
        rm -rf $FAKE_PORTAGE_CONFIGROOT
        rm $outfile
        rm $list
    fi
}

function print_atom() {
    echo "CATEGORY : " $CATEGORY
    echo "PVR      : " $PVR
    echo "P        : " $P
    echo "PR       : " $PR
    echo "PN       : " $PN
    echo "PV       : " $PV
}


function read_line() {
    while [ $# -gt 0 ]; do
        case "$1" in
            -d|--debug)
                DEBUG=true
                ;;
            -h|--help)
                SHOW_HELP=true
                ;;
            -f|--file)
                SAVE_TO_FILE=true
                FILENAME="$2"
            if [ -f "$FILENAME" ]; then
               FILE_EXISTS=true
            else
               FILE_EXISTS=false
            fi
                shift
                ;;
            -k|--keywords)
                KEYS="$2"
                shift
                ;;
            -o|--only-one-version)
                ONLY_ONE_VERSION=true
                ;;
            --vim)
                USE_VIM=true
                ;;
            *)
                TARGET="$TARGET $1"
                ;;
        esac
        shift
    done
}

main "$@"


4. How to Use

There is an help for this:

Code:

gentoo ~ # unmasker -h
unmasker: shows how to ~arch unmask a package and its dependencies,
output can be redirected to /etc/portage/package.keywords as extra
output will be redirected to stderr.

Usage: unmasker [options] [target]

 -d, --debug            : keep temporary files for debugging purpose
 -h, --help             : shows this help
 -f, --file [file]      : append output to [file]
 -k, --keywords [keys]  : set accepted keywords (defaults to '**')
 -o, --only-one-version : unmask only the needed version of each package
     --vim              : insert vim syntax autocmd

 [target]               : the package or the set of packages to unmask

Examples:

  single package        : unmasker pidgin
  specific version      : unmasker '=www-browser/mozilla-firefox-3.0-r1'
  two packages          : unmasker pidgin octave


An example is this:

Code:

gentoo ~ # unmasker -k '~x86' --vim gnome-light > /etc/portage/package.keywords/gnome
Checking for consistency...done!

gentoo ~ # cat /etc/portage/package.keywords/gnome
# vim: set ft=gentoo-package-keywords :

gnome-base/gnome-light ~x86
gnome-extra/yelp ~x86
app-text/rarian ~x86
gnome-base/gnome-panel ~x86
dev-libs/libgweather ~x86
gnome-base/gnome-control-center ~x86
gnome-base/gnome-settings-daemon ~x86
gnome-base/gnome-desktop ~x86
gnome-base/libgnomekbd ~x86
x11-wm/metacity ~x86
gnome-base/eel ~x86
gnome-base/libgnome ~x86
gnome-base/libbonobo ~x86
gnome-base/librsvg ~x86
gnome-base/gvfs ~x86
sys-apps/hal ~x86
dev-python/pyxf86config ~x86
app-misc/hal-info ~x86
net-libs/libsoup ~x86
gnome-base/gconf ~x86
dev-libs/atk ~x86
gnome-base/libbonoboui ~x86
x11-themes/gnome-themes ~x86
x11-libs/libwnck ~x86
gnome-base/nautilus ~x86
gnome-base/gnome-session ~x86
gnome-base/gnome-keyring ~x86
x11-terms/gnome-terminal ~x86
x11-themes/gnome-icon-theme ~x86
gnome-base/gnome-vfs ~x86
gnome-base/libgnomeui ~x86


or:

Code:

gentoo ~ # unmasker -k ~x86 -o gnome-light
Checking for consistency...done!

=gnome-base/gnome-light-2.22.0 ~x86
=gnome-extra/yelp-2.22.1-r1 ~x86
=app-text/rarian-0.8.0-r1 ~x86
=gnome-base/gnome-panel-2.22.2 ~x86
=dev-libs/libgweather-2.22.3 ~x86
=gnome-base/gnome-control-center-2.22.2.1 ~x86
=gnome-base/gnome-settings-daemon-2.22.2.1 ~x86
=gnome-base/gnome-desktop-2.22.3 ~x86
=gnome-base/libgnomekbd-2.22.0 ~x86
=x11-wm/metacity-2.22.0 ~x86
=gnome-base/eel-2.22.2 ~x86
=gnome-base/libgnome-2.22.0 ~x86
=gnome-base/libbonobo-2.22.0 ~x86
=gnome-base/librsvg-2.22.2 ~x86
=gnome-base/gvfs-0.2.5 ~x86
=sys-apps/hal-0.5.11-r1 ~x86
=dev-python/pyxf86config-0.3.34-r1 ~x86
=app-misc/hal-info-20080508 ~x86
=net-libs/libsoup-2.4.1 ~x86
=gnome-base/gconf-2.22.0 ~x86
=dev-libs/atk-1.22.0 ~x86
=gnome-base/libbonoboui-2.22.0 ~x86
=x11-themes/gnome-themes-2.22.2 ~x86
=x11-libs/libwnck-2.22.3 ~x86
=gnome-base/nautilus-2.22.4 ~x86
=gnome-base/gnome-session-2.22.3 ~x86
=gnome-base/gnome-keyring-2.22.3 ~x86
=x11-terms/gnome-terminal-2.22.3 ~x86
=x11-themes/gnome-icon-theme-2.22.0 ~x86
=gnome-base/gnome-vfs-2.22.0 ~x86
=gnome-base/libgnomeui-2.22.1 ~x86


x11-misc/lndir is needed to run this script.

5. Before You Start Bashing Me :D

1. I know that this could be done with a one-line mega-pipe through sed, but that was not what I wanted to do. Instead I wanted to make something simple which could be re-used if I should need to "parse" the output of `emerge --pretend' one atom a time again. And I won't pipe the output of a cryptic pipe into /etc/...
2. I know that I am not a bash guru or something and that 99.9999% of the Gentoo users are able to write something like that or even a better script... I am just sharing in case this could save you some time. :)
3. Use this stuff at YOUR OWN RISK... it does not do anything else but echoing stuff so it should be safe but you never know! :P

6. ChangeLog

ver 2.3
  • Minor aesthetical fix: vim modelines are not a special case for '--file' anymore

ver 2.2
  • Buf fix: fixed a bug with package names ending with '+'

ver 2.1
  • Bug fix: fixed the way in which the real target package list is built.

ver 2.0
  • Changed how the scripts works so that it will unmask something only if it is really needed.
  • lndir is now needed in order to make this script work.
  • --debug flag changed its behaviour and is less verbose.
  • New --file option to append the output to a file: can be used instead of the '>'.
  • --skip-check option is no more.
  • It's not possible to switch a full package set to ~arch, but anyway that was just a "collateral feature" as it is stupid to do that this way.

ver 1.1
  • Change license to GPL-2. (I thought that having the same license of portage could be a plus)
  • Fixed the command line parse when there are multiple package/keywords. Quoting is not needed anymore for multiple package (any extra input is passed directly to emerge).
  • Change --versionate in --only-one-version (sorry but I can't find a better name... :D any suggestion will be appreciated!)
  • Checks if it is really needed to unmask a package: this is done by calling emerge without dependencies checking. I didn't found a better/faster solution: parsing package.keywords in a reliable way (checking operators, versions, revisions and slots...) seems to be too complex and I feel that a full bash implementation could be even slower...
  • Puts the vim modeline on the top of the output so that appending other entries to the file results in a more coherent file

ver 1.0
  • Initial submit.


Last edited by david_e on Wed Jan 14, 2009 4:02 pm; edited 8 times in total
Back to top
View user's profile Send private message
mjf55
Guru
Guru


Joined: 25 Aug 2007
Posts: 372

PostPosted: Sat Jul 12, 2008 7:36 am    Post subject: Reply with quote

David, Thank you for taking the time and effort to create and post a script designed to make life easier for gentoo users. I have often wanted to know how many more packages do I have to add keywords for an emerge to work. Very frustrating . This should solve that. I do have a couple of thought that MAY help.
1- The -v option. In many tools, scripts, the -v means verbose. Perhaps your -debug could become your --verbose and then your versionate you can select another term. Just for better compatibility with others.
2- Your license 'AS-IS' . Perhaps you can use the standard types like GPL. If I wanted to use your script on my work machine, I would have to get management / legal approval with 'as-is', but with GPL, no problem, as permission is already granted.
3- When you generate the dependency list to add the ~arch flag, perhaps you can compare what is already in /etc/portage/package.keywords and not list ones that are already in the keywords file?

I do not think that the community will 'bash' you for being creative and helping. Again, thanks
_________________
Mark
No longer a n00b, but still a 'Kool Kid' wannabe
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 7820

PostPosted: Sat Jul 12, 2008 8:03 am    Post subject: Reply with quote

mjf55 wrote:
2- Your license 'AS-IS' . Perhaps you can use the standard types like GPL. If I wanted to use your script on my work machine, I would have to get management / legal approval with 'as-is', but with GPL, no problem, as permission is already granted.
Another option, more in keeping with the general intent of the existing license, would be to use a BSD style license, the two clause variant being the most like the current license.
Back to top
View user's profile Send private message
yabbadabbadont
Advocate
Advocate


Joined: 14 Mar 2003
Posts: 4791
Location: 2 exits past crazy

PostPosted: Sat Jul 12, 2008 8:42 am    Post subject: Reply with quote

Or even "Public Domain" would work.
Back to top
View user's profile Send private message
david_e
n00b
n00b


Joined: 27 Jul 2007
Posts: 51
Location: Berlin, Germany

PostPosted: Sat Jul 12, 2008 8:43 am    Post subject: Reply with quote

Thanks for your feedback. I really appreciate it.

1. It is true that the '-v' option is confusing: I will change it in the next version.
2. For the license I will read about the various licenses: I have read the GPLv2, but don't know much about the BSD one. Maybe I will switch in the next version.
3. Parsing the old package.keywords entries would be really nice, but I have to figure out how to implement it in a clean way (exp. for versioned entries with '>=')... I will work on it.

mjf55 wrote:
I do not think that the community will 'bash' you for being creative and helping.

I just wanted to make clear that this is not a "look how cool and 1337 I am" thread. :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum