Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[HOWTO] Using dispatch-conf with GUI diff/merge tools
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Wed Aug 15, 2007 10:23 am    Post subject: [HOWTO] Using dispatch-conf with GUI diff/merge tools Reply with quote

Hi all,

I have been using the console-based dispatch-conf for quite a while now.

But there are so many beautiful GUI tools out there like kdiff3, kompare or xxdiff which can be used for showing differences or merging - so why not using them in concert with dispatch-conf?

On the other hand, there are situations when I actually want to use a console-based tool - such as when logged into a remote account via a low-bandwidth SSH-session.

Any enhancement to dispatch-conf should therefore support both approaches.

In order to achieve this goal, I wrote two scripts which can be set as the "diff" and "merge" tool in /etc/dispatch-conf.conf and will allow to optionally use a GUI-based tool for diffing/merging when running dispatch-conf.

The scripts always ask before running a GUI tool and remember the last answer as a default for the next prompt.

It the user decides not to run the GUI tools, the usual console-based diff and sdiff tools will be run. That is, my scripts are intended to be as backwards-compatible as possible with the traditional behavior of dispatch-conf.

The scripts also try to be smart and probe for the presence of a $DISPLAY variable - if it is not set, they always use the console-based diff/merge tools and do not prompt the user.

The scripts also probe whether a GUI tool is installed before using it. If no GUI tool can be found then the scripts also fall back to the console tools without unnecessary prompts.

The current version of the scripts directly support the following GUI tools for comparison out of the box, and if more than one of those tools is installed they use the following order of preference:

  • kompare
  • kdiff3
  • xxdiff


If none of those tools is installed, the console-based "diff -u" is used as a fallback.

The current version of the scripts directly support the following GUI tools for merging out of the box, and if more than one of those tools is installed they use the following order of preference:

  • kdiff3
  • xxdiff


If none of those tools is installed, the console-based "sdiff" is used as a fallback.

The above list of comparison/merging tools can easily be extended and the order of preference can easily be customized by modifying the script functions gui_showdiff(), console_showdiff(), gui_merge() and console_merge() as desired.

Once the scripts have been installed, dispatch-conf works in the following way:

  • When comparing files, the first diff for a new configuration file is always done in console mode. This is because starting a GUI tool might involve considerable overhead, and for small changes it is often not worth it.
  • When the user does not select any specific action at the dispatch-conf prompt after the initial console prompt of dispatch-conf but rather presses the Enter key (which instructs dispatch-conf to display the diff again), the user will be prompted whether to run the GUI-based diff tool or rather again the console-based diff tool.
  • When merging files, the user will immediately be prompted whether to run the GUI-based merge tool or rather the console-based merge tool.
  • The prompts will not occur if X11 is not running (no $DISPLAY variable set) or if no GUI tools can be found.


Last edited by Guenther Brunthaler on Thu Aug 16, 2007 9:59 am; edited 1 time in total
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Wed Aug 15, 2007 10:27 am    Post subject: Instructions how to install the scripts Reply with quote


  • Copy the scripts to the /usr/local/libexec directory
  • Modify your /etc/dispatch-conf.conf as indicated in the comments at the beginning of both scripts.


In order to make it short, this is what should go into /etc/dispatch-conf.conf:
Code:

# Diff for display
# %s old file
# %s new file
diff="/usr/local/libexec/dispatch-conf-diff %s %s"

# Tool for interactive merges.
# %s output file
# %s old file
# %s new file
merge="/usr/local/libexec/dispatch-conf-merge %s %s %s"
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Wed Aug 15, 2007 10:30 am    Post subject: The diff script Reply with quote

This script will be used for comparison.

LAST UPDATE:
Revision: 901
Wed Aug 15 17:12:22 UTC 2007

Save it as "/usr/local/libexec/dispatch-conf-diff":
Code:

#! /bin/sh
# Combined gui/textmode diff for dispatch-conf
#
# Customize functions gui_showdiff and console_showdiff as desired.
#
# This script should have been saved
# as "/usr/local/libexec/dispatch-conf-diff".
#
# In order to make dispatch-conf use this script, replace
# the "diff"-setting in /etc/dispatch-conf.conf by this:
#
# > # Diff for display
# > # %s old file
# > # %s new file
# > diff="/usr/local/libexec/dispatch-conf-diff %s %s"
#
# Note: You have to remove the "# > "-prefix from each line
# after copying the above lines to /dispatch-conf.conf!
#
# Basic idea: The first time, always a command line diff is shown.
# If the diff operation is repeated for the very same file,
# the user is prompted whether to invoke a graphical tool.
# This prompt is only done if $DISPLAY has been set.
#
# $HeadURL: /caches/xsvn/uxadm/trunk/usr/local/libexec/dispatch-conf-diff $
# $Author: root $
# $Date: 2007-08-15T17:11:03.452918Z $
# $Revision: 901 $
#
# Written in 2007 by Guenther Brunthaler


OLD="$1"; NEW="$2"
STATE_DIR=/var/run/dispatch-conf


gui_showdiff() {
   if have kompare; then
      rungui kompare "$OLD" "$NEW"
   elif have kdiff3; then
      rungui kdiff3 "$OLD" "$NEW"
   elif have xxdiff; then
      rungui xxdiff "$OLD" "$NEW"
   else
      warn "No GUI tool has been installed" \
         "- falling back to console tool."
      console_showdiff
   fi
}


console_showdiff() {
   diff -Nu "$OLD" "$NEW" | run less --no-init --QUIT-AT-EOF
}


# Check whether we have tool $1 in path.
have() {
   which "$1" > /dev/null 2>& 1
}


die() {
   echo "ERROR: $*" >& 2
   exit 1
}


warn() {
   echo "WARNING: $*" >& 2
}


run() {
   "$@" && return
   die "Command >>>$*<<< failed!"
}


# Run command hiding its output
# and ignoring its return value.
rungui() {
   "$@" > /dev/null 2>& 1
}


# $1: variable name
# $2: value
varset() {
   eval "$1=$2"
}


# $1: file name
# $2: variable name
# $3: default
load_value() {
   local fn
   fn="$STATE_DIR/$1"
   varset "$2" "$3"
   test -f "$fn" && test -r "$fn" || return 0
   run read "$2" < "$fn"
}


load_state() {
   WANTED_GUI=
   test "x$DISPLAY" != x && WANTED_GUI=y
   load_value last_md5 LAST_MD5 N/A
   load_value diff_gui WANTED_GUI "$WANTED_GUI"
   CHANGED=
}


# $1: file name
# $2: value
save_value() {
   local fn
   fn="$STATE_DIR/$1"
   test -e "$fn" && run rm -f "$fn"
   run printf "%s\n" "$2" > "$fn"
}


save_state() {
   test -z "$CHANGED" && return
   test -d "$STATE_DIR" || run mkdir -m 700 "$STATE_DIR"
   save_value diff_gui "$WANTED_GUI"
   save_value last_md5 "$LAST_MD5"
}


check_file() {
   test -f "$1" && test -r "$1" && return
   die "Cannot read file '$1'!"
}


# $1: result variable
# $2 ...: files
checksum_files() {
   local var cs
   var=$1; shift
   cs=empty
   while [ $# -gt 0 ]; do
      check_file "$1"
      cs=$(
         {
            echo "$cs"
            run cat "$1"
         } | run md5sum -b | cut -c 1-32
      )
      shift
   done
   varset $var $cs
}


# $1: result variable
# $2: locale keyword to get
# $3: default if not available
locale_lookup() {
   local val
   val="$(locale "$2")"
   test "x$val" = x && val=$3
   varset $1 "$val"
}


# $1: Regex
# $2: candidate string
check_re() {
   test "$(printf "%s\n" "$2" | grep "$1" | wc -l)" = 1
}


# $1: result variable
# $2: prompt
# $3: "test -n" boolean default
yesno() {
   local preset resp ye ne
   preset="yes/No"
   test -n "$3" && preset="Yes/no"
   ye="$(locale yesexpr)"
   locale_lookup ye yesexpr "[^Yy].*"
   locale_lookup ne noexpr "[^Nn].*"
   while true; do
      read -p "$2 [$preset]? " resp || resp=
      if [ -z "$resp" ]; then
         resp=$3
      elif check_re "$ye" "$resp"; then
         resp=y
      elif check_re "$ne" "$resp"; then
         resp=
      else
         echo "Invalid answer."
         continue
      fi
      break
   done
   varset $1 $resp
}


showdiff_different() {
   local md5
   checksum_files md5 "$OLD" "$NEW"
   load_state
   if [ "$LAST_MD5" = "$md5" ]; then
      if [ "x$DISPLAY" != x ]; then
         local wg
         yesno wg "Compare files using GUI tools" \
            "$WANTED_GUI"
         if [ "x$wg" != "$WANTED_GUI" ]; then
            CHANGED=y
            WANTED_GUI=$wg
         fi
         if [ -n "$WANTED_GUI" ]; then
            gui_showdiff || return 0
         else
            console_showdiff || return 0
         fi
      else
         console_showdiff || return 0
      fi
   else
      CHANGED=y
      LAST_MD5=$md5
      console_showdiff || return 0
   fi
   save_state
}


if [ $# != 2 ]; then
   die "Usage: ${0##*/} <oldfile> <newfile>"
fi
check_file "$OLD"
check_file "$NEW"
if cmp -s "$OLD" "$NEW"; then
   console_showdiff
else
   showdiff_different
fi


Last edited by Guenther Brunthaler on Wed Aug 15, 2007 5:14 pm; edited 1 time in total
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Wed Aug 15, 2007 10:31 am    Post subject: The merge script Reply with quote

This script will be used for merging.

LAST UPDATE:
Revision: 901
Wed Aug 15 17:12:22 UTC 2007

Save it as "/usr/local/libexec/dispatch-conf-merge":
Code:

#! /bin/sh
# Combined gui/textmode merge for dispatch-conf
#
# Customize functions gui_merge and console_merge as desired.
#
# This script should have been saved
# as "/usr/local/libexec/dispatch-conf-merge".
#
# In order to make dispatch-conf use this script, replace
# the "merge"-setting in /etc/dispatch-conf.conf by this:
#
# > # Tool for interactive merges.
# > # %s output file
# > # %s old file
# > # %s new file
# > merge="/usr/local/libexec/dispatch-conf-merge %s %s %s"
#
# Note: You have to remove the "# > "-prefix from each line
# after copying the above lines to /dispatch-conf.conf!
#
# $HeadURL: /caches/xsvn/uxadm/trunk/usr/local/libexec/dispatch-conf-merge $
# $Author: root $
# $Date: 2007-08-15T17:11:03.452918Z $
# $Revision: 901 $
#
# Written in 2007 by Guenther Brunthaler


OUT="$1"; OLD="$2"; NEW="$3"
STATE_DIR=/var/run/dispatch-conf


gui_merge() {
   if have kdiff3; then
      rungui kdiff3 --merge --output "$OUT" "$OLD" "$NEW"
   elif have xxdiff; then
      rungui xxdiff --merged-filename "$OUT" "$OLD" "$NEW"
   else
      warn "No GUI tool has been installed" \
         "- falling back to console tool."
      console_merge
   fi
}


console_merge() {
   sdiff --suppress-common-lines --output="$OUT" "$OLD" "$NEW"
}


# Check whether we have tool $1 in path.
have() {

   which "$1" > /dev/null 2>& 1
}


die() {
   echo "ERROR: $*" >& 2
   exit 1
}


warn() {
   echo "WARNING: $*" >& 2
}


run() {
   "$@" && return
   die "Command >>>$*<<< failed!"
}


# Run command hiding its output
# and ignoring its return value.
rungui() {
   "$@" > /dev/null 2>& 1
}


# $1: variable name
# $2: value
varset() {
   eval "$1=$2"
}


# $1: file name
# $2: variable name
# $3: default
load_value() {
   local fn
   fn="$STATE_DIR/$1"
   varset "$2" "$3"
   test -f "$fn" && test -r "$fn" || return 0
   run read "$2" < "$fn"
}


load_state() {
   WANTED_GUI=
   test "x$DISPLAY" != x && WANTED_GUI=y
   load_value merge_gui WANTED_GUI "$WANTED_GUI"
   CHANGED=
}


# $1: file name
# $2: value
save_value() {
   local fn
   fn="$STATE_DIR/$1"
   test -e "$fn" && run rm -f "$fn"
   run printf "%s\n" "$2" > "$fn"
}


save_state() {
   test -z "$CHANGED" && return
   test -d "$STATE_DIR" || run mkdir -m 700 "$STATE_DIR"
   save_value merge_gui "$WANTED_GUI"
}


check_file() {
   test -f "$1" && test -r "$1" && return
   die "Cannot read file '$1'!"
}


# $1: result variable
# $2: locale keyword to get
# $3: default if not available
locale_lookup() {
   local val
   val="$(locale "$2")"
   test "x$val" = x && val=$3
   varset $1 "$val"
}


# $1: Regex
# $2: candidate string
check_re() {
   test "$(printf "%s\n" "$2" | grep "$1" | wc -l)" = 1
}


# $1: result variable
# $2: prompt
# $3: "test -n" boolean default
yesno() {
   local preset resp ye ne
   preset="yes/No"
   test -n "$3" && preset="Yes/no"
   ye="$(locale yesexpr)"
   locale_lookup ye yesexpr "[^Yy].*"
   locale_lookup ne noexpr "[^Nn].*"
   while true; do
      read -p "$2 [$preset]? " resp || resp=
      if [ -z "$resp" ]; then
         resp=$3
      elif check_re "$ye" "$resp"; then
         resp=y
      elif check_re "$ne" "$resp"; then
         resp=
      else
         echo "Invalid answer."
         continue
      fi
      break
   done
   varset $1 $resp
}


merge_different() {
   load_state
   if [ "x$DISPLAY" != x ]; then
      local wg
      yesno wg "Merge files using GUI tool" \
         "$WANTED_GUI"
      if [ "x$wg" != "$WANTED_GUI" ]; then
         CHANGED=y
         WANTED_GUI=$wg
      fi
      if [ -n "$WANTED_GUI" ]; then
         gui_merge || return 0
      else
         console_merge || return 0
      fi
   else
      console_merge || return 0
   fi
   save_state
}


if [ $# != 3 ]; then
   die "Usage: ${0##*/} <merged_outfile> <oldfile> <newfile>"
fi
check_file "$OLD"
check_file "$NEW"
if cmp -s "$OLD" "$NEW"; then
   console_merge
else
   merge_different
fi


Last edited by Guenther Brunthaler on Wed Aug 15, 2007 5:15 pm; edited 1 time in total
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Wed Aug 15, 2007 10:35 am    Post subject: The chechsums for the scripts Reply with quote

If the scripts don't seem to work, please first verify their MD5 checksums before reporting a bug!

LAST UPDATE:
Revision: 901
Wed Aug 15 17:12:22 UTC 2007

Code:

ee85d251960cc8febab1701546e5a558  dispatch-conf-diff
d96e29d897ce351d8cde3a56e0eda60f  dispatch-conf-merge

It is all too easy to make a mistake when copying/pasting a script from a forum article.

Update: It seems the indentation of my scripts has been screwed up by pasting it into the article! All the lines should be indented by tabs and not by spaces!

So, before trying to verify the MD5 checksums, please replace all leading spaces into tabs! (4 spaces = 1 tab.)


Last edited by Guenther Brunthaler on Wed Aug 15, 2007 5:19 pm; edited 3 times in total
Back to top
View user's profile Send private message
pilla
Administrator
Administrator


Joined: 07 Aug 2002
Posts: 7209
Location: Pelotas, BR

PostPosted: Wed Aug 15, 2007 3:04 pm    Post subject: Reply with quote

Moved from Portage & Programming to Documentation, Tips & Tricks.
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2160
Location: The Peanut Gallery

PostPosted: Fri Aug 17, 2007 6:03 am    Post subject: Reply with quote

Hi Guenther,

I haven't tried this out yet, as I am snowed under and about to crash out. (I also use etc-proposals, not cfg-update or dispatch-conf.) I was wondering would you think about how we could maybe add this to update? If it's useful for you I am sure it'd be useful to others.

One quick point: I noticed you use which; in bash type -P is quicker (as it's built-in, which is an external which means a fork.) The -P forces path lookup, but doesn't always have to be done; I am a bit hazy on when and why if I'm honest. We were using hash -t blah 2>/dev/null before in update, but it was falling over when we were coding; might be to do with the fact that the script was being edited and restarted/stopped all the time, but it was enough for us to stop using it and fallback to path lookup, and we were a bit pushed for time so I haven't got the canonical answer (although hash would be the preferred option aiui.)

help type or help hash in a terminal will give more info (try help test, that's a great one ;)

Regards,
steveL.
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Fri Aug 17, 2007 7:25 am    Post subject: Reply with quote

Hi Steve,

steveL wrote:
I was wondering would you think about how we could maybe add this to update? If it's useful for you I am sure it'd be useful to others.


Sure! If you think you could reuse part of my script for enhancing update even further, feel free to take from it what you consider useful!

I'm always happy if my work can help someone else - that's why I have been posting it.

steveL wrote:

I noticed you use which; in bash type -P is quicker


Yes, you are right.

But I generally try to avoid bash-specific commands as much as I can, because I want my scripts to be compatible with the standard UNIX sh as much as possible.

I have read autoconf's "Writing Portable Shell Scripts" and always try to adhere mostly to it.

Obviously, I'm not doing this all of the time, because otherwise I needed to use m4 instead of shell functions... but that would feel a bit extreme even to me! ;-)

So, in practice, I try to least achieve compatibility with the POSIX ksh - or with Busybox' ash - whenever I can.

BTW, you might also have noticed that I am using grep for evaluating regular expressions rather than using bash's builtin regex operator - this was done for the very same reason.

Another reason why I did not even try any speed optimization for my script is that it is not time-critical.

It's not a number crunching batch-job, and neither it is a realtime-application.

If one considers the time it will take to actually launch a GUI tool, anything else will be neglectible in comparison.

Furthermore, the real work in my script is done by external tools like md5sum, cmp and diff - those will use up most of the script's actual running time, leaving little running time left to be tuned for efficiency within the script itself.

Anyway, if you decide to pick out sections of my script for putting it into update, feel free to replace any constructs aiming for portability by more efficient bash-builtins: I certainly do understand that a script should be written either with efficiency or portability in mind, and that mixing both concepts in a single script makes no sense.
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2160
Location: The Peanut Gallery

PostPosted: Sat Aug 18, 2007 9:17 am    Post subject: Reply with quote

Guenther Brunthaler wrote:
Sure! If you think you could reuse part of my script for enhancing update even further, feel free to take from it what you consider useful!

I'm always happy if my work can help someone else - that's why I have been posting it.

Cool! We're doing some more on it later today so I'll have a look at it then.

Quote:
But I generally try to avoid bash-specific commands as much as I can, because I want my scripts to be compatible with the standard UNIX sh as much as possible.

Ugh my bad, I didn't notice the #!/bin/sh at the top. In that mode bash won't allow half the stuff I like using ;) It's amazing how many people refuse to believe that ("But /bin/sh is a link to /bin/bash!") to the extent that I added !sh to #bash greybot (I just got so tired of arguing the point.)
Quote:
I certainly do understand that a script should be written either with efficiency or portability in mind, and that mixing both concepts in a single script makes no sense.

Yeah I never thought of it like that: good point. I just really like bash; you'd enjoy talking to greycat in #bash as he's always banging on about portability; more wrt commands like sed and find, or GNU extensions which won't work everywhere, than bash vs sh, but he's quite happy to talk about making shell-scripts portable. Thing is he has bash on linux, BSD and some old HP-UX from 1995. Personally I think requiring bash is ok since it's so prevalent, can be installed on just about anything (not too good for embedded I agree, but that's kinda specialist and if you're doing serious embedded stuff, you're mostly in C and you can afford to rewrite stuff) and has an excellent man page (unlike most GNU utils.) I know others feel differently, I am not trying to get into an argument ;)

The stuff you were saying about D was quite interesting too; I wish you'd join us in #friendly-coders sometime (irc.freenode.org) as it'd be cool to talk code and language design; my nick is igli btw.
Back to top
View user's profile Send private message
Guenther Brunthaler
Apprentice
Apprentice


Joined: 22 Jul 2006
Posts: 214
Location: Vienna

PostPosted: Sat Aug 18, 2007 10:06 am    Post subject: Reply with quote

Hi Steve,
steveL wrote:
I wish you'd join us in #friendly-coders sometime (irc.freenode.org) as it'd be cool to talk code and language design

As much as I honestly appreciate your invitation, I'm afraid I'm not an IRC type of guy.

I'm a rather slow typer, so I prefer non-realtime means of communication where this does not matter.

Also there are often time-related constraints. For instance my time zone is 8 to 10 hours ahead of US time zones, which means there is only a rather small time window when IRC meetings are easily possible to be attended by all.

Nevertheless, thank your for your appreciation, which I mutually return to the fullest!
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2160
Location: The Peanut Gallery

PostPosted: Sat Aug 18, 2007 2:58 pm    Post subject: Reply with quote

Guenther Brunthaler wrote:
As much as I honestly appreciate your invitation, I'm afraid I'm not an IRC type of guy.

I'm a rather slow typer, so I prefer non-realtime means of communication where this does not matter.

Also there are often time-related constraints. For instance my time zone is 8 to 10 hours ahead of US time zones, which means there is only a rather small time window when IRC meetings are easily possible to be attended by all.

Ah well no matter, although I would say I am in the UK so only an hour difference, and also that irc is fun in the sense that it can be asynchronous, since you can always see what has been typed. If someone types slowly it doesn't matter too much, although some of the younger types make a point of typing really quickly. Thankfully I am too old for all that now. But no bother, quite happy to collaborate via forums or email.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum