bash [auto] completion made easy (sgbc)

Message

truc · Post by **truc** » Mon Dec 05, 2011 1:00 am

Hello again!

First, let me say, you'll probably find this tip a lot more usefull than my previous one, so... no special warning here!

I've written quite a few personnal bash completions and have realised that even if some completions can only be specific to one given command(to be really usefull), most of the time they're easy: they have to deal with easy stuff like short&long options optionally taking a parameter which itself can sometime be completed. So I wrote a generic, yet somewhat extensible, bash completion.

Once you have it installed, here is how you can start using it for your amazing new script:

Code: Select all

add_sgbc_for command

And that's it! You can enjoy your TAB key with it too:)

For those wondering, there's no magic here, it just uses the help message provided it's sufficiently formated. And if you'd like to enjoy this completion for a tool which has no --help message or for which it is not formated right, you're _not_ out of luck, I'll show you below how you can probably make it to work anyway.

1 - install
First you have to put the file below in your bash_completion.d directory (usually /etc/bash_completion.d/ for gentoo or /usr/share/bash/completions/ for exherbo for system wide usage). I put the file here for hthe time being, but it should also be available in the post below.

Code: Select all

wget -O sgbc https://gitorious.org/sgbc/sgbc/blobs/raw/master/sgbc

When you feel confident enough about it, copy it to your bash_completion.d directory:

Code: Select all

sudo mv -v sgbc /etc/bash_completion.d/

Note: if you put it somewhere else, you must be aware of two things:

it uses some functions of /etc/bash_completion (_get_comp_words_by_ref, _split_longopt and _filedir at least) so sgbc won't work if those function are not defined
you'll probably have to source the file explicitly in your bashrc, or the completion won't be available

2 - usage

Once you've done step 1, manually source the file or launch a new shell to load the completion, then, suppose you want to start using it for the grep and ss commands, run the following:

Code: Select all

add_sgbc_for grep ss

Now start playing with them. if you like it, just add the above command to the end of your bashrc, so you'll have them automatically in you shell sessions.

3 - quick overview on how it works

As already said it uses the help message of a command to generate the completion. This is a good thing if you're developping something: it forces you to really print _real_ informations in your help message;)

Here is an example:

Code: Select all

ss --help
Usage: ss [ OPTIONS ]
       ss [ OPTIONS ] [ FILTER ]
   -h, --help           this message
   -V, --version        output version information
...snip...
   -i, --info           show internal TCP information
   -s, --summary        show socket usage summary

   -4, --ipv4          display only IP version 4 sockets
   -6, --ipv6          display only IP version 6 sockets
...snip...
   -w, --raw            display only RAW sockets
   -x, --unix           display only Unix domain sockets
   -f, --family=FAMILY display sockets of type FAMILY

   -A, --query=QUERY
       QUERY := {all|inet|tcp|udp|raw|unix|packet|netlink}[,QUERY]

   -F, --filter=FILE   read filter information from FILE
       FILTER := [ state TCP-STATE ] [ EXPRESSION ]

With this help message, sgbc is able to propose completion for the short and long options as well as 'waiting' for a parameter for the option requiring one(eg: -f and --family=...)

And more, in the example above, see the QUERY string?

Code: Select all

   -A, --query=QUERY

If instead of this generic hint, the help message was showing this line

Code: Select all

   -A, --query={all|inet|tcp|udp|raw|unix|packet|netlink}

Then, sgbc would be able to propose completion even on the parameter! This means you can type

Code: Select all

ss -A ra

or

Code: Select all

ss --query=ra

hit [TAB] and sgbc will happily complete it to:

Code: Select all

ss -A raw
# or
ss --query=raw

As of yet, this is not working, but after reading the next section, it will:)

4 - advanced usage

So, you've got this command

which does not have the --help option but just -h, or
which has a help message not formated right to be usable by sgbc(or a help message which could be improved as we'll see with the command 'ss' below) or
which has no help message, but for which you could easily make one quickly(dynamicaly or not)
...

but you cannot modify its source, don't give up, you can probably still make use of the generic bash completion. I'm going to show you how.

Let's say you have enabled sgbc for the command my-script.sh, _before_ trying to call my-script.sh --help, sgbc checks if the function __sgbc_myscriptsh_get_help exists and runs it instead if it does.

note: my-script.sh becomes myscriptsh by removing every character that's not in [A-Za-z0-9_]

So here is the real magic:

you script does not have the --help option but the --show-help one? defined the following function(and add it to your bashrc if satisfaying)
Code: Select all
```
__sgbc_myscriptsh_get_help() { my-script.sh --show-help; }
```

the help message is poorly formated? may be you can reformat it using some sed/awk magic?

Code: Select all

__sgbc_myscriptsh_get_help() { 
   # adapt the following to your needs!
   my-script.sh --help | sed -n 'G; x; ${x;p}'
}

and so on! Your Imagination is your only limitation;)

Back to our ss example, I said, if instead of QUERY we had the possible values, it can take, enclosed in {} and separated with the pipe character, then sgbc will even be able to propose completion on the parameter. Let's do just that then!

By reading the manual, we learn that the possible values for QUERY are all, inet, tcp, udp, raw, unix, packet, netlink, unix_dgram, unix_stream, packet_raw, packet_dgram

note: from the manual, again, it looks like FAMILY is also restricted by the value it can take ( unix, inet, inet6, link, netlink)

Let's create the __sgbc_ss_get_help which will print an improved(sgbc speaking) help message:

Code: Select all

__sgbc_ss_get_help() {
   ss --help | sed '/--[^[:space:]]\+=/ {
         s/QUERY/{all|inet|tcp|udp|raw|unix|packet|netlink|unix_dgram|unix_stream|packet_raw|packet_dgram}/
         s/FAMILY/{unix|inet|inet6|link|netlin}/
      }'
}

note: You can test it by running the command

Code: Select all

__sgbc_ss_get_help

and see how the new help message looks like.

Once you've defined this function in your active shell, you can already enjoy the improved completion on the parameter! That's right! nice isn't it?

If you try to format the help message, here is how it can look

Code: Select all

   somecommand brief description

blablbbalbl
 options can be like this
  --long-option
  -l, --long-option           blahblahblah
  -l, --long-option

	and the same without the comma between the short and
the long option:
  -l  --long-option           blahblahblah

but also with restricted parameter(listed in {} separated with |
  --long-option={a|b|c}       blahblahblah
  -l, --long-option={a|b|c}

it also understand optional parameter even it the completion
really be optional
  -l, --long-option[={a|b|c}] blahblahblah

if the parameter starts with a '/' it will try to complete on filenames:
which can sometimes be wrong (e.g. the --server=/domain/ option of dnsmasq)
  -l, --long-option=/something blahblahblah

if the parameter doesn't look like the above, then there will be
no completion available on the parameter, but it will start again
on the next argument
  -l, --long-option=/something blahblahblah

  and a few more combinations of them

That's it for tonight, hope I haven't made to many typos in there!

Have fun!

EDIT: a few typos... and a trick to make you shell scripts handling --long-option=param _and_ --long-option param
EDIT: some improvements see below

truc · Post by **truc** » Mon Dec 05, 2011 1:02 am

as promised:

Script's been updated, see last version here!

Code: Select all

# samlt's generic bash completion (sgbc)
#
# *************************************************************************
# *  Copyright © 2011 samuel le thiec(aka samlt).                         *
# *  This program is free software: you can redistribute and/or modify    *
# *  it under the terms of the GNU General Public License as published by *
# *  the Free Software Foundation, versions 2 or 3 of the license.        *
# *                                                                       *
# *  This program is distributed in the hope that it will be useful,      *
# *  but WITHOUT ANY WARRANTY; without even the implied warranty of       *
# *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the        *
# *  GNU General Public License for more details.                         *
# *                                                                       *
# *  You should have received a copy of the GNU General Public License    *
# *  along with this program. If not, see <http://www.gnu.org/licenses/>. *
# *************************************************************************
#
#
# this file can be placed in your /etc/bash_completion.d/sgbc(or equivalent,
# see below) as it uses some of the functions defined in /etc/bash_completion
#
# In exherbo: put it in /usr/share/bash-completion/completions/
#


__sgbc_get_help() { # either call $prog --help or a special function if it exists to print a usable help for SABC {{{
   local prog=${COMP_WORDS[0]}
   # keel the basename and purify it so it can be used in a function name
   name=$(echo "${prog}"| sed 's#.*/## ; s/[^A-Za-z0-9_]//g')
   if command -v __sgbc_${name}_get_help &>/dev/null; then
      __sgbc_${name}_get_help
   else
      # issueing the command $1 --help ('runtime')
      "$prog" --help
   fi
}
# }}}

__sgbc_get_options_along_with_param() { # list short and long options along with their optional parameter {{{

   # options can be like this
   #  --long-option
   #  -l, --long-option           blahblahblah
   #  -l, --long-option
   #  -l  --long-option           blahblahblah
   #  --long-option={a|b|c}       blahblahblah
   #  -l, --long-option={a|b|c}
   #  -l, --long-option[={a|b|c}] blahblahblah
   #  -l, --long-option=something blahblahblah
   #  and a few more combinations of them
   #

   __sgbc_get_help | sed -rn '
         # ((-l),? )(--long-option)([?(=)(<parameters>)).*

         /^[[:space:]]*((-[^-]),?[[:space:]]+)?(--[^[:space:]=[]{2,})(\[?(=)([^][:space:]]+)]?)?.*/ {
            # split short and long options and add their optional parameters
            # in the second column
            s//\2 \6\n\3\5 \6/

            # if no short option, we need to delete the first part created above
            /^[^-]/s/.*\n//

            p
         }'
}
# }}}

__sgbc_get_options() { # print every short&long options {{{
   __sgbc_get_options_along_with_param | sed 's/[[:space:]]\+.*$//'
}
# }}}

__sgbc_is_an_option() { # return 0/1 {{{
   # long option which need a parameter are listed like '--long-option='
   # and this function should return 0 (ie true) for '--long-option' as well
   # as for '--long-option='
   __sgbc_get_options | grep -qx -- "${1}=\?"
}
# }}}

__sgbc_get_params_for_option() { # print every possible paramater for $1 when possible or some special value instead {{{
   local option="$1"
   local param_is_file='__sgbc_param_file__'
   local unknown_param='__sgbc_unknown_param__'

   __sgbc_get_options_along_with_param | sed -n '
         /^'"$option"'=\?[[:space:]]*/ {
            # just keep the parameters if any
            s///

            # no parameters: return nothing
            /^$/q

            # if parameters are like {a|b|c} print them one per line
            /^{.\+\(|.\+\)*}$/{
               s/^.//
               s/.$//
               s/|/\n/g
               p
               q
            }

            # otherwise if it starts with a / we suppose it
            # is the path to a file
            /^\// {
               s/^\/.*/'"$param_is_file"'/
               p
               q
            }

            # otherwise return something we can work with later
            s/.*/'"$unknown_param"'/
            p
            q
         }
         $ {
            s/.*/__sgbc_option_not_found__/
            p
         }'
}
# }}}

__sgbc_comp_options() { # complete on options ($1=cur) {{{
   local comp cur=$1

   actions=$(__sgbc_get_options)
   COMPREPLY=( $( compgen -W "$actions" -- $cur ) )

   if [ ${#COMPREPLY[@]} -eq 1 ]; then
      comp=${COMPREPLY[0]}
      # if it's either a short option or a long option that doesn't take
      # a parameter then finish(ie add a space) the completion
      if [ ${#comp} = 2 -o "${comp}" = "${comp%=}" ]; then
         compopt +o nospace
         return 0
      fi
      compopt -o nospace
      return 0
   elif [ ${#COMPREPLY[@]} -eq 0 ]; then
      # try completion on files/dirs instead
      _filedir
      return 0
   else
      compopt -o nospace
      return 0
   fi
}
# }}}

__sgbc_comp_params_or_options() { # complete on paameter ($1=opt, $2=cur) {{{
   local comp prev=$1 cur=$2

   actions=$(__sgbc_get_params_for_option "$prev")

   case "$actions" in
      '')
         # prev doesn't need a parameter, propose options completion
         __sgbc_comp_options "$cur"
         return
         ;;
      '__sgbc_param_file__')
         # propose something nice for this
         compopt +o nospace
         _filedir
         ;;
      '__sgbc_unknown_param__')
         # no completion available for this parameter
         return 0
         ;;
      *)
         # predefined choices for the parameter completion
         compopt +o nospace
         COMPREPLY=( $( compgen -W "$actions" -- $cur ) )
         return 0
         ;;
   esac
   return 0
}
# }}}

_sgbc_from_help() { # complete options and their parameter when possible {{{
   local actions cur prev
   local cur_prevchar
   COMPREPLY=()

   __get_cword_at_cursor_by_ref "" words cword cur
   prev=${words[cword-1]}

   if [ $cword -eq 1 ]; then
      __sgbc_comp_options "$cur"
      return 0
   fi

   # for --long=param and '--long= sthg' $prev will be =
   if [ "$prev" = '=' ]; then
      # if character before $cur in COMP_LINE is a '[[:space:]]'
      # we're dealing with the second form
      cur_prevchar=${COMP_LINE:$((COMP_POINT-${#cur}-1)):1}
      if [ "$cur_prevchar" != "${cur_prevchar/[[:space:]]}" ]; then
         # if the user typed '--long= sthg' we can safely assume the user
         # means the empty parameter for the previous options, so let's move
         # on options completion
         # _note:_ and if previous arg is not an option then that's one more
         # reason to continue with it
         __sgbc_comp_options "$cur"
         return 0
      else
         prev=${words[cword-2]}
      fi
   fi

   if __sgbc_is_an_option "$prev"; then
      # for --long-option= taking a parameter and when the
      # cursor is here    ^
      [ "$cur" = '=' ] && cur=''

      __sgbc_comp_params_or_options "$prev" "$cur"
      return 0
   else
      # $prev is not an option
      __sgbc_comp_options "$cur"
      return 0
   fi

   return 0
}
# }}}

add_sgbc_for() { # ease the definition of samlt's generic bash completion for commands {{{
   for cmd in "$@"; do
      command -v $util &>/dev/null && complete -F _sgbc_from_help $cmd
   done
}
complete -c add_sgbc_for
# }}}

# vim: set et sts=3 sw=3 filetype=sh foldmethod=marker :

truc · Post by **truc** » Mon Dec 05, 2011 8:35 am

As you'll probably notice, if your command supports this --long-option=blahblah, sgbc can handle the parameter completion for --long-option=adf but _also_ for --long-option adf, so here is an other trick for you're shell scripts so they can handle this two options easily:

when you're parsing the parameter, you can use something like the code below so your script supports both format:

Code: Select all

 

while [ $# -gt 0 ]; do
	case "$1" in
		--??*=*)
			# split --long=param => --long param
			option=${1%%=*}
			param=${1#*=}
			set -- "$@" "$option" "$param"
			;;
		-l|--long-option)
			[ $# -eq 1 ] && { echo "$2 needs a parameter"; show_help; exit 1; }
			shift
			parameter=$1
			...
			;;
		...)
			...
			;;
		-h|--help)
			# show this help message
			show_help
			exit 0
			;;
		*)
			show_help
			exit 1
			;;
	esac
	shift
done

truc · Post by **truc** » Tue Dec 06, 2011 6:56 pm

Hello Again, I'm no spammer! sorry to bump this again, but editing posts in place doesn't feel right for a big change...

Up to now I was unstatisfied by the fact the completion didn't really allow an empty parameter for an option which can take one, for example:
If your cursor is right after the '=' in the line below:

Code: Select all

mycommand --foo --long-option=

Then typing TAB will try parameter completion (providing --long-option really takes an argument), now, and this is what change, if you type a space then TAB again, sgbc will try option completion:

Code: Select all

$ ss --q                                    # hit [i]TAB[/i] once

$ ss --query=                               # hit [i]TAB[/i] twice
all           netlink       packet_dgram  raw           udp           unix_dgram
inet          packet        packet_raw    tcp           unix          unix_stream

$ ss --query=un                             # type 'un' then hit [i]TAB[/i] twice

$ ss --query=unix                           # hit [i]TAB[/i] twice
unix         unix_dgram   unix_stream

$ ss --query=unix_d                         # type '_d' then hit [i]TAB[/i] once

$ ss --query=unix_dgram -                   # hit [i]TAB[/i] twice
-0           -d           --family=    --ipv4       -n           --processes  --summary    -V
-4           --dccp       --filter=    --ipv6       --numeric    --query=     -t           --version
-6           -e           -h           -l           -o           -r           --tcp        -w
-a           --extended   --help       --listening  --options    --raw        -u           -x
-A           -f           -i           -m           -p           --resolve    --udp
--all        -F           --info       --memory     --packet     -s           --unix

$ ss --query=unix_dgram --fam               # type '-fam' then hit [i]TAB[/i] once

$ ss --query=unix_dgram --family=           # hit [i]TAB[/i] twice
inet    inet6   link    netlin  unix

$ ss --query=unix_dgram --family=           # type a ' ', then hit [i]TAB[/i] once, now completion continues on options

$ ss --query=unix_dgram --family= -         # hit [i]TAB[/i] twice
-0           -d           --family=    --ipv4       -n           --processes  --summary    -V
-4           --dccp       --filter=    --ipv6       --numeric    --query=     -t           --version
-6           -e           -h           -l           -o           -r           --tcp        -w
-a           --extended   --help       --listening  --options    --raw        -u           -x
-A           -f           -i           -m           -p           --resolve    --udp
--all        -F           --info       --memory     --packet     -s           --unix

Updating the pastebin link and/or the forum post is no pleasant task, as must be getting the script updated if, by any wonders, some of you were actually using/trying this. So I quickly set up a public git repo on gitorius.

You'll find the updated sgbc file here.

Have fun with this:)

[/code]

mv · Post by mv » Tue Dec 06, 2011 7:50 pm

This is all well, but somehow reinventing the wheel (AKA zsh). For those who maybe don't know: The zsh completion system has the possibility to parse --help strings since ages (man zshcompsys, look for _gnu_generic). Moreover, the zsh completion system also provides a function _arguments which makes it almost trivial to provide completion functionality in a readable brief syntax which gives a lot of flexibility (e.g. you can easily specify whether long and/or short options are possible, which options take which forms of arguments, whether short options can be combined, whether options may follow non-options, whether options can be repeated or exclude each other or enable other options, which help texts are printed for certain options, ...).