| View previous topic :: View next topic |
| Author |
Message |
ahsan.ae
n00b
Joined: 02 Apr 2007
Posts: 2
|
 Posted: Mon Apr 02, 2007 11:54 am Post subject: Emerge (Verbose) Output Parser - A Script |
 |
|
Hi all,
I've written a command-line utility to parse the output of the 'emerge -pv world|system' command. You just need ruby installed on your system.
The output of ./evoparser.rb -h gives a good idea of what it does.
| Code: | Usage: evoparser.rb [options] [FILENAME]
(Invoke without the filename for standard input)
Specific options:
-c, --command Print emerge command
-r, --report Print a tabular report (default)
-e, --existing Show only downloaded pkgs
-d, --pending-dl Show pkgs pending download
-t, --status [STAT] Show packages with the specified status
-s, --size Sort by package size (descending)
-S, --size-asc Sort by package size (ascending)
-A, --alphasort Sort alphabetically
-p, --precise Print emerge command with package versions
-x, --execute-command Execute 'emerge -pv' command on resulting pkgs (if -c)
-g, --debug Debug - print the lines skipped by the parser. Ignore all other options.
|
It generates a tabular report as follows:
| Code: | SIZE STATUS NAME
(kb)
0 N sys-libs/zlib-1.2.3-r1
0 N sys-libs/gpm-1.20.1-r5
0 N sys-libs/ncurses-5.5-r3
0 N app-shells/bash-3.1_p17
0 N sys-libs/readline-5.1_p4
0 N virtual/libiconv-0
0 N dev-libs/expat-1.95.8
0 N sys-devel/gettext-0.16.1
0 N sys-apps/diffutils-2.8.7-r1
0 N sys-devel/gnuconfig-20060702
0 N sys-libs/db-4.2.52_p4-r2
0 N sys-libs/gdbm-1.8.3-r3
0 N sys-devel/libperl-5.8.8-r1
0 N dev-lang/perl-5.8.8-r2
0 N dev-libs/openssl-0.9.8d
69 N perl-core/Test-Harness-2.64
0 N app-admin/perl-cleaner-1.04.3
95 N perl-core/PodParser-1.35
|
Alternatively, if I want to just emerge the packages that are already on disk, I can do:
| Code: | | emerge -pv world --deep | ./evoparser.rb -ec |
which will output the following:
| Code: | | emerge -pv sys-libs/zlib sys-libs/gpm sys-libs/ncurses app-shells/bash sys-libs/readline virtual/libiconv dev-libs/expat sys-devel/gettext sys-apps/diffutils sys-devel/gnuconfig sys-libs/db sys-libs/gdbm sys-devel/libperl dev-lang/perl dev-libs/openssl perl-core/Test-Harness app-admin/perl-cleaner perl-core/PodParser |
I can also tell evoparser to execute the command with the -x flag. Another useful flag is -p which tells evoparser to output the command with the precise package versions as follows:
| Code: | | emerge -pv =sys-libs/zlib-1.2.3-r1 =sys-libs/gpm-1.20.1-r5 =sys-libs/ncurses-5.5-r3 =app-shells/bash-3.1_p17 =sys-libs/readline-5.1_p4 =virtual/libiconv-0 =dev-libs/expat-1.95.8 =sys-devel/gettext-0.16.1 =sys-apps/diffutils-2.8.7-r1 =sys-devel/gnuconfig-20060702 =sys-libs/db-4.2.52_p4-r2 =sys-libs/gdbm-1.8.3-r3 =sys-devel/libperl-5.8.8-r1 =dev-lang/perl-5.8.8-r2 =dev-libs/openssl-0.9.8d =perl-core/Test-Harness-2.64=app-admin/perl-cleaner-1.04.3 =perl-core/PodParser-1.35 |
Since it is time-consuming to pipe the output of emerge to evoparser every time, it is also possible to do 'emerge -pv world > tmp.txt' and then: './evoparser.rb tmp.txt'.
Here's the code (please copy and paste into an editor and save it as evoparser.rb).
| Code: | #!/usr/bin/ruby
# evoparser-0.1 (Emerge (Verbose) Output Parser) by Ahsan Ali (ahsanali (dot) ae (at) gmail (dot) com)
# evoparser parses the output of
# the 'emerge -pv world|system --deep' command to report on packages that are downloaded
# or pending download, their portage status (N|UD|F|f|B|R|S), and their sizes.
# The report can be output in tabular format, or as a emerge command ready to be executed
require 'optparse'
class GentooPackage
attr_reader :name, :size, :useflags, :installed_version, :status, :category
attr_writer :name, :size, :useflags, :installed_version, :status, :category
def initialize(name)
@name = name
@size = 0
@useflags = []
@installed_version = nil
@status = ''
@category = @name.split('/').first
end
def <=>(other)
@size <=> other.size
end
def to_s
self.name
end
def name
packageName = %r{([\_\w\-0-9\.\/\+\-]+)}
pkg_regexp = %r{([\w\_0-9\-\/\+\-]+)-([rc\-\.0-9btestalpha]+)} # limited, but works for most packages
mdata = pkg_regexp.match(@name)
mdata[1] if mdata.class == MatchData
end
def name_with_version
@name
end
def get_report_row
"#@size\t#@status\t#@name"
end
end
class EVOParser # Emerge Output Parser
def initialize(options, files=[])
@options = options
@files = files
@packages = []
end
def parse(fn, stdin=false)
# [ebuild U ] gnome-base/gnome-vfs-2.16.3 [2.14.2-r1] USE="hal ssl -avahi -debug -doc -gnutls -ipv6 -samba" 0 kB [1]
patstr = '\[ebuild\s+([R UNFfBSUD]+)\s*\]\s+([\_\w\-0-9\.\/\+\-]+)\s*(\[[a-z0-9\.\_\-\w]+\])?\s*((USE|LINGUAS|VIDEO_CARDS|INPUT_DEVICES)=\"(.+)\")*\s+([0-9\,]+)\s+kB.*'
pat = Regexp.new(patstr)
if stdin then
input_file = $stdin
else
input_file = File.open(fn)
end
input_file.each_line do |line|
m = pat.match(line)
if m.class == MatchData then
gp = GentooPackage.new(m[2])
gp.status = m[1]
gp.installed_version = m[3] unless m[4].nil?
gp.useflags = m[6].split(' ') unless m[6].nil?
gp.size = m[7].delete(',').to_i
@packages << gp
else
puts line if @options[:debug]
end
end
input_file.close unless stdin
end
def get_emerge_command(list)
return "" if list.empty?
"emerge -pv #{list.join(' ')}" # otherwise ;)
end
def start
parse(@files.first, @files.empty?) # parse just the first file
puts "Warning: Ignoring #{@files.slice(1,files.length).join(' ')}" if @files.length > 1
# IGNORE all actions if in debug mode, or if there are no packages to process
unless @options[:debug] or @packages.empty?
# the interdependencies of the options are taken care of in the earlier code
@packages.reject! {|p| p.size > 0} if @options[:existing]
@packages.reject! {|p| p.size == 0 } if @options[:todownload]
@packages.reject! {|p| p.status != @options[:status]} unless @options[:status].nil?
@packages.sort! if @options[:size]
@packages.sort! {|one,two| two <=> one} if @options[:sizeasc]
@packages.sort! {|one,two| one.name <=> two.name} if @options[:alphasort]
# must be last command before printing
# as it replaces GentooPackage instances with Strings
@packages.collect! {|p| "=#{p.name_with_version}"} if @options[:precise]
# OUTPUT the report/command
if @options[:command] then
cmd = get_emerge_command(@packages)
puts cmd unless @options[:execute]
elsif @options[:report] then
puts "SIZE \tSTATUS\tNAME"
puts "(kb)"
puts
@packages.each {|p| puts p.get_report_row}
end
system(cmd) if @options[:execute]
end
end
end
def init
options = {}
op = OptionParser.new do |opts|
opts.banner = "Usage: evoparser.rb [options] [FILENAME]"
opts.separator " (Invoke without the filename for standard input)"
opts.separator ""
opts.separator "Specific options:"
opts.on('-c', '--command', 'Print emerge command ') do |e|
raise 'Specify -c OR -r' if options[:report]
options[:command] = e
end
opts.on('-r', '--report', 'Print a tabular report (default)') do |e|
raise 'Specify -c OR -r' if options[:command]
options[:report] = e
end
opts.on('-e', '--existing', 'Show only downloaded pkgs') do |e|
raise "Specify -e OR -d" if options[:todownload]
options[:existing] = e
end
opts.on('-d', '--pending-dl', 'Show pkgs pending download') do |e|
raise "Specify -e OR -d" if options[:existing]
options[:todownload] = e
end
opts.on('-t', '--status [STAT]', String ,'Show packages with the specified status') do |e|
options[:status] = e # true by default
end
opts.on('-s', '--size', 'Sort by package size (descending)') do |s|
raise 'Specify either -s OR -S or -A' if options[:sizeasc] or options[:alphasort]
options[:size] = s
end
opts.on('-S', '--size-asc', 'Sort by package size (ascending)') do |s|
raise 'Specify either -s OR -S or -A' if options[:size] or options[:alphasort]
options[:sizeasc] = s
end
opts.on('-A', '--alphasort', 'Sort alphabetically') do |s|
raise 'Specify either -s OR -S or -A' if options[:size] or options[:sizeasc]
options[:alphasort] = s
end
opts.on('-p', '--precise', 'Print emerge command with package versions ') do |e|
if options[:command] then
options[:precise] = e
else
puts "Warning: ignoring -p (needs -c)"
end
end
opts.on('-x', '--execute-command', "Execute 'emerge -pv' command on resulting pkgs (if -c)") do |e|
if options[:command] then
options[:execute] = e
else
puts "Warning: ignoring -x (needs -c)"
end
end
opts.on('-g', '--debug', 'Debug - print the lines skipped by the parser. Ignore all other options.') do |s|
options[:debug] = s
puts 'Warning: ignoring all actions due to debug mode'
end
end
files = op.parse!
# if neither -c or -r have been specified, assume -r
options[:report] = true unless options[:report] or options[:command]
parser = EVOParser.new(options, files)
parser.start
end
init
|
Please comment !
Last edited by ahsan.ae on Tue Apr 03, 2007 6:28 am; edited 1 time in total |
|
| Back to top |
|
 |
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Mon Apr 02, 2007 2:27 pm Post subject: Re: Emerge (Verbose) Output Parser - A Script |
|
|
| Quote: | | Code: | SIZE STATUS NAME
(kb)
0 N sys-libs/zlib-1.2.3-r1
0 N sys-libs/gpm-1.20.1-r5
0 N sys-libs/ncurses-5.5-r3
0 N app-shells/bash-3.1_p17
0 N sys-libs/readline-5.1_p4
0 N virtual/libiconv-0
0 N dev-libs/expat-1.95.8
0 N sys-devel/gettext-0.16.1
0 N sys-apps/diffutils-2.8.7-r1
0 N sys-devel/gnuconfig-20060702
0 N sys-libs/db-4.2.52_p4-r2
0 N sys-libs/gdbm-1.8.3-r3
0 N sys-devel/libperl-5.8.8-r1
0 N dev-lang/perl-5.8.8-r2
0 N dev-libs/openssl-0.9.8d
69 N perl-core/Test-Harness-2.64
0 N app-admin/perl-cleaner-1.04.3
95 N perl-core/PodParser-1.35
|
...
Please comment ! |
Please don't take this offensively, but I don't see what that output has over emerge. TBH bash is much better at this sort of thing, but well done for learning Ruby and getting it working. |
|
| Back to top |
|
|
ahsan.ae
n00b
Joined: 02 Apr 2007
Posts: 2
|
| Posted: Tue Apr 03, 2007 3:58 am Post subject: Why I wrote it |
|
|
Thanks for the reply, Steve.
Without a net connection on my gentoo machine I update my portage with the nightly portage snapshots, but when it comes to updating a package, I use this script to filter out the packages that need to be downloaded (I do that separately using distgrabber).
Usually, there are a lot of packages that need to be updated/installed, and emerge itself does not provide a facility to JUST emerge the packages whose source files are already on my system, or be able to run a command like 'emerge -pF' on JUST the packages that are NOT already downloaded. [see the -e and -d options]
At other times I decide to install the smallest packages first, and the big ones (qt, eclipse) later, and I need the output Sorted to decide that  [see the -S and -s options]
There are also instances when certain packages are shown to need updating but when you do 'emerge packagename' minus the version it picks up some other slot. That is also taken care of by the script [see the -p option]
That was why I wrote the script. I hope someone else also finds it helpful. If you can think of any other functionality, please drop a line.
And YES ! I'm beginning to LOVE ruby 
Cheers,
AA |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
