PORTAGE_ELOG in portage 2.1, "save" module - lots of files

[Zarhan]

I tried out the new ELOG functionality found in portage 2.1 with high hopes. However, at least when "saving" the logs (to /var/tmp/), the end result is not exactly what I was hoping for. Emerge saves an individual log file for EACH package, instead of the entire emerge run.

The end result is that after "emerge -uvDaN world" there may be hundreds of files (after something like KDE update) in there. This is especially troublesome if I forget to clean the directory after looking through the reports - how to know which reports are from the latest run?

Is there a way to configure the elog/save module so that it would concatenate all the reports from a single emerge run to a single file (still timestamped)?

Is this also how the mailed report appears? I mean, I would like to try out the mail-option, but I'm not sure I like the idea of receiving 200 emails after a world update. Why not just one, with time of emerge run and the hostname in the subject field? Or does this happen already?

[Zarhan]

...and my first try for the mailing resulted in error:

"!!! An error occured while trying to send logmail:\n(553, '5.5.4 <portage>... Domain name required for sender address portage', 'portage')"

It seems this is still pending, http://bugs.gentoo.org/show_bug.cgi?id=116637

[Genone]

You won't get hundreds of messages, I think I've received less than one hundred messages since I'm using it (= for about 6 months or so). The reason the modules behave this way is because the log processing happens at merge time in the core portage code which only deals with single packages at a time, to create one log file per emerge run this would have to be moved inside emerge which is really the wrong place.

[Zarhan]

With portage 2.1_pre9 this feature actually started to work, and yes...the end result is that after a big update (that occasionally happens, like when KDE 3.5.2 came up and split ebuilds are used) my mailbox is full on mails with subject "package xxx merged with notice"...

Is there any way to have emerge consolidate all messages from a single -uvDaN world-run into a single message? The usability is really not too good (Well, I suppose I could just do cat /var/tmp/elogs/* | mail or something, but...)

[nmbrthry]

I wrote a small viewer app in Python for elogs. It uses PyGTK. (This is my first Python program, so it might not be the "right way" to code in Python, but it works at a basic level.)

I've been meaning to add things like a "delete" button, but I haven't gotten around to it. Feel free to hack onto it.

Code: Select all

#!/usr/bin/env python

import pygtk
pygtk.require( '2.0' )
import gtk, gobject
import sys, os
from UserDict import UserDict
import re, time

class elogEntry:

	def __init__(self, logClass, stage):
		self.logClass = logClass
		self.stage = stage
		self.log = []

	def addLine(self, logLine):
		self.log.append(logLine)

inputTimeFormat = "%Y%m%d-%H%M%S.log"

def stripNl( line ):
	return re.match( r'^([^\\\n]*)', line ).group( 1 )

class elog:

	def __init__(self, file):
		(self.category, self.package, timestamp) = file.name.split( ":" )
		self.timestamp = time.strptime( timestamp, inputTimeFormat )
		self.__parseElogs( file.readlines() )


	def __parseElogs(self, lines ):
		self.elogs = []
		lastNL = False
		header = False
		headerPattern = re.compile('^(INFO|WARN|ERROR|LOG)')

		for line in lines:
			if headerPattern.match( line ):
				ls = line.split( ": " )
				currentEntry = elogEntry( ls[0], stripNl( ls[1] ) )
				lastNl = False
				header = True
			elif line == '\n':
				if not lastNl and not header:
					self.elogs.append( currentEntry )
					lastNl = True
			else:
				currentEntry.addLine( stripNl( line ) )
				lastNl = False
				header = False

	def getTimestamp(self, timeFormat):
		return time.strftime( timeFormat, self.timestamp )

class eLogViewer:
	def create_list(self):
		scrolled_window = gtk.ScrolledWindow()
		scrolled_window.set_policy( gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC )

		model = gtk.ListStore( str, str, str, gobject.TYPE_PYOBJECT )
		self.tree_view = gtk.TreeView( model )
		selection = self.tree_view.get_selection()
		selection.set_mode( gtk.SELECTION_SINGLE )
		selection.connect_object( "changed", self.insert_text, None )
		scrolled_window.add( self.tree_view )
		self.tree_view.show()

		timeFormat = "%m-%d-%Y %H:%M:%S"

		for i in self.elogs:
			model.append( [i.category, i.package, i.getTimestamp( timeFormat ), i] )
		model.set_sort_column_id( 2, gtk.SORT_DESCENDING )

		col_headers = [ "Category", "Package", "Timestamp" ]
		for i in range(0,3):
			cell = gtk.CellRendererText()
			column = gtk.TreeViewColumn( col_headers[i], cell, text=i )
			column.set_sort_column_id( i )
			self.tree_view.append_column( column )

		return scrolled_window

	def insert_text(self, data=None):
		buffer = self.view.get_buffer()
		buffer.set_text('')
		iter = buffer.get_start_iter()
		(model, liter) = self.tree_view.get_selection().get_selected()

		if liter is None:
			selected = self.elogs[1]
		else:
			selected = model.get_value( liter, 3 )
		for log in selected.elogs:
			buffer.insert( iter, "%s (%s):\n" % (log.logClass, log.stage) )
			for line in log.log:
				buffer.insert( iter, "%s\n" % line )
			buffer.insert( iter, "\n" )

	def create_text(self):
		self.view = gtk.TextView()
		self.view.set_editable( False )
		self.view.set_cursor_visible( False )
		scrolled_window = gtk.ScrolledWindow()
		scrolled_window.set_policy( gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC )
		scrolled_window.add( self.view )
		#self.insert_text()
		scrolled_window.show_all()
		return scrolled_window

	def __init__(self):
		self.elogs = []
		for f in os.listdir("/var/tmp/elogs/"):
			try:
				fileName = "/var/tmp/elogs/" + str(f)
				file = open( fileName )
				self.elogs.append( elog( file ) )
				file.close
			except:
				print "Error opening %s" % fileName

		window = gtk.Window( gtk.WINDOW_TOPLEVEL )
		window.set_title( "elog Viewer" )
		window.connect( "destroy", lambda w: gtk.main_quit() )
		window.set_border_width( 10 )
		window.set_size_request( 850, 600 )

		vpaned = gtk.VPaned()
		window.add( vpaned )
		vpaned.show()

		list = self.create_list()
		vpaned.add1( list )
		list.show()

		text = self.create_text()
		vpaned.add2( text )
		text.show()

		vpaned.set_position( 250 )
		window.show()

def main():
	gtk.main()
	return 0

if __name__ == "__main__":
	eLogViewer()
	main()

[Zarhan]

Nice! I would try pushing this to gentoolkit

[synss]

nmbrthry: please go on! I want something like that to deal with the elogs... unfortunately, I do not know python so I'm not sure I can help much, but that is a very good start!

[nmbrthry]

Note that in a recent release candidate of Portage 2.1, they moved the elog directory to /var/log/portage/elog instead of /var/tmp/elogs so you'll need to change that line. That should really be a global constant somewhere in the file, but I haven't had a chance to revisit this code for awhile (too busy using Ruby instead of Python!).

[synss]

I split, see
http://forums.gentoo.org/viewtopic-p-33 ... ml#3339221
for my version of elogviewer