Gentoo Forums :: View topic - Portage Searching Script (New Barebones Website and Source)

Portage Searching Script (New Barebones Website and Source)

View unanswered posts
View posts from last 24 hours

Gentoo Forums Forum Index

Documentation, Tips & Tricks

View previous topic :: View next topic

Author

Message

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Wed Feb 14, 2007 4:43 am Post subject: Portage Searching Script (New Barebones Website and Source)

Hello everyone,

I've written the following script which scans a Portage tree, reads the categories, ebuilds, specific ebuild files for different versions, and metadata into memory, then scans for a regex string. So far, it doesn't search the package atom's name, since that can be done more quickly with find. What do you all think?
Requires DBI, XML-DOM, and DBD-SQLite

Code:

#!/usr/bin/perl
use strict;
use File::Basename;
use XML::DOM;
use Term::ANSIColor;

my $searchall = 0;
my $colorreplace = 1;
my $insensitive = 1;
my @searchterms;
if(scalar @ARGV < 1) {
$searchall = 1;
}
else {
@searchterms = @ARGV;
}
my $portdir = "/usr/portage";
my @categories;
foreach my $category (<$portdir/*>) {
if(-d "$category") {
   my $cat = new Category;
   $cat->name(basename("$category"));
   push(@categories, $cat);
}
}
foreach my $category (@categories) {
my $catname = $category->name;
foreach my $subdir (<$portdir/$catname/*>) {
   if(-d "$subdir") {
      my $ebuild = new Ebuild;
      $ebuild->name(basename("$subdir"));
      foreach my $file (<$subdir/*>) {
         if(! -d "$file") {
            push(@{$ebuild->files}, basename("$file"));
         }
      }
      push(@{$category->ebuilds}, $ebuild);
   }
}
}
foreach my $category (@categories) {
my $catname = $category->name;
foreach my $ebuild (@{$category->ebuilds}) {
   my $ename = $ebuild->name;
   foreach my $file (@{$ebuild->files}) {
      if("$file" eq "metadata.xml") {
         my $parser = new XML::DOM::Parser;
         my $xml = $parser->parsefile("<$portdir/$catname/$ename/$file");
         my $nodes = $xml->getElementsByTagName("longdescription");
         my $n = $nodes->getLength;
         for(my $i = 0; $i < $n; $i++) {
            my $node = $nodes->item($i);
            my $lang = $node->getAttributeNode("lang");
            if($n == 1 || ($lang && $lang->getValue eq "en")) {
               my $children = $node->getChildNodes;
               my $nc = $children->getLength;
               for(my $c = 0; $c < $nc; $c++) {
                  my $subnode = $children->item($c);
                  if($subnode->getNodeType == TEXT_NODE) {
                     $ebuild->longdesc($subnode->getNodeValue);
                  }
               }
            }
         }
         $xml->dispose;
      }
      if("$file" =~ /\.ebuild$/) {
         my $version = "$file";
         my $tmpename = "$ename";
         $tmpename =~ s/\+/\\\+/g;
         $version =~ s/^$tmpename-([a-z0-9_.-]+)\.ebuild$/\1/;
         open(FILE, "<$portdir/$catname/$ename/$file");
         my @lines = <FILE>;
         chomp(@lines);
         close(FILE);
         foreach my $line (@lines) {
            if("$line" =~ /DESCRIPTION=\".*?\"/) {
               my $desc = "$line";
               $desc =~ s/^.*?DESCRIPTION=\"(.*?)\".*?$/\1/;
               ${$ebuild->shortdesc}{$version} = "$desc";
            }
         }
      }
   }
}
}
if($searchall) {
foreach my $category (@categories) {
   my $catname = $category->name;
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      print (&printinfo($ebuild, $catname));
   }
}
}
else {
foreach my $category (@categories) {
   my $catname = $category->name;
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $match = 0;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      foreach my $term (@searchterms) {
         if("$longdesc" =~ /$term/ || ("$longdesc" =~ /$term/i && $insensitive)) {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$longdesc" =~ /$term/ig;
               }
               else {
                  @captured = "$longdesc" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  $filterfound =~ s/(\W)/\\\1/g;
                  $longdesc =~ s/$filterfound/$colored/g;
               }
            }
            $match += 1;
         }
      }
      $ebuild->longdesc($longdesc);
      foreach my $version (sort keys %shortdesc) {
         foreach my $term (@searchterms) {
            if("$shortdesc{$version}" =~ /$term/ || ("$shortdesc{$version}" =~ /$term/i && $insensitive)) {
               if($colorreplace) {
                  my @captured;
                  if($insensitive) {
                     @captured = "$shortdesc{$version}" =~ /$term/ig;
                  }
                  else {
                     @captured = "$shortdesc{$version}" =~ /$term/g;
                  }
                  foreach my $found (@captured) {
                     my $colored = colored("$found", 'cyan');
                     my $filterfound = "$found";
                     $filterfound =~ s/(\W)/\\\1/g;
                     $shortdesc{$version} =~ s/$filterfound/$colored/g;
                  }
                  ${$ebuild->shortdesc}{$version} = $shortdesc{$version};
               }
               $match += 1;
            }
         }
      }
      if($match) {
         print (&printinfo($ebuild, $catname));
      }
   }
}
}

sub printinfo {
my $ebuild = shift;
my $catname = shift;
my $ename = $ebuild->name;
my $longdesc = $ebuild->longdesc;
my %shortdesc = %{$ebuild->shortdesc};
my $ret = colored("$catname/$ename\n", 'bold blue');
$ret .= "\t" . colored("Short Description\n", 'bold green');
foreach my $version (reverse sort keys %shortdesc) {
   $ret .= "\t\t" . colored("$version: ", 'bold yellow') . "$shortdesc{$version}\n";
}
if($longdesc && !("$longdesc" =~ /^[\s\t\n]*$/)) {
   my @arraydesc = split(/\n/, "$longdesc");
   chomp(@arraydesc);
   if(!$arraydesc[0]) {
      shift(@arraydesc);
   }
   $ret .= "\t" . colored("Long Description\n", 'bold red');
   foreach my $line (@arraydesc) {
      $line =~ s/^[\s\t]*/\t\t/g;
      $ret .= "$line\n";
   }
}
$ret .= "\n";
return "$ret";
}

package Category;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub ebuilds {
my $self = shift;
if(!defined $self->{EBUILDS}) {
   $self->{EBUILDS} = [ ];
}
return $self->{EBUILDS};
}
package Ebuild;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub files {
my $self = shift;
if(!defined $self->{FILES}) {
   $self->{FILES} = [ ];
}
return $self->{FILES};
}
sub longdesc {
my $self = shift;
$self->{LDESC} = shift if @_;
return $self->{LDESC};
}
sub shortdesc {
my $self = shift;
if(!defined $self->{SDESC}) {
   $self->{SDESC} = { };
}
return $self->{SDESC};
}

Here's an example search, with [\s] surrounding to search only for whole words:

Code:

$ perl portfind.pl '[\s][Ss][Vv][Gg][\s]'
app-office/dia
Short Description
0.95.1: Diagram/flowchart creation program
0.96_pre3: Diagram/flowchart creation program
Long Description
Dia is a gtk+ based diagram creation program. It can be used to
draw many different kinds of diagrams. It currently has special
objects to help draw entity relationship diagrams, UML diagrams,
flowcharts, network diagrams, and simple circuits. It is also
possible to add support for new shapes by writing simple XML
files, using a subset of SVG to draw the shape.

dev-java/batik
Short Description
1.5-r1: Java based toolkit for applications or applets that want to use images in the Scalable Vector Graphics (SVG) format for various purposes, such as viewing, generation or manipulation.
1.6-r1: Java based SVG toolkit
1.6-r2: Java based SVG toolkit
1.6-r3: Java based SVG toolkit
Long Description
Java toolkit for applications or applets that want to use
images in the Scalable Vector Graphics (SVG) format for various
purposes, such as viewing, generation or manipulation.

dev-perl/GD-SVG
Short Description
0.27: Seamlessly enable SVG output from scripts written using GD
0.28: Seamlessly enable SVG output from scripts written using GD

dev-ruby/ruby-svg
Short Description
1.0.3: Ruby SVG Generator

media-gfx/inkscape
Short Description
0.43: A SVG based generic vector-drawing program
0.43-r1: A SVG based generic vector-drawing program
0.44: A SVG based generic vector-drawing program
0.44.1: A SVG based generic vector-drawing program
0.45: A SVG based generic vector-drawing program

media-gfx/xsvg
Short Description
0.1.2: X11 SVG viewer
0.2.0: X11 SVG viewer
0.2.1: X11 SVG viewer

media-libs/libsvg
Short Description
0.1.2: A parser for SVG content in files or buffers
0.1.3: A parser for SVG content in files or buffers
0.1.4: A parser for SVG content in files or buffers

media-libs/wxsvg
Short Description
1.0_beta7: C++ library to create, manipulate and render SVG files.

x11-libs/libsvg-cairo
Short Description
0.1.4: Render SVG content using cairo
0.1.5: Render SVG content using cairo
0.1.6: Render SVG content using cairo

x11-themes/flatsvg
Short Description
1.0: Flat SVG icon set

x11-themes/gartoon
Short Description
0.5: Gartoon SVG icon theme.
0.5-r1: Gartoon SVG icon theme.
Long Description
Gartoon icon set.

x11-themes/gnome-themes-extras
Short Description
0.8.1: Additional themes for GNOME 2.2
0.9.0: Additional themes for GNOME 2.2
Long Description
GNOME Themes Extras is a collection of metathemes for the GNOME desktop environment.
This package provides GNOME users an extra set of themes that are functional and attractive,
often sporting a distinct appearance that you won't find in the primary themes package.
Most of the graphics in these themes contain SVG vector graphics.

x11-themes/nuvola
Short Description
1.0-r1: Nuvola SVG icon theme.

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

Last edited by Kenji Miyamoto on Sat Feb 24, 2007 3:07 am; edited 5 times in total

mark_alec
Bodhisattva

Joined: 11 Sep 2004
Posts: 6066
Location: Melbourne, Australia

Posted: Wed Feb 14, 2007 5:17 am Post subject:

Moved from Gentoo Chat to Documentation, Tips & Tricks. Scripts generally seem to live here. _________________ www.gentoo.org.au \|\| #gentoo-au

Dralnu
Veteran
Veteran

Joined: 24 May 2006
Posts: 1919

Posted: Wed Feb 14, 2007 5:36 am Post subject:

Not meaning to sound idiotic or anything but, whats the point of this script? The output looks nice, but isn't that what eix is for, or is this an emerge --search/desc replacement? _________________ The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Wed Feb 14, 2007 5:46 am Post subject:

Yes, but it also searches the metadata.xml file's <longdescription /> for the text, which may associate more terms with a given ebuild, making it easier to search for. Plus, the Perl rexeges makes matching easier for certain cases, and the fact it supports multiple independent search terms in one query makes it easier, at least for me. In addition, it'll match the DESCRIPTION value of previous ebuilds, and show the different values for each version. To be honest, those were my major gripes with Portage's search functionality, but its querying to check if a package installed, along with the version, was very nice. EDIT: Now it highlights the search term within each context found, but this option can be turned off. _________________ [ Kawa-kun, new and improved!! ] Alex Libman seems to be more of an anarchist than a libertarian.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Sun Feb 18, 2007 7:43 pm Post subject:

Well, a recent bit of work has added support for multiple overlays, a configuration file, and use of an SQLite backend, which allows it to complete searches within a few seconds. For those who would like to use the old, slower search, the classic_search subroutine is defined. It still uses the database, but doesn't use queries to search. The database is updated with the command portfind-sql --update. Here's an example configuration, located at /usr/portfind.conf:

Code:

Highlight: yes
CaseInsensitive: yes
Arch: amd64
Portdir: gentoo /usr/portage
Portdir: xeffects /usr/portage/local/layman/xeffects
SQLFile: /usr/portfind.dbl

This is the code:

Code:

#!/usr/bin/perl
use strict;
use File::Basename;
use XML::DOM;
use Term::ANSIColor;
use Digest::MD5 qw(md5 md5_hex);
use MIME::Base64;
use Encode;
use DBI;
use Switch;

our $searchall;
our $colorreplace;
our $insensitive;
our $arch;
our %portdirs;
our $dbfile;
our $newdb;
our @searchterms;

if(-e "/etc/portfind.conf") {
open(FILE, "</etc/portfind.conf") || die "Configuration file is unreadable.\n";
my @lines = <FILE>;
chomp(@lines);
close(FILE);
foreach my $line (@lines) {
   if("$line" =~ /^[\s]*[Ss]earch[Aa]ll[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Ss]earch[Aa]ll[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $searchall = 1 }
         else {$searchall = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $colorreplace = 1 }
         else {$colorreplace = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $insensitive = 1 }
         else {$insensitive = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Aa]rch[\:\s]+[A-z0-9\-]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Aa]rch[\:\s]+([A-z0-9\-]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $arch = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+[^\:\s]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $dbfile = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+[^\:\s]+?[\s]*$/) {
      my $key = "$line";
      $key =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+([a-z0-9]+?)[\:\s]+[^\:\s]+?[\s]*$/\1/;
      my $pre = "$line";
      $pre =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $portdirs{$key} = $pre;
   }
}
}
else {
$colorreplace = 1;
$insensitive = 1;
$arch = "amd64";
$dbfile = "/usr/portfind.dbl";
%portdirs = ("gentoo" => "/usr/portage");
# my %portdirs = ("gentoo" => "/usr/portage", "xeffects" => "/usr/portage/local/layman/xeffects");
}
if(scalar @ARGV < 1) {
$searchall = 1;
}
elsif($ARGV[0] eq "--update") {
$newdb = 1;
$searchall = 1;
}
else {
@searchterms = @ARGV;
}
if($newdb && -e "$dbfile") {
unlink("$dbfile");
}
my $dbh = DBI->connect("dbi:SQLite:$dbfile") || die "Unable to open DBFile: $dbfile\n";

if($newdb) {
my @categories = &generate_db;

$dbh->do("CREATE TABLE categories(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(32))");
$dbh->do("CREATE TABLE ebuilds(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(64), catname CHAR(32), longdesc CHAR(2048), longhash CHAR(32))");
$dbh->do("CREATE TABLE shortdescs(id INTEGER PRIMARY KEY AUTOINCREMENT, ename CHAR(64), version CHAR(32), mask CHAR(8), shortdesc CHAR(1024), shorthash CHAR(32))");

my $sth = $dbh->prepare("INSERT INTO ebuilds (name, catname, longdesc, longhash) VALUES(?, ?, ?, ?)");
my $lth = $dbh->prepare("INSERT INTO shortdescs (ename, version, mask, shortdesc, shorthash) VALUES(?, ?, ?, ?, ?)");
foreach my $category (@categories) {
   my $catname = $category->name;
   $dbh->do("INSERT INTO categories (name) VALUES(\'$catname\')");
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $ename = $ebuild->name;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      my %mask = %{$ebuild->mask};
      $longdesc = (($longdesc) ? "$longdesc" : "NULL");
      $sth->execute($ename, $catname, $longdesc, &ascii_md5_hex("$longdesc"));
      foreach my $key (sort keys %shortdesc) {
         $lth->execute($ename, $key, $mask{$key}, $shortdesc{$key}, &ascii_md5_hex("$shortdesc{$key}"));
      }
   }
}
}
else {
if($searchall) {
   my @categories = &extract_from_db($dbh);
   foreach my $category (@categories) {
      foreach my $ebuild (@{$category->ebuilds}) {
         print (&printinfo($ebuild, $category->name));
      }
   }
}
else {
#    my @categories = &extract_from_db($dbh);
#    my @found = &classic_search(@categories);
   my @found = &search_from_db($dbh);
   foreach my $sf (@found) {
      print (&printinfo($sf->ebuild, $sf->category->name));
   }
}
}
$dbh->disconnect;

sub extract_from_db {
my $dbh = shift;
my @categories;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
foreach my $dbcategory (@{$dbcategories}) {
   my $category = new Category;
   $category->name(${$dbcategory}[1]);

   my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'${$dbcategory}[1]\'");
   foreach my $dbebuild (@{$dbebuilds}) {
      my $ebuild = new Ebuild;
      $ebuild->name(${$dbebuild}[1]);
      my $ename = $ebuild->name;
      my $longdesc = ${$dbebuild}[3];
      $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);
      my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
      foreach my $dbshortdesc (@{$dbshortdescs}) {
         my $version = ${$dbshortdesc}[2];
         ${$ebuild->shortdesc}{$version} = ${$dbshortdesc}[4];
         ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
      }
      push(@{$category->ebuilds}, $ebuild);
   }
   push(@categories, $category);
}
return @categories;
}
sub search_from_db {
my $dbh = shift;
my @found;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
foreach my $dbcategory (@{$dbcategories}) {
   my $category = new Category;
   $category->name(${$dbcategory}[1]);
   foreach my $term (@searchterms) {
      my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'${$dbcategory}[1]\' AND (name GLOB \'\*$term\*\' OR longdesc GLOB \'\*$term\*\')");
      foreach my $dbebuild (@{$dbebuilds}) {
         my $ebuild = new Ebuild;
         $ebuild->name(${$dbebuild}[1]);
         my $ename = ${$dbebuild}[1];
         if($colorreplace) {
            my @captured;
            if($insensitive) {
               @captured = "$ename" =~ /$term/ig;
            }
            else {
               @captured = "$ename" =~ /$term/g;
            }
            foreach my $found (@captured) {
               my $colored = colored("$found", 'cyan');
               my $filterfound = "$found";
               my $uncolor = color 'reset';
               my $recolor = color 'bold blue';
               $filterfound =~ s/(\W)/\\\1/g;
               $ename =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
            }
         }
         $ebuild->name($ename);
         my $longdesc = ${$dbebuild}[3];
         if($longdesc ne "NULL") {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$longdesc" =~ /$term/ig;
               }
               else {
                  @captured = "$longdesc" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  $filterfound =~ s/(\W)/\\\1/g;
                  $longdesc =~ s/$filterfound/$colored/g;
               }
            }
         }
         else {
            $longdesc = undef;
         }
         $ebuild->longdesc($longdesc);
         my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'${$dbebuild}[1]\'");
         foreach my $dbshortdesc (@{$dbshortdescs}) {
            my $version = ${$dbshortdesc}[2];
            my $shortdesc = ${$dbshortdesc}[4];
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$shortdesc" =~ /$term/ig;
               }
               else {
                  @captured = "$shortdesc" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  $filterfound =~ s/(\W)/\\\1/g;
                  $shortdesc =~ s/$filterfound/$colored/g;
               }
            }
            ${$ebuild->shortdesc}{$version} = $shortdesc;
            ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
         }
         my $enc = new Encapsulated;
         $enc->ebuild($ebuild);
         $enc->category($category);
         push(@found, $enc);
      }
      my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE shortdesc GLOB \'\*$term\*\'");
      foreach my $dbshortdesc(@{$dbshortdescs}) {
         my $ename = ${$dbshortdesc}[1];
#          my $version = ${$dbshortdesc}[2];
#          my $mask = ${$dbshortdesc}[3];
#          my $shortdesc = ${$dbshortdesc}[4];
         if(&findebuild_encapsulated($ename, @found) == -1) {
            my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'${$dbcategory}[1]\' AND name = \'$ename\'");
            my $longdesc = ${${$dbebuilds}[0]}[3];
            my $ebuild = new Ebuild;
            $ebuild->name($ename);
            $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);

            my $dbebuild_shortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
            foreach my $dbebuild_shortdesc (@{$dbebuild_shortdescs}) {
               my $version = ${$dbshortdesc}[2];
               my $shortdesc = ${$dbshortdesc}[4];
               if($colorreplace) {
                  my @captured;
                  if($insensitive) {
                     @captured = "$shortdesc" =~ /$term/ig;
                  }
                  else {
                     @captured = "$shortdesc" =~ /$term/g;
                  }
                  foreach my $found (@captured) {
                     my $colored = colored("$found", 'cyan');
                     my $filterfound = "$found";
                     $filterfound =~ s/(\W)/\\\1/g;
                     $shortdesc =~ s/$filterfound/$colored/g;
                  }
               }
               ${$ebuild->shortdesc}{$version} = $shortdesc;
               ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
            }
            my $enc = new Encapsulated;
            $enc->category($category);
            $enc->ebuild($ebuild);
            push(@found, $enc);
         }
      }
   }
}
return @found;
}
sub generate_db {
my @categories;
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      foreach my $category (<$portdir/*-*>) {
         if(-d "$category" && !(&findcategory(@categories, basename("$category")))) {
            my $cat = new Category;
            $cat->name(basename("$category"));
            push(@categories, $cat);
         }
      }
      foreach my $category (@categories) {
         my $catname = $category->name;
         foreach my $subdir (<$portdir/$catname/*>) {
            if(-d "$subdir" && scalar <$subdir/*.ebuild>) {
               my $ebuild = undef;
               my $found = &findebuild(@{$category->ebuilds}, basename("$subdir"));
               if($found > -1) {
                  $ebuild = ${$category->ebuilds}[$found];
               }
               else {
                  $ebuild = new Ebuild;
               }
               $ebuild->name(basename("$subdir"));
               foreach my $file (<$subdir/*>) {
                  if(! -d "$file") {
                     push(@{$ebuild->files}, basename("$file"));
                  }
               }
               if($found > -1) {
                  ${$category->ebuilds}[$found] = $ebuild;
               }
               else{
                  push(@{$category->ebuilds}, $ebuild);
               }
            }
         }
      }
   }
}
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      foreach my $category (@categories) {
         my $catname = $category->name;
         foreach my $ebuild (@{$category->ebuilds}) {
            my $ename = $ebuild->name;
            if(-d "$portdir/$catname/$ename") {
               foreach my $file (@{$ebuild->files}) {
                  if("$file" eq "metadata.xml" && -e "$portdir/$catname/$ename/$file") {
                     my $parser = new XML::DOM::Parser;
                     my $xml = $parser->parsefile("<$portdir/$catname/$ename/$file");
                     my $nodes = $xml->getElementsByTagName("longdescription");
                     my $n = $nodes->getLength;
                     for(my $i = 0; $i < $n; $i++) {
                        my $node = $nodes->item($i);
                        my $lang = $node->getAttributeNode("lang");
                        if($n == 1 || ($lang && $lang->getValue eq "en")) {
                           my $children = $node->getChildNodes;
                           my $nc = $children->getLength;
                           for(my $c = 0; $c < $nc; $c++) {
                              my $subnode = $children->item($c);
                              if($subnode->getNodeType == TEXT_NODE) {
                                 $ebuild->longdesc($subnode->getNodeValue);
                              }
                           }
                        }
                     }
                     $xml->dispose;
                  }
                  if("$file" =~ /\.ebuild$/) {
                     my $version = "$file";
                     my $tmpename = "$ename";
                     $tmpename =~ s/\+/\\\+/g;
                     $version =~ s/^$tmpename-([a-z0-9_.-]+)\.ebuild$/\1:$key/;
                     open(FILE, "<$portdir/$catname/$ename/$file");
                     my @lines = <FILE>;
                     chomp(@lines);
                     close(FILE);
                     foreach my $line (@lines) {
                        if("$line" =~ /DESCRIPTION=\".*?\"/) {
                           my $desc = "$line";
                           $desc =~ s/^.*?DESCRIPTION=\"(.*?)\".*?$/\1/;
                           ${$ebuild->shortdesc}{$version} = "$desc";
                        }
                        elsif("$line" =~ /KEYWORDS=\".*?\"/) {
                           my $keywords = "$line";
                           my $keyword = undef;

                           if("$keywords" =~ /[\"\s]\~$arch([\"\s])/) {
                              $keyword = "~";
                           }
                           elsif("$keywords" =~ /[\"\s]$arch([\"\s])/) {
                              $keyword = "+";
                           }
                           else {
                              $keyword = "-";
                           }
                           ${$ebuild->mask}{$version} = "$keyword";
                        }
                     }
                  }
               }
            }
         }
      }
   }
}
return @categories;
}
sub classic_search {
my @categories = @_;
my @found;
foreach my $category (@categories) {
   my $catname = $category->name;
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $match = 0;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      my $ename = $ebuild->name;
      foreach my $term (@searchterms) {
         if("$ename" =~ /$term/ || ("$ename" =~ /$term/i && $insensitive)) {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$ename" =~ /$term/ig;
               }
               else {
                  @captured = "$ename" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  my $uncolor = color 'reset';
                  my $recolor = color 'bold blue';
                  $filterfound =~ s/(\W)/\\\1/g;
                  $ename =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
               }
            }
            $match += 1;
         }
         if("$longdesc" =~ /$term/ || ("$longdesc" =~ /$term/i && $insensitive)) {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$longdesc" =~ /$term/ig;
               }
               else {
                  @captured = "$longdesc" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  $filterfound =~ s/(\W)/\\\1/g;
                  $longdesc =~ s/$filterfound/$colored/g;
               }
            }
            $match += 1;
         }
      }
      $ebuild->name($ename);
      $ebuild->longdesc($longdesc);
      foreach my $version (sort keys %shortdesc) {
         foreach my $term (@searchterms) {
            if("$shortdesc{$version}" =~ /$term/ || ("$shortdesc{$version}" =~ /$term/i && $insensitive)) {
               if($colorreplace) {
                  my @captured;
                  if($insensitive) {
                     @captured = "$shortdesc{$version}" =~ /$term/ig;
                  }
                  else {
                     @captured = "$shortdesc{$version}" =~ /$term/g;
                  }
                  foreach my $found (@captured) {
                     my $colored = colored("$found", 'cyan');
                     my $filterfound = "$found";
                     $filterfound =~ s/(\W)/\\\1/g;
                     $shortdesc{$version} =~ s/$filterfound/$colored/g;
                  }
                  ${$ebuild->shortdesc}{$version} = $shortdesc{$version};
               }
               $match += 1;
            }
         }
      }
      if($match) {
         my $enc = new Encapsulated;
         $enc->category($category);
         $enc->ebuild($ebuild);
         push(@found, $enc);
      }
   }
}
return @found;
}
sub findcategory {
my $catname = pop @_;
my @categories = @_;
foreach my $category (@categories) {
   my $name = $category->name;
   if("$name" eq "$catname") {
      return 1;
   }
}
return 0;
}
sub findebuild {
my $ename = pop @_;
my @ebuilds = @_;
for(my $i = 0; $i < scalar @ebuilds; $i++) {
   my $name = $ebuilds[$i]->name;
   if("$name" eq "$ename") {
      return $i;
   }
}
return -1;
}
sub findebuild_encapsulated {
my $ename = shift;
my @encapsulated = @_;
for(my $i = 0; $i < scalar @encapsulated; $i++) {
   my $enc = $encapsulated[$i];
   my $ebuild = $enc->ebuild;
   if($ename eq $ebuild->name) {
      return $i;
   }
}
return -1;
}

sub printinfo {
my $ebuild = shift;
my $catname = shift;
my $ename = $ebuild->name;
my $longdesc = $ebuild->longdesc;
my %shortdesc = %{$ebuild->shortdesc};
my %mask = %{$ebuild->mask};
my $ret = colored("$catname/$ename\n", 'bold blue');
$ret .= "\t" . colored("Short Description\n", 'bold green');
foreach my $version (reverse sort keys %shortdesc) {
   my $maskcolor = color('reset') . colored("$mask{$version}", 'red') . color('bold yellow');
   $ret .= "\t\t" . colored("$version [$maskcolor]: ", 'bold yellow') . "$shortdesc{$version}\n";
}
if($longdesc && !("$longdesc" =~ /^[\s\t\n]*$/)) {
   my @arraydesc = split(/\n/, "$longdesc");
   chomp(@arraydesc);
   if(!$arraydesc[0]) {
      shift(@arraydesc);
   }
   $ret .= "\t" . colored("Long Description\n", 'bold red');
   foreach my $line (@arraydesc) {
      $line =~ s/^[\s\t]*/\t\t/g;
      $ret .= "$line\n";
   }
}
$ret .= "\n";
return "$ret";
}
sub ascii_md5_hex {
my $text = shift;
$text = encode("ascii", "$text");
return md5_hex("$text");
}
package Category;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub ebuilds {
my $self = shift;
if(!defined $self->{EBUILDS}) {
   $self->{EBUILDS} = [ ];
}
return $self->{EBUILDS};
}
package Encapsulated;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub category {
my $self = shift;
$self->{CATEGORY} = shift if @_;
return $self->{CATEGORY};
}
sub ebuild {
my $self = shift;
$self->{EBUILD} = shift if @_;
return $self->{EBUILD};
}
package Ebuild;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub files {
my $self = shift;
if(!defined $self->{FILES}) {
   $self->{FILES} = [ ];
}
return $self->{FILES};
}
sub longdesc {
my $self = shift;
$self->{LDESC} = shift if @_;
return $self->{LDESC};
}
sub shortdesc {
my $self = shift;
if(!defined $self->{SDESC}) {
   $self->{SDESC} = { };
}
return $self->{SDESC};
}
sub mask {
my $self = shift;
if(!defined $self->{MASK}) {
   $self->{MASK} = { };
}
return $self->{MASK};
}

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

mudrii
l33t
l33t

Joined: 26 Jun 2003
Posts: 789
Location: Singapore

Posted: Mon Feb 19, 2007 4:38 am Post subject:

Sounds very interesting I will try it . Thank you _________________ www.gentoo.ro

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Tue Feb 20, 2007 8:50 pm Post subject:

I've made another update, which alerts to Portage trees which have been updated since a database update, among other fixes fairly critical for searching. Also, this one provides --help and --help-config options.
Update Completed.

Case of update requires, when running an update:

Code:

# portfind-sql --update
/usr/portage has been modified: Please --update
/usr/portage/local/layman/xeffects has been modified: Please --update
Creating new DB at /usr/portfind.dbl

Help:

Code:

# portfind-sql --help
Usage: portfind-sql [ option | search terms ]
NOTE: Any options will override the search.
Options:
--help: Display this message
--update: Update database
--help-config: Configuration help

Configuration help:

Code:

portfind-sql Configuration:
Configuration file lies at /etc/portfind.conf: Found
Any line began with a # is a comment.
Lines are written a leading config option, and separators are : and/or spaces, ending with a newline.
E.G.: "CaseInsensitive: Yes"
SearchMode: DB or Old: Toggles whether to use SQLite Search or Search through tree from database
Currently: DB
CaseInsensitive: Yes or No: Toggles whether to search without case sensitivity.
Does not affect DB search other than highlighting, but will affect Old search
Currently: Yes
Highlight: Yes or No: Toggles whether to highlight search terms in found text.
Currently: Yes
MonitorUpdates: Yes or No: Toggles whether updates are monitored via access times.
Currently: No
Arch: The architecture as put by Portage of your current systems.
Currently: amd64
SQLFile: The location of the SQLite database
Currently: /usr/portfind.dbl: Found
PortDir: The Portage trees, including overlays and the main tree
May be specified multiple times for multiple trees.
Currently: gentoo -> /usr/portage: Found
Currently: xeffects -> /usr/portage/local/layman/xeffects: Found

Enjoy:

Code:

Highlight: yes
CaseInsensitive: yes
Arch: amd64
SeachMode: DB
Portdir: gentoo /usr/portage
Portdir: xeffects /usr/portage/local/layman/xeffects
SQLFile: /usr/portfind.dbl
MonitorUpdates: no

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Wed Feb 21, 2007 2:00 am Post subject:

Here's a major update: argument handling:

Code:

# portfind-sql --help
Usage: portfind-sql [ option | search terms ]
Options:
--help: Display this message.
--update: Update database.
--help-config: Configuration help.
The following options can be given the --no beginning. E.G.: --no-insensitive
--insensitive: Toggles case sensitivity.
--highlight: Toggles highlighting of search terms in found text.
--monitor-updates: Toggles Portage tree update monitoring by access time.
--search-db/--search-old: Toggles search mode.

Code:

# portfind-sql --help-config
portfind-sql Configuration:
Configuration file lies at /etc/portfind.conf: Found
Any line began with a # is a comment.
Lines are written a leading config option, and separators are : and/or spaces, ending with a newline.
E.G.: "CaseInsensitive: Yes"
SearchMode: DB or Old: Toggles whether to use SQLite Search or Search through tree from database
Currently: DB
CaseInsensitive: Yes or No: Toggles whether to search without case sensitivity.
Currently: Yes
Highlight: Yes or No: Toggles whether to highlight search terms in found text.
Currently: Yes
MonitorUpdates: Yes or No: Toggles whether updates are monitored via access times.
Currently: No
Arch: The architecture as put by Portage of your current systems.
Currently: amd64
SQLFile: The location of the SQLite database
Currently: /usr/portfind.dbl: Found
PortDir: The Portage trees, including overlays and the main tree
May be specified multiple times for multiple trees.
Currently: gentoo -> /usr/portage: Found
Currently: xeffects -> /usr/portage/local/layman/xeffects: Found

Otherwise, the options are the same:

Code:

#!/usr/bin/perl
use strict;
use File::Basename;
use XML::DOM;
use Term::ANSIColor;
use Digest::MD5 qw(md5 md5_hex);
use MIME::Base64;
use Encode;
use DBI;
use Switch;

our $searchall;
our $colorreplace = -1;
our $insensitive = -1;
our $arch = undef;
our %portdirs;
our %modtimes;
our $dbfile = undef;
our $findmode = -1;
our $monitorupdates = -1;
our $newdb;
our @searchterms;

if(-e "/etc/portfind.conf") {
open(FILE, "</etc/portfind.conf") || die "Configuration file is unreadable.\n";
my @lines = <FILE>;
chomp(@lines);
close(FILE);
foreach my $line (@lines) {
   if("$line" =~ /^[\s]*\#/) {
   }
   elsif("$line" =~ /^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $colorreplace = 1 }
         else {$colorreplace = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Ss]each[Mm]ode[\:\s]+([Oo]ld|[Dd][Bb])[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Ss]each[Mm]ode[\:\s]+([Oo]ld|[Dd][Bb])[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "old" {$findmode = 0 }
         case "db" {$findmode = 1 }
      }
   }
   elsif("$line" =~ /^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $insensitive = 1 }
         else {$insensitive = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Mm]onitor[Uu]pdates[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre = lc($pre);
      $pre =~ s/^[\s]*[Mm]onitor[Uu]pdates[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      switch($pre) {
         case "yes" { $monitorupdates = 1 }
         else { $monitorupdates = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Aa]rch[\:\s]+[A-z0-9\-]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Aa]rch[\:\s]+([A-z0-9\-]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $arch = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+[^\:\s]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $dbfile = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+[^\:\s]+?[\s]*$/) {
      my $key = "$line";
      $key =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+([a-z0-9]+?)[\:\s]+[^\:\s]+?[\s]*$/\1/;
      my $pre = "$line";
      $pre =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $portdirs{$key} = $pre;
   }
}
}
if($colorreplace == -1) {
$colorreplace = 1;
}
if($insensitive == -1) {
$insensitive = 1;
}
if($findmode == -1) {
$findmode = 1;
}
if($monitorupdates == -1) {
$findmode = 0;
}
if(!$arch) {
$arch = "x86";
}
if(!$dbfile) {
$dbfile = "/usr/portfind.dbl";
}
if(!scalar %portdirs) {
%portdirs = ("gentoo" => "/usr/portage");
}
my @opts;
foreach my $ARGV (@ARGV) {
if($ARGV =~ /^\-\-[^-]+.*$/) {
   push(@opts, $ARGV);
}
}
if(grep(/^\-\-insensitive$/, @opts)) {
$insensitive = 1;
}
elsif(grep(/^\-\-no\-insensitive$/, @opts)) {
$insensitive = 0;
}
if(grep(/^\-\-highlight$/, @opts)) {
$colorreplace = 1;
}
elsif(grep(/^\-\-no\-highlight$/, @opts)) {
$colorreplace = 0;
}
if(grep(/^\-\-monitor\-updates$/, @opts)) {
$monitorupdates = 1;
}
elsif(grep(/^\-\-no\-monitor\-updates$/, @opts)) {
$monitorupdates = 0;
}
if(grep(/^\-\-search\-db$/, @opts)) {
$findmode = 1;
}
elsif(grep(/^\-\-search\-old$/, @opts)) {
$findmode = 0;
}

if(-e "$dbfile" && $monitorupdates) {
my $dbh = DBI->connect("dbi:SQLite:$dbfile") || die "Unable to open DBFile: $dbfile\n";
foreach my $key (%portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      my $modtime = -A "$portdir";
      my $dbmodtime;
      my $dbdata = $dbh->selectall_arrayref("SELECT * FROM modtimes WHERE dir = \'$portdir\'");
      foreach my $dbdat (@{$dbdata}) {
         $dbmodtime = ${$dbdat}[2];
      }
      if(abs($dbmodtime - $modtime) >= 0.02) {
         print colored("$portdir has been modified", 'bold red') . ": Please " . colored("--update", 'bold yellow') . "\n";
         $modtimes{$key} = 1;
      }
      elsif(!$dbmodtime) {
         print colored("${portdir}'s modification time hasn't been recorded", 'bold red') . ": Please " . colored("--update", 'bold yellow') . "\n";
      }
      else {
         $modtimes{$key} = 0;
      }
   }
}
$dbh->disconnect;
}

if(scalar @ARGV < 1) {
$searchall = 1;
}
elsif(grep(/^\-\-update$/, @opts)) {
$newdb = 1;
$searchall = 1;
}
elsif(grep(/^\-\-help$/, @opts)) {
print "Usage: " . basename($0) . " " . colored("\[", 'bold') . " " . colored("option", 'bold green') . " " . colored("\|", 'bold') . " search terms " . colored("\]", 'bold') . "\n",
"\t" . colored("Options:\n", 'bold'),
"\t\t" . colored("--help", 'bold green') . ": Display this message.\n",
"\t\t" . colored("--update", 'bold yellow') . ": Update database.\n",
"\t\t" . colored("--help-config", 'bold red') . ": Configuration help.\n",
"\tThe following options can be given the " . colored("--no", 'bold') . " beginning. E.G.: --no-insensitive\n",
"\t\t" . colored("--insensitive", 'bold cyan') . ": Toggles case sensitivity.\n",
"\t\t" . colored("--highlight", 'bold cyan') . ": Toggles highlighting of search terms in found text.\n",
"\t\t" . colored("--monitor-updates", 'bold cyan') . ": Toggles Portage tree update monitoring by access time.\n",
"\t\t" . colored("--search-db", 'bold cyan') . "/" . colored("--search-old", 'bold cyan') . ": Toggles search mode.\n";
exit;
}
elsif(grep(/\-\-help-config/, @opts)) {
print colored(basename($0) . " Configuration:", 'bold') . "\n",
"Configuration file lies at " . colored("/etc/portfind.conf", 'bold blue') . ": " . ((-e "/etc/portfind.conf") ? colored("Found", 'bold green') : colored("Not found", 'bold red')) . "\n",
"\tAny line began with a \# is a comment.\n",
"\tLines are written a leading config option, and separators are \: and/or spaces, ending with a newline.\n",
"\t\tE.G.: \"CaseInsensitive\: Yes\"\n",
"\t" . colored("SearchMode", 'bold cyan') . ": " . colored("DB", 'bold green') . " or " . colored("Old", 'bold red') . ": Toggles whether to use SQLite Search or Search through tree from database\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . (($findmode) ? colored("DB", 'bold green') : colored("Old", 'bold red')) . "\n",
"\t" . colored("CaseInsensitive", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether to search without case sensitivity.\n",
# "\t\tDoes not affect " . colored("DB", 'bold green') . " search other than highlighting, but will affect " . colored("Old", 'bold red') . " search\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . (($insensitive) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n",
"\t" . colored("Highlight", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether to highlight search terms in found text.\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . (($colorreplace) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n",
"\t" . colored("MonitorUpdates", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether updates are monitored via access times.\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . (($monitorupdates) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n",
"\t" . colored("Arch", 'bold cyan') . ": The architecture as put by Portage of your current systems.\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . colored($arch, 'bold green') . "\n",
"\t" . colored("SQLFile", 'bold cyan') . ": The location of the SQLite database\n",
"\t\t" . colored("Currently", 'bold yellow') . ": " . colored($dbfile, 'bold green') . ": " . ((-e $dbfile) ? colored("Found", 'bold green') : colored("Not Found, please ", 'bold red') . colored("--update", 'bold yellow')) . "\n",
"\t" . colored("PortDir", 'bold cyan') . ": The Portage trees, including overlays and the main tree\n",
"\t\tMay be specified multiple times for multiple trees.\n";
if($monitorupdates) {
   foreach my $key (keys %portdirs) {
      print "\t\t" . colored("Currently", 'bold yellow') . ": " . colored("$key", 'green') . " -> " . colored("$portdirs{$key}", 'bold green') . ": " . ((-e $portdirs{$key}) ? colored("Found", 'bold green') . (($modtimes{$key}) ? "/" . colored("Update", 'bold red') : "") : colored("Not found", 'bold red')) . "\n";
   }
}
else {
   foreach my $key (keys %portdirs) {
      print "\t\t" . colored("Currently", 'bold yellow') . ": " . colored("$key", 'green') . " -> " . colored("$portdirs{$key}", 'bold green') . ": " . ((-e $portdirs{$key}) ? colored("Found", 'bold green') : colored("Not found", 'bold red')) . "\n";
   }
}
exit;
}
else {
$searchall = 0;
foreach my $ARG (@ARGV) {
   if($ARGV !=~ /^--[^-]+.*$/) {
      push(@searchterms, $ARGV);
   }
}
exit;
}
if($newdb && -e "$dbfile") {
unlink("$dbfile");
}
my $dbh = DBI->connect("dbi:SQLite:$dbfile") || die "Unable to open DBFile: $dbfile\n";

if($newdb) {
print "Creating new DB at $dbfile\n";
my @categories = &generate_db;

$dbh->do("CREATE TABLE categories(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(32))");
$dbh->do("CREATE TABLE ebuilds(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(64), catname CHAR(32), longdesc CHAR(2048), longhash CHAR(32))");
$dbh->do("CREATE TABLE shortdescs(id INTEGER PRIMARY KEY AUTOINCREMENT, ename CHAR(64), version CHAR(32), mask CHAR(8), shortdesc CHAR(1024), shorthash CHAR(32))");
$dbh->do("CREATE TABLE modtimes(id INTEGER PRIMARY KEY AUTOINCREMENT, dir CHAR(512), modtime REAL)");

foreach my $key (%portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      my $modtime = -A "$portdir";
      $dbh->do("INSERT INTO modtimes (dir, modtime) VALUES(\'$portdir\', $modtime)");
   }
}

my $sth = $dbh->prepare("INSERT INTO ebuilds (name, catname, longdesc, longhash) VALUES(?, ?, ?, ?)");
my $lth = $dbh->prepare("INSERT INTO shortdescs (ename, version, mask, shortdesc, shorthash) VALUES(?, ?, ?, ?, ?)");
foreach my $category (@categories) {
   my $catname = $category->name;
   $dbh->do("INSERT INTO categories (name) VALUES(\'$catname\')");
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $ename = $ebuild->name;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      my %mask = %{$ebuild->mask};
      $longdesc = (($longdesc) ? "$longdesc" : "NULL");
      $sth->execute($ename, $catname, $longdesc, &ascii_md5_hex("$longdesc"));
      foreach my $key (sort keys %shortdesc) {
         $lth->execute($ename, $key, $mask{$key}, $shortdesc{$key}, &ascii_md5_hex("$shortdesc{$key}"));
      }
   }
}
}
else {
if($searchall) {
   my @categories = &extract_from_db($dbh);
   foreach my $category (@categories) {
      foreach my $ebuild (@{$category->ebuilds}) {
         print (&printinfo($ebuild, $category->name));
      }
   }
}
else {
   my @found;
   if(!$findmode) {
      my @categories = &extract_from_db($dbh);
      @found = &classic_search(@categories);
   }
   else {
      @found = &search_from_db($dbh);
   }
   foreach my $sf (@found) {
      print (&printinfo($sf->ebuild, $sf->category->name));
   }
}
}
$dbh->disconnect;

sub extract_from_db {
my $dbh = shift;
my @categories;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
foreach my $dbcategory (@{$dbcategories}) {
   my $category = new Category;
   $category->name(${$dbcategory}[1]);

   my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'${$dbcategory}[1]\'");
   foreach my $dbebuild (@{$dbebuilds}) {
      my $ebuild = new Ebuild;
      $ebuild->name(${$dbebuild}[1]);
      my $ename = $ebuild->name;
      my $longdesc = ${$dbebuild}[3];
      $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);
      my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
      foreach my $dbshortdesc (@{$dbshortdescs}) {
         my $version = ${$dbshortdesc}[2];
         ${$ebuild->shortdesc}{$version} = ${$dbshortdesc}[4];
         ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
      }
      push(@{$category->ebuilds}, $ebuild);
   }
   push(@categories, $category);
}
return @categories;
}
sub search_from_db {
my $dbh = shift;
my @found;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
foreach my $dbcategory (@{$dbcategories}) {
   my $category = new Category;
   $category->name(${$dbcategory}[1]);
   my $catname = ${$dbcategory}[1];
   foreach my $term (@searchterms) {
      my $dbebuilds;
      if($insensitive) {
         $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND (name LIKE \'\%$term\%\' OR longdesc LIKE \'\%$term\%\')");
      }
      else {
         $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND (name GLOB \'\*$term\*\' OR longdesc GLOB \'\*$term\*\')");
      }
      foreach my $dbebuild (@{$dbebuilds}) {
         my $ebuild = new Ebuild;
         $ebuild->name(${$dbebuild}[1]);
         my $ename = ${$dbebuild}[1];
         $ebuild->name($ename);
         my $longdesc = ${$dbebuild}[3];
         if($longdesc ne "NULL") {
         }
         else {
            $longdesc = undef;
         }
         $ebuild->longdesc($longdesc);

         my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'${$dbebuild}[1]\'");
         foreach my $dbshortdesc (@{$dbshortdescs}) {
            my $version = ${$dbshortdesc}[2];
            my $shortdesc = ${$dbshortdesc}[4];
            ${$ebuild->shortdesc}{$version} = $shortdesc;
            ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
         }
         my $enc = new Encapsulated;
         $enc->ebuild($ebuild);
         $enc->category($category);
         push(@found, $enc);
      }

      my $dbshortdescs;
      if($insensitive) {
         $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE shortdesc LIKE \'\%$term\%\'");
      }
      else {
         $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE shortdesc GLOB \'\*$term\*\'");
      }
      foreach my $dbshortdesc(@{$dbshortdescs}) {
         my $ename = ${$dbshortdesc}[1];
         if(&findebuild_encapsulated("$catname/$ename", 0, @found) == -1) {
            my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND name = \'$ename\'");
            my $dbename = undef;
            my $longdesc = undef;
            foreach my $dbebuild (@{$dbebuilds}) {
               if($ename eq ${$dbebuild}[1] && $catname eq ${$dbebuild}[2]) {
                  $dbename = ${$dbebuild}[1];
                  $longdesc = ${$dbebuild}[3];
               }
            }
            if($dbename) {
               my $ebuild = new Ebuild;
               $ebuild->name($ename);
               $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);

               my $dbebuild_shortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
               foreach my $dbebuild_shortdesc (@{$dbebuild_shortdescs}) {
                  my $version = ${$dbshortdesc}[2];
                  my $shortdesc = ${$dbshortdesc}[4];
                  ${$ebuild->shortdesc}{$version} = $shortdesc;
                  ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
               }
               my $enc = new Encapsulated;
               $enc->category($category);
               $enc->ebuild($ebuild);
               push(@found, $enc);
            }
         }
      }
   }
}
return @found;
}
sub generate_db {
my @categories;
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      foreach my $category (<$portdir/*-*>) {
         if(-d "$category" && !(&findcategory(@categories, basename("$category")))) {
            my $cat = new Category;
            $cat->name(basename("$category"));
            push(@categories, $cat);
         }
      }
      foreach my $category (@categories) {
         my $catname = $category->name;
         foreach my $subdir (<$portdir/$catname/*>) {
            my @tfiles = <$subdir/*>; # TEMPORARY TO FIND EBUILDS
            if(-d "$subdir" && scalar @tfiles) {
               my $ebuild = undef;
               my $found = &findebuild(@{$category->ebuilds}, basename("$subdir"));
               if($found > -1) {
                  $ebuild = ${$category->ebuilds}[$found];
               }
               else {
                  $ebuild = new Ebuild;
               }
               $ebuild->name(basename("$subdir"));
               foreach my $file (<$subdir/*>) {
                  if(! -d "$file") {
                     push(@{$ebuild->files}, basename("$file"));
                  }
               }
               if($found > -1) {
                  ${$category->ebuilds}[$found] = $ebuild;
               }
               else{
                  push(@{$category->ebuilds}, $ebuild);
               }
            }
         }
      }
   }
}
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      foreach my $category (@categories) {
         my $catname = $category->name;
         foreach my $ebuild (@{$category->ebuilds}) {
            my $ename = $ebuild->name;
            if(-d "$portdir/$catname/$ename") {
               foreach my $file (@{$ebuild->files}) {
                  if("$file" eq "metadata.xml" && -e "$portdir/$catname/$ename/$file") {
                     my $parser = new XML::DOM::Parser;
                     my $xml = $parser->parsefile("<$portdir/$catname/$ename/$file");
                     my $nodes = $xml->getElementsByTagName("longdescription");
                     my $n = $nodes->getLength;
                     for(my $i = 0; $i < $n; $i++) {
                        my $node = $nodes->item($i);
                        my $lang = $node->getAttributeNode("lang");
                        if($n == 1 || ($lang && $lang->getValue eq "en")) {
                           my $children = $node->getChildNodes;
                           my $nc = $children->getLength;
                           for(my $c = 0; $c < $nc; $c++) {
                              my $subnode = $children->item($c);
                              if($subnode->getNodeType == TEXT_NODE) {
                                 $ebuild->longdesc($subnode->getNodeValue);
                              }
                           }
                        }
                     }
                     $xml->dispose;
                  }
                  if("$file" =~ /\.ebuild$/) {
                     my $version = "$file";
                     my $tmpename = "$ename";
                     $tmpename =~ s/\+/\\\+/g;
                     $version =~ s/^$tmpename-([a-z0-9_.-]+)\.ebuild$/\1:$key/;
                     open(FILE, "<$portdir/$catname/$ename/$file");
                     my @lines = <FILE>;
                     chomp(@lines);
                     close(FILE);
                     foreach my $line (@lines) {
                        if("$line" =~ /DESCRIPTION=\".*?\"/) {
                           my $desc = "$line";
                           $desc =~ s/^.*?DESCRIPTION=\"(.*?)\".*?$/\1/;
                           ${$ebuild->shortdesc}{$version} = "$desc";
                        }
                        elsif("$line" =~ /KEYWORDS=\".*?\"/) {
                           my $keywords = "$line";
                           my $keyword = undef;

                           if("$keywords" =~ /[\"\s]\~$arch([\"\s])/) {
                              $keyword = "~";
                           }
                           elsif("$keywords" =~ /[\"\s]$arch([\"\s])/) {
                              $keyword = "+";
                           }
                           else {
                              $keyword = "-";
                           }
                           ${$ebuild->mask}{$version} = "$keyword";
                        }
                     }
                  }
               }
            }
         }
      }
   }
}
return @categories;
}
sub classic_search {
my @categories = @_;
my @found;
foreach my $category (@categories) {
   my $catname = $category->name;
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $match = 0;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      my $ename = $ebuild->name;
      foreach my $term (@searchterms) {
         if("$ename" =~ /$term/ || ("$ename" =~ /$term/i && $insensitive)) {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$ename" =~ /$term/ig;
               }
               else {
                  @captured = "$ename" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  my $uncolor = color 'reset';
                  my $recolor = color 'bold blue';
                  $filterfound =~ s/(\W)/\\\1/g;
                  $ename =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
               }
            }
            $match += 1;
         }
         if("$longdesc" =~ /$term/ || ("$longdesc" =~ /$term/i && $insensitive)) {
            if($colorreplace) {
               my @captured;
               if($insensitive) {
                  @captured = "$longdesc" =~ /$term/ig;
               }
               else {
                  @captured = "$longdesc" =~ /$term/g;
               }
               foreach my $found (@captured) {
                  my $colored = colored("$found", 'cyan');
                  my $filterfound = "$found";
                  $filterfound =~ s/(\W)/\\\1/g;
                  $longdesc =~ s/$filterfound/$colored/g;
               }
            }
            $match += 1;
         }
      }
      $ebuild->name($ename);
      $ebuild->longdesc($longdesc);
      foreach my $version (sort keys %shortdesc) {
         foreach my $term (@searchterms) {
            if("$shortdesc{$version}" =~ /$term/ || ("$shortdesc{$version}" =~ /$term/i && $insensitive)) {
               if($colorreplace) {
                  my @captured;
                  if($insensitive) {
                     @captured = "$shortdesc{$version}" =~ /$term/ig;
                  }
                  else {
                     @captured = "$shortdesc{$version}" =~ /$term/g;
                  }
                  foreach my $found (@captured) {
                     my $colored = colored("$found", 'cyan');
                     my $filterfound = "$found";
                     $filterfound =~ s/(\W)/\\\1/g;
                     $shortdesc{$version} =~ s/$filterfound/$colored/g;
                  }
                  ${$ebuild->shortdesc}{$version} = $shortdesc{$version};
               }
               $match += 1;
            }
         }
      }
      if($match) {
         my $enc = new Encapsulated;
         $enc->category($category);
         $enc->ebuild($ebuild);
         push(@found, $enc);
      }
   }
}
return @found;
}
sub findcategory {
my $catname = pop @_;
my @categories = @_;
foreach my $category (@categories) {
   my $name = $category->name;
   if("$name" eq "$catname") {
      return 1;
   }
}
return 0;
}
sub findebuild {
my $ename = pop @_;
my @ebuilds = @_;
for(my $i = 0; $i < scalar @ebuilds; $i++) {
   my $name = $ebuilds[$i]->name;
   if("$name" eq "$ename") {
      return $i;
   }
}
return -1;
}
sub findebuild_encapsulated {
my $ename = shift;
my $i0 = shift;
my @encapsulated = @_;
for(my $i = $i0; $i < scalar @encapsulated; $i++) {
   my $enc = $encapsulated[$i];
   my $category = $enc->category;
   my $catname = $category->name;

   my $ebuild = $enc->ebuild;
   my $cename = $ebuild->name;
   if($ename eq "$catname/$cename") {
      return $i;
   }
}
return -1;
}
sub highlight {
my $term = shift;
my $text = shift;
my $colors = shift;
my $recolor = color shift;
my $uncolor = color 'reset';
my @captured;
if($insensitive) {
   @captured = "$text" =~ /$term/ig;
}
else {
   @captured = "$text" =~ /$term/g;
}
foreach my $found (@captured) {
   my $colored = colored("$found", $colors);
   my $filterfound = "$found";
   $filterfound =~ s/(\W)/\\\1/g;
   $text =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
}
return $text;
}

sub printinfo {
my $ebuild = shift;
my $catname = shift;
my $ename = $ebuild->name;
my $longdesc = $ebuild->longdesc;
my %shortdesc = %{$ebuild->shortdesc};
my %mask = %{$ebuild->mask};
if($colorreplace) {
   foreach my $term (@searchterms) {
      $ename = &highlight($term, $ename, 'cyan', 'bold blue');
      $longdesc = &highlight($term, $longdesc, 'cyan', 'reset');
      foreach my $version (keys %shortdesc) {
         $shortdesc{$version} = &highlight($term, $shortdesc{$version}, 'cyan', 'reset');
      }
   }
}

my $ret = colored("$catname/$ename\n", 'bold blue');
$ret .= "\t" . colored("Short Description\n", 'bold green');
foreach my $version (reverse sort keys %shortdesc) {
   my $maskcolor = color('reset') . colored("$mask{$version}", 'red') . color('bold yellow');
   $ret .= "\t\t" . colored("$version [$maskcolor]: ", 'bold yellow') . "$shortdesc{$version}\n";
}
if($longdesc && !("$longdesc" =~ /^[\s\t\n]*$/)) {
   my @arraydesc = split(/\n/, "$longdesc");
   chomp(@arraydesc);
   if(!$arraydesc[0]) {
      shift(@arraydesc);
   }
   $ret .= "\t" . colored("Long Description\n", 'bold red');
   foreach my $line (@arraydesc) {
      $line =~ s/^[\s\t]*/\t\t/g;
      $ret .= "$line\n";
   }
}
$ret .= "\n";
return "$ret";
}
sub ascii_md5_hex {
my $text = shift;
$text = encode("ascii", "$text");
return md5_hex("$text");
}
sub round {
my $number = shift;
return int($number + .5 * ($number <=> 0));
}
package Category;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub ebuilds {
my $self = shift;
if(!defined $self->{EBUILDS}) {
   $self->{EBUILDS} = [ ];
}
return $self->{EBUILDS};
}
package Encapsulated;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub category {
my $self = shift;
$self->{CATEGORY} = shift if @_;
return $self->{CATEGORY};
}
sub ebuild {
my $self = shift;
$self->{EBUILD} = shift if @_;
return $self->{EBUILD};
}
package Ebuild;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub files {
my $self = shift;
if(!defined $self->{FILES}) {
   $self->{FILES} = [ ];
}
return $self->{FILES};
}
sub longdesc {
my $self = shift;
$self->{LDESC} = shift if @_;
return $self->{LDESC};
}
sub shortdesc {
my $self = shift;
if(!defined $self->{SDESC}) {
   $self->{SDESC} = { };
}
return $self->{SDESC};
}
sub mask {
my $self = shift;
if(!defined $self->{MASK}) {
   $self->{MASK} = { };
}
return $self->{MASK};
}

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Wed Feb 21, 2007 7:04 am Post subject:

Here's an interesting note:

Code:

# time portfind-sql --no-insensitive myth
...
real 0m12.773s
user 0m8.942s
sys 0m3.101s

Code:

# time portfind-sql --insensitive myth
...
real 0m56.250s
user 0m37.810s
sys 0m13.482s

A case-sensitive search is far faster.
_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

msalerno
Veteran
Veteran

Joined: 17 Dec 2002
Posts: 1338
Location: Sweating in South Florida

Posted: Thu Feb 22, 2007 5:25 am Post subject:

That will always be the case. But since you have the term, you could just copy it to another variable, set everything to lowercase and then when you have the results, you could just replace the lowercase term with the original term.

I didn't run your script, and I didn't test my code, but regardless, I'm sure you get the idea.

Code:

sub highlight {
my $term = shift;
my $text = shift;
my $colors = shift;
my $recolor = color shift;
my $uncolor = color 'reset';
my $originalterm = $term;
@captured = lc($text) =~ /\L$term\E/g;

foreach my $found (@captured) {
   $found =~ s/\L$term\E/$originalterm/g
   my $colored = colored("$found", $colors);
   (my $filterfound = $found) =~ s/(\W)/\\\1/g;
   $text =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
}
return $text;
}

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Thu Feb 22, 2007 8:29 pm Post subject:

The slowness is attributed to not the regex insensitive search, but using the SQLite LIKE command, which is far slower than GLOB.

I've redone some of the code, to add rudimentary USE flag support, in addition to progress indicators:

Code:

# time portfind-usql cow
...
real 0m9.637s
user 0m7.082s
sys 0m2.373s
# time portfind-usql --insensitive cow
...
real 0m10.177s
user 0m7.513s
sys 0m2.324s

Needless to say, it is much faster. I'm also working on a more advanced ebuild parser written in C and using an associative array struct/object I wrote.

Here's the new portfind-usql code:

Code:

#!/usr/bin/perl
use strict;
use File::Basename;
use XML::DOM;
use Term::ANSIColor;
use Digest::MD5 qw(md5 md5_hex);
use MIME::Base64;
use Encode;
use DBI;
use Switch;

our $searchall;
our $colorreplace = -1;
our $insensitive = -1;
our $arch = undef;
our %portdirs;
our %eclasses;
our %modtimes;
our $dbfile = undef;
# OLD SEARCH CODE
#our $findmode = -1;
our $monitorupdates = -1;
our $newdb;
our $nocolor = -1;
our @searchterms;

if(-e "/etc/portfind.conf") {
open(FILE, "</etc/portfind.conf") || die "Configuration file is unreadable.\n";
my @lines = <FILE>;
chomp(@lines);
close(FILE);
foreach my $line (@lines) {
   if("$line" =~ /^[\s]*\#/) {
   }
   elsif("$line" =~ /^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Hh]ighlight[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $colorreplace = 1 }
         else {$colorreplace = 0 }
      }
   }
# OLD SEARCH CODE
#    elsif("$line" =~ /^[\s]*[Ss]each[Mm]ode[\:\s]+([Oo]ld|[Dd][Bb])[\s]*$/) {
#       my $pre = "$line";
#       $pre =~ s/^[\s]*[Ss]each[Mm]ode[\:\s]+([Oo]ld|[Dd][Bb])[\s]*$/\1/;
#       $pre = lc($pre);
#       switch($pre) {
#          case "old" {$findmode = 0 }
#          case "db" {$findmode = 1 }
#       }
#    }
   elsif("$line" =~ /^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Cc]ase[Ii]nsensitive[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      $pre = lc($pre);
      switch($pre) {
         case "yes" { $insensitive = 1 }
         else {$insensitive = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Mm]onitor[Uu]pdates[\:\s]+([Yy]es|[Nn]o)[\s]*$/) {
      my $pre = "$line";
      $pre = lc($pre);
      $pre =~ s/^[\s]*[Mm]onitor[Uu]pdates[\:\s]+([Yy]es|[Nn]o)[\s]*$/\1/;
      switch($pre) {
         case "yes" { $monitorupdates = 1 }
         else { $monitorupdates = 0 }
      }
   }
   elsif("$line" =~ /^[\s]*[Aa]rch[\:\s]+[A-z0-9\-]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Aa]rch[\:\s]+([A-z0-9\-]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $arch = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+[^\:\s]+?[\s]*$/) {
      my $pre = "$line";
      $pre =~ s/^[\s]*[Ss][Qq][Ll][Ff]ile[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $pre = lc($pre);
      $dbfile = "$pre";
   }
   elsif("$line" =~ /^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+[^\:\s]+?[\:\s]+[^\:\s]+?[\s]*$/) {
      my $key = "$line";
      $key =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+([a-z0-9]+?)[\:\s]+[^\:\s]+?[\:\s]+[^\:\s]+?[\s]*$/\1/;
      my $pre = "$line";
      $pre =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+([^\:\s]+?)[\:\s]+[^\:\s]+?[\s]*$/\1/;
      my $eclass = "$line";
      $eclass =~ s/^[\s]*[Pp]ort[Dd]ir[\:\s]+[a-z0-9]+?[\:\s]+[^\:\s]+?[\:\s]+([^\:\s]+?)[\s]*$/\1/;
      $portdirs{$key} = $pre;
      $eclasses{$key} = $eclass;
   }
}
}
if($colorreplace == -1) {
$colorreplace = 1;
}
if($insensitive == -1) {
$insensitive = 1;
}
# OLD SEARCH CODE
#if($findmode == -1) {
# $findmode = 1;
#}
if($nocolor == -1) {
$nocolor = 0;
}
if($monitorupdates == -1) {
$monitorupdates = 0;
}
if(!$arch) {
$arch = "x86";
}
if(!$dbfile) {
$dbfile = "/usr/portfind.dbl";
}
if(!scalar %portdirs) {
%portdirs = ("gentoo" => "/usr/portage");
%eclasses = ("gentoo" => "/usr/portage/eclass");
}

my @opts;
foreach my $ARG (@ARGV) {
if($ARG =~ /^\-\-[^-]+.*$/) {
   push(@opts, $ARG);
}
}

if(grep(/^\-\-insensitive$/, @opts)) {
$insensitive = 1;
}
elsif(grep(/^\-\-no\-insensitive$/, @opts)) {
$insensitive = 0;
}
if(grep(/^\-\-highlight$/, @opts)) {
$colorreplace = 1;
}
elsif(grep(/^\-\-no\-highlight$/, @opts)) {
$colorreplace = 0;
}
if(grep(/^\-\-monitor\-updates$/, @opts)) {
$monitorupdates = 1;
}
elsif(grep(/^\-\-no\-monitor\-updates$/, @opts)) {
$monitorupdates = 0;
}
# OLD SEARCH CODE
#if(grep(/^\-\-search\-db$/, @opts)) {
# $findmode = 1;
#}
#elsif(grep(/^\-\-search\-old$/, @opts)) {
# $findmode = 0;
#}

if(grep(/^\-\-no-color$/, @opts)) {
$nocolor = 1;
}

if(-e "$dbfile" && $monitorupdates) {
my $dbh = DBI->connect("dbi:SQLite:$dbfile") || die "Unable to open DBFile: $dbfile\n";
foreach my $key (%portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      my $modtime = -A "$portdir";
      my $dbmodtime;
      my $dbdata = $dbh->selectall_arrayref("SELECT * FROM modtimes WHERE dir = \'$portdir\'");
      foreach my $dbdat (@{$dbdata}) {
         $dbmodtime = ${$dbdat}[2];
      }
      if(abs($dbmodtime - $modtime) >= 0.02) {
         print colored("$portdir has been modified", 'bold red') . ": Please " . colored("--update", 'bold yellow') . "\n";
         $modtimes{$key} = 1;
      }
      elsif(!$dbmodtime) {
         print colored("${portdir}'s modification time hasn't been recorded", 'bold red') . ": Please " . colored("--update", 'bold yellow') . "\n";
      }
      else {
         $modtimes{$key} = 0;
      }
   }
}
$dbh->disconnect;
}

if(scalar @ARGV < 1) {
$searchall = 1;
}
elsif(grep(/^\-\-update$/, @opts)) {
$newdb = 1;
$searchall = 1;
}
elsif(grep(/^\-\-help$/, @opts)) {
my $text = "Usage: " . basename($0) . " " . colored("\[", 'bold') . " " . colored("option", 'bold green') . " " . colored("\|", 'bold') . " search terms " . colored("\]", 'bold') . "\n";
$text .= "\t" . colored("Options:\n", 'bold');
$text .= "\t\t" . colored("--help", 'bold green') . ": Display this message.\n";
$text .= "\t\t" . colored("--update", 'bold yellow') . ": Update database.\n";
$text .= "\t\t" . colored("--help-config", 'bold red') . ": Configuration help.\n";
$text .= "\t\t" . colored("--search-db", 'bold cyan') . "/" . colored("--search-old", 'bold cyan') . ": Toggles search mode.\n";
$text .= "\t\t" . colored("--no-color", 'bold blue') . ": Disables colored output.\n";
$text .= "\tThe following options can be given the " . colored("--no", 'bold') . " beginning. E.G.: --no-insensitive\n";
$text .= "\t\t" . colored("--insensitive", 'bold cyan') . ": Toggles case sensitivity.\n";
$text .= "\t\t" . colored("--highlight", 'bold cyan') . ": Toggles highlighting of search terms in found text.\n";
$text .= "\t\t" . colored("--monitor-updates", 'bold cyan') . ": Toggles Portage tree update monitoring by access time.\n";
if($nocolor) {
   print &nocolor("$text");
}
else {
   print "$text";
}
exit;
}
elsif(grep(/^\-\-help-config$/, @opts)) {
my $text = colored(basename($0) . " Configuration:", 'bold') . "\n";
$text .= "Configuration file lies at " . colored("/etc/portfind.conf", 'bold blue') . ": " . ((-e "/etc/portfind.conf") ? colored("Found", 'bold green') : colored("Not found", 'bold red')) . "\n";
$text .= "\tAny line began with a \# is a comment.\n";
$text .= "\tLines are written a leading config option, and separators are \: and/or spaces, ending with a newline.\n";
$text .= "\t\tE.G.: \"CaseInsensitive\: Yes\"\n";
# OLD SEARCH CODE
# $text .= "\t" . colored("SearchMode", 'bold cyan') . ": " . colored("DB", 'bold green') . " or " . colored("Old", 'bold red') . ": Toggles whether to use SQLite Search or Search through tree from database\n";
# $text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . (($findmode) ? colored("DB", 'bold green') : colored("Old", 'bold red')) . "\n";
$text .= "\t" . colored("CaseInsensitive", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether to search without case sensitivity.\n";
$text .= "\t\t" . colored("Note", 'bold') . ": The case sensitive search is much quicker.\n";
$text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . (($insensitive) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n";
$text .= "\t" . colored("Highlight", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether to highlight search terms in found text.\n";
$text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . (($colorreplace) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n";
$text .= "\t" . colored("MonitorUpdates", 'bold cyan') . ": " . colored("Yes", 'bold green') . " or " . colored("No", 'bold red') . ": Toggles whether updates are monitored via access times.\n";
$text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . (($monitorupdates) ? colored("Yes", 'bold green') : colored("No", 'bold red')) . "\n";
$text .= "\t" . colored("Arch", 'bold cyan') . ": The architecture as put by Portage of your current systems.\n";
$text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . colored($arch, 'bold green') . "\n";
$text .= "\t" . colored("SQLFile", 'bold cyan') . ": The location of the SQLite database\n";
$text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . colored($dbfile, 'bold green') . ": " . ((-e $dbfile) ? colored("Found", 'bold green') : colored("Not Found, please ", 'bold red') . colored("--update", 'bold yellow')) . "\n";
$text .= "\t" . colored("PortDir", 'bold cyan') . ": The Portage trees, including overlays and the main tree\n";
$text .= "\t\tMay be specified multiple times for multiple trees.\n";
$text .= "\t\tPortDir: name /path/to/tree /path/to/eclass\n";
if($monitorupdates) {
   foreach my $key (keys %portdirs) {
      $text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . colored("$key", 'green') . " -> " . colored("$portdirs{$key}", 'bold green') . ": " . ((-e $portdirs{$key}) ? colored("Found", 'bold green') . (($modtimes{$key}) ? "/" . colored("Update", 'bold red') : "") : colored("Not found", 'bold red')) . " (" . colored("Eclass: ", 'bold yellow') . colored("$eclasses{$key}", 'bold green') . ": " . ((-e $eclasses{$key}) ? colored("Found", 'bold green') : colored("Not Found", 'bold red')) . ")\n";
   }
}
else {
   foreach my $key (keys %portdirs) {
      $text .= "\t\t" . colored("Currently", 'bold yellow') . ": " . colored("$key", 'green') . " -> " . colored("$portdirs{$key}", 'bold green') . ": " . ((-e $portdirs{$key}) ? colored("Found", 'bold green') : colored("Not found", 'bold red')) . " (" . colored("Eclass: ", 'bold yellow') . colored("$eclasses{$key}", 'bold green') . ": " . ((-e $eclasses{$key}) ? colored("Found", 'bold green') : colored("Not Found", 'bold red')) . ")\n";
   }
}
if($nocolor) {
   print &nocolor("$text");
}
else {
   print "$text";
}
exit;
}
else {
$searchall = 0;
foreach my $ARG (@ARGV) {
   if(!($ARG =~ /^--[^-]+.*$/)) {
      push(@searchterms, $ARG);
   }
}
}
if($newdb && -e "$dbfile") {
unlink("$dbfile");
}

my $dbh = DBI->connect("dbi:SQLite:$dbfile") || die "Unable to open DBFile: $dbfile\n";

if($newdb) {
STDOUT->autoflush(1);
print "Creating new DB at $dbfile...\n";
my @categories = &generate_db;

$dbh->do("CREATE TABLE categories(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(32))");
$dbh->do("CREATE TABLE ebuilds(id INTEGER PRIMARY KEY AUTOINCREMENT, name CHAR(64), catname CHAR(32), longdesc CHAR(2048), longhash CHAR(32))");
$dbh->do("CREATE TABLE shortdescs(id INTEGER PRIMARY KEY AUTOINCREMENT, ename CHAR(64), version CHAR(32), mask CHAR(8), shortdesc CHAR(1024), shorthash CHAR(32), use CHAR(512))");
$dbh->do("CREATE TABLE modtimes(id INTEGER PRIMARY KEY AUTOINCREMENT, dir CHAR(512), modtime REAL)");

foreach my $key (%portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      my $modtime = -A "$portdir";
      $dbh->do("INSERT INTO modtimes (dir, modtime) VALUES(\'$portdir\', $modtime)");
   }
}

my $sth = $dbh->prepare("INSERT INTO ebuilds (name, catname, longdesc, longhash) VALUES(?, ?, ?, ?)");
my $lth = $dbh->prepare("INSERT INTO shortdescs (ename, version, mask, shortdesc, shorthash, use) VALUES(?, ?, ?, ?, ?, ?)");
print "Building SQLite database.\n";
my $precent = 0;
my $len = scalar @categories;
for(my $i = 0; $i < $len; $i++) {
   my $category = $categories[$i];
   my $catname = $category->name;
   $dbh->do("INSERT INTO categories (name) VALUES(\'$catname\')");
   my @ebuilds = @{$category->ebuilds};
   foreach my $ebuild (@ebuilds) {
      my $ename = $ebuild->name;
      my $longdesc = $ebuild->longdesc;
      my %shortdesc = %{$ebuild->shortdesc};
      my %use = %{$ebuild->use};
      my %mask = %{$ebuild->mask};
      $longdesc = (($longdesc) ? "$longdesc" : "NULL");
      $sth->execute($ename, $catname, $longdesc, &ascii_md5_hex("$longdesc"));
      foreach my $key (sort keys %shortdesc) {
         $lth->execute($ename, $key, $mask{$key}, $shortdesc{$key}, &ascii_md5_hex("$shortdesc{$key}"), $use{$key});
      }
   }
   my $percent = (($i + 1) / $len) * 100;
   if(abs($precent - $percent) >= 0.7) {
      print "\b\b\b\b" . &round($percent) . "%";
#       print &round($percent) . "%\n";
      $precent = $percent;
   }
}
print "\nComplete.\n";
}
else {
if($searchall) {
   my @categories = &extract_from_db($dbh);
   if($nocolor) {
      foreach my $category (@categories) {
         foreach my $ebuild (@{$category->ebuilds}) {
            print &nocolor(&printinfo($ebuild, $category->name));
         }
      }
   }
   else {
      foreach my $category (@categories) {
         foreach my $ebuild (@{$category->ebuilds}) {
            print (&printinfo($ebuild, $category->name));
         }
      }
   }
}
else {
   my @found;
   STDOUT->autoflush(1);
# OLD SEARCH CODE
#    if(!$findmode) {
#       my @categories = &extract_from_db($dbh);
#       @found = &classic_search(@categories);
#    }
#    else {
      @found = &search_from_db($dbh);
#    }
   STDOUT->autoflush(0);
   if($nocolor) {
      if(scalar @found == 0) {
         print "No results.\n";
      }
      else {
         foreach my $sf (@found) {
            print &nocolor(&printinfo($sf->ebuild, $sf->category->name));
         }
      }
   }
   else {
      if(scalar @found == 0) {
         print colored("No results.", 'bold red') . "\n";
      }
      else {
         foreach my $sf (@found) {
            print (&printinfo($sf->ebuild, $sf->category->name));
         }
      }
   }
}
}
$dbh->disconnect;

sub extract_from_db {
my $dbh = shift;
my @categories;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
foreach my $dbcategory (@{$dbcategories}) {
   my $category = new Category;
   $category->name(${$dbcategory}[1]);

   my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'${$dbcategory}[1]\'");
   foreach my $dbebuild (@{$dbebuilds}) {
      my $ebuild = new Ebuild;
      $ebuild->name(${$dbebuild}[1]);
      my $ename = $ebuild->name;
      my $longdesc = ${$dbebuild}[3];
      $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);
      my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
      foreach my $dbshortdesc (@{$dbshortdescs}) {
         my $version = ${$dbshortdesc}[2];
         ${$ebuild->shortdesc}{$version} = ${$dbshortdesc}[4];
         ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
      }
      push(@{$category->ebuilds}, $ebuild);
   }
   push(@categories, $category);
}
return @categories;
}
sub search_from_db {
my $dbh = shift;
my @found;
my $dbcategories = $dbh->selectall_arrayref("SELECT * FROM categories");
print STDERR "Initiating search.\n";
my $len = scalar @{$dbcategories};
my $precent = 0;
for(my $i = 0; $i < $len; $i++) {
   my $dbcategory = ${$dbcategories}[$i];
   my $category = new Category;
   $category->name(${$dbcategory}[1]);
   my $catname = ${$dbcategory}[1];
   foreach my $term (@searchterms) {
      my $dbebuilds;
      if($insensitive) {
         $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND (name LIKE \'\%$term\%\' OR longdesc LIKE \'\%$term\%\')");
      }
      else {
         $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND (name GLOB \'\*$term\*\' OR longdesc GLOB \'\*$term\*\')");
      }
      foreach my $dbebuild (@{$dbebuilds}) {
         my $ebuild = new Ebuild;
         $ebuild->name(${$dbebuild}[1]);
         my $ename = ${$dbebuild}[1];
         $ebuild->name($ename);
         my $longdesc = ${$dbebuild}[3];
         if($longdesc ne "NULL") {
         }
         else {
            $longdesc = undef;
         }
         $ebuild->longdesc($longdesc);

         my $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'${$dbebuild}[1]\'");
         foreach my $dbshortdesc (@{$dbshortdescs}) {
            my $version = ${$dbshortdesc}[2];
            my $shortdesc = ${$dbshortdesc}[4];
            ${$ebuild->shortdesc}{$version} = $shortdesc;
            ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
            ${$ebuild->use}{$version} = ${$dbshortdesc}[6];
         }
         my $enc = new Encapsulated;
         $enc->ebuild($ebuild);
         $enc->category($category);
         push(@found, $enc);
      }

      my $dbshortdescs;
      if($insensitive) {
         $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE shortdesc LIKE \'\%$term\%\'");
      }
      else {
         $dbshortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE shortdesc GLOB \'\*$term\*\'");
      }
      foreach my $dbshortdesc(@{$dbshortdescs}) {
         my $ename = ${$dbshortdesc}[1];
         if(&findebuild_encapsulated("$catname/$ename", 0, @found) == -1) {
            my $dbebuilds = $dbh->selectall_arrayref("SELECT * FROM ebuilds WHERE catname = \'$catname\' AND name = \'$ename\'");
            my $dbename = undef;
            my $longdesc = undef;
            foreach my $dbebuild (@{$dbebuilds}) {
               if($ename eq ${$dbebuild}[1] && $catname eq ${$dbebuild}[2]) {
                  $dbename = ${$dbebuild}[1];
                  $longdesc = ${$dbebuild}[3];
               }
            }
            if($dbename) {
               my $ebuild = new Ebuild;
               $ebuild->name($ename);
               $ebuild->longdesc(($longdesc eq "NULL") ? undef : $longdesc);

               my $dbebuild_shortdescs = $dbh->selectall_arrayref("SELECT * FROM shortdescs WHERE ename = \'$ename\'");
               foreach my $dbebuild_shortdesc (@{$dbebuild_shortdescs}) {
                  my $version = ${$dbshortdesc}[2];
                  my $shortdesc = ${$dbshortdesc}[4];
                  ${$ebuild->shortdesc}{$version} = $shortdesc;
                  ${$ebuild->mask}{$version} = ${$dbshortdesc}[3];
                  ${$ebuild->use}{$version} = ${$dbshortdesc}[6];
               }
               my $enc = new Encapsulated;
               $enc->category($category);
               $enc->ebuild($ebuild);
               push(@found, $enc);
            }
         }
      }
   }
   my $percent = (($i + 1) / $len) * 100;
   if(abs($precent - $percent) >= 0.7) {
      print STDERR "\b\b\b\b" . &round($percent) . "%";
#       print STDERR &round($percent) . "%\n";
      $precent = $percent;
   }
}
print STDERR "\nSearch complete.\n";
return @found;
}
sub generate_db {
my @categories;
print "Reading directory tree.\n";
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   if(-e "$portdir") {
      print "\nFor $portdir:\nFinding categories\n";
      my @categorydirs = <$portdir/*-*>;
      my $len = scalar @categorydirs;
      my $precent = 0;
      for(my $i = 0; $i < $len; $i++) {
         my $category = $categorydirs[$i];
         if(-d "$category" && !(&findcategory(@categories, basename("$category")))) {
            my $cat = new Category;
            $cat->name(basename("$category"));
            push(@categories, $cat);
         }

         my $percent = (($i + 1) / $len) * 100;
         if(abs($precent - $percent) >= 0.7) {
            print "\b\b\b\b" . &round($percent) . "%";
#             print &round($percent) . "%\n";
            $precent = $percent;
         }
      }
      print "\nFinding ebuilds\n";

      my $precent = 0;
      my $len = scalar @categories;
      for(my $i = 0; $i < $len; $i++) {
         my $category = $categories[$i];
         my $catname = $category->name;
         foreach my $subdir (<$portdir/$catname/*>) {
            my @tfiles = <$subdir/*>; # TEMPORARY TO FIND EBUILDS
            if(-d "$subdir" && scalar @tfiles) {
               my $ebuild = undef;
               my $found = &findebuild(@{$category->ebuilds}, basename("$subdir"));
               if($found > -1) {
                  $ebuild = ${$category->ebuilds}[$found];
               }
               else {
                  $ebuild = new Ebuild;
               }
               $ebuild->name(basename("$subdir"));
               foreach my $file (<$subdir/*>) {
                  if(! -d "$file") {
                     push(@{$ebuild->files}, basename("$file"));
                  }
               }
               if($found > -1) {
                  ${$category->ebuilds}[$found] = $ebuild;
               }
               else {
                  push(@{$category->ebuilds}, $ebuild);
               }
            }
         }
         my $percent = (($i + 1) / $len) * 100;
         if(abs($precent - $percent) >= 0.7) {
            print "\b\b\b\b" . &round($percent) . "%";
#             print &round($percent) . "%\n";
            $precent = $percent;
         }
      }
   }
}
print "\nCreating database from files.";
foreach my $key (keys %portdirs) {
   my $portdir = $portdirs{$key};
   my $eclass = $eclasses{$key};
   if(-e "$portdir") {
      print "\nFor $portdir:\n";
      my $precent = 0;
      my $len = scalar @categories;
      for(my $i = 0; $i < $len; $i++) {
         my $category = $categories[$i];
         my $catname = $category->name;
         foreach my $ebuild (@{$category->ebuilds}) {
            my $ename = $ebuild->name;
#             print "$catname/$ename\n";
            if(-d "$portdir/$catname/$ename") {
               foreach my $file (@{$ebuild->files}) {
                  if("$file" eq "metadata.xml" && -e "$portdir/$catname/$ename/$file") {
                     my $parser = new XML::DOM::Parser;
                     my $xml = $parser->parsefile("<$portdir/$catname/$ename/$file");
                     my $nodes = $xml->getElementsByTagName("longdescription");
                     my $n = $nodes->getLength;
                     for(my $i = 0; $i < $n; $i++) {
                        my $node = $nodes->item($i);
                        my $lang = $node->getAttributeNode("lang");
                        if($n == 1 || ($lang && $lang->getValue eq "en")) {
                           my $children = $node->getChildNodes;
                           my $nc = $children->getLength;
                           for(my $c = 0; $c < $nc; $c++) {
                              my $subnode = $children->item($c);
                              if($subnode->getNodeType == TEXT_NODE) {
                                 $ebuild->longdesc($subnode->getNodeValue);
                              }
                           }
                        }
                     }
                     $xml->dispose;
                  }
                  if("$file" =~ /\.ebuild$/) {
                     my $version = "$file";
                     my $tmpename = "$ename";
                     my $descfound = 0;
                     my $maskfound = 0;
                     $tmpename =~ s/\+/\\\+/g;
                     $version =~ s/^$tmpename-([a-z0-9_.-]+)\.ebuild$/\1:$key/;
                     open(FILE, "<$portdir/$catname/$ename/$file");
                     my @lines = <FILE>;
                     chomp(@lines);
                     close(FILE);
                     my $use = &get_use("$portdir/$catname/$ename/$file", $eclass, 0);
                     foreach my $line (@lines) {
                        if("$line" =~ /DESCRIPTION=\".*?\"/) {
                           my $desc = "$line";
                           $desc =~ s/^.*?DESCRIPTION=\"(.*?)\".*?$/\1/;
                           ${$ebuild->shortdesc}{$version} = "$desc";
                           $descfound = 1;
                        }
                        elsif("$line" =~ /KEYWORDS=\".*?\"/) {
                           my $keywords = "$line";
                           my $keyword = undef;

                           if("$keywords" =~ /[\"\s]\~$arch([\"\s])/) {
                              $keyword = "~";
                           }
                           elsif("$keywords" =~ /[\"\s]$arch([\"\s])/) {
                              $keyword = "+";
                           }
                           else {
                              $keyword = "-";
                           }
                           ${$ebuild->mask}{$version} = "$keyword";
                           $maskfound = 1;
                        }
                     }
                     ${$ebuild->use}{$version} = "$use";
                     if(!$descfound) {
                        ${$ebuild->shortdesc}{$version} = undef;
                        if(!$maskfound) {
                           ${$ebuild->mask}{$version} = "-";
                        }
                     }
                  }
               }
            }
         }
         my $percent = (($i + 1) / $len) * 100;
         if(abs($precent - $percent) >= 0.7) {
            print "\b\b\b\b" . &round($percent) . "%";
#             print &round($percent) . "%\n";
            $precent = $percent;
         }
      }
   }
}
print "\nGeneration complete.\n";
return @categories;
}
# OLD SEARCH CODE
#sub classic_search {
# my @categories = @_;
# my @found;
# foreach my $category (@categories) {
#    my $catname = $category->name;
#    my @ebuilds = @{$category->ebuilds};
#    foreach my $ebuild (@ebuilds) {
#       my $match = 0;
#       my $longdesc = $ebuild->longdesc;
#       my %shortdesc = %{$ebuild->shortdesc};
#       my $ename = $ebuild->name;
#       foreach my $term (@searchterms) {
#          if("$ename" =~ /$term/ || ("$ename" =~ /$term/i && $insensitive)) {
#             if($colorreplace) {
#                my @captured;
#                if($insensitive) {
#                   @captured = "$ename" =~ /$term/ig;
#                }
#                else {
#                   @captured = "$ename" =~ /$term/g;
#                }
#                foreach my $found (@captured) {
#                   my $colored = colored("$found", 'cyan');
#                   my $filterfound = "$found";
#                   my $uncolor = color 'reset';
#                   my $recolor = color 'bold blue';
#                   $filterfound =~ s/(\W)/\\\1/g;
#                   $ename =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
#                }
#             }
#             $match += 1;
#          }
#          if("$longdesc" =~ /$term/ || ("$longdesc" =~ /$term/i && $insensitive)) {
#             if($colorreplace) {
#                my @captured;
#                if($insensitive) {
#                   @captured = "$longdesc" =~ /$term/ig;
#                }
#                else {
#                   @captured = "$longdesc" =~ /$term/g;
#                }
#                foreach my $found (@captured) {
#                   my $colored = colored("$found", 'cyan');
#                   my $filterfound = "$found";
#                   $filterfound =~ s/(\W)/\\\1/g;
#                   $longdesc =~ s/$filterfound/$colored/g;
#                }
#             }
#             $match += 1;
#          }
#       }
#       $ebuild->name($ename);
#       $ebuild->longdesc($longdesc);
#       foreach my $version (sort keys %shortdesc) {
#          foreach my $term (@searchterms) {
#             if("$shortdesc{$version}" =~ /$term/ || ("$shortdesc{$version}" =~ /$term/i && $insensitive)) {
#                if($colorreplace) {
#                   my @captured;
#                   if($insensitive) {
#                      @captured = "$shortdesc{$version}" =~ /$term/ig;
#                   }
#                   else {
#                      @captured = "$shortdesc{$version}" =~ /$term/g;
#                   }
#                   foreach my $found (@captured) {
#                      my $colored = colored("$found", 'cyan');
#                      my $filterfound = "$found";
#                      $filterfound =~ s/(\W)/\\\1/g;
#                      $shortdesc{$version} =~ s/$filterfound/$colored/g;
#                   }
#                   ${$ebuild->shortdesc}{$version} = $shortdesc{$version};
#                }
#                $match += 1;
#             }
#          }
#       }
#       if($match) {
#          my $enc = new Encapsulated;
#          $enc->category($category);
#          $enc->ebuild($ebuild);
#          push(@found, $enc);
#       }
#    }
# }
# return @found;
#}
sub findcategory {
my $catname = pop @_;
my @categories = @_;
foreach my $category (@categories) {
   my $name = $category->name;
   if("$name" eq "$catname") {
      return 1;
   }
}
return 0;
}
sub findebuild {
my $ename = pop @_;
my @ebuilds = @_;
for(my $i = 0; $i < scalar @ebuilds; $i++) {
   my $name = $ebuilds[$i]->name;
   if("$name" eq "$ename") {
      return $i;
   }
}
return -1;
}
sub findebuild_encapsulated {
my $ename = shift;
my $i0 = shift;
my @encapsulated = @_;
for(my $i = $i0; $i < scalar @encapsulated; $i++) {
   my $enc = $encapsulated[$i];
   my $category = $enc->category;
   my $catname = $category->name;

   my $ebuild = $enc->ebuild;
   my $cename = $ebuild->name;
   if($ename eq "$catname/$cename") {
      return $i;
   }
}
return -1;
}
sub highlight {
my $term = shift;
my $text = shift;
my $colors = shift;
my $recolor = color shift;
my $uncolor = color 'reset';
my @captured;
if($insensitive) {
   @captured = "$text" =~ /$term/ig;
}
else {
   @captured = "$text" =~ /$term/g;
}
foreach my $found (@captured) {
   my $colored = colored("$found", $colors);
   my $filterfound = "$found";
   $filterfound =~ s/(\W)/\\\1/g;
   $text =~ s/$filterfound/${uncolor}${colored}${recolor}/g;
}
return $text;
}

sub printinfo {
my $ebuild = shift;
my $catname = shift;
my $ename = $ebuild->name;
my $longdesc = $ebuild->longdesc;
my %shortdesc = %{$ebuild->shortdesc};
my %mask = %{$ebuild->mask};
my %use = %{$ebuild->use};
if($colorreplace) {
   foreach my $term (@searchterms) {
      $ename = &highlight($term, $ename, 'cyan', 'bold blue');
      $longdesc = &highlight($term, $longdesc, 'cyan', 'reset');
      foreach my $version (keys %shortdesc) {
         $shortdesc{$version} = &highlight($term, $shortdesc{$version}, 'cyan', 'reset');
      }
   }
}

my $ret = colored("$catname/$ename\n", 'bold blue');
$ret .= "\t" . colored("Short Description\n", 'bold green');
foreach my $version (reverse sort keys %shortdesc) {
   my $maskcolor = color('reset') . colored("$mask{$version}", 'red') . color('bold yellow');
   $ret .= "\t\t" . colored("$version [$maskcolor]: ", 'bold yellow') . "$shortdesc{$version}\n";
   if($use{$version}) {
      $ret .= "\t\t\t" . colored("USE:", 'bold blue') . " " . colored($use{$version}, 'green') . "\n";
   }
}
if($longdesc && !("$longdesc" =~ /^[\s\t\n]*$/)) {
   my @arraydesc = split(/\n/, "$longdesc");
   chomp(@arraydesc);
   if(!$arraydesc[0]) {
      shift(@arraydesc);
   }
   $ret .= "\t" . colored("Long Description\n", 'bold red');
   foreach my $line (@arraydesc) {
      $line =~ s/^[\s\t]*/\t\t/g;
      $ret .= "$line\n";
   }
}
$ret .= "\n";
return "$ret";
}
sub ascii_md5_hex {
my $text = shift;
$text = encode("ascii", "$text");
return md5_hex("$text");
}
sub round {
my $number = shift;
return int($number + .5 * ($number <=> 0));
}
sub nocolor {
my $text = shift;
$text =~ s/\e\[.+?m//g;
return $text;
}

# sub get_use {
# my $ebuild = shift;
# my $eclass = shift;
# my %vars = &resolve_vars("$ebuild", $eclass, 0);
# my $iuse;
# foreach my $key (keys %vars) {
#    if($key eq "IUSE") {
#       $iuse = $vars{$key};
#    }
# }
# my @flags = split(/ /, $iuse);
# my $use;
# foreach my $flag (@flags) {
#    if(!($flag =~ /^[\s\t\n]*$/ || $flag =~ /\$\{[A-z_]+\}/ || $flag =~ /\_/)) {
#       $use .= (($use) ? " $flag" : "$flag");
#    }
# }
# return $use;
#}
sub get_use {
my $ebuild = shift;
my $eclass = shift;
my $iuse = &resolve_use("$ebuild", $eclass, 0);
my @flags = split(/ /, $iuse);
my $use;
foreach my $flag (@flags) {
   if(!($flag =~ /^[\s\t\n]*$/ || $flag =~ /\$\{[A-z_]+\}/ || $flag =~ /\_/)) {
      $use .= (($use) ? " $flag" : "$flag");
   }
}
return $use;
}
# sub get_use {
# my $ebuild = shift;
# my $eclass = shift;
# my %vars = &resolve_vars("$ebuild", $eclass, 0);
# my $iuse;
# foreach my $key (keys %vars) {
#    if($key eq "IUSE") {
#       $iuse = $vars{$key};
#    }
# }
# my @flags = split(/ /, $iuse);
# my $use;
# foreach my $flag (@flags) {
#    if(!($flag =~ /^[\s\t\n]*$/ || $flag =~ /\$\{[A-z_]+\}/ || $flag =~ /\_/)) {
#       $use .= (($use) ? " $flag" : "$flag");
#    }
# }
# return $use;
#}

#sub resolve_vars {
sub resolve_use {
my $efile = shift;
my $eclass = shift;
my $level = shift;
if($level > 2) {
   return;
}
my $outuse = undef;
# my %outvars;
# $outvars{"IUSE"} = undef;
if(-e "$efile") {
   open(FILE, "<$efile") || die "Could not open: $efile\n";
   my @data = <FILE>;
   close(FILE);
   my $text = "@data";
   my @found = $text =~ /IUSE\=\"[^\"]*?\"/mg;
   my @inheritlist = $text =~ /inherit[\s]+[^\.\,\)\(]+?\n/g;
   chomp(@inheritlist);
   my @inherits;
   foreach my $inn (@inheritlist) {
      my @inherit = split(/[ ]+?/, "$inn");
      foreach my $in (@inherit) {
         if($in ne "inherit" && !($in =~ /^[\s\t\n]*$/)) {
            push(@inherits, $in);
         }
      }
   }
   foreach my $inherit (@inherits) {
#       if(!grep(/$inherit/, @blacklist)) {
#          my %inherited = &resolve_vars("$eclass/${inherit}.eclass", $level + 1);
         $outuse .= " " . &find_iuse("$eclass/${inherit}.eclass", $eclass, $level + 1) . " ";
#          ${outvars}{"IUSE"} .= " " . &find_iuse("$eclass/${inherit}.eclass", $eclass, $level + 1) . " ";
#          my $iuse = ${outvars}{"IUSE"};
#          foreach my $key (keys %inherited) {
#             if($key eq "IUSE") {
#                print "\t\t$key: $inherited{$key}\n";
#                $outvars{$key} = $inherited{$key};
#             }
#          }
#       }
   }

   foreach my $ff (@found) {
      my $name = "$ff";
      $name =~ s/^([A-Z_]+?)\=\"[^\"]*?\"$/\1/;
      my $value = $ff;
      $value =~ s/^[A-Z_]+?\=\"([^\"]*?)\"$/\1/;
      my @vars = $value =~ /\$\{[A-Z_]+?\}/g;
      foreach my $var (@vars) {
         my $sname = "$var";
         $sname =~ s/^\$\{(.*?)\}$/\1/;
         my $svalue = undef;
#          foreach my $key (keys %outvars) {
#             if($key eq $sname) {
#                $svalue = $outvars{$key};
#             }
#             $svalue =~ s/(\W)/\\\1/g;
#             $value =~ s/\$\{$sname\}/$svalue/g;
#          }
      }
      $outuse .= " $value ";
#       $outvars{$name} .= " $value ";
   }
}
# return %outvars;
return $outuse;
}

sub find_iuse {
my $file = shift;
my $eclass = shift;
my $level = shift;
my $iuse = undef;
open(FILE, "<$file");
my @data = <FILE>;
close(FILE);
my $text = "@data";
my @found = $text =~ /IUSE\=\"[^\"]*?\"/mg;
my @inheritlist = $text =~ /inherit[\s]+[^\.\,\)\(]+?\n/g;
chomp(@inheritlist);
my @inherits;
foreach my $inn (@inheritlist) {
   my @inherit = split(/[ ]+?/, "$inn");
   foreach my $in (@inherit) {
      if($in ne "inherit" && !($in =~ /^[\s\t\n]*$/)) {
         push(@inherits, $in);
      }
   }
}
foreach my $inherit (@inherits) {
#    if(!grep(/$inherit/, @blacklist)) {
      $iuse .= " " . &find_iuse("$eclass/${inherit}.eclass", $level + 1) . " ";
#    }
}
foreach my $ff (@found) {
   my $name = $ff;
   $name =~ s/^([A-Z_]+)\=\"[^\"]*?\"$/\1/;
   if($name eq "IUSE") {
      my $value = $ff;
      $value =~ s/^[A-z_]+?\=\"([^\"]*?)\"$/\1/;
      $iuse .= " $value ";
   }
}
return "$iuse";
}

package Category;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub ebuilds {
my $self = shift;
if(!defined $self->{EBUILDS}) {
   $self->{EBUILDS} = [ ];
}
return $self->{EBUILDS};
}
package Encapsulated;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub category {
my $self = shift;
$self->{CATEGORY} = shift if @_;
return $self->{CATEGORY};
}
sub ebuild {
my $self = shift;
$self->{EBUILD} = shift if @_;
return $self->{EBUILD};
}
package Ebuild;
sub new {
my $self = { };
my $class = shift;
return bless($self, $class);
}
sub name {
my $self = shift;
$self->{NAME} = shift if @_;
return $self->{NAME};
}
sub files {
my $self = shift;
if(!defined $self->{FILES}) {
   $self->{FILES} = [ ];
}
return $self->{FILES};
}
sub longdesc {
my $self = shift;
$self->{LDESC} = shift if @_;
return $self->{LDESC};
}
sub shortdesc {
my $self = shift;
if(!defined $self->{SDESC}) {
   $self->{SDESC} = { };
}
return $self->{SDESC};
}
sub use {
my $self = shift;
if(!defined $self->{USE}) {
   $self->{USE} = { };
}
return $self->{USE};
}
sub mask {
my $self = shift;
if(!defined $self->{MASK}) {
   $self->{MASK} = { };
}
return $self->{MASK};
}

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Sat Feb 24, 2007 3:07 am Post subject:

I've created a website for the project, with the newest version of it: http://guhnoo.org/~neil/ _________________ [ Kawa-kun, new and improved!! ] Alex Libman seems to be more of an anarchist than a libertarian.

Kenji Miyamoto
Veteran
Veteran

Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

Posted: Sun Feb 25, 2007 5:16 am Post subject:

Yet another update, with a new method based on Bash to extract the USE flags, and a configuration option to choose either that or the old method. _________________ [ Kawa-kun, new and improved!! ] Alex Libman seems to be more of an anarchist than a libertarian.

Display posts from previous:

	Gentoo Forums Forum Index Documentation, Tips & Tricks	All times are GMT
Page 1 of 1

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Copyright 2001-2024 Gentoo Foundation, Inc. Designed by Kyle Manna © 2003; Style derived from original subSilver theme. | Hosting by Gossamer Threads Inc. © | Powered by phpBB 2.0.23-gentoo-p11 © 2001, 2002 phpBB Group
Privacy Policy