Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
portagedb-0.2.1_alpha1 - with overlay support
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Thu Aug 26, 2004 5:25 am    Post subject: portagedb-0.2.1_alpha1 - with overlay support Reply with quote

portagedb is now eix
the new discussion can be found here: https://forums.gentoo.org/viewtopic.php?t=278819


Portagedb is a small program for searching the portage tree. Similar to app-portage/esearch it uses an index file (/var/cache/portagedb) which makes it very fast.

Usage:
Code:
xoa@pluto ~ $ portagedb
Usage: portagedb [options] command <search string>
Commands: u(pdate)           Updates the database -> /var/cache/portagedb
          s(earch)           Search for ebuilds (posix regex)
          S(earchdesc)       Search for ebuilds + descriptions (posix regex)
Options:  -n,--nocolor       Do not use ANSI color codes
          -c,--compact       Compact search results
          -v,--verbose       Verbose search results


Example output:
Code:
pluto ~ # pdb s bogo
* app-text/bogosort
     Available versions:  ~0.4.2
     Installed:           no
     Homepage:            http://www.lysator.liu.se/~qha/bogosort/
     Description:         A file sorting program which uses the bogosort algorithm

* mail-filter/bogofilter
     Available versions:  0.92.8  [M]~0.93.2
     Installed:           no
     Homepage:            http://bogofilter.sourceforge.net/
     Description:         Bayesian spam filter designed with fast algorithms, and tuned for speed.
Screenshot: https://sourceforge.net/project/screenshots.php?group_id=117574

Project page: http://portdb.sourceforge.net
Installation: emerge portagedb
(Latest version with overlay support: http://prdownloads.sourceforge.net/portdb/portagedb-0.2.1_alpha1.tar.bz2?download)

A little benchmark:
#!/bin/bash
MEMSIZE=512
flushcache() { dd if=/dev/hda of=/dev/null ibs=1048576 count=$[ MEMSIZE*2 ] 2>/dev/null; }
gettime_real( ) {
echo -n "flushcache; "
flushcache
echo -e -n $1 "\r\t\t\t\t\t"
(time $1 >/dev/null) 2>&1 |egrep real |egrep "[[:digit:]]*m.*s" -o
}
gettime_real "portagedb u"
gettime_real "eupdatedb"
gettime_real "portagedb S rare"
gettime_real "esearch -S rare"
gettime_real "emerge --searchdesc rare"


Updates:
Code:
-------------------------------------------------------------------------------
Changelog for portagedb
-------------------------------------------------------------------------------
Project page: http://portdb.sf.net
Discussion: http://forums.gentoo.org/viewtopic.php?t=214547&highlight=portagedb
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Ideas for future versions:
-------------------------------------------------------------------------------

- Support slotted versions, bug #1025517
  https://sourceforge.net/tracker/index.php?func=detail&aid=1025517&group_id=117574&atid=678491
- Provide a way to automatically "portagedb u" after emerge
- Tests on other platforms than x86
- Option for arbitrary formatting
- Replicate esearch's esync.py
- Option '-i' for restricting search results to installed packages
- Improve error handling

-------------------------------------------------------------------------------
v0.2.1_alpha1:
-------------------------------------------------------------------------------
* Execution speed of "portagedb s" is back to normal again,
  i.e. no unnecessary ~1 sec delay when the program is started.

* Portage overlays now are supported!

-------------------------------------------------------------------------------
v0.2.0:
-------------------------------------------------------------------------------
- nothing changed. alpha4 appears to be stable on x86 and amd64.
  we're in the portage tree now :)

-------------------------------------------------------------------------------
v0.2.0_alpha4:
-------------------------------------------------------------------------------
- xororand * Fixed memory management issues which lead to a crash on amd64
  (Discovered valgrind and removed all delete/delete[]/free() mismatches)

-------------------------------------------------------------------------------
v0.2.0_alpha3:
-------------------------------------------------------------------------------
- LinuxCommando * Fixed bug: "portagedb s" now throws an error message

- LinuxCommando * Added: "portagedb u" checks for root rights

- LinuxCommando * Added:
  "portagedb s abc xyz" searches for all packages with either abc or xyz in their name
  equivalent to: portagedb s "\(abc\)\|\(xyz\)"

- xororand * Fixed: /etc/portage/package.unmask is now used

-------------------------------------------------------------------------------
v0.2.0_alpha2:
-------------------------------------------------------------------------------

* Fixed bug #1025515: Version numbers now are sorted properly
  https://sourceforge.net/tracker/index.php?func=detail&aid=1025515&group_id=117574&atid=678491
  Reference: http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1#doc_chap2

* Fixed bug #1025519: Ebuild is available
  https://sourceforge.net/tracker/index.php?func=detail&aid=1025519&group_id=117574&atid=678491

* Stability and masking of packages are distinguished:
  Masked:      Red / [M]
  Stable:      Green / No prefix
  Unstable:   Brown / ~

* Different levels of verbosity:
  -c   Compact: One line per result
  -v   Verbose: Print licenses

* Color of packages changed from light green to normal green
  -> looks better in black-on-white terminals

* The arch is currently determined by a call to "portageq envvar ARCH" which is kinda
  slow, taking about 1 second on my system. If someone knows a faster, _reliable_ way,
  please tell me.

------------------------------------------------------------------------------
v0.1.3-r1:
------------------------------------------------------------------------------
* Fixed bug #1048527: 'portagedb s' searches in package names AND descriptions
  https://sourceforge.net/tracker/index.php?func=detail&aid=1048527&group_id=117574&atid=678491

-------------------------------------------------------------------------------
v0.1.3:
-------------------------------------------------------------------------------

* PORTDIR in /etc/make.conf is checked

* Colored text only uses foreground color so it looks nice on
  Black-On-White terminals

* Added new option "--nocolor"

-------------------------------------------------------------------------------
v0.1.2:
-------------------------------------------------------------------------------

* fixed a stupid bug with directory handling. thank you wiebel for debugging.


Last edited by xororand on Wed Jan 19, 2005 10:32 pm; edited 47 times in total
Back to top
View user's profile Send private message
dannycool
Tux's lil' helper
Tux's lil' helper


Joined: 13 Aug 2004
Posts: 111
Location: Karlsruhe Germany

PostPosted: Thu Aug 26, 2004 9:27 am    Post subject: Reply with quote

Code:
df@kahlann ~ $ psearch() { locate $* | grep "^/usr/portage/[^/\-]*-.*/" | cut -d/ -f4,5 | sort | uniq; }
df@kahlann ~ $ psearch bonobo
app-doc/ebook-bonobo
dev-cpp/libbonobomm
dev-cpp/libbonobouimm
dev-python/bonobo-python
gnome-base/bonobo
gnome-base/bonobo-activation
gnome-base/libbonobo
gnome-base/libbonoboui
        (...)


*mumble* UNIX *mumble* destined to reinvent *mumble*

SCNR :P
Back to top
View user's profile Send private message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Thu Aug 26, 2004 12:04 pm    Post subject: Reply with quote

Quote:
SCNR

np :)
Back to top
View user's profile Send private message
Pythonhead
Developer
Developer


Joined: 16 Dec 2002
Posts: 1801
Location: Redondo Beach, Republic of Calif.

PostPosted: Fri Aug 27, 2004 7:20 pm    Post subject: Reply with quote

It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf

Last edited by Pythonhead on Sat Aug 28, 2004 1:51 am; edited 1 time in total
Back to top
View user's profile Send private message
asph
l33t
l33t


Joined: 25 Aug 2003
Posts: 741
Location: Barcelona, Spain

PostPosted: Sat Aug 28, 2004 12:49 am    Post subject: Reply with quote

just instaled it, VERY nice!!
now i can search ebuilds fast :)
_________________
gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep
Back to top
View user's profile Send private message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Sat Aug 28, 2004 1:42 pm    Post subject: Reply with quote

Pythonhead wrote:
It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf

PORTDIR is now checked in version 0.1.3. Thanks for pointing that out.
Back to top
View user's profile Send private message
Pythonhead
Developer
Developer


Joined: 16 Dec 2002
Posts: 1801
Location: Redondo Beach, Republic of Calif.

PostPosted: Sat Aug 28, 2004 4:32 pm    Post subject: Reply with quote

xororand wrote:
Pythonhead wrote:
It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf

PORTDIR is now checked in version 0.1.3. Thanks for pointing that out.


Thanks. A lot of developers only use CVS, not 'emerge sync' so they don't use /usr/portage.
Back to top
View user's profile Send private message
devon
l33t
l33t


Joined: 23 Jun 2003
Posts: 943

PostPosted: Sat Aug 28, 2004 7:48 pm    Post subject: Reply with quote

I did not see an ebuild for this on Bugzilla, so I submitted one. :) Worked fine for me on my x86 system using /usr/local/portage/app-portage/ as the directory.
Back to top
View user's profile Send private message
asph
l33t
l33t


Joined: 25 Aug 2003
Posts: 741
Location: Barcelona, Spain

PostPosted: Thu Sep 16, 2004 2:20 pm    Post subject: Reply with quote

i think it would be great to incorporate more functions to modify how the output is done, do you think it would be posible to incorporate a function like the "c" (short listing) option in esearch? it is very useful and fast to read, since using |more on long searches is a not very practical

keep up good work :)

*edit*
it seems to be a little bug, since even if you use only the "s" option, it also looks at the descriptions, as if you used "sS"
*/edit*
_________________
gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Fri Sep 24, 2004 9:32 am    Post subject: Reply with quote

Is it planned to integrate such indexing functionality into portage?
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Tue Sep 28, 2004 11:49 am    Post subject: Reply with quote

Maybe my question wasn't clear enough. Doing an esync updates the database so that esearch lists the current state of all applications. If I then install and application, eg. foo, and then do an esearch foo, it is still shown as uninstalled. It stays this way until I do another esync.

It would be great if this database was integrated into portage, so that emerge foo updates the database.
Back to top
View user's profile Send private message
GrayFox
Apprentice
Apprentice


Joined: 24 Jan 2004
Posts: 183
Location: Germany

PostPosted: Tue Sep 28, 2004 1:11 pm    Post subject: Reply with quote

Strange but a very cool thing here:
Code:
sudo ./portagedb u  0.77s user 0.66s system 85% cpu 1.680 total

I don't know any other way to check how long it took and I was to lazy to copy your script. Updating in <2s ? nice :D
Code:
./portagedb s notexistant  0.05s user 0.01s system 91% cpu 0.066 total

really nice! I bet you could somehow manage that the shell appends a portagedb u after every emerge command.
_________________
Random Stuff
Back to top
View user's profile Send private message
LifesizeKenDoll
n00b
n00b


Joined: 17 Jun 2004
Posts: 27

PostPosted: Wed Oct 06, 2004 2:33 am    Post subject: Reply with quote

I'd have to say this is pretty cool, although I had to do a little hacking to change the packagename text from white to default (cause I have a black-on-white xterm)
Back to top
View user's profile Send private message
vdboor
Guru
Guru


Joined: 03 Dec 2003
Posts: 592
Location: The Netherlands

PostPosted: Wed Oct 06, 2004 9:17 pm    Post subject: Reply with quote

Just wondering, the esearch package also has a tool called "esync". This tool compares the old cached portage tree with the new one. Do you think you can reproduce that functionality as well? :)
_________________
The best way to accelerate a windows server is by 9.81M/S²
Linux user #311670 and Yet Another Perl Programmer

[ screenies | Coding on KMess ]
Back to top
View user's profile Send private message
nsahoo
l33t
l33t


Joined: 17 Jul 2003
Posts: 618

PostPosted: Thu Oct 07, 2004 1:17 am    Post subject: Reply with quote

it apparently doesn't index the portage_overlay yet. It could be really useful if it can index them as well. :)

thanks.
Back to top
View user's profile Send private message
Goya13
n00b
n00b


Joined: 12 Aug 2004
Posts: 59

PostPosted: Wed Oct 27, 2004 8:18 pm    Post subject: Reply with quote

WOW, what an awesome app! I love it.

Just one request - licenses in the output???

thanks so much
Back to top
View user's profile Send private message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Mon Nov 08, 2004 1:11 pm    Post subject: Version 0.1.3-r1 and 0.1.4 Reply with quote

Thanks for your encouragement! So, portagedb hasn't moved alot during the last 2 months. That will change :) I have implemented some of your suggestions and also some other new features.

devon wrote:
I did not see an ebuild for this on Bugzilla, so I submitted one. Smile Worked fine for me on my x86 system using /usr/local/portage/app-portage/ as the directory.

An ebuild will come with 0.1.4.

mvila wrote:
it seems to be a little bug, since even if you use only the "s" option, it also looks at the descriptions, as if you used "sS"

fixed in 0.1.3-r1 -> added 5 bytes to the sourcecode :D

LifeSizeKenDoll wrote:
I'd have to say this is pretty cool, although I had to do a little hacking to change the packagename text from white to default (cause I have a black-on-white xterm)
I'll check that again in 0.1.4.

vdboot wrote:
Just wondering, the esearch package also has a tool called "esync". This tool compares the old cached portage tree with the new one. Do you think you can reproduce that functionality as well?
Later, sorry.

nsahoo wrote:
it apparently doesn't index the portage_overlay yet. It could be really useful if it can index them as well.
That is planned for the version after 0.1.4.

Goya13 wrote:
Just one request - licenses in the output???
Version 0.1.4 :)

Here's the current ChangeLog:
Code:

-------------------------------------------------------------------------------
Changelog for portagedb
-------------------------------------------------------------------------------
Project page: http://portdb.sf.net
Discussion: http://forums.gentoo.org/viewtopic.php?t=214547&highlight=portagedb
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Ideas for future versions:
-------------------------------------------------------------------------------

-   Support slotted versions, bug #1025517
   https://sourceforge.net/tracker/index.php?func=detail&aid=1025517&group_id=117574&atid=678491

- Support for portage overlays

- Provide a way to automatically update portagedb after emerges

- Test functionality on other platforms than x86

- Option for arbitrary formatting

- Replicate esearch's esync.py

-   Option '-i' for restricting search results to installed packages

- Bash completion for emerge/genlop etc.

- Test performance with Intel's C++ Compiler

-------------------------------------------------------------------------------
v0.1.4:
-------------------------------------------------------------------------------

- Fixed bug #1025515: Version numbers now are sorted properly
   https://sourceforge.net/tracker/index.php?func=detail&aid=1025515&group_id=117574&atid=678491

- Fixed bug #1025519: Ebuild is available
   https://sourceforge.net/tracker/index.php?func=detail&aid=1025519&group_id=117574&atid=678491

- Stability and masking of packages are distinguished:
   Stable-Unmasked:      Green / No prefix
   Unstable-Unmasked:   Brown / ~
   Masked:                     Red / [M]

- Different levels of verbosity:
   -c   Compact: One line per result
   -v   Verbose: [insert desc]

- Improved error handling


50-70% of the planned code is done. I hope to be ready in about 1 or 2 weeks.
Back to top
View user's profile Send private message
SuperJudge
Apprentice
Apprentice


Joined: 08 Jun 2004
Posts: 177
Location: Albany, GA

PostPosted: Thu Nov 25, 2004 1:04 am    Post subject: Reply with quote

I was frightened how quick that was. 8O I had gotten used to starting emerge -S and not paying attention to it for a long time.
_________________
---Powered by Honda
Back to top
View user's profile Send private message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Sun Dec 12, 2004 2:55 am    Post subject: Reply with quote

I added an ebuild file of version 0.1.3-r1 for easier installation.
Get it here.
Back to top
View user's profile Send private message
skally
n00b
n00b


Joined: 16 Mar 2004
Posts: 33

PostPosted: Thu Dec 16, 2004 9:03 pm    Post subject: Reply with quote

0.1.4!
0.1.4!
This is what we're waiting for!
:wink:
_________________
-- skally
Back to top
View user's profile Send private message
Pythonhead
Developer
Developer


Joined: 16 Dec 2002
Posts: 1801
Location: Redondo Beach, Republic of Calif.

PostPosted: Thu Dec 16, 2004 10:24 pm    Post subject: Reply with quote

I had a couple problems with it. First it failed with libtool version mismatches:

Quote:
*** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.10, ltmain.sh = 1.5a) ***


Changing src_compile() to this fixes it:
Code:

src_compile() {
   aclocal
   libtoolize --force --copy
   autoconf
   econf || die "econf failed"
   emake || die "emake failed"
}


Second, it doesn't work unless you run
Code:
/usr/bin/portagedb u

from your root directory

Quote:
# portagedb u
Updating portage database
Using portage cache: usr/portage/metadata/cache/
Error: scandir64("usr/portage/metadata/cache/", ..) failed
Database update failed



Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1.

Runs amazingly fast, nice job. Number one feature request: overlay directories
Back to top
View user's profile Send private message
SuperJudge
Apprentice
Apprentice


Joined: 08 Jun 2004
Posts: 177
Location: Albany, GA

PostPosted: Sun Dec 19, 2004 4:47 pm    Post subject: Reply with quote

Pythonhead wrote:
I had a couple problems with it. First it failed with libtool version mismatches:

Quote:
*** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.10, ltmain.sh = 1.5a) ***


Changing src_compile() to this fixes it:
Code:

src_compile() {
   aclocal
   libtoolize --force --copy
   autoconf
   econf || die "econf failed"
   emake || die "emake failed"
}


Second, it doesn't work unless you run
Code:
/usr/bin/portagedb u

from your root directory

Quote:
# portagedb u
Updating portage database
Using portage cache: usr/portage/metadata/cache/
Error: scandir64("usr/portage/metadata/cache/", ..) failed
Database update failed



Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1.

Runs amazingly fast, nice job. Number one feature request: overlay directories
Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back.
_________________
---Powered by Honda
Back to top
View user's profile Send private message
Pythonhead
Developer
Developer


Joined: 16 Dec 2002
Posts: 1801
Location: Redondo Beach, Republic of Calif.

PostPosted: Sun Dec 19, 2004 6:11 pm    Post subject: Reply with quote

SuperJudge wrote:
Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back.


libtool isn't broken, ebuilds are:
http://article.gmane.org/gmane.linux.gentoo.devel/23449/match=gentoo+libtoolize

So its going to be a pain for a while but should fix some real errors in the long run.
Back to top
View user's profile Send private message
SuperJudge
Apprentice
Apprentice


Joined: 08 Jun 2004
Posts: 177
Location: Albany, GA

PostPosted: Sun Dec 19, 2004 7:37 pm    Post subject: Reply with quote

Pythonhead wrote:
SuperJudge wrote:
Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back.


libtool isn't broken, ebuilds are:
http://article.gmane.org/gmane.linux.gentoo.devel/23449/match=gentoo+libtoolize

So its going to be a pain for a while but should fix some real errors in the long run.
I figured it was the ebuild, but I couldn't search out a good fix, and it seemed to work fine when I used an older ebuild. Thanks for the info.
_________________
---Powered by Honda
Back to top
View user's profile Send private message
xororand
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 119

PostPosted: Fri Dec 31, 2004 2:49 pm    Post subject: Reply with quote

Pythonhead wrote:
I had a couple problems with it. First it failed with libtool version mismatches. Changing src_compile() to this fixes it:
Thanks for the detailed notice. I'll fix it when 0.2.0 is final. The 0.1.3-r1 eBuild will be fixed as soon as possible.

Quote:
Second, it doesn't work unless you run
Code:
/usr/bin/portagedb u
from your root directory
Unfortunately, I can't reproduce this behaviour. Do you use a custom PORTDIR (make.conf)?

Quote:
Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1.

Thanks again for the hint. Meanwhile I've studied the portage help document and learned about proper version numbering. The new version is 0.2.0 - patch releases will be named with a _p suffix from now on.

By the way, 0.2.0_alpha2 is available on Sourceforge :) I hope it works.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum