View previous topic :: View next topic |
Author |
Message |
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Thu Aug 26, 2004 5:25 am Post subject: portagedb-0.2.1_alpha1 - with overlay support |
|
|
portagedb is now eix
the new discussion can be found here: https://forums.gentoo.org/viewtopic.php?t=278819
Portagedb is a small program for searching the portage tree. Similar to app-portage/esearch it uses an index file (/var/cache/portagedb) which makes it very fast.
Usage: Code: | xoa@pluto ~ $ portagedb
Usage: portagedb [options] command <search string>
Commands: u(pdate) Updates the database -> /var/cache/portagedb
s(earch) Search for ebuilds (posix regex)
S(earchdesc) Search for ebuilds + descriptions (posix regex)
Options: -n,--nocolor Do not use ANSI color codes
-c,--compact Compact search results
-v,--verbose Verbose search results |
Example output:
Code: | pluto ~ # pdb s bogo
* app-text/bogosort
Available versions: ~0.4.2
Installed: no
Homepage: http://www.lysator.liu.se/~qha/bogosort/
Description: A file sorting program which uses the bogosort algorithm
* mail-filter/bogofilter
Available versions: 0.92.8 [M]~0.93.2
Installed: no
Homepage: http://bogofilter.sourceforge.net/
Description: Bayesian spam filter designed with fast algorithms, and tuned for speed. | Screenshot: https://sourceforge.net/project/screenshots.php?group_id=117574
Project page: http://portdb.sourceforge.net
Installation: emerge portagedb
(Latest version with overlay support: http://prdownloads.sourceforge.net/portdb/portagedb-0.2.1_alpha1.tar.bz2?download)
A little benchmark:
#!/bin/bash
MEMSIZE=512
flushcache() { dd if=/dev/hda of=/dev/null ibs=1048576 count=$[ MEMSIZE*2 ] 2>/dev/null; }
gettime_real( ) {
echo -n "flushcache; "
flushcache
echo -e -n $1 "\r\t\t\t\t\t"
(time $1 >/dev/null) 2>&1 |egrep real |egrep "[[:digit:]]*m.*s" -o
}
gettime_real "portagedb u"
gettime_real "eupdatedb"
gettime_real "portagedb S rare"
gettime_real "esearch -S rare"
gettime_real "emerge --searchdesc rare"
Updates:
Code: | -------------------------------------------------------------------------------
Changelog for portagedb
-------------------------------------------------------------------------------
Project page: http://portdb.sf.net
Discussion: http://forums.gentoo.org/viewtopic.php?t=214547&highlight=portagedb
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Ideas for future versions:
-------------------------------------------------------------------------------
- Support slotted versions, bug #1025517
https://sourceforge.net/tracker/index.php?func=detail&aid=1025517&group_id=117574&atid=678491
- Provide a way to automatically "portagedb u" after emerge
- Tests on other platforms than x86
- Option for arbitrary formatting
- Replicate esearch's esync.py
- Option '-i' for restricting search results to installed packages
- Improve error handling
-------------------------------------------------------------------------------
v0.2.1_alpha1:
-------------------------------------------------------------------------------
* Execution speed of "portagedb s" is back to normal again,
i.e. no unnecessary ~1 sec delay when the program is started.
* Portage overlays now are supported!
-------------------------------------------------------------------------------
v0.2.0:
-------------------------------------------------------------------------------
- nothing changed. alpha4 appears to be stable on x86 and amd64.
we're in the portage tree now :)
-------------------------------------------------------------------------------
v0.2.0_alpha4:
-------------------------------------------------------------------------------
- xororand * Fixed memory management issues which lead to a crash on amd64
(Discovered valgrind and removed all delete/delete[]/free() mismatches)
-------------------------------------------------------------------------------
v0.2.0_alpha3:
-------------------------------------------------------------------------------
- LinuxCommando * Fixed bug: "portagedb s" now throws an error message
- LinuxCommando * Added: "portagedb u" checks for root rights
- LinuxCommando * Added:
"portagedb s abc xyz" searches for all packages with either abc or xyz in their name
equivalent to: portagedb s "\(abc\)\|\(xyz\)"
- xororand * Fixed: /etc/portage/package.unmask is now used
-------------------------------------------------------------------------------
v0.2.0_alpha2:
-------------------------------------------------------------------------------
* Fixed bug #1025515: Version numbers now are sorted properly
https://sourceforge.net/tracker/index.php?func=detail&aid=1025515&group_id=117574&atid=678491
Reference: http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1#doc_chap2
* Fixed bug #1025519: Ebuild is available
https://sourceforge.net/tracker/index.php?func=detail&aid=1025519&group_id=117574&atid=678491
* Stability and masking of packages are distinguished:
Masked: Red / [M]
Stable: Green / No prefix
Unstable: Brown / ~
* Different levels of verbosity:
-c Compact: One line per result
-v Verbose: Print licenses
* Color of packages changed from light green to normal green
-> looks better in black-on-white terminals
* The arch is currently determined by a call to "portageq envvar ARCH" which is kinda
slow, taking about 1 second on my system. If someone knows a faster, _reliable_ way,
please tell me.
------------------------------------------------------------------------------
v0.1.3-r1:
------------------------------------------------------------------------------
* Fixed bug #1048527: 'portagedb s' searches in package names AND descriptions
https://sourceforge.net/tracker/index.php?func=detail&aid=1048527&group_id=117574&atid=678491
-------------------------------------------------------------------------------
v0.1.3:
-------------------------------------------------------------------------------
* PORTDIR in /etc/make.conf is checked
* Colored text only uses foreground color so it looks nice on
Black-On-White terminals
* Added new option "--nocolor"
-------------------------------------------------------------------------------
v0.1.2:
-------------------------------------------------------------------------------
* fixed a stupid bug with directory handling. thank you wiebel for debugging. |
Last edited by xororand on Wed Jan 19, 2005 10:32 pm; edited 47 times in total |
|
Back to top |
|
|
dannycool Tux's lil' helper
Joined: 13 Aug 2004 Posts: 111 Location: Karlsruhe Germany
|
Posted: Thu Aug 26, 2004 9:27 am Post subject: |
|
|
Code: | df@kahlann ~ $ psearch() { locate $* | grep "^/usr/portage/[^/\-]*-.*/" | cut -d/ -f4,5 | sort | uniq; }
df@kahlann ~ $ psearch bonobo
app-doc/ebook-bonobo
dev-cpp/libbonobomm
dev-cpp/libbonobouimm
dev-python/bonobo-python
gnome-base/bonobo
gnome-base/bonobo-activation
gnome-base/libbonobo
gnome-base/libbonoboui
(...)
|
*mumble* UNIX *mumble* destined to reinvent *mumble*
SCNR |
|
Back to top |
|
|
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Thu Aug 26, 2004 12:04 pm Post subject: |
|
|
np |
|
Back to top |
|
|
Pythonhead Developer
Joined: 16 Dec 2002 Posts: 1801 Location: Redondo Beach, Republic of Calif.
|
Posted: Fri Aug 27, 2004 7:20 pm Post subject: |
|
|
It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf
Last edited by Pythonhead on Sat Aug 28, 2004 1:51 am; edited 1 time in total |
|
Back to top |
|
|
asph l33t
Joined: 25 Aug 2003 Posts: 741 Location: Barcelona, Spain
|
Posted: Sat Aug 28, 2004 12:49 am Post subject: |
|
|
just instaled it, VERY nice!!
now i can search ebuilds fast _________________ gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep |
|
Back to top |
|
|
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Sat Aug 28, 2004 1:42 pm Post subject: |
|
|
Pythonhead wrote: | It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf |
PORTDIR is now checked in version 0.1.3. Thanks for pointing that out. |
|
Back to top |
|
|
Pythonhead Developer
Joined: 16 Dec 2002 Posts: 1801 Location: Redondo Beach, Republic of Calif.
|
Posted: Sat Aug 28, 2004 4:32 pm Post subject: |
|
|
xororand wrote: | Pythonhead wrote: | It fails if PORTDIR != /usr/portage so it should get that value from /etc/make.conf |
PORTDIR is now checked in version 0.1.3. Thanks for pointing that out. |
Thanks. A lot of developers only use CVS, not 'emerge sync' so they don't use /usr/portage. |
|
Back to top |
|
|
devon l33t
Joined: 23 Jun 2003 Posts: 943
|
Posted: Sat Aug 28, 2004 7:48 pm Post subject: |
|
|
I did not see an ebuild for this on Bugzilla, so I submitted one. Worked fine for me on my x86 system using /usr/local/portage/app-portage/ as the directory. |
|
Back to top |
|
|
asph l33t
Joined: 25 Aug 2003 Posts: 741 Location: Barcelona, Spain
|
Posted: Thu Sep 16, 2004 2:20 pm Post subject: |
|
|
i think it would be great to incorporate more functions to modify how the output is done, do you think it would be posible to incorporate a function like the "c" (short listing) option in esearch? it is very useful and fast to read, since using |more on long searches is a not very practical
keep up good work
*edit*
it seems to be a little bug, since even if you use only the "s" option, it also looks at the descriptions, as if you used "sS"
*/edit* _________________ gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep |
|
Back to top |
|
|
bludger Guru
Joined: 09 Apr 2003 Posts: 389
|
Posted: Fri Sep 24, 2004 9:32 am Post subject: |
|
|
Is it planned to integrate such indexing functionality into portage? |
|
Back to top |
|
|
bludger Guru
Joined: 09 Apr 2003 Posts: 389
|
Posted: Tue Sep 28, 2004 11:49 am Post subject: |
|
|
Maybe my question wasn't clear enough. Doing an esync updates the database so that esearch lists the current state of all applications. If I then install and application, eg. foo, and then do an esearch foo, it is still shown as uninstalled. It stays this way until I do another esync.
It would be great if this database was integrated into portage, so that emerge foo updates the database. |
|
Back to top |
|
|
GrayFox Apprentice
Joined: 24 Jan 2004 Posts: 183 Location: Germany
|
Posted: Tue Sep 28, 2004 1:11 pm Post subject: |
|
|
Strange but a very cool thing here:
Code: | sudo ./portagedb u 0.77s user 0.66s system 85% cpu 1.680 total
|
I don't know any other way to check how long it took and I was to lazy to copy your script. Updating in <2s ? nice
Code: | ./portagedb s notexistant 0.05s user 0.01s system 91% cpu 0.066 total |
really nice! I bet you could somehow manage that the shell appends a portagedb u after every emerge command. _________________ Random Stuff |
|
Back to top |
|
|
LifesizeKenDoll n00b
Joined: 17 Jun 2004 Posts: 27
|
Posted: Wed Oct 06, 2004 2:33 am Post subject: |
|
|
I'd have to say this is pretty cool, although I had to do a little hacking to change the packagename text from white to default (cause I have a black-on-white xterm) |
|
Back to top |
|
|
vdboor Guru
Joined: 03 Dec 2003 Posts: 592 Location: The Netherlands
|
Posted: Wed Oct 06, 2004 9:17 pm Post subject: |
|
|
Just wondering, the esearch package also has a tool called "esync". This tool compares the old cached portage tree with the new one. Do you think you can reproduce that functionality as well? _________________ The best way to accelerate a windows server is by 9.81M/S²
Linux user #311670 and Yet Another Perl Programmer
[ screenies | Coding on KMess ] |
|
Back to top |
|
|
nsahoo l33t
Joined: 17 Jul 2003 Posts: 618
|
Posted: Thu Oct 07, 2004 1:17 am Post subject: |
|
|
it apparently doesn't index the portage_overlay yet. It could be really useful if it can index them as well.
thanks. |
|
Back to top |
|
|
Goya13 n00b
Joined: 12 Aug 2004 Posts: 59
|
Posted: Wed Oct 27, 2004 8:18 pm Post subject: |
|
|
WOW, what an awesome app! I love it.
Just one request - licenses in the output???
thanks so much |
|
Back to top |
|
|
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Mon Nov 08, 2004 1:11 pm Post subject: Version 0.1.3-r1 and 0.1.4 |
|
|
Thanks for your encouragement! So, portagedb hasn't moved alot during the last 2 months. That will change I have implemented some of your suggestions and also some other new features.
devon wrote: | I did not see an ebuild for this on Bugzilla, so I submitted one. Smile Worked fine for me on my x86 system using /usr/local/portage/app-portage/ as the directory. |
An ebuild will come with 0.1.4.
mvila wrote: | it seems to be a little bug, since even if you use only the "s" option, it also looks at the descriptions, as if you used "sS" |
fixed in 0.1.3-r1 -> added 5 bytes to the sourcecode
LifeSizeKenDoll wrote: | I'd have to say this is pretty cool, although I had to do a little hacking to change the packagename text from white to default (cause I have a black-on-white xterm) | I'll check that again in 0.1.4.
vdboot wrote: | Just wondering, the esearch package also has a tool called "esync". This tool compares the old cached portage tree with the new one. Do you think you can reproduce that functionality as well? | Later, sorry.
nsahoo wrote: | it apparently doesn't index the portage_overlay yet. It could be really useful if it can index them as well. | That is planned for the version after 0.1.4.
Goya13 wrote: | Just one request - licenses in the output??? | Version 0.1.4
Here's the current ChangeLog:
Code: |
-------------------------------------------------------------------------------
Changelog for portagedb
-------------------------------------------------------------------------------
Project page: http://portdb.sf.net
Discussion: http://forums.gentoo.org/viewtopic.php?t=214547&highlight=portagedb
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Ideas for future versions:
-------------------------------------------------------------------------------
- Support slotted versions, bug #1025517
https://sourceforge.net/tracker/index.php?func=detail&aid=1025517&group_id=117574&atid=678491
- Support for portage overlays
- Provide a way to automatically update portagedb after emerges
- Test functionality on other platforms than x86
- Option for arbitrary formatting
- Replicate esearch's esync.py
- Option '-i' for restricting search results to installed packages
- Bash completion for emerge/genlop etc.
- Test performance with Intel's C++ Compiler
-------------------------------------------------------------------------------
v0.1.4:
-------------------------------------------------------------------------------
- Fixed bug #1025515: Version numbers now are sorted properly
https://sourceforge.net/tracker/index.php?func=detail&aid=1025515&group_id=117574&atid=678491
- Fixed bug #1025519: Ebuild is available
https://sourceforge.net/tracker/index.php?func=detail&aid=1025519&group_id=117574&atid=678491
- Stability and masking of packages are distinguished:
Stable-Unmasked: Green / No prefix
Unstable-Unmasked: Brown / ~
Masked: Red / [M]
- Different levels of verbosity:
-c Compact: One line per result
-v Verbose: [insert desc]
- Improved error handling
|
50-70% of the planned code is done. I hope to be ready in about 1 or 2 weeks. |
|
Back to top |
|
|
SuperJudge Apprentice
Joined: 08 Jun 2004 Posts: 177 Location: Albany, GA
|
Posted: Thu Nov 25, 2004 1:04 am Post subject: |
|
|
I was frightened how quick that was. I had gotten used to starting emerge -S and not paying attention to it for a long time. _________________ ---Powered by Honda |
|
Back to top |
|
|
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Sun Dec 12, 2004 2:55 am Post subject: |
|
|
I added an ebuild file of version 0.1.3-r1 for easier installation.
Get it here. |
|
Back to top |
|
|
skally n00b
Joined: 16 Mar 2004 Posts: 33
|
Posted: Thu Dec 16, 2004 9:03 pm Post subject: |
|
|
0.1.4!
0.1.4!
This is what we're waiting for!
_________________ -- skally |
|
Back to top |
|
|
Pythonhead Developer
Joined: 16 Dec 2002 Posts: 1801 Location: Redondo Beach, Republic of Calif.
|
Posted: Thu Dec 16, 2004 10:24 pm Post subject: |
|
|
I had a couple problems with it. First it failed with libtool version mismatches:
Quote: | *** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.10, ltmain.sh = 1.5a) ***
|
Changing src_compile() to this fixes it:
Code: |
src_compile() {
aclocal
libtoolize --force --copy
autoconf
econf || die "econf failed"
emake || die "emake failed"
} |
Second, it doesn't work unless you run
Code: | /usr/bin/portagedb u |
from your root directory
Quote: | # portagedb u
Updating portage database
Using portage cache: usr/portage/metadata/cache/
Error: scandir64("usr/portage/metadata/cache/", ..) failed
Database update failed
|
Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1.
Runs amazingly fast, nice job. Number one feature request: overlay directories |
|
Back to top |
|
|
SuperJudge Apprentice
Joined: 08 Jun 2004 Posts: 177 Location: Albany, GA
|
Posted: Sun Dec 19, 2004 4:47 pm Post subject: |
|
|
Pythonhead wrote: | I had a couple problems with it. First it failed with libtool version mismatches:
Quote: | *** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.10, ltmain.sh = 1.5a) ***
|
Changing src_compile() to this fixes it:
Code: |
src_compile() {
aclocal
libtoolize --force --copy
autoconf
econf || die "econf failed"
emake || die "emake failed"
} |
Second, it doesn't work unless you run
Code: | /usr/bin/portagedb u |
from your root directory
Quote: | # portagedb u
Updating portage database
Using portage cache: usr/portage/metadata/cache/
Error: scandir64("usr/portage/metadata/cache/", ..) failed
Database update failed
|
Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1.
Runs amazingly fast, nice job. Number one feature request: overlay directories | Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back. _________________ ---Powered by Honda |
|
Back to top |
|
|
Pythonhead Developer
Joined: 16 Dec 2002 Posts: 1801 Location: Redondo Beach, Republic of Calif.
|
Posted: Sun Dec 19, 2004 6:11 pm Post subject: |
|
|
SuperJudge wrote: | Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back. |
libtool isn't broken, ebuilds are:
http://article.gmane.org/gmane.linux.gentoo.devel/23449/match=gentoo+libtoolize
So its going to be a pain for a while but should fix some real errors in the long run. |
|
Back to top |
|
|
SuperJudge Apprentice
Joined: 08 Jun 2004 Posts: 177 Location: Albany, GA
|
Posted: Sun Dec 19, 2004 7:37 pm Post subject: |
|
|
Pythonhead wrote: | SuperJudge wrote: | Is there a way to fix libtool, every other damn ebuild needs to be modified because of libtool version errors. I used to have an older version of libtool installed, but I'm tired of switching back. |
libtool isn't broken, ebuilds are:
http://article.gmane.org/gmane.linux.gentoo.devel/23449/match=gentoo+libtoolize
So its going to be a pain for a while but should fix some real errors in the long run. | I figured it was the ebuild, but I couldn't search out a good fix, and it seemed to work fine when I used an older ebuild. Thanks for the info. _________________ ---Powered by Honda |
|
Back to top |
|
|
xororand Tux's lil' helper
Joined: 22 Jul 2004 Posts: 119
|
Posted: Fri Dec 31, 2004 2:49 pm Post subject: |
|
|
Pythonhead wrote: | I had a couple problems with it. First it failed with libtool version mismatches. Changing src_compile() to this fixes it: | Thanks for the detailed notice. I'll fix it when 0.2.0 is final. The 0.1.3-r1 eBuild will be fixed as soon as possible.
Quote: | Second, it doesn't work unless you run Code: | /usr/bin/portagedb u | from your root directory | Unfortunately, I can't reproduce this behaviour. Do you use a custom PORTDIR (make.conf)?
Quote: | Also, it'd be easier if you named your tarball files without the -r1. Its just a little confusing since you have S=${WORKDIR}/${P}-r1 and the ebuild named with -r1. |
Thanks again for the hint. Meanwhile I've studied the portage help document and learned about proper version numbering. The new version is 0.2.0 - patch releases will be named with a _p suffix from now on.
By the way, 0.2.0_alpha2 is available on Sourceforge I hope it works. |
|
Back to top |
|
|
|