Portage utilities not in portage

[Pythonhead]

Mod edit: Unstuck, list is unmaintained and the script repo is discontinued. --kallamej 2009-10-22

I'm making a list of portage utilities that aren't in portage. These are utilities or applications that would fit in the app-portage category, not general Gentoo related software. Private Message me and/or add your utility to this thread and I'll update the list.

If you sent me a utility to list but you don't see it, please send it again. I may have overlooked it.

THIS LIST IS FROZEN!
Please visit the new script repository:
http://gentooexperimental.org/script/repo/list

I won't be updating this list any longer. The script repository now has all scripts in this list (and more) available for direct download.

Last updated: Mar 24, 2005

Now listed by Category:

Newest additions, not categorized yet:

enotice Tackles the most popular portage feature request ever: per-package message logging.
The patches are outdated, so use the second method. It only takes a minute to setup, its painless and its just what you need.
http://dev.gentoo.org/~eldad/

Feb 18, 2005:
Genu is a portage front-end written in mono.
http://www.markus-jonsson.com/projects/genu/
http://forums.gentoo.org/viewtopic-t-297256.html

Feb 09, 2005:
prlock.py (Portage Rsync Lockdown) creates a exclude list for portage's rsync. This can dramatically reduce the size of your rsync file list.
WARNING: Follow the directions. If you get some kind of dependency error you'd better comment RSYNC_EXCLUDEFROM and do a full sync before reporting any bugs on bugs.gentoo.org
http://forums.gentoo.org/viewtopic.php?t=55031

Jan 20, 2005:
rtkmerge is back from the grave and has been updated
to work with current versions of portage:
http://rtkmerge.webhop.net/
or
http://www.forchheimer.se/rtkmerge/

Jan 05, 2005:
portage browser is a php script to browse your local portage tree.
http://forums.gentoo.org/viewtopic.php?t=274819

Dec 14, 2004:
md5check is a portage md5 checksum verification tool. It can show you which files have changed or been removed since you emerged a package.
http://forums.gentoo.org/viewtopic.php?t=266365

unclepine helps you with un-merging recursive dependencies.
http://forums.gentoo.org/viewtopic.php?t=260866

Nov 16, 2004:
regenpkgdb will attempt to re-generate a "Oops! I needed that?" /var/db/pkg
http://dev.gentoo.org/~spyderous/scripts/regenpkgdb

Nov 12, 2004:
portage toys is perhaps the name of a script for per-package CFLAGS. See these for info:
http://forums.gentoo.org/viewtopic.php?p=1749445
http://thread.gmane.org/gmane.linux.gentoo.devel/21760

USE & CFLAGS related utilities

• bininfo will give you USE, CFLAGS, etc from packages you created with quickpkg:
  http://forums.gentoo.org/viewtopic.php?t=230287
• used is a ufed replacement.
  http://forums.gentoo.org/viewtopic.php?t=232973
• usechange (Obsolete with latest version of portage's --newuse ). uses the information in /var/db/pkg/* to determine the USE flags with which each installed package was compiled and checks those against the ones currently in effect
  http://www.coe.uncc.edu/~danderse/www/usechange
• usetool will give you a detailed analysis of package USE flags.
  Update: New version available, re-written from scratch
  in Python:
  http://xchris.altervista.org/GECHI/usetool/
  http://forums.gentoo.org/viewtopic.php?t=203823
• Gufo searches for packages compiled with selected cflags and recompiles them.
  https://sourceforge.net/projects/gfr/

etc-update replacements
Ever get the feeling people don't like etc-update?

• dispatch-conf is the "etc-update"-like tool in python, which uses an RCS repository for configuration backups.
  http://www.blorf.net/dispatch-conf/
• cfg-update is a replacement for etc-update which uses an external grpahical diff progam of your choice.
  http://forums.gentoo.org/viewtopic.php?t=86622
• config-rename
  Yet another replacement for etc-update
  http://iaindb.customer.netspace.net.au/

emerge utilities

• portal is like pye (Pick Your Emerge) or rtkmerge. Portal lets you choose which packages you want to emerge from the output of "emerge --pretend world" (or system etc). You can also add packages to the various /etc/portage/* files and edit per package USE flags and add them to /etc/portage/package.use
  Screenshot: http://portal.sf.net//portal.png
  http://portal.sf.net/
• remerge is a web browser based interface to /usr/bin/emerge
  http://remerge.sourceforge.net/
• pye
  pye (Pick Your Emerge) gives you a numbered list based on the output of emerge -pU world. You can then choose what to emerge by typing in the corresponding numbers.
  http://dev.gentoo.org/~strangedr/files/pye
  http://dev.gentoo.org/~strangedr/
  Discussion:
  http://forums.gentoo.org/viewtopic.php?t=135968
• efetch-wrapper
  Extends emerge's use of --fetch (wget) so that you can download over ssh or rsync-ssh.
  http://iaindb.customer.netspace.net.au/
• appear downloads packages while emerge is compiling
  http://l.armory.com/~cerise/appear-latest.tbz2 Bad link. Anyone still have a copy of this?
• emerge-uri.sh generates a list from emerge -pf on a computer with no net connection. download-uri.sh downloads the packages for you on a computer with a connection to the ineternet.
  http://forums.gentoo.org/viewtopic.php?t=79884
• gportage is a simple GTK-based portage package browser.
  http://www.stacken.kth.se/~foo/gentoo/
  (Discontinued in favor of Porthole which is now in portage)
• semerge downloads packages while compiling
  http://forums.gentoo.org/viewtopic.php?t=37606
• glcu - gentoo linux cron update - Not your average emerge sync from cron script. Does GLSA checks, email on failed emerges and more.
  http://forums.gentoo.org/viewtopic.php?p=1304452
  website: http://www.panhorst.com/glcu/
• eMenu provides common emerge commands, and, more usefully, captures messages outputed by portage so you can view them later.
  http://www.dopesmoker.net/files/emenu.sh
• femerge allows you to filter through what emerge wants to install to deselect packages you didn't want after all (e.g. after an emerge -up world). It still tracks dependencies, of course. This is powered by the more general script "pfilter", which can filter package listings output from emerge.
  http://forums.gentoo.org/viewtopic.php?t=45827
• bemerge allows you to browse the portage tree either amongst all available or all installed packages, and perform some emerge command on any subset of these. This is powered by pbrowser, which is a generic script that lets you select packages from the ones in /usr/portage or in /var/db/pkg (i.e. installed packages) and output them.
  http://forums.gentoo.org/viewtopic.php?t=46794
• ehush is a wrapper for emerge that supresses compiling information and only prints the "important" stuff. It logs the full output if you need to see it. (This horrible hack should be forgotten in favor of enotice.)
  See this thread for more info:
  http://forums.gentoo.org/viewtopic.php?t=76161
  http://abeni.sf.net/ebuilds/ehush
• bumper "bumps up" a version of a package in portage when there is a new download available but no ebuild. It copies the ebuild to the overlay directory with the higher version number and creates a digest. See this post:
  http://forums.gentoo.org/viewtopic.php?p=445501#445501
  http://dev.gentoo.org/~pythonhead/bumper/bumper-0.0.10
• rtkmerge is a GUI for emerge. When doing 'emerge world' (or system etc) the user is presented with a list of packages. You can easily select which packages you want and don't want. A new and interesting approach.
  http://rtkmerge.webhop.net/
  or
  http://www.forchheimer.se/rtkmerge/

Cleaning utilities

• unmerge-rdep is a patch to portage to do a better job of unmerging dependencies:
  http://forums.gentoo.org/viewtopic.php?t=210288
• pkdep removes unused dependencies for packages. See this thread for the script and info:
  http://forums.gentoo.org/viewtopic.php?t=83563
• eclean is yet another cleaning script for Gentoo. It is based on some ideas from the Cleaning out stale distfiles thread, that in fact also apply well to binary package cleaning, and thus can do both. Nothing really new inside in fact, i've just tried to give a clean and fast answer to this now classic problem.
  http://bugs.gentoo.org/show_bug.cgi?id=33877
• distclean.py is a Python script that removes all source files (e.g. tarballs) that are not associated with installed packages from the DISTDIR directory.
  http://www.stacken.kth.se/~foo/gentoo/
• source_clean.sh removes source files from DISTDIR which are not in the world file anymore.
  http://forums.gentoo.org/viewtopic.php?p=429213#429213
• misc dist tidiers is a thread with several snippets of code that do what distclean and source_clean do. Actually dist_clean was born there.
  http://forums.gentoo.org/viewtopic.php?t=3011

Informational utilities

• portagedb is similar to app-portage/eseach but was coded in hopes of faster update and search times. Its written in C++ and has its own Sourceforge page so I think we may have a serious contender.
  Now in portage as app-portage/eix
  http://portdb.sourceforge.net/
  Discussion of portagedb:
  http://forums.gentoo.org/viewtopic.php?p=1469897
• find-deps.py shows all packages that have a dependency on a given package. Similar to "qpkg -q", but that only shows installed pacakges.
  http://dev.gentoo.org/~ferringb/find-deps.py
• earch shows the ARCH status for each ebuild when run in that package's directory.
  http://dev.gentoo.org/~eldad/earch/earch
• gest calculates the time for a package to be re-emerged based on previous emerges. It will work on several packages at once. See this thread:
  http://forums.gentoo.org/viewtopic.php?t=70172
  http://mywebpages.comcast.net/dmsmith/gest
• portlog-info is a small bash script that can parse content of your /var/log/portage (or whatever else is defined as PORT_LOGDIR in /etc/make.conf) to retrieve different kind of messages (einfo, emerge errors, etc.) in per-package emerge logs.
  http://tdegreni.free.fr/gentoo/portlog-info
• eprogress
  Ever wondered how long an ebuild is going to take? Or perhaps you'd like to see a list of all previous failed/interrupted ebuilds?
  http://forums.gentoo.org/viewtopic.php?t=197646
• compileTime shows compile time, date etc of emerge
  http://forums.gentoo.org/viewtopic.php?p=410143#410143
• fastsearch is a lightning fast search script for the portage tree.
  http://gentoo.devel-net.org/download/fastsearch
• ecat allows you to select an ebuild from a list of available versions then view the ebuild.
  http://forums.gentoo.org/viewtopic.php?p=565237#565237

Portage replacements (database backends)

• portage-c
  This isn't a portage utility, this is... portage! Written in C. With an sqlite backend. Zooooom.
  http://sourceforge.net/projects/portage-c/
• portagesql is an implementation of portage using SQL
  http://breakmygentoo.net/
  (Seems to be dead)

Miscellaneous

• dynamic deltup server is a revival of the old and known deltup but with a download-wrapper script transparent to portage and a server that creates the delta-files on demand (if not available in its cache)
  Make sense? Good. Get it:
  Now in portage! Congratulations.
  http://linux01.gwdg.de/~nlissne/deltup- ... atime.html
  And the announcement:
  http://forums.gentoo.org/viewtopic.php?t=215262
• einjected searches your system for packages you emerged with --inject
  This is useful with the switch to using /etc/porgate/package.provided in the latest portage version
  http://www.stud.uni-karlsruhe.de/~uxhz/ ... /einjected
• distdir_overlay allows multiple distfile directories, similar to PORTDIR_OVERLAY. It appears to be in the testing phase. The authors say there will be a webpage for the project soon, with English:
  http://forums.gentoo.org/viewtopic.php?t=222541
• GLIS is the Gentoo Linux Install script
  http://glis.sourceforge.net/
  Seems to be inactive. See this if you're interested:
  http://www.gentoo.org/proj/en/releng/in ... /index.xml
• license-checker
  http://gentoo.devel-net.org/portage/license-checker
  Remove packages by LICENSE
• licenses2kill.sh
  http://gentoo.kicks-ass.org/scripts/licenses2kill.sh
  Remove packages by LICENSE
  See this thread for discussion:
  http://forums.gentoo.org/viewtopic.php? ... highlight=
• kdedelta.sh
  This shell script will download an xdelta from the KDE mirrors, greatly reducing download size.
  http://linux01.gwdg.de/~nlissne/kdedelta.sh
  http://forums.gentoo.org/viewtopic.php?t=146973
• imlate seeks to make arch maintainance easier by comparing the current stable version of an ebuild for one arch with another. x86 is the default comparison.
  http://dev.gentoo.org/~gmsoft/tools/imlate
• ecatmur's stuff
  This is a directory of portage utilities mixed in with some non-portage things:
  http://home.jesus.ox.ac.uk/~ecatmur/my-bin/
• These unmasking scripts may be obsolete due to improvements in portage (See /etc/portage/package.unmask,package.keywords etc.)
• unmask is a package unmask-erator. See this thread for details:
  http://forums.gentoo.org/viewtopic.php?t=42845
  http://www.gforum.org/unmask
• merge_mask is another pacakage unmask-erator.
  http://www.breakmygentoo.net/ebuilds/merge_mask

[Genone]

nice summary, some notes
- deltup is in portage (sys-apps/deltup), but there are only a few delta files available
- splat was added yesterday (app-admin/splat)
- your repoman-local link pointed to the first version, I fixed this and symlinked it to version 6 on the server (for current stable portage).
- additional package: genlop (in portage: app-admin/genlop, URL: http://pollycoke.org/genlop.html ), another logfile parser, IIRC splat is based on this.

[Pythonhead]

nice summary, some notes
- deltup is in portage (sys-apps/deltup), but there are only a few delta files available
- splat was added yesterday (app-admin/splat)
- your repoman-local link pointed to the first version, I fixed this and symlinked it to version 6 on the server (for current stable portage).
- additional package: genlop (in portage: app-admin/genlop, URL: http://pollycoke.org/genlop.html ), another logfile parser, IIRC splat is based on this.

Thanks for corrections/additions.

I'll modify the original post. Since I don't know of any list of all the portage utilities in portage either I think I'll divide the list into utils in portage and not in portage.

[haceye]

Hi,

I just saw the 'fastsearch' script and tried it. It seems as if it does the same job than my 'esearch' script.
Please don't understand me wrong, I don't want to praise my script, but I think it's more useful than 'fastsearch'. On my computer it generates the index a few times faster, and it has the advantage, that the output (should) be exactly the same as "emerge -s".

I would be very happy if you could take a look at it. The link to the ebuild is in my signature.

Thanks,

David

[Genone]

At one point a new category app-portage (or sys-portage) would be useful, I'll propose this on the -dev list (I found at least 15 portage-related packages in the tree, enough for a new category).

[Pythonhead]

Hey haceye,

I really like the output of esearch. I timed fastsearch and esearch doing an update of the database on my old clunker (700mhz w/ IDE drive):

esearch: 18 minutes
fastsearch: 3 minutes

On your website it shows an update time of 18 seconds. How did you get that amazing result?

[Genone]

I just saw the 'fastsearch' script and tried it. It seems as if it does the same job than my 'esearch' script.
Please don't understand me wrong, I don't want to praise my script, but I think it's more useful than 'fastsearch'. On my computer it generates the index a few times faster, and it has the advantage, that the output (should) be exactly the same as "emerge -s".

Well, my goal was not to replace emerge -s, something I don't like about it (beside the slowness) is that the result list can get very long because there are at least 6 lines per package, and I often only want to get the exact package name or the description (I might include a --verbose switch in a future version).

I just checked your script, some notes:
- it lacks a description-only search (not that important)
- the index generation is faster ? It says that it needed 20 minutes on my desktop (while fastsearch only needs 1 or 2).
- python regular expressions vs. grep expressions and options (I don't know if there are substantial differences)
- I don't like some parts of the update code (why unlinking the database source ?)
- relies on portage for database updates
- stores the database in /usr/lib
- oh, and of course my script is a bit faster and the db file smaller

I don't say it is bad but it doesn't fit my needs.

[pjp]

Moved from Portage & Programming.

[haceye]

Well, my goal was not to replace emerge -s, something I don't like about it (beside the slowness) is that the result list can get very long because there are at least 6 lines per package, and I often only want to get the exact package name or the description (I might include a --verbose switch in a future version).

Ok, that's an argument, but it is pretty easy to rewrite the output code, so it displays the results in a more compact way.

I just checked your script, some notes:
- it lacks a description-only search (not that important)

I never needed this feature and I didn't see that in emerge, I think it's really not that important...

- the index generation is faster ? It says that it needed 20 minutes on my desktop (while fastsearch only needs 1 or 2).

I just re-checked this. Your script is sometimes faster sometimes myone is (both about a minute on my 2,6Ghz P IV), I think these 18 seconds were measured after a previous call of 'eupdatedb', so maybe portage or my harddisk/filesystem cached some of the date somewhere, I don't know

- python regular expressions vs. grep expressions and options (I don't know if there are substantial differences)

Again, I used the python regular expressions because emerge did so, too. Secondly, my script is written in python, so why use grep?

- I don't like some parts of the update code (why unlinking the database source ?)

You're right, it really doesn't make any sense (ok, maybe to save some kilobytes on harddisk:-))

- relies on portage for database updates

Ok, but...? If you don't have portage, you probably won't search the portage tree?!

- stores the database in /usr/lib

Sorry, really didn't knew a better place to store it...

- oh, and of course my script is a bit faster and the db file smaller

That's a bit unfair . My script has to store the latest version (installed), size of download and the homepage, too.

Ok, but I have to confess. 20 minutes to update the index-file is definately too long!

David

[Jimbow]

I have a Dell inspiron 8200 laptop with a 1.6 GHz P4. eupdatedb took 55 secs. Kudos to you both. I love the faster searches!

[TGL]

Thanks Pythonhead for this very useful summary.

I would add this to your list:
http://www.blorf.net/dispatch-conf/
It's a special version of "dispatch-conf" (the "etc-update"-like tool in python), which use a RCS repository for configuration backups.

Note also that it will probably soon be an "app-portage" category for this kind of tools: http://article.gmane.org/gmane.linux.gentoo.devel/10274
It will be easier to know what does exists in the tree this way.

[Genone]

Note also that it will probably soon be an "app-portage" category for this kind of tools: http://article.gmane.org/gmane.linux.gentoo.devel/10274
It will be easier to know what does exists in the tree this way.

Hey, I just proposed it, so far only one comment from liquidx, waiting for further input.

[TGL]

Hey, I just proposed it

Oops, sorry, I've not seen your previous "I'll propose this on the -dev list", and have thought this was only a coincidence

so far only one comment

Probably because everybody think it's a too obviously good idea

[Pythonhead]

I have a Dell inspiron 8200 laptop with a 1.6 GHz P4. eupdatedb took 55 secs. Kudos to you both. I love the faster searches!

Sheesh. Guess its time for a new pc for me.

I have been using both fastsearch and esearch extensively while coding ebuilds and they both work great and fill a slightly different need. Variety is good.

One feature I'd like to see for developers is a switch to allow just PORTDIR_OVERLAY get searched. I guess this would take two separate databases; I haven't looked at the code for either. I only emerge sync once a week or so but I'm constantly doing work in PORTDIR_OVERLAY. Not a big deal, but it would be nice if it wouldn't take a lot of effort to add it.

It would also be very sweet to have a switch to show what etcat -v shows. Of course this would get out of date very quickly as you emerge packages, so could be misleading to rely on. Maybe not so good an idea, unless you emerge sync every day.

Another feature would be to show the time you last updated the database and the last time you did emerge sync every time it displays the search results.

[Yinchie]

...
I timed fastsearch and esearch doing an update of the database on my old clunker (700mhz w/ IDE drive):

esearch: 18 minutes
....

celeronator esearch # eupdatedb
* esearch-index generated in 452 seconds
* indexed 4932 ebuilds
* size of esearch-index: 759 kB

452 seconds = ~7.5 minutes.
(433 Mhz celeronator w/ IDE drive)

[Genone]

See my other thread for further fastsearch discussion.

[Genone]

Ok, I bugged seemant about it, looks like the category will be added (but he liked my link to http://mozex.mozdev.org more )

[floam]

You can take the unmasking stuff out of the list, since portage can do that nativly now.

[epretorious]

Are these utilities available as a package?

[haceye]

Hi,

Thanks for your feedback Pythonhead.

I have been using both fastsearch and esearch extensively while coding ebuilds and they both work great and fill a slightly different need. Variety is good.

Yes, that's right. And we won't try to find the better one

One feature I'd like to see for developers is a switch to allow just PORTDIR_OVERLAY get searched. I guess this would take two separate databases; I haven't looked at the code for either. I only emerge sync once a week or so but I'm constantly doing work in PORTDIR_OVERLAY. Not a big deal, but it would be nice if it wouldn't take a lot of effort to add it.

Genone has already answered to this in his fastsearch-thread. Looking through his code, I think it wouldn't be that much work to seperate PORTDIR and PORTDIR_OVERLAY, but I also think that's not that important. As I said above, the index-generation should be a lot faster, if you use it in short time intervals, maybe that could help you a little bit.

Btw.: In esearch it's harder to realize this feature, because it depends on the portage code (portage.portdb.cp_all()) to get the packages, and portage makes no difference between the two directories, i think.

It would also be very sweet to have a switch to show what etcat -v shows. Of course this would get out of date very quickly as you emerge packages, so could be misleading to rely on. Maybe not so good an idea, unless you emerge sync every day.

In the new version of esearch I added the option --compact. It compresses the output to a minimal.
Take a look at the example on http://david-peter.de/esearch.

(Thanks to Genone for the idea of having a one-line output for each package )

Another feature would be to show the time you last updated the database and the last time you did emerge sync every time it displays the search results.

Well, there are several scripts in the forums doing this job (didn't you write one, once? )
I think it would just bloat the code for the index-generation.



esearch-0.3.0
So, this is the new version of esearch. I changed a few things:
* database is stored in /var/cache/edb (Thanks to Genone, again)
* duration of index-generation is displayed in seconds or minutes and seconds if time > 60secs
* --nocolor switch, self-explaining
* --compact option, minimizing the output to one line per package
* bugfix, for some packages there are no ebuilds. Not including them in database.
* Not unlinking the source-file, anymore

Homepage: http://david-peter.de/esearch/
EBuild: http://david-peter.de/esearch/esearch-0.3.0.ebuild

David


Edit: If you're installing the new version, you have to delete the old database-files (/usr/lib/esearch/esearchdb.py and /usr/lib/esearch/esearchdb.pyc), thanks for testing

[epretorious]

Seeing this list of "helpers" makes me wonder about...

1. The pace and direction of portage development*, and
   The documentation or lack thereof for portage

Any thoughts on these vexing questions?

*IMHO - A really well-thougt-out package management system won't need helpers.

[Pythonhead]

Are these utilities available as a package?

No, thats the whole point of this thread, to make a list of all the utilities that aren't in packages in portage. Hopefully as people try them out and they get popular they will get into portage, as a few already have.

[Pythonhead]

Pythonhead wrote:
Another feature would be to show the time you last updated the database and the last time you did emerge sync every time it displays the search results.

Well, there are several scripts in the forums doing this job (didn't you write one, once? )
I think it would just bloat the code for the index-generation.

If it showed the last time you updated the database and the last time you did 'emerge sync' you'd know how accurate your search results would be. Not a big deal if you simply eupdatedb right after portage, but I'm always adding ebuilds in PORTDIR_OVERLAY. So, no, its not critical for users not adding their own ebuilds.

Thanks

[Mystilleef]

If all these changes make it into portage, it will rapidly be becoming an indeed advanced, intelligent and sophisticated package manager. Kudos to you all. I particularly found esearch amazing!

Regards,

Mystilleef

[Genone]

The main problem with portage at the moment is that there is only one dev really involved with its development, he currently has about 400 open bugs. This is subject to change, but it will take some time until the changes show results. I hope it explains the current issues with portage.