portage implementation discussion

[adsmith]

Okay, okay -- enough with the pointless your-language's-mom-is-soooo-fat banter.

Does anyone know if the Devs are actively working on refactoring the portage tree into a sane DB-driven system?

I have a thesis to finish [okay.. start], but the image of a small DB for the entire portage tree with dep/rdep/use/... info is just incredibly delicious.

[Genone]

Okay, okay -- enough with the pointless your-language's-mom-is-soooo-fat banter.

Does anyone know if the Devs are actively working on refactoring the portage tree into a sane DB-driven system?

We don't (maybe with the cache, but definitely not the tree itself).

[Vininim]

Anyone who thinks writing a largely string proccessing application in C over Python is a good thing should be slapped.

Hard.

I guess compilers,

Are not string proccessors, they immediately break down things to tokens, which are for the most part constant. An ebuild on the other hand is widly varrying.

browsers,

For the most part should not be written in C/C++. In fact, there are very very few applications that should be written in C.

interpreters don't fit in that category... OR we need to slap a bunch of people around.

Same boat as compilers. What they process is constant and predictable.


The speed gain you would get, if any, you would get from re-writing portage in C/C++ is miniscule compared to what you would achieve through a restructuring of the portage tree(If I use KDE, why do I need to keep the 8 billion GNOME package ebuilds on my machine up to date? Or if I don't play games/run a webserver/any other of a number of specialized roles) And, you would incur a huge overhead in testing and development.

If you are so obsessed about wasted time, use a binary distro. I guarantee the small performance increase you gain from using source would be HUGELY offset by the fact you don't have to compile, IE, how long would you have to use something like KDE with your optimizations before you make up the 2 days in lost time it took you to compile it vs download a binary.

My obsession was never wasted time/perfomance, but the depedency of python in a base-system application. I am not the one advocating C for speed, I am talking about portability. Since people are eager to recreate portage using a different scheme(using a database instead of filesystem repository way), there could be other changes, including languages and having ebuild in a more 'token' way.

[Drysh]

I also don't know why don't portage developers use things like Psyco, optimizers that can increase Python apps's speed painlessly before trying any port to C++.

I read the whole thread trying to find an answer... So... Let's ask again:
Why don't portage use Psyco??? (Maybe a use key to let it run with psyco?)
(I was searching about Psyco when I found this.)

But since I'm posting here... From the point of view of a user, Portage would benefit a lot if:

- A DB alowed us to stop using a tree-like way, and start using search keys to find packages. Instead of guessing if a package is under gnome-extra or x11-apps, I would search for a package that has the keywords gnome, app and x11. Not a tree, but a search in a db.

- We had a definition of each use key. For instance, +mozilla is used for the mozilla suite and any mozilla browser (firefox or suite); so when I set +mozilla, I have to edit package.use to exclude it from some applications that want the mozilla suite (like openoffice).

- The names in portage tree were writen in the singular or plural. (that's not a joke) ... I was trying to find out why my package.mask wasn't working: it's x11-apps (not x11-app)! But since we have app-*... Ok... You see what I'm talking about.

- Why php, and php4, and php5 ??? A db, please!

- Something simple - The date (of last edit or something) in the search result. I'm tired of emerging something just to find out its 2 years old (or having to look into the e-build to find out when it was last edited). Things grow old very fast in software development.

[Shadow Skill]

Just split the damned tree and get rid of stupid files like package.keywords and package.unmask I get really tired of typing echo because I need to force something when it wouls make a great deal more sense to just enable the cvs tree for a given class of programs and go from there. Portage forces you to use a whole song and dance routine just because of the nature of the tree when you want to install certain things and you have to go ~arch or use a cvs ebuild. It would also be alot easier to ensure that only gnome apps get installed or only kde apps get installed if the tree was split in such a way that one could just enable the gnome tree to get all the gnome related apps [this would in no way nessecarily include gtk applications in theory at least.] or enable the kde tree to get all kde related applications. if you enabled a flag that required another tree be enabled to satisfy a dependency you should be prompted to have portage enable it on the fly and if you want it enabled with no prompt on the fly there should be a CLI switch and something one can set in make.conf.

Please for the love of God default Portage to emerge --ask and abstract that flag away, and put in a --yes flag. Without interactive USE flags ala Sourcemage one is going to probably want to use pretend alot to see if flags are being enabled or disabled properly or if there are new flags available for a program; it makes more sense to use emerge --ask since if all is well I can just hit "y" and let Portage work its magic, instead of typing two commands and stringing them together with a semi colon or a dual ampersand. Not to mention it would help prevent people from accidentally doing something bad to thier system if it defaults to requiring user input before execution by default, to not have it this way at all is just stupid.

I think this is actually planned for the next major version of Portage but could Portage please download all pertinent tarballs before it starts compiling stuff...Oh lets not forget the obligatory inline reverse dependency resolution request; a DB of course like the previous poster said as well, and the different tree indexes need to be compressed downloading hundreds or even thousands of ebuilds per tree is not acceptable. For package.mask and package.use being forced to do category/appname needs to be eliminated entirely Portage should be able to figure that out on its own unless two different programs in different categories share the same name, which does happen occassionally.

[Carlo]

Why don't portage use Psyco???

!!!

Please for the love of God default Portage to emerge --ask

Write yourself a bash script or alias that calls emerge --ask.

not [to] have it this way at all is just stupid.

Wrong. Most people don't want to be the slave sitting next to their box during emerge runs, confirming all and everything.

[Shadow Skill]

Why don't portage use Psyco???

!!!

Please for the love of God default Portage to emerge --ask

Write yourself a bash script or alias that calls emerge --ask.

not [to] have it this way at all is just stupid.

Wrong. Most people don't want to be the slave sitting next to their box during emerge runs, confirming all and everything.

Writing a script or alias is stupid because there would be no way to take off the --ask flag if I wanted it to just go ahead let things operate without being prompted. Only someone who is not thinking straight creates a system that defaults to no user interaction then puts a flag in to turn on user interaction then never puts a flag to override the question. In order for a script or alias to work I would have to make an entirely new command alias when the program itself should already be setup to do that. You don't turn on confirmation you turn it off at will, you talk about not being a slave to emerge runs but I am already a slave to it because I have to do a song and dance routine to get the thing to download and then install ebuilds, I have to do another song and dance to see what use flags are new and what they do; some flags don't even have any information for some incoprehensible reason. Do you honestly think I want to sit at my machine and do

Code: Select all

emerge --pv foo; emerge -f foo; USE=foo emerge foo

when I could have Portage prompt for new flags or prompt for all disabled flags ignoring the ones that are enabled depending on how I turn on the feature in make.conf? Which do you think takes less time and eliminates the need for certain commands and makes me less of a slave to emerge runs, a few questions that also tell you what the flag does or a song and dance routine where I have to pretend first then do equery uses foo [which only seems to work for installed packages currently.] then do two seperate emerge commands because Portage doesn't download things first as I said earlier?

[Genone]

I also don't know why don't portage developers use things like Psyco, optimizers that can increase Python apps's speed painlessly before trying any port to C++.

I read the whole thread trying to find an answer... So... Let's ask again:
Why don't portage use Psyco??? (Maybe a use key to let it run with psyco?)
(I was searching about Psyco when I found this.)

x86 only, huge memory waste, crashes randomly, ...

- A DB alowed us to stop using a tree-like way, and start using search keys to find packages. Instead of guessing if a package is under gnome-extra or x11-apps, I would search for a package that has the keywords gnome, app and x11. Not a tree, but a search in a db.

Duplicate package names become problems (assuming you want to use package names as keys), reduces flexibility a lot.

- We had a definition of each use key. For instance, +mozilla is used for the mozilla suite and any mozilla browser (firefox or suite); so when I set +mozilla, I have to edit package.use to exclude it from some applications that want the mozilla suite (like openoffice).

I don't get what you're trying to say here.

- The names in portage tree were writen in the singular or plural. (that's not a joke) ... I was trying to find out why my package.mask wasn't working: it's x11-apps (not x11-app)! But since we have app-*... Ok... You see what I'm talking about.

A DB won't fix any semantic srewups.

- Why php, and php4, and php5 ??? A db, please!

A DB won't help with any transition problems.

[Drysh]

I'll explain myself... As a matter of fact none of my suggestions really requires a DB, I was mixing implementation with the idea. So, I'll break them in 2 different ideas:

1. A keyword based search engine instead of a tree based one (or both).
- Put all e-builds in the same directory (there is an all directory in portage already). Forget about the tree. All packages should have unique names, like they have today, but instead of something/something make it (for instance) something-something.
- Create some keywords to add in the e-build to find the packages. Like: x11, gnome, kde, app, browser, plug-in, etc... I'll give one example using evolution, the gnome mail-client:

Lets add these keywords to the following e-builds:
mail-client/evolution: x11, gnome, mail, mail-client, office
gnome-extra/evolution-data-server: x11, gnome, office
gnome-extra/evolution-exchange: x11, gnome, office, mail, plug-in

So, when we make a search for gnome, we will find all the 3 packages. IMHO, the tree only causes confusion if you have a lot of packages (and Gentoo is growing). The keyword approach (maybe using a DB to speed up the searches, but not necessary) may be better for a huge amount of packages.

2. The Uniform Keyword (UK) project.
Since we have more e-builds each day, the keywords sometimes mean different things for different packages. For instance, the mozilla use keyword means that it should connect specifically with the mozilla suite for OpenOffice (requiring mozilla package) and it only means to build with support to any mozilla browser (read firefox) for other e-builds (thus requiring firefox or mozilla packages). Once you learn that you should use mozilla and firefox keywords for some and only firefox for others it works fine, but I think that a manual or something to help the e-build developers (and users) would be nice.
Another problem is that different people use different standards to name things. That increases the probability of mistakes. Example: app and apps.
My idea to minimize that: a wiki page with keywords descriptions used and updated by the developers; some guidelines to the e-build developers (always use singular, read the wiki before creating a new keyword, post there any new use of a keyword, etc).
This is more a political issue than a technical one. But it has some serious technical implications.

3. The "last edited". (I know I said it would be 2 ideas...)
Just include the last edited date in the output of emerge -s.

I'm posting these suggestions because Gentoo is growing (and fast). It doesn't mean its broken, it means the more packages we have it becomes harder to find and manage them. And we should expect the number of packages increasing more and more. None of these are critical now, but now seems a good time to start thinking about it, since it will be harder to do that later (when this becomes a critical issue).

Thks for the answers (I guess I'll have to think twice before using psyco). Cheers.

BTW: I'm still new to gentoo (and linux); I have some experience with DB; I code in C and I'm learning python. I'm willing to help improving portage. How should I become more actively involved with the development? (Right now I'm still trying to understand how it works.)

[Genone]

1. A keyword based search engine instead of a tree based one (or both).
- Put all e-builds in the same directory (there is an all directory in portage already). Forget about the tree. All packages should have unique names, like they have today, but instead of something/something make it (for instance) something-something.

What's the gain of replacing / with - ? FYI, there were similar proposals like this in the past and been shot down as it creates a multitude of other problems (cvs updates/repoman checks become more difficult for example).
Btw, the "All" directory is only in the PKGDIR (so not in the tree) and is likely going away sooner or later.

- Create some keywords to add in the e-build to find the packages. Like: x11, gnome, kde, app, browser, plug-in, etc... I'll give one example using evolution, the gnome mail-client:

Lets add these keywords to the following e-builds:
mail-client/evolution: x11, gnome, mail, mail-client, office
gnome-extra/evolution-data-server: x11, gnome, office
gnome-extra/evolution-exchange: x11, gnome, office, mail, plug-in

Be careful with the term "keywords", it has a special meaning in ebuild contexts, and using it in the general sense makes things confusing
Basically what you want is just another metadata variable similar to DESCRIPTION (just that it contains some predefined tags instead of freeform text).

2. The Uniform Keyword (UK) project.
Since we have more e-builds each day, the keywords sometimes mean different things for different packages. For instance, the mozilla use keyword means that it should connect specifically with the mozilla suite for OpenOffice (requiring mozilla package) and it only means to build with support to any mozilla browser (read firefox) for other e-builds (thus requiring firefox or mozilla packages). Once you learn that you should use mozilla and firefox keywords for some and only firefox for others it works fine, but I think that a manual or something to help the e-build developers (and users) would be nice.

As said before, be careful with the "keyword" usage in ebuild context. Other than that I'M not quite sure what you're trying to say, if it's just that use flags don't always have the exactly same meaning, that's known and discussed/decided on a case by case base as there is no technical solution.

Another problem is that different people use different standards to name things. That increases the probability of mistakes. Example: app and apps.
My idea to minimize that: a wiki page with keywords descriptions used and updated by the developers; some guidelines to the e-build developers (always use singular, read the wiki before creating a new keyword, post there any new use of a keyword, etc).

Actually it's just the same issue as above, different people having different preferences, and again not technically solvable (keep in mind this is about portage implementations, not Gentoo policies).

This is more a political issue than a technical one. But it has some serious technical implications.

It has? I don't think so.

3. The "last edited". (I know I said it would be 2 ideas...)
Just include the last edited date in the output of emerge -s.

Define "edited" ... The last time
- any ebuild of the package was edited
- any file of the package was edited
- the ebuild selected by the search was edited
- any file belonging to the selected ebuild was edited
Also what time should be used, the one from the CVS header, the local mtime, some time written into the ebuild by a dev?
IMO all of these would be pretty useless as the first two are updated on any change (including for example added keywords that are irrelevant to you) and the last is simply too errorprone (and I'm pretty sure that ebuild devs would shoot it down immidiately anyway).

BTW: I'm still new to gentoo (and linux); I have some experience with DB; I code in C and I'm learning python. I'm willing to help improving portage. How should I become more actively involved with the development? (Right now I'm still trying to understand how it works.)

First step would be to join the gentoo-portage-dev mailing list and join the #gentoo-portage IRC channel.

[cokey]

If every idea is being shot down here, what - if anything - is being done to make portage better. Everyone gets so defensive when it comes to ideas to help out.

[Genone]

If every idea is being shot down here, what - if anything - is being done to make portage better. Everyone gets so defensive when it comes to ideas to help out.

Well, I can't help that the same ideas are being proposed over and over again. Some problems with portage are just due to bad coding (e.g. take a look at the version comparison in 2.0, but be warned that you'll get sick), but most/all proposals I see in the forums involve sometimes drastic design changes. Some desgin changes are being done in the 3.0 branch, but right now I'm not involved with that one so don't ask me about details.
As for changing the tree structure, this is something that's very unlikely to happen simply due to transistion issues (old portage versions have to stay working, our VCS has to work good with it, ebuild devs need to be comfortable with it, ...).
It's just that it's very easy to say "hey, this might improve stuff" if you don't know all the consequences and of course you become a bit frustrated if someone shows them to you (or just says that it won't fly, I'm sometimes simply too tired to explain things in detail).
Maybe a few hints for future proposals:
1) Definitely explain what the exact benefits of your proposal would be (as, say, "using a DB" isn't a benefit by itself), if you have numbers supporting your case even better
2) Always think about if this is a compatible change, and if not, how a transition could be done (e.g. if you change something in the tree, how does this afect older portage versions)
3) Try to cover as many use cases as you can think of. For example we recently had a bug about showing if a package is upgraded to a ~arch keyword (forgot the number, sorry). Looks very simple at first, but then you get into the different masking types and similar things which makes this pretty much impossible (keyword here is "semantic issues").
4) Avoid special cases. Special cases are one of the most evil things in code.
5) Patches are always helpful

Often you'll probably see by yourself that things won't be that simple if you check these points.
Oh, and just for completeness: While my opinion has some weight currently I'm not the most important portage developer. I'm just the one who happens to read the forums regulary Maybe you can convince Brian and Jason that your idea is a brilliant one and contradict my comments, however you'll have to do this on IRC or the mailing list.

[cokey]

Genome, you and harring should rewrite portage

Anyway, one idea that I think would be of great use would be a dedicated portage testing team so the ideas about databases and whatever (even different languages) could be tested in a very small scale environment. 5 dedicated devs who aren't in any teams apart from the Portage-Testing group aiming to test idea to see if they would eventually be viable.

[Genone]

Genome, you and harring should rewrite portage

He already does, I still have to get into savior
As for the testing team, while the idea is good, the problem with for example the DB (for the tree) proposal is that even a prototype would require a large amount of work, and the problems are quite obvious, so I have doubts you can motivate enough capable people. And to repeat, anyone really interested in portage development should join the mailing list and the IRC channel.

[cokey]

Genome, you and harring should rewrite portage

He already does, I still have to get into savior
As for the testing team, while the idea is good, the problem with for example the DB (for the tree) proposal is that even a prototype would require a large amount of work, and the problems are quite obvious, so I have doubts you can motivate enough capable people. And to repeat, anyone really interested in portage development should join the mailing list and the IRC channel.

well that is the idea of a testing team, there would need to be plenty of capable people. Remember though, most changes wouldn't require a complete re-write, code could be taken from existing projects and changed.

[Lechium]

I dont see how Python is Portage's bottleneck speed-wise.

You spend most of the time wating on gcc and such, while Python does its job of downloading stuff, calculating depndancies, etc in avery short times compared to what it takes to compile stuff. Re-writing it to C or C++ may save you 5 seconds a day, but is it really worth it?

[cokey]

I dont see how Python is Portage's bottleneck speed-wise.

You spend most of the time wating on gcc and such, while Python does its job of downloading stuff, calculating depndancies, etc in avery short times compared to what it takes to compile stuff. Re-writing it to C or C++ may save you 5 seconds a day, but is it really worth it?

yes, thats 30mins a year!!!!!!!!!!!

[Nihilus]

http://gentoo-wiki.com/TIP_Speed_up_portage_with_Psyco FTW!

[enderandrew]

I can't say that I know enough about various languages to be really useful in the debate of rewriting in various languages.

However, we have two partially working DB implementations of Portage currently, and both already offer large speed increases. I really hope the official version of Portage moves to a DB backend eventually. I think we can all agree this is a good move.

Keep the tree as it helps with backwards compatibility and transition. However, a DB would allow for a simple search regardless of tree. If I want to know where a program is, a simply query can find where it lies within the tree. Isn't that simple?

I know this is quite a bit of work, but I'm going to request it anyway. I'd like to see a standard Gentoo menu system. When you install an program, you shouldn't have to be hassled to add it to your DM's menu. Gnome and KDE are both on the same standard now. Not everyone uses these two, but most do. The standard is the official standard either way.

Some have attempted to make programs that filter out x11, or other programs that shouldn't get menu entries. Honestly, instead of compiling a filter list, the best thing to do is to have an entry in each ebuild for placing said item in the menu or not.

Quite frankly, I think it is silly how many divergent methods we have for installing software on Linux. I think there should be two systems, one for binary installations, and one for installing from source. Should I use apt-get? Should I use emerge? Should I go with rpm? It's all fairly silly. I think emerge could stand out from the crowd and become the definitive method for installing applications from source (and from binaries) if it improved in a few ways. Cutting down on hdd space, improving speed, and automating entries into the menu will all go a long way to improving emerge, and gentoo as a distro.

[gentoo_lan]

http://gentoo-wiki.com/TIP_Speed_up_portage_with_Psyco FTW!

Psycho just covers up bad, broken code. Get over it.

[cokey]

rewrite portage in assembly so we can sort out all the idiots who start setting asflags

[Aynjell]

I've since rescinded my beleifs that portage shuld be in C or whatnot. It should be in python... too much file queerying to be managable in C or C++.

[Drysh]

3. The "last edited". (I know I said it would be 2 ideas...)
Just include the last edited date in the output of emerge -s.

Define "edited" ... The last time
- any ebuild of the package was edited
- any file of the package was edited
- the ebuild selected by the search was edited
- any file belonging to the selected ebuild was edited
Also what time should be used, the one from the CVS header, the local mtime, some time written into the ebuild by a dev?
IMO all of these would be pretty useless as the first two are updated on any change (including for example added keywords that are irrelevant to you) and the last is simply too errorprone (and I'm pretty sure that ebuild devs would shoot it down immidiately anyway).

Maybe we could use what we use for the site.
For instance, gentollkit-0.2.1_rc1 has the date "Sat Nov 12 15:53:37 2005". As we can see here! So we don't have to define which one to use, as it is already used. The only thing that could be changed is to show that when you do an "emerge -vs package". It is also in the ebuild (header line):
# $Header: /var/cvsroot/gentoo-x86/app-portage/gentoolkit/gentoolkit-0.2.1_rc1.ebuild,v 1.1 2005/11/12 15:53:37 fuzzyray Exp $
IMHO it isn't prone to error, as it clearly indicates what I want to know: Is that project active? Is it newer or older than another package? When they say it is necessary to run with something else, are they talking about the current version?
The point is, information degrades very fast in IT. And sometimes the description is talking about something that changed. For instance, I emerged a version of wine that was patched to run with NWN Toolset (a game editor), but later I discovered that it was patched because there was a bug in the previous version of wine. If I had seen the date, I would notice this was older than the current ebuild of wine, and I would have searched for that bug. I don't think this would be usefull for old gentooers, as they already know which package is out-dated, but this would help noobs.
BTW: Thanks for removing the out-dated packages from the tree, whoever did it. I was searching for that ebuild and I could not find it.

Genome: What I posted are suggestions. I recognize the excelent work you've done with portage. I'm not even posting as a requested feat, but only as a suggestion. That's because I'm too new here to be able to understand all the implications of something. But sometimes, the point of view of a new user has some insights that the old users cannot see anymore, because it is so easy to them to do it the current way that they cannot imagine how difficult it is to the new user. I believe the date is something like this: old users wouldn't need it, but new users would love it. I know I would. Anyway, thanks for the answers, and I hope I don't offend anyone when I express my ideas to improve gentoo, even if they are stupid ideas (yet!).

[Genone]

3. The "last edited". (I know I said it would be 2 ideas...)
Just include the last edited date in the output of emerge -s.

Define "edited" ... The last time
- any ebuild of the package was edited
- any file of the package was edited
- the ebuild selected by the search was edited
- any file belonging to the selected ebuild was edited
Also what time should be used, the one from the CVS header, the local mtime, some time written into the ebuild by a dev?
IMO all of these would be pretty useless as the first two are updated on any change (including for example added keywords that are irrelevant to you) and the last is simply too errorprone (and I'm pretty sure that ebuild devs would shoot it down immidiately anyway).

Maybe we could use what we use for the site.
For instance, gentollkit-0.2.1_rc1 has the date "Sat Nov 12 15:53:37 2005". As we can see here! So we don't have to define which one to use, as it is already used. The only thing that could be changed is to show that when you do an "emerge -vs package". It is also in the ebuild (header line):
# $Header: /var/cvsroot/gentoo-x86/app-portage/gentoolkit/gentoolkit-0.2.1_rc1.ebuild,v 1.1 2005/11/12 15:53:37 fuzzyray Exp $

That's an answer for the second question, but not for the first one.

IMHO it isn't prone to error, as it clearly indicates what I want to know: Is that project active? Is it newer or older than another package? When they say it is necessary to run with something else, are they talking about the current version?

As I said above, the date from the CVS header won't tell you that, could be just someone marking a two year old ebuild for the s390 arch. And regarding your last statement: date information isn't a good indicator to select the revision, just the the actual revision number for that.

The point is, information degrades very fast in IT. And sometimes the description is talking about something that changed. For instance, I emerged a version of wine that was patched to run with NWN Toolset (a game editor), but later I discovered that it was patched because there was a bug in the previous version of wine. If I had seen the date, I would notice this was older than the current ebuild of wine, and I would have searched for that bug. I don't think this would be usefull for old gentooers, as they already know which package is out-dated, but this would help noobs.

See above: cvs header doesn't really tell you that (another example: the sylpheed-claws-1.0.5-r1 ebuild is (from the CVS header) "newer" than the 1.9.100 ebuild, but upstream will tell you "wow, you're using an obsolete version")

Genome: What I posted are suggestions. I recognize the excelent work you've done with portage. I'm not even posting as a requested feat, but only as a suggestion. That's because I'm too new here to be able to understand all the implications of something. But sometimes, the point of view of a new user has some insights that the old users cannot see anymore, because it is so easy to them to do it the current way that they cannot imagine how difficult it is to the new user. I believe the date is something like this: old users wouldn't need it, but new users would love it. I know I would. Anyway, thanks for the answers, and I hope I don't offend anyone when I express my ideas to improve gentoo, even if they are stupid ideas (yet!).

No problem, I'm just pointing out that adding a date doesn't really give you the information you're looking for, as you want an indicator how active the upstream package is, and there is no easy way to get that information as far as I can see.

PS: those are really two 'n' in my nickname

[Sir No]

Probably not, but the fun is in trying.

I'm with you about this one, but it all comes down to one's definition of "fun". I don't think I would have much fun rewriting the whole Portage core in C, having to fight against infancy bugs and maybe see halfway that the real problem was in old portage's lack of DB structure, and that I could have written 90% less code and implemented 90% more features in half the time using Python or Ruby... Or realizing in the end that a better thing to do would have been locating the incriminated part of Portage and refactoring it.

Anyway, it would be nice to hear opinions from the only people that will make the decision in the end: Portage developers. Are you out there?

In the world of fast machines with big hard drives Python is not that fat, nor is Perl (I know, I know, please spare me the ultimate truth...)

But the combination of Perl + C + underlying DB has already proven its usefulness (hints: apt, Debian, Ubuntu). I can safely say that because I REALLY KNOW how is it to run "apt-get update" on a 650MHz sparc64 machine with 256MB of RAM. It flies with the speed of light compared to doing "emerge --sync" on my AMD 2800+ laptop with 512 MB DDR... And searching the packages through "emerge -s sth", right after the system booted and the portage tree hasn't been loaded into memory yet... This delay can kill one day, I tell you.

So, if anyone is restructuring the portage code, I'm all with you, guys! Do it, do it, do it!!! There IS a light at the end of this tunnel...