portage implementation discussion

[Alejandro Nova]

I also don't know why don't portage developers use things like Psyco, optimizers that can increase Python apps's speed painlessly before trying any port to C++.
_________________
Becoming someone beautiful, through my music, my silent devotion...
Alejandro Nova™.

[Valheru]

[Redeeman]

i agree, portage is becoming too slow, but its not entirely pythons fault, its because portage just wasnt really designed to scale as much as its being asked to, the number if ebuilds have exploded over the past few years. recoding portage in python would be okay too.. but it should be entirely restructured..

instead of having all the ebuilds on your own box, they should be on a mirror.

the only thing you should have locally is a sqlite or some other small db with the metadata, the packagename, keywords, dependency information and stuff.. this way it would only be a database approximately the size of 10mb at most, and when you sync, you only get a diff from the last time you synced (this would work with some commit id, possibly like svn, where you update from revision <nr> of the metadata database..) and then it fetches ebuilds and eventual patches and init scripts when you emerge..

ofcourse you should have the ability to do overlays and stuff, and also download the ebuild, make changes and then emerge it..

but this way, syncing alot wouldnt hurt the gentoo mirrors so much, as they would roughly probably smash out 200 or 300kb of data for a sync between a day, instead of ALOT
and it would be smashingly fast, and also, it wouldnt take up so much space, which is, if you think about it quite stupid as it does it now

[Raniz]

[hoschi]

[nevynxxx]

[hoschi]

Okay,
you must see, to solve dependencies via a software-managment is a good thing, but to have no dependencies is always better and simpler.
why i should install a big and "slow" dependencie like python, it blows up my system.
sure, portage with python is not slow, but it will be never so fast as and c/c++ app, on the other side: portage is not something really heavy thing like gnome-nautilus or inkscape

the reason why i don't like programs with dependencies like python or ruby is, that the are not slower than c/c++ apps (the are often small, no big speed difference), it is because i don't like fat ans slow systems, with many not necessary software on it.



if i can replace portage (python), with portage (c/c++, maybe too if it based on the bash) i will do it
u will say *what* a portage based on bash?
yes!
i dont like dependencies, and the bash is always installed on nearly every system...


Deps must die, Deps must die, Deps must die
_________________
Just you and me strogg!

[tempest]

What's this dependency phobia you are suffering of, I don't think it's rational at all. I can't understand either why adding Python to a base system would made it "fat and slow".

Apart from these things, I think we have some dangerous effort duplication here. sportage and portage-c might be among the best softwares around, but I don't think they're the right way to address current Portage's empasse. A nice refactoring of Portage is what is needed, first of all introducing database use (sqlite?), then I think that lots of speed issues would simply go away or be easily fixed.
_________________
For every problem there is a solution that is simple, neat, and wrong.

[wizardofos]

I think so too, but nevertheless I think the current Portage code (I looked a bit through it) has grown enormously in the past and is very difficult to understand and change now. Therefore it'll be a lot easier to reimplement portage from scratch. And if somebody does that I understand that he/she chooses c/c++ over python. Why not, when you have to code everything from scratch anyway...
I do believe that many many people want to see a portage without python dependency.

[EzInKy]

[tempest]

In this topic I read a lot of personal opinions that are senselessly promoted as absolute truths.

Before *even* thinking about reimplementing Portage, what about sitting for a moment and locate the actual problem(s)? Rewriting a software from scratch just because we hope this time it will come out better will lead to time loss rewriting a software that already works, stillness of standard Portage and a lot of bugs to fix in the new one as the development goes through its natural cycle.

Besides that, what are the rational and objective arguments against Python for Portage, apart reasonless dependency phobia? Why would C (!!!) or C++ be "better" than Python for Portage?

EzInKy: I'm happy that you managed to install your LFS, and I'm sorry that you had to handle complexity caused by dependencies. That's not a problem with Gentoo since Portage always came integrated with the distribution and it's a task that has to be performed only once.

[EzInKy]

[tempest]

[lamekain]

let's try to do something creative instead of argumenting over personal opinions. I checked the size of /usr/portage first time today and I must say that 1.4G is a bit much.. Let's try to find the best (or most promising) solutions for a "future portage".

I too like the ideas that Redeeman pointed out earlier. Databases make handling ebuilds a lot faster and save space. Any other opinions on what the storage mechanism should be?

In my opinion C/C++ is well suited for this because:
- it's fast
- gcc is always installed (and g++)

On the other hand python/perl (perhaps bash. not much experience):
- is easier to learn and to use
- powerful and fast to develop on

With a good architecture we could easily use multiple languages. Not forgetting stuff like mono, which would make it even easier. Any suggestions for an architecture? Anybody with good experience doing DB-diagrams?
_________________
"Only when the last tree has died, the last river been poisoned,
and the last fish been caught, will we realize we cannot eat money"
-Global Terrorism.org

[hoschi]

[tempest]

hoschi, I really don't get your point, if any, so I'll try to make mine clear. When a project starts and a platform (language/architecture/whatever) for it has to be chosen, priority should to go to the platform who helps developers get the job done well and quickly, not to the one that minimizes dependencies on runtime environments (to what advantage, anyway?).
_________________
For every problem there is a solution that is simple, neat, and wrong.

[EzInKy]

[Jeremy_Z]

That is not necessarily true.While coding in C or other low level language it is your responsibility to produce portable code, using portable libraries, supporting different compilers on different architecture. Using an interpreted language (or pseudo interpreted) your code is directly portable everywhere there is an interpreter (or VM).
_________________
"Because two groups of consumers drive the absolute high end of home computing: the gamers and the porn surfers." /.
My gentoo projects, Kelogviewer and a QT4 gui for etc-proposals

[EzInKy]

[EzInKy]

And to emphasize the problems with unnecessary dependencies I just ran into this bug while guiding a friend through a stage 1 install.
_________________
Time is what keeps everything from happening all at once.

[Ibn al-Hazardous]

[hoschi]

[Dlareh]

[hoschi]

Oh no, no discussion about the stages
_________________
Just you and me strogg!

[Evilguru]

The reason that portage is so slow is not because of python, but because it does not use a database. By using a database it would be a lot faster, something like sqlite, which is only 30k of C code and is a fully featured, server-less database system. That would solve most of our problems. Getting portage to use a database as the back-end would not only save us space but would also be much faster. I am not sure why it has not been done before.