View previous topic :: View next topic |
Is there a more effective way to install than emerge? |
Yes |
|
10% |
[ 1 ] |
No |
|
90% |
[ 9 ] |
|
Total Votes : 10 |
|
Author |
Message |
LIsLinuxIsSogood Veteran
Joined: 13 Feb 2016 Posts: 1179
|
Posted: Tue Jul 10, 2018 5:15 am Post subject: Gentoo linux okbu not gentoo linux actually yes gentoo linux |
|
|
I would like to know how to save all that dependency calculation work that Portage does in order to minimize time when running --pretend or --ask (followed by a no) and not actually merging anything. Why does Portage not act smart later when I want it to do the same function and using the same system environment, tree for sources, and everything else like that? Why does it go through all those calculations again in order to merge stuff that could have been saved to some sort of cache?
Does anyone have any hints for creating such a cache that would save the "state" of Portage in the midst of building (say several hundred packages), or else just prior to that?
Motivation: If sometimes I am getting ready to leave the house and I run the command, so I had to wait to see what the output would look like. Now later I come home and want to review the list of packages maybe read up on them, and then ultimately merge them. That is up to three times that I might have to run that command.
If Portage takes 3 minutes each time in figuring it out, those add up and pretty soon it becomes a drag or at least discouragement for trying different cases or scenarios (like use flags, environment variables, etc.).
Not to mention the times when I maybe meant to press "y" and instead "n" after emerge --ask and now i have to wait for portage to figure all that stuff out again?
My suggestion would be either to help users with a smarter interface (no i don't mean flashy or shiny buttons although i don't see how that would hurt), but what I mean is if you aren't a pro system admin, which I am not, if you aren't a developer (I am really not) then it seems like the only way to fix problems in terms of troubleshooting Portage is to turn to people on this forum for help. I really appreciate the help I get here, but at the end of it all when a package is failing to build or portage won't move forward there seems like a pretty large gap there for what the package manager could do in terms of presenting the user with some viable alternatives (either helpful or non-helpful, but by presenting several options that could be a better case scenario than just some generic statement about Build Conflicts and how those can get resolved, etc.)
This is my rant. I don't mean to take it out on any of the hard working developers who built the package management system...that is hardly my goal, but I want to continue getting the benefit just without some of the "slow downs" that are provided by emerge. I remember seeing some pretty cool scripting ways to work with the system update, like with eix and then piping cmmands to download and install packages from source. If anyone has some helpful scripts for that a URL explainaing a different way to handle updates on my Gentoo system that doesn't have me sitting around more often than not waiting for Portage output...then I would like that!
Maybe it would also hellp me to get clear on some of the emerge options like the --pretend option just in case I can't be sure what it does other than to run conflict checks and stuff like that. But I don't think it will be able to account or predict for the situation of a package that won't build because of some other missing library or something like that. Is that what other package managers are for? I haven't tried any others within Gentoo other than Portage. I know that there should be a way to just create my own ebuilds or not even have to and just use ebuild command...right for the more advanced interaction am I on target with that last statement? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Tue Jul 10, 2018 11:54 am Post subject: |
|
|
Moved from Off the Wall to Other Things Gentoo.
Its not really OTW material and it is a reasonable technical question too.
I voted No ... the alternative is dependency hell, where you manage all the depenencies yourself.
Binary distros can do it as they only give you one choice - take it or leave it. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Tue Jul 10, 2018 12:24 pm Post subject: |
|
|
"No" also for me. It's been discussed before, but there are issues with caching the dependency graph. Just to mention a couple:- If you change a USE flag, then a significant amount of the dependency graph might need to be recalculated.
- If you emerge a new package, then the cached dependency graph will be incomplete as the new package will most likely bring new dependencies into the graph.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21633
|
Posted: Wed Jul 11, 2018 2:45 am Post subject: |
|
|
I would love for the system to be better at caching results, but it is a hard problem. If the cache manager is not aggressive enough at invalidating stale results, you get weird and confusing output when the system reuses data that should have been discarded (for example, what if Portage's cache sometimes incorrectly ignored new packages being installed, so you got an output saying you need to install Foo, but you just finished installing Foo?). If the cache manager is too aggressive at invalidating stale results, you get little benefit from the cache. Most people won't be rerunning exactly the same command, with no intervening USE flag changes, mask changes, other installs, etc., so the obviously safe cache invalidation solution (discard on any system state change) will usually not help. To get good results, you quickly descend into very selective invalidation rules in an attempt to invalidate as little as you can. That is not only difficult to get right, but fragile since it depends on assumptions about what types of system state change are relevant. For example, consider the mess if the cache were developed before the invention of package USE dependencies, and therefore the invalidation algorithm assumed (correctly at the time of its invention, but incorrectly now) that changing the USE flag on Foo cannot impact the dependency resolution of Bar. |
|
Back to top |
|
|
LIsLinuxIsSogood Veteran
Joined: 13 Feb 2016 Posts: 1179
|
Posted: Sat Jul 14, 2018 8:34 am Post subject: |
|
|
I seem to get the idea that caching the output is going to be very complex and perhaps unnecessarily so. But maybe a different approach rather not caching it but somehow saving the amount of time that Portage calculates some commands that are similar relatively given a nearly exactly similar system. Now I get that the tree is going to always be the reason for recalculation but there just doesn't seem to enough changes from one day to the next in the tree to not be able to consider some realistic time saving in calculations perhaps if even based on a cached tree??
Anyway, I've been using Gentoo for a while now and I did want to test out some other package management options so now may be the time.
Would you recommend going the route of actually installing another PM or else just make some attempts to use the built in Portage tools like ebuild to start managing the packages a little differently. If there is a clear shortcoming for me it is the concept of sets because I can't seem to get good working use out of any of them other than 1 or 2 of them (system and world)...and I don't really get what the point of those are most of the time since --deep like some other flags seem to "hide" a lot of the work that is done and make the resulting output confusing.
Basically I mostly just run into the updates with world or system (to save time)...but now I am thinking what would save time might be to write some scripts of my own that could query if those updates are even available and then go from there.
I assume that the package management options for Gentoo are somewhat limited (Paludis, Portage, are there more?) but with Portage at least I can hopefully begin to get used to installing packages from source just like emerge would but with ebuild instead. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Sat Jul 14, 2018 9:17 am Post subject: |
|
|
LIsLinuxIsSogood wrote: | If there is a clear shortcoming for me it is the concept of sets because I can't seem to get good working use out of any of them other than 1 or 2 of them (system and world)... |
LIsLinuxIsSogood ... if you look at the output of 'emerge --list-sets' then you'll see sets other than @system and @world. So, something like @x11-module-rebuild can be used when x11 is updated, and rebuilding 'modules' (ie, xf86-input-*, xf86-video-*) is required. There is also @preserved-rebuild which can be used for rebuilding packages against an updated library but who's the previous version was retained because some package in @preserved-rebuild was built against that version.
LIsLinuxIsSogood wrote: | Basically I mostly just run into the updates with world or system (to save time)...but now I am thinking what would save time might be to write some scripts of my own that could query if those updates are even available and then go from there. |
I very much doubt this will improve the situation, the PM will still need to calculate the dependency graph.
BTW, one way to reduce calculation time is to add '--ignore-built-slot-operator-deps=y' to the command, or EMERGE_DEFAULT_OPTS.
LIsLinuxIsSogood wrote: | I assume that the package management options for Gentoo are somewhat limited (Paludis, Portage, are there more?) but with Portage at least I can hopefully begin to get used to installing packages from source just like emerge would but with ebuild instead. |
plaudis was given last rites, and so the only other option is sys-apps/pkgcore ... this is faster but lacks some features available in portage.
best ... khay |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Sat Jul 14, 2018 10:13 am Post subject: |
|
|
LIsLinuxIsSogood,
Using ebuild to install packages is like Code: | emerge --nodeps <package> |
That makes you the package manager. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Sat Jul 14, 2018 10:16 am Post subject: |
|
|
you can also cut down calculations if you tell portage there's just nothing to calculate, just remember to properly use onehost
emerge -p packagA (getting the list)
packageB
packageA...
emerge packageA packageB.... --nodeps --oneshot (no more busy)
ps: like what NeddySeagon said, but slower |
|
Back to top |
|
|
LIsLinuxIsSogood Veteran
Joined: 13 Feb 2016 Posts: 1179
|
Posted: Sat Jul 14, 2018 10:58 am Post subject: |
|
|
I'm going to install pkgcore on my desktop pc, which is where I run stable packages so less likely to have issues there I suppose than on my virtual Gentoo Box that houses pretty much double the packages of that one.
In response to sets the @preserved-rebuild set is something that I still struggle with in terms of deciding what it is that should be done.
E.g. right now I have a package listed in there with a dependency that has since been updated but that I had patched in order to prevent some gui problems that abiword had on that version. I've been reluctant to upgrade since without a separate slot in existence for the package in gentoo I would either have to build from source or create a "shell" of ebuild with new slot to be able to do that. Either way it is a lot more work. So in a way in regards to comparing different PM and pkgcore missing slot dependency check seems like in this case Portage itself has no real advantage. Unless there is some easy command that I am unaware to install a secondary or third build of the same package on the system and have Portage keep track of each one. But I assume that isn't the purpose of the package manager in Gentoo.
Neddy is right and I like the idea of being the package manager for my system, but I guess the point would be to build in to the situation a certain fail-safe on my own calculations. There seems to be plenty in Portage to do this, so I think I will stick with that for now. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Sat Jul 14, 2018 11:09 am Post subject: |
|
|
LIsLinuxIsSogood,
You know that portage can apply patches for you? _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
LIsLinuxIsSogood Veteran
Joined: 13 Feb 2016 Posts: 1179
|
Posted: Sat Jul 14, 2018 11:25 am Post subject: |
|
|
Yes that is how I managed to fix the application in the first place. My concern is the patch could be different for the newer package and I haven't checked (just lazy) but I guess I would need to find out before applying it to the version bump...is that right?
The real problem with Portage as a package manager isn't the functionality really it is that it just is more than I need these days.
UPDATE: The file looks good without the patch, from what I can tell with the version bump of abiword to 3.0.2-r1 no screen flickering which was an issue that got in the way about 6 months ago or 1 year.
I initially enjoyed aspects of customizing to the highest degree possible with portage, but more and more my needs these days are actually becoming common given a better foundational understanding of more tools to get the job done, for example application space, userland(?) and kernel tools, all being able to work together even gui in order to fix a rough problem with a system.
But if there is something that really doesn't play in much at all to the experience of a working system it is package management. That is why I started this thread, and would like to know about what other tools or workarounds exist so that I can continue to keep my Gentoo boxes up and running, smoothly with as little effort possible. For really many years now I have been able to basically rely on occasional updates, and a very rare running of --depclean. That's prety much what I've been doing for years now and it seems to work, but would be nice to have some other options. So I think I will be trying pkgcore on at least one of the systems.
DISREGARD BELOW THIS LINE SINCE THE UPGRADE WORKED TO ABIWORD-3.0.2-r1
Any recommendations in terms of checking if a patch will work before upgrading, or else keeping the previously patched version so that in case things go wrong I can always revert back to the patched one. That seems fairly easy at least (I think) |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Sat Jul 14, 2018 12:41 pm Post subject: |
|
|
LIsLinuxIsSogood,
Read up on pkgcore before you deploy it. It is known to be feature incomplete.
Does it support EAPI 7?
You should use --depclean every update. If you don't, portage will maintain the orphaned packages on your system until you do depclean them.
That's extra time to calculate the dependency tree, build and install packages that you don't actually need.
You should also check your world file for pollution from time to time.
--depclean will not remove any packages that are listed here but its all too easy to forget --oneshot when you are looking into a build failure.
This pollution can keep things around that you don't need ... and the dependencies. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Sat Jul 14, 2018 2:58 pm Post subject: |
|
|
NeddySeagoon wrote: | That's extra time to calculate the dependency tree, build and install packages that you don't actually need. |
No NeddySeagoon, that's just 0.00% extra time ; dependencies/program that are useless takes nothing from portage because they are just not pull in the graph at all. But they do take space. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Sat Jul 14, 2018 3:01 pm Post subject: |
|
|
krinn,
Well caught. I was thinking ahead to extra entries in @world. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
LIsLinuxIsSogood Veteran
Joined: 13 Feb 2016 Posts: 1179
|
Posted: Sun Jul 15, 2018 2:41 am Post subject: |
|
|
How best to do this world set cleanup so to speak? Obviously --depclean should work, but I've had past scenarios before I really knew how to fix it where running that command often created a lot of weird circular dependencies.
Maybe there is another wya? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Sun Jul 15, 2018 10:05 am Post subject: |
|
|
LIsLinuxIsSogood,
Cleaning world consists of removing entries that should not be there.
There is a wiki page and a discussion cleaning the world file (wiki) - check the script
Essentally, for every entry in the world file, check that nothing depends in it. If the package is pulled in as a dependency, it should not be in world.
That gets a list of candidates for removal. You cannot just remove things, its your system. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10589 Location: Somewhere over Atlanta, Georgia
|
Posted: Sun Jul 15, 2018 3:32 pm Post subject: |
|
|
LIsLinuxIsSogood wrote: | How best to do this world set cleanup so to speak? Obviously --depclean should work, but I've had past scenarios before I really knew how to fix it where running that command often created a lot of weird circular dependencies. | Just to clarify, --depclean doesn't clean the world set at all. What it does do is to remove installed packages that no other installed package depends on.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
|