Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
What is --backtrack
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Tue Apr 18, 2017 12:49 am    Post subject: What is --backtrack Reply with quote

on a recent issue i raised on this forum, --backtrack=30 was recommended to me, and it resolved my problem by recompiling fifty-six packages instead of the initial two.

i have never needed to use --backtrack before. i can't seem to find much information, apart from one sentence in the man page which i don't completely understand.
Code:
$ man emerge

       --backtrack=COUNT
              Specifies an integer number of times to backtrack if dependency calculation fails due to a conflict or an unsatisfied dependency (default: ´3´).


can anyone explain what --backtrack does?
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Tue Apr 18, 2017 2:07 am    Post subject: Reply with quote

It backtracks through the dependency tree to the level specified by the provided integer. When I do my monthly world updates, I use an insane number like 1000 to make sure nothing is missing or broken.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Tue Apr 18, 2017 7:47 am    Post subject: Reply with quote

audiodef wrote:
When I do my monthly world updates, I use an insane number like 1000 to make sure nothing is missing or broken.

i do my world updates almost daily, and have not done any other updates at all. i find that if i leave too long a gap, sometimes i end up in situations i don't completely understand.

if something is missing or broken, wouldn't it show up without any backtrack? i have never before used it. is this something i should be using more regularly.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54248
Location: 56N 3W

PostPosted: Tue Apr 18, 2017 8:36 am    Post subject: Reply with quote

josephg,

When dependency resolution comes to dead end, portage has two choices
[*]tell you it failed and provide a sometimes less than helpful error message
[*]go back some branches in the dependency tree and look for a different route

The default is --backtrack=3. The higher the setting, the further back in the dependency tree portage is allowed to go to try to find a solution.
Notice the 'allowed to go'.
If the default --backtrack=3 would of worked --backtrack=30 should produce the same solution in the same amount of time. (I've not tested that)
When --backtrack=3 fails and --backtrack=30 works, portage has gone back more that three levels in the dependency tree to reach a solution but you don't know how far, maybe only 4?

The downside to large --backtrack= settings is that it takes longer to get to the error message when there is no solution to the dependency graph.

Testing --backtrack= for 0,1 and 2 is left as an exercise to the reader.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Tue Apr 18, 2017 9:08 am    Post subject: Reply with quote

NeddySeagoon wrote:
If the default --backtrack=3 would of worked --backtrack=30 should produce the same solution in the same amount of time. (I've not tested that)

so i guess backtrack is incremental. if it takes the same time amount of time, why would it matter to me whether i use --backtrack=3 or --backtrack=30?

NeddySeagoon wrote:
When --backtrack=3 fails and --backtrack=30 works, portage has gone back more that three levels in the dependency tree to reach a solution but you don't know how far, maybe only 4?

what can i infer from --backtrack=4 reaches a solution vs --backtrack=30?

NeddySeagoon wrote:
Testing --backtrack= for 0,1 and 2 is left as an exercise to the reader.

i tested with 0, 1, 2, 30, 1000, and all produce the same output.
Code:
Nothing to merge; quitting.

i suppose i should defer this testing for another time, perhaps till conflicts arise again.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54248
Location: 56N 3W

PostPosted: Tue Apr 18, 2017 10:49 am    Post subject: Reply with quote

josephg,

The difference should only be apparent when the default --backtrack=3 results in a conflict error message and --backtrack>3 finds a solution.
If backtrack=0 works, portage got it right first time and no backtracking was required.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
kornhs4
Tux's lil' helper
Tux's lil' helper


Joined: 27 Jun 2004
Posts: 86
Location: Austria

PostPosted: Sun Mar 29, 2020 5:30 pm    Post subject: Reply with quote

The difference is the following: Assume there is actually no solution to your current set of package dependencies. If you set --backtrack=N for a very high N then you force it to look for it, minutes, hours, years, for the age of the universe, until it finds that there is no solution. (The computational problem considered here is NP hard, so the age of the universe is no exaggeration.)

Instead you may want to leave N at its default (which is 10 according to `man emerge` here). Then it tells you after some "reasonable" amount of time that it could not fulfill your wishes and tells you about the conflict. (Which is often not easy to read for a human.) Anyhow, you as a human may (or may not) easily see what the "real" problem is as a package is mentioned that is now hard masked you do not care about anymore, so uninstall it. And suddenly all works out very quickly.
_________________
_________________
Life would be easier
if i had the source code
Back to top
View user's profile Send private message
lburrow
n00b
n00b


Joined: 18 Jul 2023
Posts: 1

PostPosted: Tue Jul 18, 2023 5:47 pm    Post subject: That can't be right Reply with quote

Sorry to wake such an old thread but this has to be bad information and I had to say something.

OK, Here's my routine.

I use an alias for my updates.
alias eu='emerge -u -D --keep-going --backtrack=100 --newuse --with-bdeps y world'

My routine is:
• open a screen session
• eix-sync
• eu -p
• eu
• dispatch-conf
• emerge --depclean -p
• emerge --depclean
• emerge @preserved-rebuild -p
• emerge @preserved-rebuild
• revdep-rebuild -p
• revdep-rebuild

I do this almost daily.

Ok. So that alias is something I have built up over the years in a quest to get a most trouble-free, consistent and complete update. This alias has worked pretty well for me but I can't say I entirely understand it, with that --backtrack= being one of the parts I least understand.

I have RTFM but that just leaves me with the same conclusions others have responded here, none of which match my experience at all!

So, that alias... I added the --backtrack, and kept raising the number until I stopped at 100 because it seems to give me fewer blockers this way. It is slower, but I don't care. It's usually running in the background while I am working but also I can walk away.

So far that matches the explanations others have given right?

But every once in a while (I'm talking several months apart) there is something tricky and it comes up with blockers anyway. Increasing that number above 100 has never helped. But.. sometimes just running "emerge -u world" makes the problem just go away. It emerges some stuff and the next time I do "eu" it's back to just working.

If it worked like described above there shouldn't be any situations where running with --backtrack set to 100 does not resolve the problem but running without --backtrack set does.

Today I have an even weirder situation. For the last week or so "eu" has resulted in a ton of blockers. For some reason updating qtgui and qtcore were blocked by a bunch of qt apps. I want to say I was stuck on 5.15.9 although 5.15.10 was available.

Anyway, the really weird thing was all those apps that supposedly were blockers either only required qt be >=5 or had their own updates also available that would have resolved the conflict. This made no sense to me.

So.. I ran "emerge -u world". Nothing! It didn't even find that there was anything to do at all! But finally, shooting in the dark I tried "emerge -u --backtrack=10 world". Now it seems to be emerging all that qt stuff. It does nothing at the default setting, breaks at 100 but works at 10 (which one person thought WAS the default setting). I'm currently happy to be building qtwebengine, 48 of 59 but more convinced than ever that this does not follow any sort of logic.

Then there is that emerge @preserved-rebuild step.

It used to be there were almost always packages to emerge at that step. Now it seems to be much more rare although it does happen. So I don't get that one either. It seems to be packages that were not upgraded but need to be re-emerged because libraries they link to were? But during the "eu" or "emerge --upgrade" step I see all sorts of packages marked as being R-rebuilds, not U-updates. So why do some fall into this @preserved-rebuild while others just get rebuilt as part of the update step? Clearly emerge knows they need rebuilt in order to add them to the set. Is "leave my system broken" a valid wish that a user might have? Why isn't it just emerging them in the first place all on it's own???

The best I can tell from all this is that Gentoo is awesome, Portage is awesome but there is magic there. There are some things that just don't make sense to anyone at all and it is best just to know what to do even if not why. --backtrack is a feature that makes finding updates and dependencies slower but better, USUALLY more complete and with fewer conflicts. And usually being not-always one should be ready to try again without backtrack or with a lower backtrack. One can know to do this but understanding it.. nope, magic.
Back to top
View user's profile Send private message
sam_
Developer
Developer


Joined: 14 Aug 2020
Posts: 1678

PostPosted: Tue Jul 18, 2023 6:21 pm    Post subject: Reply with quote

audiodef wrote:
It backtracks through the dependency tree to the level specified by the provided integer. When I do my monthly world updates, I use an insane number like 1000 to make sure nothing is missing or broken.


No, it's a backtrack limit.

But failure to reach a solution will not make something broken. At worst, it leads to skipped parts of the depgraph in a consistent way - i.e. missed updates, not anything broken.
Back to top
View user's profile Send private message
sam_
Developer
Developer


Joined: 14 Aug 2020
Posts: 1678

PostPosted: Tue Jul 18, 2023 6:23 pm    Post subject: Reply with quote

NeddySeagoon wrote:
josephg,

When dependency resolution comes to dead end, portage has two choices
[*]tell you it failed and provide a sometimes less than helpful error message
[*]go back some branches in the dependency tree and look for a different route

The default is --backtrack=3. The higher the setting, the further back in the dependency tree portage is allowed to go to try to find a solution.
Notice the 'allowed to go'.
If the default --backtrack=3 would of worked --backtrack=30 should produce the same solution in the same amount of time. (I've not tested that)
When --backtrack=3 fails and --backtrack=30 works, portage has gone back more that three levels in the dependency tree to reach a solution but you don't know how far, maybe only 4?

The downside to large --backtrack= settings is that it takes longer to get to the error message when there is no solution to the dependency graph.

Testing --backtrack= for 0,1 and 2 is left as an exercise to the reader.


This is right, just note that the default is --backtrack=10, not 3 nowadays.

josephg wrote:
audiodef wrote:
When I do my monthly world updates, I use an insane number like 1000 to make sure nothing is missing or broken.

i do my world updates almost daily, and have not done any other updates at all. i find that if i leave too long a gap, sometimes i end up in situations i don't completely understand.

if something is missing or broken, wouldn't it show up without any backtrack? i have never before used it. is this something i should be using more regularly.


Yes, as per my earlier reply, this is wrong and please don't be scared by it. See NeddySeagoon's and kornhs4's replies too.
Back to top
View user's profile Send private message
sam_
Developer
Developer


Joined: 14 Aug 2020
Posts: 1678

PostPosted: Tue Jul 18, 2023 6:31 pm    Post subject: Re: That can't be right Reply with quote

lburrow wrote:
Sorry to wake such an old thread but this has to be bad information and I had to say something.


Yes, there's some bad information in this thread I've just corrected.

lburrow wrote:

OK, Here's my routine.

I use an alias for my updates.
alias eu='emerge -u -D --keep-going --backtrack=100 --newuse --with-bdeps y world'


You could just do 'emerge -uvDU @world --keep-going' or 'emerge -uvDN @world --keep-going'. --with-bdeps is on by default where sensible (only changes if using binpkgs) nowadays anyway.

lburrow wrote:

My routine is:
• open a screen session
• eix-sync
• eu -p
• eu
• dispatch-conf
• emerge --depclean -p
• emerge --depclean
• emerge @preserved-rebuild -p
• emerge @preserved-rebuild
• revdep-rebuild -p
• revdep-rebuild


You could just use --ask/-a instead of -p here. You also shouldn't need revdep-rebuild nowadays. It's pretty rare that it's needed. I'll get on to why in a second.

lburrow wrote:

[...]
But every once in a while (I'm talking several months apart) there is something tricky and it comes up with blockers anyway. Increasing that number above 100 has never helped. But.. sometimes just running "emerge -u world" makes the problem just go away. It emerges some stuff and the next time I do "eu" it's back to just working.

If it worked like described above there shouldn't be any situations where running with --backtrack set to 100 does not resolve the problem but running without --backtrack set does.

Today I have an even weirder situation. For the last week or so "eu" has resulted in a ton of blockers. For some reason updating qtgui and qtcore were blocked by a bunch of qt apps. I want to say I was stuck on 5.15.9 although 5.15.10 was available.

Anyway, the really weird thing was all those apps that supposedly were blockers either only required qt be >=5 or had their own updates also available that would have resolved the conflict. This made no sense to me.

So.. I ran "emerge -u world". Nothing! It didn't even find that there was anything to do at all! But finally, shooting in the dark I tried "emerge -u --backtrack=10 world". Now it seems to be emerging all that qt stuff. It does nothing at the default setting, breaks at 100 but works at 10 (which one person thought WAS the default setting). I'm currently happy to be building qtwebengine, 48 of 59 but more convinced than ever that this does not follow any sort of logic.


In general, if you're having trouble, you should post your own thread instead of suffering. Sometimes things like Qt upgrades require more backtracking for Portage to solve it.

Sometimes this is because Portage already has to backtrack a bit for regular upgrades, but the complexity of Qt upgrades tips it over the edge. Running a command like 'emerge -pev @world --backtrack=0' purely for diagnostic purposes can help find problems with packages Portage can't rebuild (packages in world file which don't exist anymore, etc).

I imagine that "emerge -u --backtrack=10 world" succeeded because you didn't use --deep or similar. Very hard for me to say without seeing all of the relevant commands and output in full (which is again why starting your own thread when you're having trouble is what you should do).

And yes, more backtracking can sometimes lead to skipping too much of the depgraph, but it's somewhat rare (backtracking almost always leads to better outcomes, not worse ones). Not much else I can say here really without specifics.

lburrow wrote:

Then there is that emerge @preserved-rebuild step.

It used to be there were almost always packages to emerge at that step. Now it seems to be much more rare although it does happen. So I don't get that one either. It seems to be packages that were not upgraded but need to be re-emerged because libraries they link to were? But during the "eu" or "emerge --upgrade" step I see all sorts of packages marked as being R-rebuilds, not U-updates. So why do some fall into this @preserved-rebuild while others just get rebuilt as part of the update step? Clearly emerge knows they need rebuilt in order to add them to the set. Is "leave my system broken" a valid wish that a user might have? Why isn't it just emerging them in the first place all on it's own???


Back to the point about revdep-rebuild from earlier.

In the olden Gentoo times, Portage had no way of automatically triggering rebuilds of packages when ABI changed. ABI breakage in packages would mean we had to write a news item (sometimes) and users would have to run revdep-rebuild manually.

EAPI 5 introduced slot operators and subslots which allows packages to ask to be rebuilt when something's subslot changes. Portage can only actually act on this if a package has a := dep, and the subslot in the relevant package changed. Before this, Portage had no way of not breaking people's systems, because it had no idea what ABI breakage was or how or when to force a rebuild.

This is also why some people tend to say "there's more conflicts nowadays" - yes, because Portage refuses to let people break their systems.

Portage also has a feature called 'preserve-libs' which keeps behind libraries if something on the system still needs it, but it doesn't know why. It might be because ABI changed and a rebuild is needed (and maybe someone forgot to update the subslot in a library, or didn't add the right dependency), or it might be because the dependency is automagic.

So, unless emerge got interrupted during the world upgrade process (causing rebuilds to not happen), seeing anything for preserved-rebuild to do is a bug. There is generally no need to run revdep-rebuild unless instructed to by e.g. a news item.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum