Is Running 'sudo emerge --sync' Permissible?

[Genone]

Well, you have FEATURES=usersync enabled (probably the default by now), so verify:

usersync
Drop privileges to the owner of PORTDIR for emerge(1) --sync operations. Note that this feature assumes that all subdirectories of PORTDIR have the same ownership as PORTDIR itself. It is the user's responsibility to ensure correct ownership, since otherwise portage would have to waste time validating ownership for each and every sync operation.

[sk3l]

PORTDIR for me is (the canonical?) /usr/portage, and the subtree under /usr/portage is all portage:portage.

The other thing I saw about usersync is that the PORTAGE_TMPDIR should be portage:portage & 755 as outlined in this thread.

My tmp dir is root:root and crazy 777s, heck if I know why it's that way; I do have it as tmpfs but that shouldn't affect ownership/permissions. That said, I am not seeing the same symptoms as mentioned in the above thread, so I'm skeptical this is the underlying cause, although I will rectify it anyhow.

[Genone]

Well, another way to make sure it is not related to usersync would be to disable it temporarily with FEATURES=-usersync in make.conf (commandline override probably doesn't work with sudo).

[sk3l]

Someone else earlier in the thread mentioned something, and at the time it didn't register, but now I'm rethinking it.

For overlays, what is supposed to be the ownership of the subtree under /var/lib/layman? I only have the "kde" and "bleeding-edge" overlays (just jettisoned bleeding-edge, as it's no longer needed), but their ownership is root:root. Could that possibly be the issue? The only thing I'm using from kde's overlay is kdeconnect, but there hasn't been any new ebuilds in quite a while, so there should be no reason to delete anything during --sync AFAICT.

[Anon-E-moose]

AFAIK, emerge --sync shouldn't do anything with overlays.
I run layman -S so that I sync up those separately.

[mv]

Just a general remark without reading the whole thread:

If sudo does not work but running as root does work, this can practically only be caused by sudo not passing all environment variables (which normally is good). You can try by a binary search which variables need to be exported. A crucial first attempt would be using

Code: Select all

sudo -H

instead of sudo so that by a proper HOME actually ~root/.* files are accessed and not ~user/.*

[sk3l]

Yeah, you're right Moose.

IDK, I'm grasping at straws on this one.

I actually re-ran 'sudo emerge --sync' this morning, and although not much in the portage tree had changed, it worked without issue, even deleting a number of outdated ebuilds.

So whatever the problem is, it is an intermittent one.

[mv]

I remember now that I had a similar problem once: To my suprise it was related to some special handling of SHLVL. It seems that portage is somewhat picky if that is too large/small.

[sk3l]

Any idea what shell level portage wants? I mean, I'm not doing anything exotic, simply executing sudo emerge from within konsole, so $SHLVL should be 2.

[sk3l]

I ran emerge --sync for the first time in around a month or so today.

This time, I sync'd as root, and I tee'd the sync output into a log file to capture trace information from rsync, just in case there was another issue. Sure enough, the same problem occurs, which is some deletion failure, so none of the old ebuilds are removed, resulting in digest errors when attempting to emerge world.

So apparently this is unrelated to sudo execution, and as Hu posited, is due to some fundamental issue on my system. This problem seems most likely to manifest the longer I wait between sync's.

Here is the emerge log that I captured while sync'ing. Here is a snippet of some interesting lines (beginning at ln 1164) in which it appears rsync encounters some problem deleting files, and then apparently throws up its hands:

Code: Select all

[receiver] receiving flist for dir 202
[receiver] receiving flist for dir 231
recv_generator(app-accessibility,172)
delete_in_dir(app-accessibility)
IO error encountered -- skipping file deletion
recv_generator(app-accessibility/metadata.xml,173

As mentioned before, everything in /usr/portage is ownership portage:portage, and all directories are 755. The metadata.xml files are portage:portage and 644.

Appreciate a fresh look and any thoughts/suggestions/theories.

[sk3l]

This is insane.

After waiting an hour or so after my initial --sync, I again ran --sync, and this time it seems portage successfully deleted all the old files.

I compared the sync logs, and even though there is no obvious file troubles that would suggest a cause for the reported IO error, e.g. permissions, bad symlinks, etc., I see a subtle difference comparing the failed vs. successful --sync logs. It seems as if during the failed run, there are multiple concurrent 'flists' operations occurring for directories, almost as if rsync was performing these ops asynchronously. However, in the successful run, the 'flist' operations seem to be serialized. My wild ass guess is that perhaps during the failed run, there is some resource contention/locking that is happening for the concurrent flists, and this leads to a blocked attempt at deleting one of the directories. I have researched the parameters for rsync, and there appears to be no functionality controlling concurrent operation. For my purposes, I am not passing any special parameters to rsync, only increase verbosity to record more debug info.

Here is an excerpt from the failed --sync, just before the initial delete_in_dir(app-accessibility). Notice the interweaving "receiving filst" entries:

[receiver] receiving flist for dir 229
[generator] i=191 2 app-accessibility/festival-it/ mode=040755 len=88 flags=4
[generator] i=192 2 app-accessibility/festival-ru/ mode=040755 len=85 flags=4
[generator] i=193 2 app-accessibility/festival/ mode=040755 len=123 flags=4
[generator] i=194 2 app-accessibility/flite/ mode=040755 len=4096 flags=4
[receiver] receiving flist for dir 197
[generator] i=195 2 app-accessibility/freetts/ mode=040755 len=155 flags=4
[generator] i=196 2 app-accessibility/gnome-mag/ mode=040755 len=86 flags=4
[generator] i=197 2 app-accessibility/gnome-speech/ mode=040755 len=104 flags=4
[generator] i=198 2 app-accessibility/gok/ mode=040755 len=80 flags=4
[generator] i=199 2 app-accessibility/java-access-bridge/ mode=040755 len=146 flags=4
[receiver] receiving flist for dir 198
[generator] i=200 2 app-accessibility/julius/ mode=040755 len=94 flags=4
[generator] i=201 2 app-accessibility/mbrola/ mode=040755 len=116 flags=4
[generator] i=202 2 app-accessibility/morseall/ mode=040755 len=84 flags=4
[generator] i=203 2 app-accessibility/nfbtrans/ mode=040755 len=125 flags=4
[generator] i=204 2 app-accessibility/orca/ mode=040755 len=130 flags=4
[receiver] receiving flist for dir 199
[generator] i=205 2 app-accessibility/perlbox-voice/ mode=040755 len=126 flags=4
[generator] i=206 2 app-accessibility/pidgin-festival/ mode=040755 len=89 flags=4
[generator] i=207 2 app-accessibility/pocketsphinx/ mode=040755 len=86 flags=4
[generator] i=208 2 app-accessibility/powiedz/ mode=040755 len=96 flags=4
[receiver] receiving flist for dir 200
[generator] i=209 2 app-accessibility/simon/ mode=040755 len=118 flags=4
[generator] i=210 2 app-accessibility/sound-icons/ mode=040755 len=85 flags=4
[generator] i=211 2 app-accessibility/speakup/ mode=040755 len=150 flags=4
[generator] i=212 2 app-accessibility/speech-dispatcher/ mode=040755 len=4096 flags=4
[receiver] receiving flist for dir 201
[generator] i=213 2 app-accessibility/speech-tools/ mode=040755 len=4096 flags=4
[generator] i=214 2 app-accessibility/speechd-el/ mode=040755 len=84 flags=4
[generator] i=215 2 app-accessibility/speechd-up/ mode=040755 len=99 flags=4
[generator] i=216 2 app-accessibility/sphinx2/ mode=040755 len=93 flags=4
[generator] i=217 2 app-accessibility/sphinx3/ mode=040755 len=120 flags=4
[receiver] receiving flist for dir 230
[generator] i=218 2 app-accessibility/sphinxbase/ mode=040755 len=124 flags=4
[generator] i=219 2 app-accessibility/yasr/ mode=040755 len=95 flags=4
recv_file_list done
[receiver] receiving flist for dir 202
[receiver] receiving flist for dir 231
recv_generator(app-accessibility,172)
delete_in_dir(app-accessibility)
IO error encountered -- skipping file deletion
recv_generator(app-accessibility/metadata.xml,173)

Now here is an excerpt from the successful --sync. Notice the missing flists this time:

[generator] flist start=173, used=47, low=0, high=46
[generator] i=173 2 app-accessibility/metadata.xml mode=0100644 len=1652 flags=0
[generator] i=174 2 app-accessibility/SphinxTrain/ mode=040755 len=133 flags=4
[generator] i=175 2 app-accessibility/accerciser/ mode=040755 len=117 flags=4
[generator] i=176 2 app-accessibility/at-spi2-atk/ mode=040755 len=4096 flags=4
[generator] i=177 2 app-accessibility/at-spi2-core/ mode=040755 len=4096 flags=4
[generator] i=178 2 app-accessibility/brltty/ mode=040755 len=122 flags=4
[generator] i=179 2 app-accessibility/caribou/ mode=040755 len=84 flags=4
[generator] i=180 2 app-accessibility/dasher/ mode=040755 len=81 flags=4
[generator] i=181 2 app-accessibility/edbrowse/ mode=040755 len=152 flags=4
[generator] i=182 2 app-accessibility/eflite/ mode=040755 len=94 flags=4
[generator] i=183 2 app-accessibility/emacspeak-ss/ mode=040755 len=100 flags=4
[generator] i=184 2 app-accessibility/emacspeak/ mode=040755 len=146 flags=4
[generator] i=185 2 app-accessibility/epos/ mode=040755 len=96 flags=4
[generator] i=186 2 app-accessibility/espeak/ mode=040755 len=87 flags=4
[generator] i=187 2 app-accessibility/espeakup/ mode=040755 len=122 flags=4
[generator] i=188 2 app-accessibility/festival-fi/ mode=040755 len=102 flags=4
[generator] i=189 2 app-accessibility/festival-freebsoft-utils/ mode=040755 len=141 flags=4
[generator] i=190 2 app-accessibility/festival-hts/ mode=040755 len=86 flags=4
[generator] i=191 2 app-accessibility/festival-it/ mode=040755 len=88 flags=4
[generator] i=192 2 app-accessibility/festival-ru/ mode=040755 len=85 flags=4
[generator] i=193 2 app-accessibility/festival/ mode=040755 len=123 flags=4
[generator] i=194 2 app-accessibility/flite/ mode=040755 len=4096 flags=4
[generator] i=195 2 app-accessibility/freetts/ mode=040755 len=155 flags=4
[generator] i=196 2 app-accessibility/gnome-mag/ mode=040755 len=86 flags=4
[generator] i=197 2 app-accessibility/gnome-speech/ mode=040755 len=104 flags=4
[generator] i=198 2 app-accessibility/gok/ mode=040755 len=80 flags=4
[generator] i=199 2 app-accessibility/java-access-bridge/ mode=040755 len=146 flags=4
[generator] i=200 2 app-accessibility/julius/ mode=040755 len=94 flags=4
[generator] i=201 2 app-accessibility/mbrola/ mode=040755 len=116 flags=4
[generator] i=202 2 app-accessibility/morseall/ mode=040755 len=84 flags=4
[generator] i=203 2 app-accessibility/nfbtrans/ mode=040755 len=125 flags=4
[generator] i=204 2 app-accessibility/orca/ mode=040755 len=130 flags=4
[generator] i=205 2 app-accessibility/perlbox-voice/ mode=040755 len=126 flags=4
[generator] i=206 2 app-accessibility/pidgin-festival/ mode=040755 len=89 flags=4
[generator] i=207 2 app-accessibility/pocketsphinx/ mode=040755 len=86 flags=4
[generator] i=208 2 app-accessibility/powiedz/ mode=040755 len=96 flags=4
[generator] i=209 2 app-accessibility/simon/ mode=040755 len=118 flags=4
[generator] i=210 2 app-accessibility/sound-icons/ mode=040755 len=85 flags=4
[generator] i=211 2 app-accessibility/speakup/ mode=040755 len=150 flags=4
[generator] i=212 2 app-accessibility/speech-dispatcher/ mode=040755 len=4096 flags=4
[generator] i=213 2 app-accessibility/speech-tools/ mode=040755 len=4096 flags=4
[generator] i=214 2 app-accessibility/speechd-el/ mode=040755 len=84 flags=4
[generator] i=215 2 app-accessibility/speechd-up/ mode=040755 len=99 flags=4
[generator] i=216 2 app-accessibility/sphinx2/ mode=040755 len=93 flags=4
[generator] i=217 2 app-accessibility/sphinx3/ mode=040755 len=120 flags=4
[generator] i=218 2 app-accessibility/sphinxbase/ mode=040755 len=124 flags=4
[generator] i=219 2 app-accessibility/yasr/ mode=040755 len=95 flags=4
recv_file_list done
recv_generator(app-accessibility,172)
delete_in_dir(app-accessibility)
[generator] make_file(app-accessibility/julius,*,2)
[generator] make_file(app-accessibility/speech-tools,*,2)

[Anon-E-moose]

Then sudo is likely not the problem.

[sk3l]

I ran emerge --sync for the first time in around a month or so today.

This time, I sync'd as root, and I tee'd the sync output into a log file to capture trace information from rsync, just in case there was another issue. Sure enough, the same problem occurs, which is some deletion failure, so none of the old ebuilds are removed, resulting in digest errors when attempting to emerge world.

So apparently this is unrelated to sudo execution, and as Hu posited, is due to some fundamental issue on my system. This problem seems most likely to manifest the longer I wait between sync's.

Here is the emerge log that I captured while sync'ing. Here is a snippet of some interesting lines (beginning at ln 1164) in which it appears rsync encounters some problem deleting files, and then apparently throws up its hands:

Code: Select all

[receiver] receiving flist for dir 202
[receiver] receiving flist for dir 231
recv_generator(app-accessibility,172)
delete_in_dir(app-accessibility)
IO error encountered -- skipping file deletion
recv_generator(app-accessibility/metadata.xml,173

As mentioned before, everything in /usr/portage is ownership portage:portage, and all directories are 755. The metadata.xml files are portage:portage and 644.

Appreciate a fresh look and any thoughts/suggestions/theories.

[Anon-E-moose]

This is insane.

After waiting an hour or so after my initial --sync, I again ran --sync, and this time it seems portage successfully deleted all the old files.

I compared the sync logs, and even though there is no obvious file troubles that would suggest a cause for the reported IO error, e.g. permissions, bad symlinks, etc., I see a subtle difference comparing the failed vs. successful --sync logs. It seems as if during the failed run, there are multiple concurrent 'flists' operations occurring for directories, almost as if rsync was performing these ops asynchronously. However, in the successful run, the 'flist' operations seem to be serialized. My wild ass guess is that perhaps during the failed run, there is some resource contention/locking that is happening for the concurrent flists, and this leads to a blocked attempt at deleting one of the directories. I have researched the parameters for rsync, and there appears to be no functionality controlling concurrent operation. For my purposes, I am not passing any special parameters to rsync, only increase verbosity to record more debug info.

It's possible that it's some conflict with rsync.

I personally do my rsync, in the middle of the night when I'm usually sleeping, from cron.
I also have emerge --sync run from cron daily and they run about an hour apart.

I don't know if rsync locks files it is copying, but it would make sense to do so at least in a directory it's trying to sync up.
Doing a simple google search on rsync locking, it appears that it does NOT do that.

Edit to add: It's possible that it still has something to do with lvm. I don't know enough of how lvm does what it does.
Does lvm do snapshots and if so, it may be doing the write locking (deletions are just another form of write)

[sk3l]

Thanks Anon-E-moose.

Yeah, my usage pattern is a bit different, I tend to perform --syncs much less frequently. This is my desktop PC, but it is not always on, and I might go days or even a week without powering it on. I would think that even though most people seem to incrementally sync & emerge in shorter periods, my use case is probably not incredibly rare, where people only sync + emerge 1x every few weeks.

I am looking at the source now for rsync. There is a global io_error variable, and once it's set, then any subsequent deletes are skipped. I sure would love it if the exact code value of the io_error was logged during the initial error reporting so I could get a better sense of what's causing this. I don't see anything in the preceding lines in my --sync log, either warning or errors, that give me a clue as to what the underlying cause is. I also read that there was a recent patch in rsync in the area of not setting the io_error flag for so called "vanished files" so as to avoid killing deletions in that scenario. This patch was in rsync 3.1, and I'm on 3.09, which AFAIK is the most recent stable rev in the tree. I could upgrade to unstable 3.1 and see if that fixes my issue. Alternatively, I could build my own local copy of rsync and add some additional trace logging to try to flush out the issue, but I'd rather not go there yet.

Does anyone else log their --sync operations? If so, it would help if you could provide me with a copy to compare against my own, to see if there are any key differences.