Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage not checking md5sums/timestamps when cleaning?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
n0n
Guru
Guru


Joined: 13 Jun 2002
Posts: 355

PostPosted: Thu Oct 11, 2007 4:50 pm    Post subject: Portage not checking md5sums/timestamps when cleaning? Reply with quote

Hey, I'm experiencing an odd problem with Portage 2.1.3.9 which may or may not be related to binary packages.

Essentially, it looks like Portage isn't properly checking md5sums and timestamps when cleaning packages after some installs. One specific case is a box where I'm upgrading to a new debianutils, which depends on sys-apps/mktemp instead of having /bin/mktemp bundled into debianutils itself. Before the install, my /var/db/pkg/sys-apps/debianutils-2.15-r1/CONTENTS contains the following:
Code:
obj /bin/mktemp 884a3b58d7d30898a7a1f1e1dcd130e2 1172858141

That's correct - the md5sum matches, as does the timestamp. When mktemp gets installed (before bebianutils), I get the following in /var/db/pkg/sys-apps/mktemp-1.5/CONTENTS -
Code:
obj /bin/mktemp 455b1b406780eea7426ed0ad701279da 1190739808

This, too, is correct. However, when the debianutils upgrade happens, and Portage cleans out the old version of debianutils, it ends up removing /bin/mktemp despite the fact that neither the md5sum nor timestamp actually match:
Code:
<<<          obj /bin/mktemp


I can reproduce this in reverse as well... After that upgrade, I've got debianutils-2.23.1 and mktemp-1.5 installed. If I downgrade back to debianutils-2.15-r1 and then remove the mktemp package, the mktemp unmerge removes /bin/mktemp even though the md5sum and timestamp no longer match.

So what gives here? The packages I'm installing are binary packages, I don't know if that has anything to do with it. Do I have to do anything specifically to get Portage to behave itself here?

Code:
# emerge --info
Portage 2.1.3.9 (default-linux/x86/no-nptl/2.4, gcc-3.4.6, glibc-2.3.6-r4, 2.4.34.1 i686)
=================================================================
System uname: 2.4.34.1 i686 Intel(R) Xeon(TM) CPU 2.66GHz
Timestamp of tree: Tue, 25 Sep 2007 15:00:01 +0000
app-shells/bash:     3.2_p17
dev-java/java-config: 1.2.11-r1
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.4-r7
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.4.19, 2.4.26-r1
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-mcpu=pentium3 -O3 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/local/www/apache/conf"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-mcpu=pentium3 -O3 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo/"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="alsa apm arts berkdb bitmap-fonts cli cracklib crypt cups dri eds emboss encode foomaticdb fortran gdbm gif gpm gstreamer gtk2 iconv imap imlib innodb ipv6 isdnlog jpeg kde libg++ libwww mad midi mikmod motif mp3 mpeg mudflap mysql ncurses nls ogg old-linux opengl openmp oss pam pcre perl png pppd python qt3 qt4 quicktime readline reflection sasl sdl session snmp spell spl ssl tcpd truetype truetype-fonts type1-fonts vorbis x86 xml xorg xv zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 i810 imstt mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Back to top
View user's profile Send private message
n0n
Guru
Guru


Joined: 13 Jun 2002
Posts: 355

PostPosted: Thu Oct 11, 2007 5:03 pm    Post subject: Reply with quote

Here's a log of what I'm doing, in case the above wasn't clear:

Code:
box ~ # grep /bin/mktemp /var/db/pkg/sys-apps/debianutils-2.15-r1/CONTENTS
obj /bin/mktemp 884a3b58d7d30898a7a1f1e1dcd130e2 1171647276

box ~ # stat -c "%Y" /bin/mktemp
1171647276

box ~ # md5sum /bin/mktemp
884a3b58d7d30898a7a1f1e1dcd130e2  /bin/mktemp

box ~ # emerge -K1O mktemp

>>> Emerging (1 of 1) sys-apps/mktemp-1.5 to /
>>> Extracting info
>>> Extracting mktemp-1.5
>>> Merging sys-apps/mktemp-1.5 to /
--- /bin/
>>> /bin/mktemp
--- /usr/
--- /usr/share/
--- /usr/share/man/
--- /usr/share/man/man1/
>>> /usr/share/man/man1/mktemp.1.bz2
--- /usr/share/doc/
>>> /usr/share/doc/mktemp-1.5/
>>> /usr/share/doc/mktemp-1.5/README.bz2
>>> /usr/share/doc/mktemp-1.5/RELEASE_NOTES.bz2
>>> sys-apps/mktemp-1.5 merged.

>>> No packages selected for removal by clean
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
 * GNU info directory index is up-to-date.

box ~ # grep /bin/mktemp /var/db/pkg/sys-apps/mktemp-1.5/CONTENTS
obj /bin/mktemp 455b1b406780eea7426ed0ad701279da 1190739808

box ~ # stat -c "%Y" /bin/mktemp
1190739808

box ~ # md5sum /bin/mktemp
455b1b406780eea7426ed0ad701279da  /bin/mktemp

box ~ # grep /bin/mktemp /var/db/pkg/sys-apps/debianutils-2.15-r1/CONTENTS
obj /bin/mktemp 884a3b58d7d30898a7a1f1e1dcd130e2 1171647276

box ~ # emerge -K1O debianutils

>>> Emerging (1 of 1) sys-apps/debianutils-2.23.1 to /
>>> Extracting info
>>> Extracting debianutils-2.23.1
>>> Merging sys-apps/debianutils-2.23.1 to /
--- /bin/
>>> /bin/tempfile
>>> /bin/run-parts
--- /sbin/
>>> /sbin/installkernel
--- /usr/
--- /usr/sbin/
>>> /usr/sbin/savelog
>>> /usr/sbin/mkboot
--- /usr/share/
--- /usr/share/man/
--- /usr/share/man/man1/
>>> /usr/share/man/man1/tempfile.1.bz2
--- /usr/share/man/man8/
>>> /usr/share/man/man8/run-parts.8.bz2
>>> /usr/share/man/man8/savelog.8.bz2
>>> /usr/share/man/man8/installkernel.8.bz2
>>> /usr/share/man/man8/mkboot.8.bz2
--- /usr/share/doc/
>>> /usr/share/doc/debianutils-2.23.1/
>>> /usr/share/doc/debianutils-2.23.1/changelog.bz2
>>> /usr/share/doc/debianutils-2.23.1/control.bz2
>>> sys-apps/debianutils-2.23.1 merged.

 sys-apps/debianutils
    selected: 2.15-r1
   protected: 2.23.1
     omitted: none

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

>>> Unmerging sys-apps/debianutils-2.15-r1...
No package files given... Grabbing a set.
<<<          obj /usr/share/man/man8/savelog.8.gz
<<<          obj /usr/share/man/man8/run-parts.8.gz
<<<          obj /usr/share/man/man8/mkboot.8.gz
<<<          obj /usr/share/man/man8/installkernel.8.gz
--- replaced dir /usr/share/man/man8
<<<          obj /usr/share/man/man1/tempfile.1.gz
<<<          obj /usr/share/man/man1/mktemp.1.gz
--- replaced dir /usr/share/man/man1
--- replaced dir /usr/share/man
<<<          obj /usr/share/doc/debianutils-2.15-r1/control.gz
<<<          obj /usr/share/doc/debianutils-2.15-r1/changelog.gz
--- replaced dir /usr/share/doc
--- replaced dir /usr/share
--- replaced obj /usr/sbin/savelog
--- replaced obj /usr/sbin/mkboot
--- replaced dir /usr/sbin
--- replaced dir /usr
--- replaced obj /sbin/installkernel
--- replaced dir /sbin
--- replaced obj /bin/tempfile
--- replaced obj /bin/run-parts
<<<          obj /bin/mktemp
--- replaced dir /bin
<<<          dir /usr/share/doc/debianutils-2.15-r1
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
 * GNU info directory index is up-to-date.

box ~ # ls /bin/mktemp
ls: cannot access /bin/mktemp: No such file or directory

So there you have it. I should also note that it's not just this one package - I've noticed this behavior in a few others as well.
Back to top
View user's profile Send private message
n0n
Guru
Guru


Joined: 13 Jun 2002
Posts: 355

PostPosted: Thu Oct 11, 2007 5:34 pm    Post subject: Reply with quote

Well, that's interesting... I was digging around /usr/lib/portage/pym/portage.py to see if I could find out what was going on, and it looks like, when portage removes files on this system, it's happening in this block (line 7483):
Code:
                # Don't unlink symlinks to directories here since that can
                # remove /lib and /usr/lib symlinks.
                if unmerge_orphans and \
                    lstatobj and not stat.S_ISDIR(lstatobj.st_mode) and \
                    not (islink and statobj and stat.S_ISDIR(statobj.st_mode)) and \
                    not self.isprotected(obj):
                    try:
                        # Remove permissions to ensure that any hardlinks to
                        # suid/sgid files are rendered harmless.
                        if statobj and not islink:
                            os.chmod(obj, 0)
                        os.unlink(obj)
                    except EnvironmentError, e:
                        pass
                    show_unmerge("<<<", "", file_type, obj)
                    continue

... instead of a bit lower down during the block starting with
Code:
elif pkgfiles[objkey][0] == "obj":
, like I'd expect.

So that's certainly why md5sums and timestamps aren't being checked on my system. Now to find out why that's happening...
Back to top
View user's profile Send private message
n0n
Guru
Guru


Joined: 13 Jun 2002
Posts: 355

PostPosted: Thu Oct 11, 2007 5:42 pm    Post subject: Reply with quote

Aha! From make.conf's manpage:
Code:
              unmerge-orphans
                     If a file is not claimed by another package in the same slot and  it  is  not  protected  by  CON-
                     FIG_PROTECT,  unmerge  it even if the modification time or checksum differs from the file that was
                     originally installed.

That certainly sounds exactly like what's going on here. I wonder why that's apparently set by default, though... Seems rather potentially destructive.
Back to top
View user's profile Send private message
n0n
Guru
Guru


Joined: 13 Jun 2002
Posts: 355

PostPosted: Thu Oct 11, 2007 6:23 pm    Post subject: Reply with quote

Well, I may as well continue this long string of Talking To Myself to mention that I opened up a bug on this. IMO this really shouldn't be a default FEATURE. http://bugs.gentoo.org/show_bug.cgi?id=195527
Back to top
View user's profile Send private message
rizzo
Retired Dev
Retired Dev


Joined: 30 Apr 2002
Posts: 1067
Location: Manitowoc, WI, USA

PostPosted: Thu Oct 11, 2007 6:27 pm    Post subject: Reply with quote

<3
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum