Portage does more security checks? Digest verification fails

iTux · Guru Joined: 07 Sep 2004 Posts: 586 Location: Toronto

I did an emerge sync. I did not upgrade portage or other system utilities and I did not modified my config.

I noticed that portage does more checking that it used before:

iTux · Guru Joined: 07 Sep 2004 Posts: 586 Location: Toronto

Seems to be due to FEATURES=strict that is enabled by default. I guess it was disabled by default before?

iTux

Earthwings · Posted: Sun Apr 24, 2005 9:59 am Post subject:

Yes, it was changed recently. http://www.gentoo.org/cgi-bin/viewcvs.cgi/profiles/base/make.defaults?r1=1.4&r2=1.5

Here's a short summary what changed and what can be done against "Digest verification failure".

FEATURES=strict (which is now enabled on your system if you ran emerge --sync in the last days and don't have FEATURES=-strict in make.conf) tells emerge to check the digest (md5 sum) not only for the files in SRC_URI, but also for ebuilds. This is more secure and therefore a good thing, but you may experience "digest verification failure" more often. Here is how to handle this situation.

First, determine whether the digest failed for a file from SRC_URI (most often a .gz or .bz2 or .rpm). If this is the case try the following:
- delete the file from /usr/portage/distfiles and restart emerge. It will be downloaded again from the mirror.
- delete the file from /usr/portage/distfiles and download in manually from another mirror, cause the mirror may contain a broken file.
- run emerge --sync and try again. The problem might be already fixed
- search forums.gentoo.org and bugs.gentoo.org to see if other people experience the same problem and the recorded md5 sum is wrong
- write a bug report if none exists or contact a developer in irc at irc.freenode.net
- If you are absolutely sure that the file you downloaded is ok, you can generate a new digest with ebuild /path/to/ebuild digest.

If you get a digest failure for a file from the Manifest (typically a .ebuild), do the following:
- run emerge --sync to see if the problem is already fixed.
- search forums.gentoo.org and bugs.gentoo.org to see if other people experience the same problem and the recorded md5 sum is wrong
- write a bug report if none exists or contact a developer in irc at irc.freenode.net
- if you verified the file is ok (have a look at it), you can generate a new digest with ebuild /path/to/ebuild digest.
_________________
KDE

Bob P · Posted: Sun Apr 24, 2005 12:05 pm Post subject:

ARGH!

i'm performing an emerge -e system to rebuild the toolkit on a fleet of systems (every possible x86 architecture) after downloading the most recent portage snapshot. would anyone care to guess how often this error pops up:

Earthwings · Posted: Sun Apr 24, 2005 12:56 pm Post subject:

I think I would disable strict in emerge -e until everyone uses repoman for checking things in. FEATURES=-strict emerge -e system
_________________
KDE

kimchi_sg · Advocate Joined: 26 Nov 2004 Posts: 2968

In a perfect world, this would be a Good Thing.

Bob P · Posted: Sun Apr 24, 2005 1:03 pm Post subject:

Bob P · Posted: Sun Apr 24, 2005 7:06 pm Post subject:

well, i have to admit, i'm giving up on the Boy Scout approach of doing it the hard way and reporting all of the bugs to Bugzilla. unfortunately, its impossible to perform an "emerge --resume --skipfirst" following a critical stop during an "emerge -e system". because there's no way to recover, and because its pointless to keep starting the whole emerge over again, i'm just proceeding with "-strict" in make.conf. other people will have to report bugs as they encounter them.

_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks

rhill · Posted: Mon Apr 25, 2005 5:23 am Post subject:

heh.

https://bugs.gentoo.org/show_bug.cgi?id=89388
_________________
by design, by neglect
for a fact or just for effect

moocha · Watchman Joined: 21 Oct 2003 Posts: 5722

Most of the

Bob P · Posted: Mon Apr 25, 2005 10:37 pm Post subject:

in my case, i was rebuilding freshly extracted tarballs, rebuilding their toolkits for an entire array of architectures, using the most current portage snapshot. there was no old portage tree to replace, and i was already using the most recent portage snapshot.

the idea of syncing is well advised. but in my case, i had the most current snapshots and i was trying to build binary packages for distribution. in a situation like that, syncing doesn't help, as the users have to be given a snapshot that matches their binaries. turning off that horrible "strict" feature was the only way around the problem.

otherwise, our entire Jackass! production effort would have been torpedoed by the introduction of strict checking as a "feature" in portage.

i can see where strict checking would be an asset as default portage behavior ... that is, if ebuild maintainers were willing to take the effort to fix their ebuilds in time to have them ready for the "strict" rollout. but that wasn't the case -- strict got implemented, ebuilds weren't ready, and plenty of crappy ebuilds broke as a result. one of the ebuilds i found had contained a stray file that was 2 years old. that's just sloppy programming, and it shows that there wasn't much effort to synchronize actions on the part of portage developers and ebuild maintainers.

i honestly think that the implementation of strict checking would have been better received if there had been a heads-up that it was coming. it seemed to come out of nowhere, and hit everybody like it was a sucker punch. :cry:

_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks

Bob P · Posted: Mon Apr 25, 2005 10:44 pm Post subject:

Genone · Posted: Tue Apr 26, 2005 1:35 am Post subject:

Hmm, reminds me to put it in make.globals (it should never have been in make.defaults IMO).

opentaka · l33t Joined: 18 Feb 2005 Posts: 840 Location: Japan

cool, but i saw something saying "Gentoo Portage Backdoor" thing somewhere in the web.. I think it fixed by now
_________________
"Being defeated is often a temporary condition. Giving up is what makes it permanent" - Marilyn vos Savant

vonhelmet · Posted: Fri May 06, 2005 11:11 am Post subject:

I've just run into this problem with files not listed in the manifest.

I don't have a net connection at home, so I download snapshots and distfiles at work. I "sync" by untarring the whole tree over the /usr/portage directory. This isn't a real sync, as old files aren't deleted, which then causes problems because there's loads of old ebuilds and stuff that aren't in the manifest.

Anyone got any suggestions on how I can "sync" without running into this manifest problem? For the time being I've been deleting all the files in portage (except distfiles and a couple of others) and untarring the new tarball in the nearly empty directory. I'm thinking I might try to put together a perl script that will "sync" with a tarball offline in some way, but that sounds a bit complex to me.
_________________
My blog
nvtuner software - enhance your AGP Geforce 6800 or 6200!

Ciaran · n00b Joined: 10 May 2005 Posts: 2 Location: England, UK

FWIW, I've never had any problem with the new strict checking...
_________________
Please note: I'm not ciaranm. This is an entirely different Ciaran. ;-)

nexus780 · Posted: Tue May 10, 2005 1:44 pm Post subject:

mikenerone · Posted: Tue May 10, 2005 11:04 pm Post subject:

paultorbens · n00b Joined: 12 May 2005 Posts: 2

thanks for the script - it worked great for me ...

vonhelmet · Posted: Mon May 16, 2005 7:56 am Post subject:

Yeah, cheers, that did the job and made sync'ing offline a lot easier.
_________________
My blog
nvtuner software - enhance your AGP Geforce 6800 or 6200!

dundas · Guru Joined: 16 Dec 2004 Posts: 317 Location: China, Earth

I had this prob after emerge --sync

adding
FEATURES=-strict
in make.conf

worked fine for me, thx buddy.

sinewalker

I stumbled across this issue when adding new packages for the first time on my bandwidth-impared home PC. I have updated the wiki to use mikenerone's shell script after downloading a snapshot but before doing emerge. It works wonders.

So now I can emerge using work's bandwidth and still be "strict".

I am really impressed by the Gentoo comunity -- so many answers solved quickly!

dundas · Guru Joined: 16 Dec 2004 Posts: 317 Location: China, Earth

Dear all:

Here is the problem while I updating world today,

[NOTE] I dont have features="strict" in my make.conf

dundas · Guru Joined: 16 Dec 2004 Posts: 317 Location: China, Earth

even if I rm -rf /var/tmp/portage/*, rm -rf /usr/portage/distfiles/* and emerge --sync again, I couldn't get it working, still same errors

[SOLUTION]
The only thing worked for me is to put the "-strict" in my features

p.s. I did use -strict before but after some while, I tried to use strict, but now....guess I'm back to the old way again....feeling dizzy....I need security....

anyways, thx for merging my post.
_________________
Appreciate Gentoo: Best Devs, Best Forums. YOU could help too: Help Answer

Bob P · Posted: Mon Aug 15, 2005 2:11 am Post subject:

i'm going to rant a little about this problem, which hasn't gone away after a period of months. the long and short of it is that ebuild maintainers are just doing a half-baked job at maintaining their ebuilds. unfortunately, there are ALOT of ebuilds that are gaining status as offenders in this regard, and the problem seems to be getting worse rather than better. such wouldn't be the case if the people maintaining ebuilds bothered to check their work before releasing it into the portage tree.

<rant>
a conscientious ebuild maintainer would actually test their ebuild before publishing it, by performing a test emerge after finishing the build. if such testing were performed, a light bulb would go on over their heads as they emerged their ebuild and saw the bright red letters warning about files existing that are not in the manifest, and the emerge stopped with a critical error. the fact that these mistakes are slipping through means that error checks aren't being performed. how could this happen?

well, for one, it appears that the offending ebuild maintainers have to be using FEATURES=-strict settings on their development boxes. this effectively suppresses the error warnings and the critical stops so that they never see them. in contrast, conscientious users who try to protect their systems by not using the FEATURES=-strict settings get jammed when they try to emerge the ebuilds.

the fact that these problems are continuously popping up in the ebuilds means that there are quite a few ebuild maintainers who just don't bother to perform QA. sheesh. this happened to me four times tonight trying to perform a basic install, and included ebuilds such as grub, hotplug or coldplug, acpid and gentoo-sources-2.6.12-r6.

this is a very simple problem to fix. the ebuild maintainers should STOP using "FEATURES=-strict" on their development boxes. that will result in the error not slipping by them, and then we'll stop having the problem. until EVERY ebuild maintainer does this, the problem will never go away.

</rant>
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks