Forums

Skip to content

Advanced search
  • Quick links
    • Unanswered topics
    • Active topics
    • Search
  • FAQ
  • Login
  • Register
  • Board index Discussion & Documentation Documentation, Tips & Tricks
  • Search

Cleaning World-file pollution - new fast script

Unofficial documentation for various parts of Gentoo Linux. Note: This is not a support forum.
Post Reply
Advanced search
10 posts • Page 1 of 1
Author
Message
Goverp
Advocate
Advocate
User avatar
Posts: 2402
Joined: Wed Mar 07, 2007 6:41 pm

Cleaning World-file pollution - new fast script

  • Quote

Post by Goverp » Fri Feb 20, 2026 7:50 pm

Updated - added explanation and caveats to the report, and Colour.
There's an existing script to identify portage world-file entries that are dependencies of other packages, which can be useful if you've forgotten to specify "--one-shot" while fixing emerge issues, and similar.

However, it is now broken, as it relies on output from "qdepends -Q" that has changed significantly since the script was written, and also it performs like a dog as it issues "emerge --pretend --depclean" for each world file entry the isn't identified by the "qdepends" test.

It turns out that "emerge --pretend --quiet --depclean" works if you feed it the entire world file contents in the command line, and this takes not much longer than each of the individual runs in the old script. You can parse the output to generate a couple of reports, one of packages that are not dependencies of anything else (and therefore you may wish to retain in your world file), and one of packages that are dependencies, and therefore are candidates for consideration as "pollution" that can be removed with "emerge --deselect y".

Here's a new script to do that:

Code: Select all

#!/bin/sh

### Report on Portage world-file packages
# Paul Gover, Feb 2026

# A couple of essential portage paths
world_file=/var/lib/portage/world
vdb=$(portageq vdb_path)

# Identify myself for use in tempfile names
ME="${0##*/}"	# strip off any path
ME="${ME%%.*}"	# strip off any extension

# Some ANSI colour sequences
BLUE=$(printf '\033[34;1m')
WHITE=$(printf '\033[37;1m')
RESET=$(printf '\033[0m')

# We do everything in the portage installed package database
cd "$vdb" || { printf 'cd %s failed.\n' "$vdb" >&2 ; exit 1 ; }

root_pkgs() {
	# A list of world-set packages that are required by nothing else
	# The output up to the first blank line is a series of category/package lines
	# shellcheck disable=SC2046
	emerge -pq --depclean $(cat "$world_file")
}

disqualify() {
	# Remove any qualification text following colons from a stream of files
	# (for example, www-client/firefox:rapid, or
	# games-util/steam-launcher::steam),
	# but discarding all contents from the first blank line onwards
	local finished pkg rest
	while read -r pkg rest
	do
		[ "$finished" ] && continue
		[ "" = "$pkg" ] && finished="true" && continue
		printf '%s\n' "${pkg%%:*}"
	done
}

describe() {
	# Print the package name and its description from the portage database
	local pkg rest

	while read -r pkg rest
	do
		printdesc "$pkg" "$pkg"*/DESCRIPTION # This may match several files
	done
}

printdesc() {
	# Get the first Descriptions file, if any
	# First parameter is the package name, second is a series of paths in the vdb
	local pkg path cpn cp desc

	pkg="$1"

	shift 

	for path
	do
		cpn="${path%/DESCRIPTION}"
		cp="${cpn%%-[0-9]*}"		# p is everything up to the first version number field
		[ "$pkg" = "$cp" ] || break	# The path expansion in describe() might include "A" and "A-B" names

		[ -r "$path" ] && read -r desc<"$path"  # The last matching path will be the most up to date
	done

	if [ "$desc" ]
	then printf '%-35s %s%s%s\n' "$pkg" "$BLUE" "$desc" "$RESET"
	else printf '%s\n' "$pkg"
	fi
}

tempf() {
	# Create a tempf, using /run ramdisk if possible
        local id dir
        id="$(id -u)"

        if      [ "$id" = "0" ]
        then
                dir="${XDG_RUNTIME_DIR:-/run}"
                [ -d "$dir" ] || dir=""
                mktemp -p "$dir" "tmp.$ME$1.XXXXXXXXXX"
        else    mktemp -p "${XDG_RUNTIME_DIR}" "tmp.$ME$1.XXXXXXXXXX"
        fi
}

# A couple of tmp files to fill
roots=$(tempf '-roots')
clean_world=$(tempf '-world')

# Filter the world file and emerge -pv --depclean output
# and sort and remove duplicate entries (depclean can definitely produce some)
root_pkgs | disqualify | sort -u > "$roots"
disqualify < "$world_file" | sort -u > "$clean_world"

# Packages in both files are world entries that are not dependencies
cat << END

The following reports list package names from your portage @world set,
for you to review and possibly remove if they are unnecessary.

Note that the reports considers neither package slots nor
repository names.  If necessary, clarify package names using
	equery list -F '\$cpv:\$slot::\$repo' <packname>
(Note the quotes are essential in the above command.)

Note also that this program identifies neither packages in a circular dependency,
nor dependents of virtual packages.

${WHITE}WORLD-FILE ENTRIES THAT ARE NOT DEPENDENCIES OF OTHER PACKAGES${RESET}

These should be packages that you want to retain.
But sometimes you get things included because you forgot
emerge's "--one-shot" (or "-1") parameter when you made changes
in response to a problem merging a particular package
(such as needing a specific USE flag or keywording).

Then there may be packages you installed for a test,
haven't used since, but never got around to deleting them.
Now is your chance to "emerge --depclean" them.

In particular, you are unlikely to need libraries unless unless
you are developing code.

END

comm -12 "$clean_world" "$roots" | describe

# Packages only in the roots file are dependencies of other packages
cat << END2

${WHITE}WORLD-FILE ENTRIES THAT ARE DEPENDENCIES OF OTHER PACKAGES${RESET}

These may be unnecessary entries that should be removed with "emerge --deselect y",
but not always; sometimes you install a package, only later to install another that
depends on the first.
It's not a crime to leave them in your world file,
but if you don't think you actively and explicitly use them,
it should be safe to deselect them.

END2

comm -23 "$clean_world" "$roots" | describe

# Cleanup
rm "$roots" "$clean_world"
Last edited by Goverp on Mon Feb 23, 2026 5:54 pm, edited 3 times in total.
Greybeard
Top
NeddySeagoon
Administrator
Administrator
User avatar
Posts: 56085
Joined: Sat Jul 05, 2003 9:37 am
Location: 56N 3W

  • Quote

Post by NeddySeagoon » Fri Feb 20, 2026 9:25 pm

Goverp

Code: Select all

        # Remove any text following colons from a stream of files
        # (for example, www-client/firefox:rapid,
It's <category>/<package>:<slot>
Slots are valid in the world file, e.g. for kernels.
You probably want to retain slots if they are there and maype the repo too.
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Top
pietinger
Administrator
Administrator
Posts: 6620
Joined: Tue Oct 17, 2006 5:11 pm
Location: Bavaria

  • Quote

Post by pietinger » Fri Feb 20, 2026 9:50 pm

Goverp,

you probably know I love your scripts ... but maybe there is a problem: Some packages have dependencies only because there is a virtual ...

I just tested it and got these:

Code: Select all

app-admin/sysklogd
sys-process/cronie
->

Code: Select all

virtual/logger-0-r2 requires app-admin/sysklogd
virtual/cron-0-r3 requires sys-process/cronie
https://wiki.gentoo.org/wiki/User:Pietinger --> https://wiki.gentoo.org/wiki/User:Pieti ... _at_Gentoo
Top
b11n
Guru
Guru
User avatar
Posts: 303
Joined: Wed Mar 26, 2003 8:15 am
Location: New Zealand

Re: Cleaning World-file pollution - new fast and accurate

  • Quote

Post by b11n » Fri Feb 20, 2026 10:23 pm

Goverp said it, but it probably deserves a bit more emphasis:
Goverp wrote:and therefore are candidates for consideration as "pollution" that can be removed with "emerge --deselect y".
There will be packages which are deps of others which might still be desirable in the world file in their own right, e.g. wpa_supplicant, pulled in by NetworkManager, but not by netifrc etc.
Is there gas in the caaaaar?
Yes, there's gas in the caaaar
Top
Goverp
Advocate
Advocate
User avatar
Posts: 2402
Joined: Wed Mar 07, 2007 6:41 pm

  • Quote

Post by Goverp » Fri Feb 20, 2026 11:05 pm

NeddySeagoon wrote:Goverp

Code: Select all

        # Remove any text following colons from a stream of files
        # (for example, www-client/firefox:rapid,
It's <category>/<package>:<slot>
Slots are valid in the world file, e.g. for kernels.
You probably want to retain slots if they are there and maype the repo too.
I would if it were easy! The output of "--depclean -pq" is one line per package, but AFAIR without slot or repos suffices. The info may be there without the "-q", but then there's a lot more parsing to do, as it's several lines per item. My thinking was, if the slot or repos matters, the user will likely already be aware.

Net, I probably ought to make the disclaimer bigger, maybe add some colour. Certainly nobody should be blindly taking the output and pasting it into "emerge --deselect y".
I also now have a tweaked version which adds the package description, which is also a worthwhile extra, so I may do a V2, if the above was V1
Greybeard
Top
Goverp
Advocate
Advocate
User avatar
Posts: 2402
Joined: Wed Mar 07, 2007 6:41 pm

  • Quote

Post by Goverp » Fri Feb 20, 2026 11:16 pm

pietinger wrote:Goverp,

you probably know I love your scripts ... but maybe there is a problem: Some packages have dependencies only because there is a virtual ...

I just tested it and got these:

Code: Select all

app-admin/sysklogd
sys-process/cronie
->

Code: Select all

virtual/logger-0-r2 requires app-admin/sysklogd
virtual/cron-0-r3 requires sys-process/cronie
Glad you like them. I actually rather dislike shell script as a language - too many fiddly syntax and system-specific details - but it's fun seeing what you can do with it, and it's much better than Bash in those respects.
(I've just written some Rust code; that may not be system-specific, but gosh is it (a) every bit as hard to get to grips with as shell, and (b) much harder work than garbage-collecting languages like Smalltalk.)

Interesting - on my box: "--depclean -p" fails for cronie, "--depclean -pv" tells me about virtual/cron, as you say, but for some reason "--depclean -pq" reports nothing. :-(
I'm not fixing "emerge", so it looks like I need to warn about virtual packages not being caught as well.
For the time being, I've removed the "and accurate" from the thread title!
Greybeard
Top
pietinger
Administrator
Administrator
Posts: 6620
Joined: Tue Oct 17, 2006 5:11 pm
Location: Bavaria

  • Quote

Post by pietinger » Sat Feb 21, 2026 12:50 am

Goverp wrote:[...] but for some reason "--depclean -pq" reports nothing. :-(
For me it feels logical ... there is nothing to do and with -q emerge is quiet ... :lol:

If you do a -cpqv (the -v gives the information WHY it is not possible to remove it) then you will get only this information (although -q is also enable):

Code: Select all

# emerge -cp cronie

Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed:   1171
Packages in world:    96
Packages in system:   50
Required packages:    1171
Number to remove:     0
# 
# emerge -cpv cronie

Calculating dependencies... done!
  sys-process/cronie-1.7.2 pulled in by:
    virtual/cron-0-r3 requires sys-process/cronie

>>> No packages selected for removal by depclean
Packages installed:   1171
Packages in world:    96
Packages in system:   50
Required packages:    1171
Number to remove:     0
# 
# emerge -cpq cronie
# 
# emerge -cpqv cronie
  sys-process/cronie-1.7.2 pulled in by:
    virtual/cron-0-r3 requires sys-process/cronie
https://wiki.gentoo.org/wiki/User:Pietinger --> https://wiki.gentoo.org/wiki/User:Pieti ... _at_Gentoo
Top
dmpogo
Advocate
Advocate
Posts: 3713
Joined: Thu Sep 02, 2004 9:21 pm
Location: Canada

  • Quote

Post by dmpogo » Sat Feb 21, 2026 2:13 am

There is also the case of circular dependencies that one needs to be careful about. For instance


dev-tex/minted
dev-tex/latex2pydata
dev-texlive/texlive-latexextra

all depend on each other. I happen to have the last one in the world file, but it is not that I can take it out unless I have app-text/texlive with +extra USE flag
Top
Goverp
Advocate
Advocate
User avatar
Posts: 2402
Joined: Wed Mar 07, 2007 6:41 pm

  • Quote

Post by Goverp » Mon Feb 23, 2026 5:47 pm

I've posted an updated version of the script as an edit to the first post to this thread.
It now adds the description of each package, and a load of explanation and caveats to the report, and a bit of colour.

I've not fixed any of the issues raised above - I'm not sure if the have easy fixes,
and more to the point, you should not blindly use the report as a list of things to remove from portage.
You need to think about each deletion. It's easy to discover "Oops, I didn't mean that."
Greybeard
Top
pietinger
Administrator
Administrator
Posts: 6620
Joined: Tue Oct 17, 2006 5:11 pm
Location: Bavaria

  • Quote

Post by pietinger » Thu Mar 05, 2026 1:09 pm

Moved from Portage & Programming to Documentation, Tips & Tricks.
https://wiki.gentoo.org/wiki/User:Pietinger --> https://wiki.gentoo.org/wiki/User:Pieti ... _at_Gentoo
Top
Post Reply

10 posts • Page 1 of 1

Return to “Documentation, Tips & Tricks”

Jump to
  • Assistance
  • ↳   News & Announcements
  • ↳   Frequently Asked Questions
  • ↳   Installing Gentoo
  • ↳   Multimedia
  • ↳   Desktop Environments
  • ↳   Networking & Security
  • ↳   Kernel & Hardware
  • ↳   Portage & Programming
  • ↳   Gamers & Players
  • ↳   Other Things Gentoo
  • ↳   Unsupported Software
  • Discussion & Documentation
  • ↳   Documentation, Tips & Tricks
  • ↳   Gentoo Chat
  • ↳   Gentoo Forums Feedback
  • ↳   Duplicate Threads
  • International Gentoo Users
  • ↳   中文 (Chinese)
  • ↳   Dutch
  • ↳   Finnish
  • ↳   French
  • ↳   Deutsches Forum (German)
  • ↳   Diskussionsforum
  • ↳   Deutsche Dokumentation
  • ↳   Greek
  • ↳   Forum italiano (Italian)
  • ↳   Forum di discussione italiano
  • ↳   Risorse italiane (documentazione e tools)
  • ↳   Polskie forum (Polish)
  • ↳   Instalacja i sprzęt
  • ↳   Polish OTW
  • ↳   Portuguese
  • ↳   Documentação, Ferramentas e Dicas
  • ↳   Russian
  • ↳   Scandinavian
  • ↳   Spanish
  • ↳   Other Languages
  • Architectures & Platforms
  • ↳   Gentoo on ARM
  • ↳   Gentoo on PPC
  • ↳   Gentoo on Sparc
  • ↳   Gentoo on Alternative Architectures
  • ↳   Gentoo on AMD64
  • ↳   Gentoo for Mac OS X (Portage for Mac OS X)
  • Board index
  • All times are UTC
  • Delete cookies

© 2001–2026 Gentoo Foundation, Inc.

Powered by phpBB® Forum Software © phpBB Limited

Privacy Policy

 

 

magic