View previous topic :: View next topic |
Author |
Message |
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Mon Oct 13, 2003 10:46 pm Post subject: How can I find and remove non OSI licensed packages? |
|
|
I would like to remove all the packages on my system that aren't covered under an OSI approved license, since I want my system to be 100% OSS.
Is there any easy way to do this?
Last edited by hbmartin on Sat Mar 12, 2011 12:48 am; edited 1 time in total |
|
Back to top |
|
|
nephros Advocate
Joined: 07 Feb 2003 Posts: 2139 Location: Graz, Austria (Europe - no kangaroos.)
|
Posted: Mon Oct 13, 2003 11:57 pm Post subject: |
|
|
I don't think there is an automated way to do it, but there should be files like this:
/var/db/pkg/pkg-category/pkg-name/LICENSE
which contain the names of the licenses of all installed packages. Using some find & sed magic you should be able to make a list of all non-free packages installed.
You can find all possible licenses in /usr/portage/licenses/
hope that helps. _________________ Please put [SOLVED] in your topic if you are a moron. |
|
Back to top |
|
|
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Wed Jan 21, 2004 12:06 am Post subject: |
|
|
I don't know enough about how portage works (and I don't know Python, just Perl) but would anyone know how to hack something the selected all my installed packages and then sorted them be license?
Last edited by hbmartin on Sat Mar 12, 2011 12:48 am; edited 1 time in total |
|
Back to top |
|
|
sumin k'adra n00b
Joined: 20 Sep 2003 Posts: 65 Location: santa fe, nm
|
Posted: Wed Jan 21, 2004 7:06 am Post subject: |
|
|
in the next version of portage (2.0.50) you should be able to select which licenses you find acceptable in make.conf. Try searching the forums for the thread on it.
-sk |
|
Back to top |
|
|
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Wed Jan 21, 2004 2:32 pm Post subject: |
|
|
sumin k'adra wrote: | in the next version of portage (2.0.50) you should be able to select which licenses you find acceptable in make.conf. Try searching the forums for the thread on it.
-sk |
That still won't help me with software that's already installed.
Last edited by hbmartin on Sat Mar 12, 2011 12:49 am; edited 1 time in total |
|
Back to top |
|
|
Genone Retired Dev
Joined: 14 Mar 2003 Posts: 9558 Location: beyond the rim
|
Posted: Wed Jan 21, 2004 6:18 pm Post subject: |
|
|
sumin k'adra wrote: | in the next version of portage (2.0.50) you should be able to select which licenses you find acceptable in make.conf. Try searching the forums for the thread on it. |
Where did you get this information ? 2.0.50 has many new features, but not that one.
Ok, I've written a small script to check your installed packages for licenses you don't want, you can get it at http://gentoo.devel-net.org/portage/license-checker . It is completely unofficial atm, so don't file bugs about it, if you find something odd either report it to me directly or ignore it
Oh, I'm not going to provide any list for OSI or GNU compatible licenses as you should know what you're looking for. |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Wed Jan 21, 2004 7:11 pm Post subject: |
|
|
Edit: The latest version of the script can be found here:
http://gentoo.kicks-ass.org/scripts/licenses2kill.sh
I've written a little script that will go through your packages and will get rid of those with unwanted licenses.
Just add the licenses you're happy with in the oklicenses variable. The script will then do a pretend unmerge for those packages with unacceptable licenses. Change the script to take out the p option of emerge only when you're sure you wont break your system.
Code: | #!/bin/bash
# Program to find licenses of installed packages
# Space separated list of acceptable licenses. See /usr/portage/licenses
oklicenses="GPL-1 GPL-2 as-is"
pkgdir=/var/db/pkg
filelist=`mktemp`
for i in `find $pkgdir -name LICENSE`; do
license=`cat "$i"`
echo -n "$license " >> $filelist
echo $i | cut -f5-6 -d/ >> $filelist
done
sort $filelist >> ${filelist}.sorted
for i in $oklicenses; do
grep $i ${filelist}.sorted >> ${filelist}.good
done
diff -u ${filelist}.sorted ${filelist}.good | grep -v '^--' | grep '^-' | cut -f2- -d/ >> ${filelist}.bad
# Remove the p at your own peril! Check it with the p first just in case you missed something
for i in `cat ${filelist}.bad`; do
emerge -pC =${i}
done
# Comment this line if you want to keep the lists (/tmp/tmpXXXX)
rm -f ${filelist}*
|
Sometimes scripts get errors when cut+pasted from webpages so you can also get this version of the script here:
http://gentoo.kicks-ass.org/scripts/licensegrep-0.1.sh _________________ Search | Read | Answer | Report | Strip
Last edited by tomk on Thu Apr 01, 2004 9:58 pm; edited 3 times in total |
|
Back to top |
|
|
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Thu Jan 22, 2004 4:04 am Post subject: |
|
|
Thanks a bunch tomk
I'm having fun hacking on it.
Would you mind explaining the diff line to me?
Maybe you should submit it to be part of the gentoolkit
Last edited by hbmartin on Sat Mar 12, 2011 12:48 am; edited 1 time in total |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Thu Jan 22, 2004 1:13 pm Post subject: |
|
|
No probs, I had fun hacking it too.
When it gets to the diff you've got two lists, sorted and good. Both are in the format:
License categogy/package-version
The diff then sees what's in sorted but not in good, i.e all the bad stuff. The first grep just gets rid of the name of the file being grepped. The second grep gets the bad lines and the cut removes the license and category. I was going to leave the category, but some packages have more than one license so had to cut it somewhere else.
I'd have to tidy it up a bit before I submitted it. The main problem being that it does each emerge seperately. If I can change it so it does an emerge -pC of everything, so you get a nice list and/or an easier way of removing packages from the list. Maybe if it was reversed so you specify which licenses you don't like rather than the ones you like.
I'll play around with it a bit more _________________ Search | Read | Answer | Report | Strip |
|
Back to top |
|
|
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Thu Jan 22, 2004 3:59 pm Post subject: |
|
|
I prefer being able to specify which licenses I like that having to dig through each license.
I'm having a problem, when I set the oklicenses to
Code: | oklicenses="GPL-1 GPL-2 as-is Apache-1.1 Artistic BSD BZIP2 DIVX EDB LGPL-2.1 FDL-1.1 FLEX FTL Info-ZIP MIT X11 MOTIF MPL-1.1 OPL OpenSoftware PSF-2.2 PYTHON Sudo W3C ZLIB fontconfig fmod freedist povlegal-3.5 public-domain wxWinLL-3"
|
Things like mod_perl and xscreensaver are showing up in /tmp/tmpXX.bad
Any ideas what might be going wrong?
Besides that I haven't changed the script, except to comment out the emerge loop and the rm tmp files.
Last edited by hbmartin on Sat Mar 12, 2011 12:49 am; edited 1 time in total |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Thu Jan 22, 2004 4:35 pm Post subject: |
|
|
Edit: The latest version of the script can be found here:
http://gentoo.kicks-ass.org/scripts/licenses2kill.sh
OK found the problem, the good list needs sorting before it's diffed against the sorted list. I've also cleaned it up a bit so that it does all the emerges at once, gives you a chance to look at the bad list and will do a real unmerge if you tell it to. This version (0.2) should work ok, as before you can download it:
WARNING: If you tell it to this version will unmerge packages, some of which may be vital to your system. Please check the bad list before unmerging anything
http://gentoo.kicks-ass.org/scripts/licensegrep-0.2.sh
Code: | #!/bin/bash
# Program to find and remove packages with unwanted licenses
# Space separated list of acceptable licenses. See /usr/portage/licenses
oklicenses="GPL-1 GPL-2 as-is BSD"
pkgdir=/var/db/pkg
filelist=`mktemp`
emergeopts="-pC"
for i in `find $pkgdir -name LICENSE`; do
license=`cat "$i"`
echo -n "$license " >> $filelist
echo $i | cut -f5-6 -d/ >> $filelist
done
sort $filelist >> ${filelist}.sorted
for i in $oklicenses; do
grep $i ${filelist}.sorted >> ${filelist}.good
done
cat ${filelist}.good | sort -u >> ${filelist}.tmp && mv ${filelist}.{tmp,good}
diff -u ${filelist}.sorted ${filelist}.good | grep -v '^--' | grep '^-' | cut -c2- >> ${filelist}.bad
for i in `cat ${filelist}.bad | cut -f2- -d/` ; do
emergefiles=${emergefiles:-""}"=${i} "
done
read -n1 -p "Do you want to see the list of bad packages? (y/n) " reply
echo
if [[ $reply = 'y' ]]; then
less ${filelist}.bad
fi
echo "Pretend unmerge of packages:"
emerge $emergeopts $emergefiles
read -n1 -p "Continue with real unmerge? Saying yes will permanently remove packages (y/n) " reply2
if [[ $reply2 != 'y' ]]; then
echo && exit 0
fi
emergeopts="-C"
emerge $emergeopts $emergefiles
# Comment this line if you want to keep the lists (/tmp/tmpXXXX)
rm -f ${filelist}*
|
_________________ Search | Read | Answer | Report | Strip
Last edited by tomk on Thu Apr 01, 2004 10:00 pm; edited 2 times in total |
|
Back to top |
|
|
hbmartin Guru
Joined: 12 Sep 2003 Posts: 386 Location: Home is where the boxen are
|
Posted: Thu Jan 22, 2004 8:09 pm Post subject: |
|
|
I just unmerged a bunch of packages with 0.2, it works great!
Nice interface, it's alot easier.
Here's my code for .21
Code: |
#!/bin/bash
# Program to find and remove packages with unwanted licenses
# Space separated list of acceptable licenses. See /usr/portage/licenses
#OSI Compatible (+ as-is & public-domain)
#I couldn't find some of the OSI licenses in /usr/portage/licenses
#http://www.opensource.org/licenses/
oklicenses="as-is public-domain Academic Apache-1.1 Apple Artistic Artistic-2 BSD GPL-1 GPL-2 LGPL-2 LGPL-2.1 PLAN9 IBM MIT MPL-1.0 MPL-1.1 nethack OpenSoftware PHP PYTHON PSF-2.1.1 PSF-2.2 QPL QPL-1.0 Sleepycat SISSL-1.1 SPL ncsa-1.3 W3C wxWinLL-3 ZLIB"
#We should probably do OSI compatible + whatever Gentoo needs
#My system's
#Hopefully I'll narrow this down, since most of these licenses only cover one or two pacakges
#oklicenses="GPL-1 GPL-2 as-is Apache-1.1 Artistic BSD BZIP2 DIVX EDB LGPL-2.1 FDL-1.1 FLEX FTL Info-ZIP MIT X11 MOTIF MPL-1.1 OPL OpenSoftware PSF-2.2 PYTHON W3C ZLIB fontconfig fmod freedist povlegal-3.5 public-domain wxWinLL-3 CRACKLIB DB"
pkgdir=/var/db/pkg
filelist=`mktemp`
emergeopts="-C"
for i in `find $pkgdir -name LICENSE`; do
license=`cat "$i"`
echo -n "$license " >> $filelist
echo $i | cut -f5-6 -d/ >> $filelist
done
sort $filelist >> ${filelist}.sorted
for i in $oklicenses; do
grep $i ${filelist}.sorted >> ${filelist}.good
done
cat ${filelist}.good | sort -u >> ${filelist}.tmp && mv ${filelist}.{tmp,good}
diff -u ${filelist}.sorted ${filelist}.good | grep -v '^--' | grep '^-' | cut -c2- >> ${filelist}.bad
for i in `cat ${filelist}.bad | cut -f2- -d/` ; do
emergefiles=${emergefiles:-""}"=${i} "
done
badpackages=`wc -l ${filelist}.bad | cut -f1 -d\ `
if [ $badpackages = '0' ]; then
echo "No packages found outside of license parameters"; exit 2;
else
echo "Found ${badpackages} packages outside of license parameters";
fi
read -n1 -p "Do you want to see the list of bad packages? (y/n) " reply
echo
if [[ $reply = 'y' ]]; then
less ${filelist}.bad
fi
echo "Pretend unmerge of packages:"
emerge -p $emergeopts $emergefiles
read -n1 -p "Continue with real unmerge? Saying yes will permanently remove packages (y/n) " reply2
if [[ $reply2 != 'y' ]]; then
echo; exit 1
fi
emerge $emergeopts $emergefiles
cp ${filelist}.bad /var/log/unmerged.license
# Comment this line if you want to keep the lists (/tmp/tmpXXXX)
rm -f ${filelist}*
exit 0
|
http://home.earthlink.net/~cocoadev/licensegrep.sh
Changes:
oklicenses is OSI by default (see the comments)
emergeopts only has to be specifed once
checking to see it there are no bad packages
s/echo && exit 0/echo; exit0/
exit codes (0 = sucess, 1 = user canceled unmerge, 2 = no bad packages found)
List of unmerged pacakges copied to /var/log/unmerged.license
Can we get this into CVS somewhere? I love working with CVS (and I think we might need it swoon), so I'd be happy to get a project from sf.net
Actually, nevermind. I want to be able to blame someone's code if this messes up my system
I assume you're going to GPL this?
Last edited by hbmartin on Sat Mar 12, 2011 12:49 am; edited 1 time in total |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Fri Jan 23, 2004 4:27 pm Post subject: |
|
|
hbmartin wrote: | I assume you're going to GPL this? |
Yep GPL, otherwise it would probably delete itself _________________ Search | Read | Answer | Report | Strip |
|
Back to top |
|
|
maxcow Tux's lil' helper
Joined: 04 Jul 2003 Posts: 126
|
Posted: Sat Jan 24, 2004 1:51 am Post subject: |
|
|
maybe Vim's license, bzip2 license, Xfree license and free documentation license should also be included as default too. |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Fri Feb 20, 2004 12:17 pm Post subject: |
|
|
Me and hbmartin have been working on this script, making quite a few improvements. It's still a work in progress, but it's usable. You are now given a list of the packages which have incompatable licenses, giving you the option to keep certain packages or licenses.
We've also renamed the script to licenses2kill, you can get it here:
http://gentoo.kicks-ass.org/scripts/licenses2kill.sh
Here's the code:
Code: | #!/bin/bash
# $Id: licenses2kill.sh,v 1.6 2004/01/30 19:57:06 tom Exp $
# Program to find and remove packages with unwanted licenses
# Copyright (c) Tom Knight (tomk@forums.gentoo.org)
# Harold Martin (cocoadev@earthlink.net)
# This program is distributed under the terms of GPL version 2.
# Space separated list of acceptable licenses. See /usr/portage/licenses
#OSI Compatible (+ as-is & public-domain)
#I couldn't find some of the OSI licenses in /usr/portage/licenses
#http://www.opensource.org/licenses/
oklicenses="as-is public-domain Academic Apache-1.1 Apple Artistic Artistic-2 BSD GPL-1 GPL-2 LGPL-2 LGPL-2.1 PLAN9 IBM MIT MPL-1.0 MPL-1.1 nethack OpenSoftware PHP PYTHON PSF-2.1.1 PSF-2.2 QPL QPL-1.0 Sleepycat SISSL-1.1 SPL ncsa-1.3 W3C wxWinLL-3 ZLIB"
#We should probably do OSI compatible + whatever Gentoo needs
#My system's
#Hopefully I'll narrow this down, since most of these licenses only cover one or two pacakges
#oklicenses="GPL-1 GPL-2 as-is Apache-1.1 Artistic BSD BZIP2 DIVX EDB LGPL-2.1 FDL-1.1 FLEX FTL Info-ZIP MIT X11 MOTIF MPL-1.1 OPL OpenSoftware PSF-2.2 PYTHON W3C ZLIB fontconfig fmod freedist povlegal-3.5 public-domain wxWinLL-3 CRACKLIB DB"
cleanup ()
{
rm -f ${filelist}*
}
pkgdir=/var/db/pkg
filelist=`mktemp`
emergeopts="-C"
logfile=/var/log/unmerged.license
echo "Searching licenses"
for i in `find $pkgdir -name LICENSE`; do
license=`cat "$i"`
echo -n "$license " >> $filelist
echo $i | cut -f5-6 -d/ >> $filelist
done
sort $filelist >> ${filelist}.sorted
for i in $oklicenses; do
grep $i ${filelist}.sorted >> ${filelist}.good
done
cat ${filelist}.good | sort -u >> ${filelist}.tmp && mv ${filelist}.{tmp,good}
diff -u ${filelist}.sorted ${filelist}.good | grep -v '^--' | grep '^-' | cut -c2- >> ${filelist}.bad
badpackages=`wc -l ${filelist}.bad | awk '{print $1}'`
if [[ $badpackages = '0' ]]; then
echo "No packages found outside of license parameters"; cleanup; exit 2;
else
echo "Found ${badpackages} packages outside of license parameters";
fi
read -n1 -p "Do you want to see the list of bad packages? (y/n) " reply
echo
if [[ $reply = 'y' ]]; then
while :; do
nl -s ") " ${filelist}.bad
read -p "Enter the number of a package you wish to keep, or d to continue " keep_package_no
if [[ ${keep_package_no} != 'd' ]]; then
if (( ${keep_package_no} > 0 )) && (( ${keep_package_no} <= ${badpackages} )); then
keep_package=`head -n ${keep_package_no} ${filelist}.bad | tail -n1 | cut -f2- -d/`
keep_license=`head -n ${keep_package_no} ${filelist}.bad | tail -n1 | cut -f1 -d\ `
echo "Keeping ${keep_package} package"
badleft=`wc -l ${filelist}.bad | awk '{print $1}'`
declare -i headn
declare -i tailn
headn=${keep_package_no}-1
tailn=${badleft}-${keep_package_no}
head -n ${headn} ${filelist}.bad > ${filelist}.tmp && \
tail -n ${tailn} ${filelist}.bad >> ${filelist}.tmp && \
mv ${filelist}.{tmp,bad}
# Maybe this should add the license to oklicenses for future use?
read -n1 -p "Keep all packages with ${keep_license} license? " keep_all
echo
if [[ $keep_all = 'y' ]]; then
echo "Keeping all ${keep_license} packages"
grep -v ^${keep_license} ${filelist}.bad > ${filelist}.tmp && \
mv ${filelist}.{tmp,bad}
fi
else
echo "Number out of range"
fi
else
break
fi
done
fi
for i in `cat ${filelist}.bad | cut -f2- -d/` ; do
emergefiles=${emergefiles:-""}"=${i} "
done
read -n1 -p "Continue with unmerge? Saying yes will permanently remove packages (y/n) " reply2
if [[ $reply2 != 'y' ]]; then
echo; cleanup; exit 1
fi
emerge $emergeopts $emergefiles
echo "Packages removed by licenses2kill" >> $logfile
echo `date +%c` >> $logfile
echo >> $logfile
cat ${filelist}.bad >> $logfile
echo >> $logfile
cleanup
exit 0
|
_________________ Search | Read | Answer | Report | Strip |
|
Back to top |
|
|
HolyCoitus n00b
Joined: 16 Jun 2003 Posts: 21 Location: Fort Wayne, Indiana
|
Posted: Tue Jul 20, 2004 6:39 pm Post subject: |
|
|
Very nice script so far. I changed a few things to my likings, so I might as well post them back to the forums. Going to do code snippets so that the thread doesn't get too confusing with pasted code.
http://mywebpages.comcast.net/HolyCoitus/licenses2kill.sh
Changes:
Added output so you can have flat text files to view afterwards if you so choose.
Added a dump of the licenses that are not on the list, so you can get an idea much easier of what they are
Added a check at the end to see if you are root or in the portage group before attempting to unmerge anything
Code: | # Blank for no logging, otherwise it will output to the file you specify
licenselog=""
packagelog="" |
Code: | # separate the licenses from the packages that they associate with, and then save the variable for later use
badlicenses=`cat "${filelist}.bad" | cut -f1 -d\ | sort -u | tee ${filelist}.liccrop | wc -l`
echo "There are $badlicenses licenses outside of parameters"
read -n1 -p "Do you want to view the list of licenses that are being rejected? (y/n)" licrep
echo
if [[ $licrep = 'y' ]];
then
if [[ $licenselog != "" ]]
then
nl -s ") " ${filelist}.liccrop | tee $licenselog
else
nl -s ") " ${filelist}.liccrop
fi
fi |
Code: | if [[ $packagelog != "" ]]
then
nl -s ") " ${filelist}.bad | tee $packagelog
else
nl -s ") " ${filelist}.bad
fi |
Code: | # Check to see if the user is root or in the portage group before you try to unmerge things.
if [[ `/usr/bin/whoami` = 'root' || `groups | grep -c "portage"` -ge 1 ]]
then |
_________________ Celebrate this chance to be alive and breathing! |
|
Back to top |
|
|
tomk Bodhisattva
Joined: 23 Sep 2003 Posts: 7221 Location: Sat in front of my computer
|
Posted: Fri Oct 08, 2004 5:56 pm Post subject: |
|
|
Seems I missed this when you first posted. I like the changes you've made, especially dumping the list of licenses. _________________ Search | Read | Answer | Report | Strip |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20112
|
Posted: Fri Oct 08, 2004 6:01 pm Post subject: |
|
|
Moved from Other Things Gentoo. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
|