Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Script to clean /usr/portage/distfiles
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
haylocki
Tux's lil' helper
Tux's lil' helper


Joined: 10 Apr 2002
Posts: 85
Location: London

PostPosted: Thu Jul 24, 2003 12:57 pm    Post subject: Script to clean /usr/portage/distfiles Reply with quote

Hi,

I have been working on a script to clean old source files from the distfiles directory.

Running the following script should leave just the files which are in the world file, in the distfiles directory.

Other files are moved into distfiles_backup. This will include source for files which were emerged with the inject option.

Old versions of files which still exist in the distfiles directory are deleted from the distfiles_backup directory.

The script works by parsing the output of 'emerge -ef world'. So it would be a good idea to run 'emerge -ef world before this script, just to make sure that all the packages needed are still available, and that emerge is working o.k.

The script uses /tmp to store its temporary files, so this directory must already exist (doesn't it exist on all Gentoo systems ?)

Oh, and of course you need to be root to run the script.

Only tested with Portage 2.0.48-r5.

O.K. heres the script, just save it as something like /usr/bin/source_clean.sh

Code:

#!/usr/bin/env bash

read_file="packages.txt"
write_file1="edit1.txt"
write_file2="edit2.txt"
write_dir="/tmp"
source_dir="/usr/portage/distfiles"
backup_dir="/usr/portage/distfiles_backup"

# Create package list
echo Creating package list
emerge -ef world  > ${write_dir}/${read_file}

# Remove un-needed text from list
cat ${write_dir}/${read_file} |
while read filename
    do
   case ${filename} in
       ">>> md5 src_uri"*)
           echo ${filename##*' '} >> ${write_dir}/${write_file1};;   
        esac
    done
# Remove duplicate lines
sort -u ${write_dir}/${write_file1} > ${write_dir}/${write_file2}
   
# Move packages to backup file
mkdir -p ${backup_dir}
mv  -f ${source_dir}/* ${backup_dir}

# Remove cvs-src directory if it exists
if [ -d  ${backup_dir}/cvs-src ]
    then
        rm -rf  ${backup_dir}/cvs-src
fi

# Move required packages back to source directory
cat ${write_dir}/${write_file2} |
while read filename
    do
         mv ${backup_dir}/${filename} ${source_dir}/
    done

# Remove old versions from backup directory
sort -u ${write_dir}/${write_file2} > ${write_dir}/${write_file1}
rm ${write_dir}/${write_file1}
cat ${write_dir}/${write_file2} |
while read filename
    do   
   filename=${filename%-*}"*"
   if [ -f ${backup_dir}/${filename} ]
       then
      rm  ${backup_dir}/${filename}
        fi
    done

# Remove temporary files
rm -f ${write_dir}/${write_file1}
rm -f ${write_dir}/${write_file2}
rm -f ${write_dir}/${read_file}

echo Distfiles directory now cleaned


Hope someone finds it helpful, (and it works :wink: )

Ian
Back to top
View user's profile Send private message
Pythonhead
Developer
Developer


Joined: 16 Dec 2002
Posts: 1801
Location: Redondo Beach, Republic of Calif.

PostPosted: Thu Jul 24, 2003 3:26 pm    Post subject: Reply with quote

I added your script to this list of portage utilities:

http://forums.gentoo.org/viewtopic.php?t=67849

There is one there called distclean.py that is similar to yours.

People may change their DISTDIR directory though, so you could add this:

Code:
source /etc/make.conf


Then you have ${DISTDIR}


Last edited by Pythonhead on Tue Mar 08, 2005 6:06 pm; edited 2 times in total
Back to top
View user's profile Send private message
pilla
Administrator
Administrator


Joined: 07 Aug 2002
Posts: 7214
Location: Pelotas, BR

PostPosted: Thu Jul 24, 2003 3:32 pm    Post subject: Reply with quote

Moving to Docs, Tips & Tricks, as it is not a support request.
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Thu Jul 24, 2003 3:48 pm    Post subject: Reply with quote

See also, Cleaning out stale distfiles
Back to top
View user's profile Send private message
Calamarain
n00b
n00b


Joined: 27 Jul 2004
Posts: 23
Location: Netherlands

PostPosted: Wed Aug 11, 2004 3:08 pm    Post subject: Reply with quote

I get this error in the end:

/usr/bin/source_clean.sh: line 50: [: too many arguments
/usr/bin/source_clean.sh: line 50: [: /usr/portage/distfiles_backup/enigmail-0.83.6.tar.gz: binary operator expected
/usr/bin/source_clean.sh: line 50: [: /usr/portage/distfiles_backup/gentoo-sources-2.4.26-r1.tar.bz2: binary operator expected
/usr/bin/source_clean.sh: line 50: [: /usr/portage/distfiles_backup/ipc-1.0.5.tar.gz: binary operator expected
/usr/bin/source_clean.sh: line 50: [: /usr/portage/distfiles_backup/m4-1.4.tar.gz: binary operator expected
/usr/bin/source_clean.sh: line 50: [: /usr/portage/distfiles_backup/tvtime-0.9.12.tar.gz?download&failedmirror=easynews.dl.sourceforge.net: binary operator expected
Distfiles directory now cleaned


Also, why does the script redownload all the files I already have? Or is that a one time effort?
Back to top
View user's profile Send private message
g4c9z
Apprentice
Apprentice


Joined: 03 Jun 2004
Posts: 178

PostPosted: Tue Dec 07, 2004 4:09 pm    Post subject: Reply with quote

I got a similar error when I pressed Ctrl+C:
./source_clean.sh: line 50: [: /usr/portage/distfiles_backup/portage-2.0.51-r2.tar.bz2: binary operator expected

And I'm also wondering why it needs to download all 290 of my installed packages.

This script is a great idea though. In fact, I believe something like this should be put in portage and run by default after doing package management. The reason is that a program should never be allowed to gradually fill up a disk more and more as time goes on. The amount of space being used should always be fixed by some user decided constant, in this case by the set of packages they choose to install.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Tue Dec 07, 2004 4:16 pm    Post subject: Reply with quote

Moved from Documentation, Tips & Tricks as this no longer seems maintained.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum