| View previous topic :: View next topic |
| Author |
Message |
donguss
n00b
Joined: 18 Apr 2018
Posts: 22
|
 Posted: Sun Jul 15, 2018 2:55 pm Post subject: Is it possible to make Portage not to keep package sources? |
 |
|
I'm talking about a sh*t ton of source tarballs corresponding to currently installed packages residing in /usr/portage/distfiles.
This directory is taking about half of all the used disk space on my machine. Even though I'm not actually going to do this since I have enough disk space for now, I should at least know if it's possible for Portage to remove sources immediately after the package is merged.
Is it achieveable? |
|
| Back to top |
|
 |
asturm
Developer
Joined: 05 Apr 2007
Posts: 8936
|
| Posted: Sun Jul 15, 2018 3:05 pm Post subject: |
|
|
| Run eclean regularly. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sun Jul 15, 2018 3:06 pm Post subject: |
|
|
| donguss wrote: | | Is it achieveable? |
No because it's bad netiquette. The bandwidth to re-download those distfiles is donated; Gentoo users need to be good stewards of that donation.
That said, a periodic cleaning of that directory, especially of packages you no longer have installed, can be a good thing. See eclean from app-portage/gentoolkit. A reasonable invocation would be | Code: | | eclean --time-limit=3m distfiles |
which would preserved all distfiles for currently installed packages and anything that has been installed in the last 3 months.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
donguss
n00b
Joined: 18 Apr 2018
Posts: 22
|
| Posted: Sun Jul 15, 2018 3:08 pm Post subject: |
|
|
| asturm wrote: | | Run eclean regularly. |
IIRC eclean only cleans out outdated tarballs, the ones that correspond to currently installed packages are still there. I'm looking for a way to get rid of all of them completely. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sun Jul 15, 2018 3:11 pm Post subject: |
|
|
Like I said, it's bad netiquette. But if you want to be that indiscriminate, the solution is easy. Left as an exercise for the student.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
Jaglover
Watchman
Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana
|
|
| Back to top |
|
|
donguss
n00b
Joined: 18 Apr 2018
Posts: 22
|
| Posted: Sun Jul 15, 2018 3:19 pm Post subject: |
|
|
| John R. Graham wrote: | Like I said, it's bad netiquette. But if you want to be that indiscriminate, the solution is easy. Left as an exercise for the student.
- John |
As said in the OP, i'm not planning to do that in the first place, it's theoretical question - and it was about whether portage allows such thing out of the box, like "keep-sources=false" in the configs or something. If the exercise you're talking about includes something crutchy like manually editing portage scripts or putting DISTDIR removal in the cron, then the answer to my question is "no, you can't". |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sun Jul 15, 2018 3:21 pm Post subject: |
|
|
And I already answered in my first reply.
| Jaglover wrote: | | You can share them over NFS if you have more than one Gentoo box. |
This is in fact what I do. The stationary machines mount distfiles over NFS from the master and the laptops rsync from the same master.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
Jaglover
Watchman
Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Sun Jul 15, 2018 3:36 pm Post subject: |
|
|
| donguss wrote: | | John R. Graham wrote: | Like I said, it's bad netiquette. But if you want to be that indiscriminate, the solution is easy. Left as an exercise for the student.
- John |
If the exercise you're talking about includes something crutchy like manually editing portage scripts or putting DISTDIR removal in the cron, then the answer to my question is "no, you can't". |
The simple solution "cd /usr/portage/distfiles" then "rm -r *"
Me, every month or two, I run "eclean-dist -p" to see what would be deleted and then run without the pretend flag, this will clean out about 80%-90% of the files that aren't currently installed packages.
Or you could set up a backup disk to put all the files from distfiles in it. I've been doing that for years.
Edit to add: you don't need ANYTHING in distfiles after you've installed the package ... well that is until the next rebuild of "whatever" package.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Last edited by Anon-E-moose on Sun Jul 15, 2018 3:39 pm; edited 1 time in total |
|
| Back to top |
|
|
Jaglover
Watchman
Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana
|
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21635
|
| Posted: Sun Jul 15, 2018 3:39 pm Post subject: |
|
|
If your complaint is that it is occupying half of all used disk space, I suggest enabling FEATURES=nostrip and building everything with debug symbols. That will consume substantial amounts of disk space for things that are not source files, bringing the overall percentage down considerably. 
Otherwise, the answer is that it is not particularly difficult, but it is not exposed as a configuration knob for the exact reasons that JRG explained. In practice, unless you have an extremely fast network connection to the distfile server (which you might, if it was hosted on your LAN), you're better off keeping recent sources anyway. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54239
Location: 56N 3W
|
| Posted: Sun Jul 15, 2018 3:48 pm Post subject: |
|
|
I keep all my distfiles. You never know when they will be needed again.
| Code: | $ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg-distfiles 148G 123G 19G 88% /usr/portage/distfiles |
Thats from April 2009.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sun Jul 15, 2018 3:56 pm Post subject: |
|
|
Same here, but only on my aforementioned master server. I only appear to have accumulated about half of what you have, though, in my case since 2005.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
asturm
Developer
Joined: 05 Apr 2007
Posts: 8936
|
| Posted: Sun Jul 15, 2018 4:06 pm Post subject: |
|
|
| Update frequency and arch vs. ~arch can make a big difference, even if you supposedly use the same packages. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Jul 15, 2018 7:06 pm Post subject: |
|
|
| Quote: | LANG=C df /usr/portage/distfiles/ -h
Filesystem Size Used Avail Use% Mounted on
faramir:/export/distfiles 143G 60G 76G 45% /usr/portage/distfiles
|
well, i should probably run that eclean a few more, but the problem with eclean is that it only take the host it is run on (which would clean sources i have need of for other hosts). |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21635
|
| Posted: Sun Jul 15, 2018 7:11 pm Post subject: |
|
|
| You can compensate for that with some hard link tricks. Instead of having each host download to the same directory, give each their own directory, but prefer hardlinking the file from a sibling directory over downloading it from the mirrors. Then, run eclean on each host, each time targeting that host's distfiles directory. As long as at least one host needs the file, there will be a hard link to it from that host's area. When every host declares the file obsolete, all the hard links will be removed and the file will be deleted. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sun Jul 15, 2018 7:15 pm Post subject: |
|
|
| krinn wrote: | | but the problem with eclean is that it only take the host it is run on |
Some years ago, eclean had even a lot more problems like not taking you package-specific USE-flags into account. I don't know whether this has been fixed meanwhile.
A reliable way (solving also your problems with multiple hosts) is to use trickyfetch.
It might be a bit hard to understand how it works, but after configuring portage to use trickyfetch (on every host), essentially you just call the obsolete script and then emerge -ef @world on every host (after having mounted the $DISTDIR with e.g. sshfs). Then the data you can remove remains in $DISTDIR/.obsolete |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Jul 15, 2018 7:25 pm Post subject: |
|
|
i will probably use subdir in that nfs dir with my hosts name, so everyone will have its own space in it.
but i still have plenty space, so it's low in my priorities. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9532
Location: beyond the rim
|
| Posted: Tue Jul 17, 2018 8:34 am Post subject: |
|
|
| donguss wrote: | | As said in the OP, i'm not planning to do that in the first place, it's theoretical question - and it was about whether portage allows such thing out of the box, like "keep-sources=false" in the configs or something. |
There is no config setting or commandline option if that's what you're after, because that would be utterly stupid (hint: a single source archive might be used by more than a single package or version).
If one really wants this there are many ways to achieve it however, just requires a little bit of extra work. |
|
| Back to top |
|
|
|
Portage & Programming
