| View previous topic :: View next topic |
| Author |
Message |
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
 Posted: Sun Jan 09, 2005 10:20 pm Post subject: What is the Syntax for rsync_excludes? |
 |
|
I'm experimenting with RSYNC_EXCLUDEFROM in Portage. my objective is to exclude some packages that i do not want to be upgraded from showing up in the portage tree and being updated in a poorly though-out emerge world.
i've looked at the Gentoo Installation Handbook and the MAN RSYNC pages, and i've searched the forums, but i can't seem to find a solution that works to keep a newer version of "linux26-headers" from finding its way into my portage tree.
what is the correct format for the contents of the /etc/portage/rsync_excludes file? i'm hoping that someone can clarify this, as it takes a long time to test every possible syntax structure when you have to sync a new portage tree to perform every test.
TIA.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
 |
Halcy0n
Developer
Joined: 17 Sep 2003
Posts: 1682
Location: Freehold, NJ
|
| Posted: Sun Jan 09, 2005 11:11 pm Post subject: |
|
|
Why don't you just use /etc/portage/package.mask and mask out the versions you don't want? Also: | man rsync wrote: | --exclude=PATTERN
This option allows you to selectively exclude certain files from
the list of files to be transferred. This is most useful in com-
bination with a recursive transfer.
You may use as many --exclude options on the command line as you
like to build up the list of files to exclude.
See the EXCLUDE PATTERNS section for detailed information on
this option.
--exclude-from=FILE
This option is similar to the --exclude option, but instead it
adds all exclude patterns listed in the file FILE to the exclude
list. Blank lines in FILE and lines starting with ';' or '#'
are ignored. If FILE is - the list will be read from standard
input. |
Look in man rsync for how the patterns work.
_________________
Mark Loeser
http://www.halcy0n.com |
|
| Back to top |
|
|
Ox-
Guru
Joined: 19 Jun 2003
Posts: 305
|
| Posted: Mon Jan 10, 2005 1:18 am Post subject: |
|
|
| Halcy0n wrote: | | Why don't you just use /etc/portage/package.mask and mask out the versions you don't want? |
Why would masking packages that aren't even emerge'd help? The point of rsync_excludes is to greatly speed up emerge sync times by preventing huge unused chunks of the portage tree from being downloaded in the first place. It's also friendlier on the portage mirrors.
This was my /etc/portage/rsync_exludes for a production server at one time (haven't tested it in a while so some modifications might be needed): | Code: | # exclude all games
app-games/
# exclude all X11, KDE, and Gnome packages
x11-*/
gnome-*/
kde-*/
xfce-*/
# exclude multimedia
media-*/
# exclude dev-lisp, dev-utils (except for dev-utils/cvs)
dev-lisp/
dev-utils/*
+ dev-utils/cvs/
# exclude edit applications, except for console mode vi
app-emacs/
app-xemacs/
app-editors/*
+ app-editors/vi/
app-office/
app-misc/
app-i18n/
# I've seen someone use ** in front as another way to ignore metadata/categories
# Not sure if it works
**app-i18n
**app-antivirus
**app-benchmarks
**app-laptop
**app-pda
**app-xemacs
**dev-ada
**dev-embedded
|
|
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 2:05 am Post subject: |
|
|
| Halcy0n wrote: | | Why don't you just use /etc/portage/package.mask and mask out the versions you don't want? |
the reason that i'm not using package.mask is because it doesn't work the way i need it to:
say that i have the following toolkit components emerged onto my system, and i do not want them updated so that my toolkit will not become corrupted:
| Code: | linux26-headers-2.6.8.1-r1
glibc-2.3.4.20041102
binutils-2.15.92.0.2-r2
gcc-3.4.3-r1 |
how would you mask the packages so that they are never re-emerged during an emerge -uD world?
take linux26-headers for example.
scenario 1: if you put this in package.mask:
| Code: | | =sys-kernel/linux26-headers-2.6.8.1-r1 |
you get this output:
| Code: | | [ebuild U] sys-kernel/linux26-headers-2.6.8.1-r2 [2.6.8.1-r1] |
that doesn't help.
scenario 2: if you put this in package.mask:
| Code: | | >=sys-kernel/linux26-headers-2.6.8.1-r1 |
you get this output:
| Code: | | [ebuild UD] sys-kernel/linux26-headers-2.6.8.1 [2.6.8.1-r1] |
that doesn't help.
scenario 3: if you put this in package.mask:
| Code: | | <sys-kernel/linux26-headers-2.6.8.1-r1 |
you get this output:
| Code: | | [ebuild U] sys-kernel/linux26-headers-2.6.8.1-r2 [2.6.8.1-r1] |
that doesn't help.
scenario 4: if you put this in package.mask:
| Code: | | =sys-kernel/linux26-headers-2.6.8.1 |
you get this output:
| Code: | | [ebuild U] sys-kernel/linux26-headers-2.6.8.1-r2 [2.6.8.1-r1] |
that doesn't help.
scenario 5: if you put this in package.mask:
| Code: | | >=sys-kernel/linux26-headers-2.6.8.1 |
you get this output:
| Code: | | [ebuild UD] sys-kernel/linux26-headers-2.6.7-r4 [2.6.8.1-r1] |
that doesn't help.
scenario 6: if you put this in package.mask:
| Code: | | <sys-kernel/linux26-headers-2.6.8.1 |
you get this output:
| Code: | | [ebuild U] sys-kernel/linux26-headers-2.6.8.1-r2 [2.6.8.1-r1] |
that doesn't help.
scenario 7: if you put this in package.mask:
| Code: | | =sys-kernel/linux26-headers-2.6.8.1-r2 |
you get this output:
| Code: | | [ebuild UD] sys-kernel/linux26-headers-2.6.8.1 [2.6.8.1-r1] |
that doesn't help.
scenario 8: if you put this in package.mask:
| Code: | | >=sys-kernel/linux26-headers-2.6.8.1-r2 |
you get this output:
| Code: | | [ebuild UD] sys-kernel/linux26-headers-2.6.8.1 [2.6.8.1-r1] |
that doesn't help.
scenario 9: if you put this in package.mask:
| Code: | | <sys-kernel/linux26-headers-2.6.8.1-r2 |
you get this output:
| Code: | | [ebuild U] sys-kernel/linux26-headers-2.6.8.1-r2 [2.6.8.1-r1] |
that doesn't help either.
so unless i'm really missing something (i hope that i am), package.mask doesn't do anything to prevent a newer version of linux26-headers from being emerged onto my system.
so it looks like rsync_excludes is the only way to solve the problem.
| Halcy0n wrote: | | Quote: | --exclude-from=FILE
This option is similar to the --exclude option, but instead it
adds all exclude patterns listed in the file FILE to the exclude
list. Blank lines in FILE and lines starting with ';' or '#'
are ignored. If FILE is - the list will be read from standard
input. |
Look in man rsync for how the patterns work. |
i guess you missed the part where i mentioned that i had read the man pages. the problem is that the section you quoted refers to passing command line parameters to portage, not to populating a data file with excludes. fwiw, the command line syntax does not work when placed in a data file. i tried it. that is why i started the post to ask for an example.
thanks for your help.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks
Last edited by Bob P on Mon Jan 10, 2005 2:23 am; edited 1 time in total |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 2:11 am Post subject: |
|
|
| Ox- wrote: | | This was my /etc/portage/rsync_exludes for a production server at one time (haven't tested it in a while so some modifications might be needed)... |
thanks. that's exactly what i needed to know. 
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Halcy0n
Developer
Joined: 17 Sep 2003
Posts: 1682
Location: Freehold, NJ
|
| Posted: Mon Jan 10, 2005 2:20 am Post subject: |
|
|
| Bob P wrote: | | Code: | linux26-headers-2.6.8.1-r1
glibc-2.3.4.20041102
binutils-2.15.92.0.2-r2
gcc-3.4.3-r1 |
how would you mask the packages so that they are never re-emerged during an emerge -uD world?
take linux26-headers for example.
.......
so unless i'm really missing something (i hope that i am), package.mask doesn't do anything to prevent a newer version of linux26-headers from being emerged onto my system.
so it looks like rsync_excludes is the only way to solve the problem. |
So, you don't want anything newer than linux26-headers-2.6.8.1-r1. So put: | Code: | | >linux26-headers-2.6.8.1-r1 |
in package.mask and move the ebuild to your overlay so it doesn't get deleted when you emerge sync if the ebuild gets removed. Problem solved.
The reason none of yours worked is because you don't have the ebuild anymore, so you are going to continue to have a problem. You can get it from the cvsweb interface on http://www.gentoo.org
_________________
Mark Loeser
http://www.halcy0n.com |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 2:38 am Post subject: |
|
|
Thanks for your help, Mark.
I'm reading the GIH, Section C, "Working with Portage." In section 5 there is a section entitled, "Diverting from the Official Tree. Accorrding to that section of the manual, you're supposed to use rsync to exclude unwanted packages/categories, and use an overlay directory to add ebuilds that aren't available via the normal portage tree. So I have to sheepishly admit that I don't understand the approach that you had recommended.
I was thinking about using a statement like | Code: | | sys-kernel/linux26-headers*/ |
in the /etc/portage/rsync_excludes file.
i don't understand what you're recommending to do with package.mask and the overlay directory.
thanks.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Ox-
Guru
Joined: 19 Jun 2003
Posts: 305
|
| Posted: Mon Jan 10, 2005 3:44 am Post subject: |
|
|
Ah, ok, I didn't realize you actually were trying to use rsync_excludes to act like package masking.
In that case, I can verify that Halcyon is correct. I have a single windows program I run once in a blue moon and I could only get it to work with one specific version of wine.
So, I copied that wine ebuild to portage overlay: | Code: | # mkdir -p /usr/local/portage/app-emulation/wine
# cp -p /usr/portage/app-emulation/wine/wine-20030618-r1.ebuild /usr/local/portage/app-emulation/wine
# ebuild /usr/local/portage/app-emulation/wine-20030618-r1.ebuild digest |
Then, in /etc/portage/package.mask I put: | Code: | | >app-emulation/wine-20030618-r1 |
I regularly use "emerge -uDv world" and wine never shows up on the list.
This should be a LOT easier than trying to exclude newer versions from rsync. |
|
| Back to top |
|
|
Gentree
Watchman
Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe
|
| Posted: Mon Jan 10, 2005 1:10 pm Post subject: |
|
|
I gave up last time I looked at this because the man page was so lacking in detail as to be useless, so thanks for the examples.
I tried a small excludes to see:
| Code: | # exclude edit applications, except for console mode vi
app-emacs/
app-xemacs/
kde-misc/
games-*/
+games-simulation/
# exclude edit applications
app-emacs/
app-xemacs/
kde-base/kdeedu
kde-base/games
kde-base/kdepim
kde-base/kdesdk
|
however, I just started emerge sync and it reported 104,200 files and I saw all the games stuff I thought I had excluded
It appears not to have affected anything.
(Yes I did uncomment the RSYNC_EXCLUDEFROM line in /etc/make.conf)
Any ideas?
TIA 
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 5:33 pm Post subject: |
|
|
| Halcy0n wrote: | So, you don't want anything newer than linux26-headers-2.6.8.1-r1. So put: | Code: | | >linux26-headers-2.6.8.1-r1 |
in package.mask and move the ebuild to your overlay so it doesn't get deleted when you emerge sync if the ebuild gets removed. Problem solved.
The reason none of yours worked is because you don't have the ebuild anymore, so you are going to continue to have a problem. You can get it from the cvsweb interface on http://www.gentoo.org |
i guess that i need to change the way i'm thinking about this. i'm stuck in the rut of doing what i'm told in the GIH. Accorrding to the GIH you're supposed to use rsync to exclude unwanted packages/categories, and use an overlay directory to add ebuilds that aren't available via the normal portage tree.
as i understand it, your methodology treats the desired ebuild as if it were an "unofficial" or 3rd party ebuild by placing it in the overlay directory. by putting it there, it is protected from being erased by a future ebuild that would supercede it. by adding the package mask statement, you're telling portage not to display any of the ebuilds that would supercede it. that seems simple enough, now that i understand the logic.
from a practical standpoint, how does this really differ from leaving the ebuild in portage, and adding its branch on the portage tree to the rsync_excludes file? conceptually, it would seem just as simple (actually even simpler) to add a statement like "sys-kernel/linux26-headers*/" to rsync_excludes and leave the ebuild in the portage tree.
does this amount to two ways to skin a cat, or am i missing something?
finally, what if anything would i gain by putting the description of my ebuild in package.provided?
thanks.
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Gentree
Watchman
Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe
|
| Posted: Mon Jan 10, 2005 6:34 pm Post subject: |
|
|
| Quote: | | om a practical standpoint, how does this really differ from leaving the ebuild in portage, and adding its branch on the portage tree to the rsync_excludes file? |
rsync synchonises your directory contents against the server it is pointed at . That includes _deleteing_ anything on yours that aint on the server.
rsync_excludes will prevent fetching listed directories but afaik it wont protect what you have, hence the need for overlay in this case.
I think Ox suggested using overlay because if you want to stick with an older version it eventually get removed from the mirrors at which point it will become "unofficial" and disappear.
HTH
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 7:39 pm Post subject: |
|
|
| Gentree wrote: | | I think Ox suggested using overlay because if you want to stick with an older version it eventually get removed from the mirrors at which point it will become "unofficial" and disappear. |
that does help -- thanks!
now can anyone shed light on what Portage does with the entries in the package.provided file?
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 7:48 pm Post subject: |
|
|
| Gentree wrote: | I gave up last time I looked at this because the man page was so lacking in detail as to be useless, so thanks for the examples.
I tried a small excludes to see:
| Code: | # exclude edit applications, except for console mode vi
app-emacs/
app-xemacs/
kde-misc/
games-*/
+games-simulation/
# exclude edit applications
app-emacs/
app-xemacs/
kde-base/kdeedu
kde-base/games
kde-base/kdepim
kde-base/kdesdk
|
however, I just started emerge sync and it reported 104,200 files and I saw all the games stuff I thought I had excluded
It appears not to have affected anything.
(Yes I did uncomment the RSYNC_EXCLUDEFROM line in /etc/make.conf)
Any ideas?
TIA |
i just tried doing an "excludes" to keep things like KDE and games from being sync'd onto a PC that i will be using as a router. i added the following lines to my /etc/portage/rsync_excludes file:
| Code: | # exclude games
games-*/
#exclude KDE
kde-*/ |
to perform an acid-test of the system, after editing the excludes file i deleted my entire portage tree and then emerge --sync'd it. during the sync my file count dropped appropriately. when i resync'd my portage tree, those directories were indeed gone.
maybe your system is working, and you need to delete the directories that you don't want to be sync'd to notice the difference?
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 8:39 pm Post subject: |
|
|
| Ox- wrote: | So, I copied that wine ebuild to portage overlay: | Code: | # mkdir -p /usr/local/portage/app-emulation/wine
# cp -p /usr/portage/app-emulation/wine/wine-20030618-r1.ebuild /usr/local/portage/app-emulation/wine
# ebuild /usr/local/portage/app-emulation/wine-20030618-r1.ebuild digest |
|
I've tried masking four packages. two of them have worked perfectly, but two of them are giving strange errors when its time to rebuild the digest, so i'm hoping that someone can comment on this.
for example, my package.mask file contains the following entries:
| Code: | >sys-kernel/linux26-headers-2.6.8.1-r2
>sys-libs/glibc-2.3.4.20041102
>sys-devel/binutils-2.15.92.0.2-r2
>sys-devel/gcc-3.4.3-r1
|
I've followed Ox-'s directions in order to mask the linux26-headers and binutils ebuilds, and everything worked exactly as i had expected. the digests were properly built and those two ebuilds have been successfully masked from portage.
i had a harder time with two of the ebuilds though: glibc and gcc.
when attempting to package.mask glibc and gcc, i ran into errors while building the digests. i'm hoping that someone can look at the results and offer a comment about what could be wrong. my only idea at this point is that the ebuilds are broken.
i've performed the following steps, which result in the following errors:
for glibc:
| Code: | # mkdir -p /usr/local/portage/sys-libs/glibc
# cp -p /usr/portage/sys-libs/glibc/glibc-2.3.4.20041102.ebuild /usr/local/portage/sys-libs/glibc
# ebuild /usr/local/portage/sys-libs/glibc/glibc-2.3.4.20041102.ebuild digest
>>> Generating digest file ...
<<< glibc-2.3.3.tar.bz2
<<< glibc-manpages-2.3.4.tar.bz2
<<< glibc-infopages-2.3.4.tar.bz2
<<< glibc-hppa-patches-2004-09-30.tar.gz
!!! We have a source URI, but no file...
!!! File: /usr/portage/distfiles/glibc-hppa-patches-2004-09-30.tar.gz
|
for gcc:
| Code: | # mkdir -p /usr/local/portage/sys-devel/gcc
# cp -p /usr/portage/sys-devel/gcc/gcc-3.4.3-r1.ebuild /usr/local/portage/sys-devel/gcc
# ebuild /usr/local/portage/sys-devel/gcc/gcc-3.4.3-r1.ebuild digest
>>> Generating digest file ...
<<< gcc-3.4.3.tar.bz2
<<< gcc-3.4.3-branch-update-20041125.patch.bz2
<<< protector-3.4.3-0.tar.bz2
<<< gcc-3.4.3-manpages.tar.bz2
<<< gcc-3.4.3-patches-1.1.tar.bz2
<<< gcc-3.4.3-piepatches-v8.7.7.tar.bz2
<<< bounds-checking-gcc-3.4.2-1.00.patch.bz2
!!! We have a source URI, but no file...
!!! File: /usr/portage/distfiles/bounds-checking-gcc-3.4.2-1.00.patch.bz2
|
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks
Last edited by Bob P on Mon Jan 10, 2005 11:21 pm; edited 1 time in total |
|
| Back to top |
|
|
Gentree
Watchman
Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe
|
| Posted: Mon Jan 10, 2005 8:49 pm Post subject: |
|
|
didnot work here, I deleted most of those branches I indicated and got a file count of 104,177 , almost the same as earlier.
There are at least 200 files in app-emacs alone and I saw all that coming back in.
I've got more important issues to sort out so I'll live with it.
Thanks.
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3355
Location: Jackass! Development Labs
|
| Posted: Mon Jan 10, 2005 11:23 pm Post subject: |
|
|
i've successfully completed the "ebuild <path-to-ebuild> digest" procedure for 2 of 4 masked ebuilds. on the other two, errors were generated in preparing digests. the error encountered was:
| Code: | | !!! We have a source URI, but no file... |
this appears to represent a case where some of the files in the ebuild appear to be missing. if anyone has an idea why i'm getting these strange error messages, please let me know.
i mistakenly posted the results as a correction to my most recent post (above). details can be found in the above post. thanks!
_________________
.
Stage 1/3 | Jackass! | Rockhopper! | Thanks | Google Sucks |
|
| Back to top |
|
|
|
Portage & Programming
