Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Central Gentoo Mirror for your Internal Network
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Russel-Athletic
n00b
n00b


Joined: 07 Jan 2005
Posts: 47

PostPosted: Tue Mar 22, 2005 9:13 pm    Post subject: Reply with quote

I have one question.
I want the solution with nfs share (only 2 clients) but i don't know which direcotries i have to share. i want to do a emerge sync only on the server maschine and emerge -u world manually.
Ok there is /usr/portage and /usr/portage/distfiles but something else? Something in /var ?
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Tue Mar 22, 2005 9:45 pm    Post subject: Reply with quote

You see. That is why we have forums. I haven't read that particular man page in a long time. That'll solve part of the problem. Satisfactorily (spelling ...) for now. And probably for a long time to come.

Next question, we maintain some of our own ebuilds, some of these are queued for inclusion or we're pushing to get them in, but some of these will never get included. We have a seperate rsync section for this called gentoo-local-portage to which we can manually rsync. Since we are running the sync's out of cron'ed scripts I suppose I can just add an additional rsync command to those scripts, but it would be nice if portage has some way of doing this for me, ie, if I can ask it to sync /usr/local/portage against rsync://some.server/gentoo-local-portage whenever emerge sync gets called :). Any ideas? And this time I promise that I have completely read and understood man 5 make.conf and has not seen such an option.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
yetano
n00b
n00b


Joined: 02 Mar 2005
Posts: 2

PostPosted: Wed Mar 23, 2005 12:28 am    Post subject: Reply with quote

Russel-Athletic wrote:
I have one question.
I want the solution with nfs share (only 2 clients) but i don't know which direcotries i have to share. i want to do a emerge sync only on the server maschine and emerge -u world manually.
Ok there is /usr/portage and /usr/portage/distfiles but something else? Something in /var ?

AFAIK sharing /usr/portage (including all subdirectories) should be fine. With the default setup portage uses /var only for hostspecific (e.g. /var/lib/portage/world) and temporary files (e.g. /var/tmp/portage/*).

@jkroon

My first guess was moving /usr/local/portage to a subdirectory of /usr/portage and then add this to PORTDIR_OVERLAY, but after a small test I had to find out that PORTDIR_OVERLAY isn't working the way make.conf(5) implies.

make.conf(5) wrote:
Defines the directories in which user made ebuilds may be stored and not overwritten when 'emerge --sync' is run. [...]

Code:
# /etc/make.conf
PORTDIR="/var/portage/sys"
PORTDIR_OVERLAY="/var/portage/sys/dummy"


During 'emerge sync' /var/portage/sys/dummy is deleted with all files in it and at the end emerge complains about PORTDIR_OVERLAY not being a valid path (anymore). Well, at least none of the files has been overwritten :lol:.

Nevertheless, /usr/local/portage could still be moved to a subdirectory of /usr/portage, but should be added to the file pointed to by RSYNC_EXCLUDEFROM (actually the name of the subdirectory, not the full path and don't use '/local' as this is excluded in any case). This way 'emerge sync' on the clients will include your own ebuilds, but won't touch them on the server. No need for a 'gentoo-local-portage', though the solution you already mentioned might be the better one.

Basically, the problem is that emerge can sync with one server/module only, unless SYNC (and PORTDIR) is changed every time.
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Wed Mar 23, 2005 5:35 am    Post subject: Reply with quote

I think I'll just go with the additional rsync command in our nightly scripts - it is imho the cleaner solution. Also make it easier to distinguish which is custom ebuilds and which are portage ebuilds. It doesn't solves the problem for all but a small number of desktops, which isn't that big a problem - we can probably just write a small script that does both emerge sync and rsync -rav --delete rsync://out.rsync.server/gentoo-local-portage /usr/local/portage in one go.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Mon Mar 28, 2005 3:18 pm    Post subject: Reply with quote

Ok, torpage 0.2.0 is available for testing. It still lacks ACLs but who cares? For most of us this is simply a caching solution and EXCLUDEFROM can be used to exclude games-* (which is what I wanted to have ACLs for).

The new version instructs the server to fetch a specific file for a specific package and version, the server then manually parses the appropriate digest files to check whether these files are actually in those packages, and then does some nifty parsing and sourcing of files to determine the upstream mirrors that portage would have fetched from and then manually fetches and does the checksumming. I do this by hand since portage cannot fetch a particular file for me at this point in time - nor is the emerge command available on non-gentoo servers (and yes, I do have the need that torpage run on non-gentoo systems on the server side).

Additionally torpage doesn't take over your fetching entirely any more, it now actually specifies a seperate torpage:// protocol, so you can have multiple upstream torpage servers (ok, that is just leetness factor and doesn't imho have much real use) along with other standard ftp:// and http:// servers and torpage won't interfere with these. This allows for fallback should torpage fail for some reason or another. I've even managed to configure it to use different upstream torpage servers depending on the network I'm connected to, or to silently fail if I'm not connected to one of these networks.

Anyway, please take a look and let me know of any shortcommings (both in my code and in documentation).

URI: http://www.kroon.co.za/torpage.php

Thanks for any and all feedback.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
mndar
n00b
n00b


Joined: 04 Apr 2005
Posts: 49

PostPosted: Tue Apr 05, 2005 4:50 am    Post subject: How do I get back my original emerge cache ? Reply with quote

HI,
Is there any way I could get back my original emerge cache/ portage tree, after doing emerge --sync
Sorry, if this question is too silly to be replied to, by all the GURUs out there. I recently installed Gentoo from the universal CD (install-x86-universal-2005.0.iso) and then installed KDE, GNOME, etc from the package CD (packages-x86-2005.0.iso) using emerge --usepkg <package-name> This installed KDE 3.3.2.
Well, then I did emerge --sync. Now I can't install any kde-3.3.2 package since all the entries in the emerge cache/portage tree point to kde-3.4. Thus I can't use the package CD. I want to be able to install packages from the Package CD which has got kde-3.3.2. So, is there a way by which I could get back my original emerge cache or portage tree. I was thinking emerge --regen. Would that do it ?
Also I am not very sure whats the difference (if any) between emerge cache and portage tree.
_________________
Gentoo on Freerunner 2014-02-25 http://mndar.phpnet.us/freerunner/gentoo/
GStreamer XOverlay and VideoOverlay http://mndar.phpnet.us/gstreamer
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Tue Apr 05, 2005 6:29 am    Post subject: Reply with quote

You presumably got your orriginal portage tree from a tarball on the CD? I suspect the following sequence of commands will solve you problem in that case:

Code:
# mount /mnt/cdrom
# mv /usr/portage/distfiles /tmp/distfiles
# rm -rf /usr/portage/distfiles
# tar xjvpf /mnt/cdrom/path/to/that/tarball/which/I/can/never/remember # check the install manual for the command
# rm -rf /usr/portage/distfiles # just in case
# mv /tmp/distfiles /usr/portage/distfiles


Something along those lines might work - and if not, emerge --sync will get you back where you are now :(.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
mndar
n00b
n00b


Joined: 04 Apr 2005
Posts: 49

PostPosted: Tue Apr 05, 2005 8:35 am    Post subject: Reply with quote

Thanks jkroon, it worked!!
Following were the sequence of commands I had to issue

Code:
mv /usr/portage/distfiles /tmp/distfiles
mount -o loop /downloads/gentoo/install-x86-universal-2005.0.iso /mnt/iso 
tar -xvjf /mnt/iso/snapshots/portage-20050303.tar.bz2 -C /usr
mv /tmp/distfiles /usr/portage/distfiles

_________________
Gentoo on Freerunner 2014-02-25 http://mndar.phpnet.us/freerunner/gentoo/
GStreamer XOverlay and VideoOverlay http://mndar.phpnet.us/gstreamer
Back to top
View user's profile Send private message
yaneurabeya
Veteran
Veteran


Joined: 13 May 2004
Posts: 1754
Location: Seattle

PostPosted: Tue Jun 07, 2005 1:42 am    Post subject: Reply with quote

If you have several machines as well with different configurations, and several flagship machines that do compiling, etc... then you might want to consider something like this...

Server
/ \
/ \
/ \
PC(0) [....] PC(n)

Where PC(0) through PC(n) have NFS access and are trusted for authentication. Given that each and every machine can mount and access the distfiles NFS share, you can setup a simple cron command where the trusted machines will 'sync' with the server by running cp /usr/portage/distfiles/*. Then you can run rsync to your heart's content. Why not just use NFS? Because NFS requires an additional service for all the non-flagship machines and rsync doesn't require any additional kernel modules.
Back to top
View user's profile Send private message
jasperbg
n00b
n00b


Joined: 02 Mar 2005
Posts: 62
Location: Christchurch, New Zealand

PostPosted: Tue Jun 07, 2005 5:51 am    Post subject: Re: HOWTO: Central Gentoo Mirror for your Internal Network Reply with quote

Grimthorn wrote:
Portage uses two methods to keep an updated Portage tree and retrieve current Gentoo packages (distfiles). Rsync (rsync) is used for the tree and wget is used for the packages. I’m not aware of the motivations behind these protocol choices but they work very well.


Great article, but wget is not a protocol.


Last edited by jasperbg on Tue Jun 07, 2005 6:00 am; edited 1 time in total
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Tue Jun 07, 2005 5:55 am    Post subject: Reply with quote

You really don't want to be doing any kind of NFS unless your _entire_ network can be trusted. It is simply too insecure. Take for example a university (Yes, I'm a sysad at a university) where I'll need to export /usr/portage rw to the entire campus (well, a large part thereof anyway) to make your trick work. Not something I'm willing to do since this means some cracker can go and put a new baselayout along with a trojanned archive on the server and everybody would install it.

Nope, not a good idea imho, plus there is locking issues even if you do trust the entire network.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
yaneurabeya
Veteran
Veteran


Joined: 13 May 2004
Posts: 1754
Location: Seattle

PostPosted: Thu Jun 09, 2005 2:49 am    Post subject: Reply with quote

Yes, and that's why something should be done for authentication dealing with trusted keys and so forth like kerberos auth.
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Thu Jun 09, 2005 5:41 am    Post subject: Reply with quote

yaneurabeya wrote:
Yes, and that's why something should be done for authentication dealing with trusted keys and so forth like kerberos auth.


Ok, we're going off topic, but OpenAFS seems to be the only viable alternative at this point in time, and for heavens sake, don't use the version in portage. Anyhow, kerberos itself has some severe flaws in as well with regards to the way it works making replay attacks possible. But much better than NFS. NFSv4 (kerborised NFS) might also be an alternative if it should get stable at some point.

Anyway, with regards to a central repository a read-only NFS export of /usr/portage (And $DISTDIR if different from the default /usr/portage/distfiles) and then mounting that and using something like torpage is about the best solution I've managed to come up with. Using it both at home and at work and it's like a charm, it just works. Well, at work I'm only using it as a on-demand mirror (ie, download from outside once, download to servers/workstations from there). At home I've got a bit of a mixture atm (need to get everything back to ro NFS).
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
matt.matolcsi
n00b
n00b


Joined: 03 Apr 2005
Posts: 5

PostPosted: Thu Jun 23, 2005 7:11 pm    Post subject: NFS mounting /usr/portage ro Reply with quote

First off, thanks to everyone for the well-written, thoughtful, and useful comments.

Here at work we've got something of a similar situation: five recycled Ultra 5's do most of the dirty server work, with one of them being the designated binary package build machine and portage sync'er. Because they are all very similar hardware (only processor speed and RAM differ), they all use the same make.conf and general filesystem layout. Actually, I've got a neat rsync setup that lets the other machines pull this and some other files automatically from the server.
At first, I setup the machines to use binary package downloading using emerge -g, but this had some drawbacks:

1. It's slow: every time I wanted to emerge a package, some 'metadata pickle' had to be downloaded and parsed, and this took ages.
2. It required the setup of an http server on the package host, which was one more thing to worry about that I didn't want.

I also had that host serve as the rsync mirror, which had another few disadvantages:

1. It came off of an actual hard drive, which meant that it was probably 5-10x slower than syncing to an official mirror (which act like they're on RAM drives.)
2. It caused a ton of grinding each time the tree was sync'd! This worries me because I want these servers to go the next 5 years with minimal adjustments on the administrator's part (I'll be leaving soon), and all that grinding * 4 for each machine that had to sync just gave me a bad feeling.

Solution? Share /usr/portage read-only over NFS to the local network. The other machines automount /usr/portage, and I can use emerge --buildpkg (actually, I have it set in FEATURES in /etc/make.conf to build binary packages) to build packages on the main host, and then install them on the others using emerge --usepkg.

Actually installing packages has worked great, but I'm worried because emerge goes through a huge round of 'quarterly updates' when I first automounted /usr/portage, and sometimes it acts like it wants to write into those directories:

Performing Global Updates: /usr/portage/profiles/updates/4Q-2004
(Could take a couple of minutes if you have a lot of binary packages.)
.='update pass' *='binary update' @='/var/db move'
s='/var/db SLOT move' S='binary SLOT move' p='update /etc/portage/package.*'
......................................!!! Cannot update readonly binary: sys-devel/gcc-3.3.5-r1
!!! Cannot update readonly binary: sys-devel/gcc-3.3.5.20050130-r1
.....!!! Cannot update readonly binary: sys-devel/libtool-1.5.2-r5
..........................................................................................................................................................................................................................

Does anyone know if indeed anything *needs* write access within /usr/portage? It seems like nothing should, since all the ebuilds get synced, but it still worries me that emerge complains.

Thanks,

Matt
Back to top
View user's profile Send private message
yaneurabeya
Veteran
Veteran


Joined: 13 May 2004
Posts: 1754
Location: Seattle

PostPosted: Fri Jun 24, 2005 6:59 pm    Post subject: Reply with quote

Uhm, you need +w for distfiles and certain sections of portage (individual branches, ie patches) I think.
Back to top
View user's profile Send private message
matt.matolcsi
n00b
n00b


Joined: 03 Apr 2005
Posts: 5

PostPosted: Mon Jun 27, 2005 6:55 pm    Post subject: Reply with quote

distfiles I'm not worried about, since only one machine will sync with the outside world and download/build packages; but what process or program needs to be able to write to specific ebuilds?
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Sun Jul 03, 2005 11:31 am    Post subject: Reply with quote

I've submitted a patch that removes the need for +w to ${DISTDIR}, but then you need some other method to fetch files (like torpage - http://www.kroon.co.za/torpage.php works nicely). I like the --usepkg and --buildpkg idea, I quess one can even use the same idea that torpage uses for fetching distfiles for building packages ... (aka, packages gets built on demand whilst the client waits).
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
bigdave1
Tux's lil' helper
Tux's lil' helper


Joined: 03 May 2005
Posts: 106
Location: Raleigh, NC

PostPosted: Tue Jan 10, 2006 2:02 am    Post subject: Reply with quote

I am having a problem setting this up as described in Setup #3. I have set up rsync exactly as described and setup my client accordingly. However, whenever I do an emerge --sync, I get the following error:

Code:

linuxclient etc # emerge --sync
>>> starting rsync with rsync://192.168.1.4/gentoo-portage...
>>> checking server timestamp ...
@ERROR: Unknown module 'gentoo-portage'
rsync: connection unexpectedly closed (52 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(189)
>>> retry ...


My rsyncd.conf file looks like this:

Code:

# Copyright 1999-2004 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/app-admin/gentoo-rsync-mirror/files/rsyncd.c$

#uid = nobody
#gid = nobody
use chroot = no
max connections = 10
pid file = /var/run/rsyncd.pid
motd file = /etc/rsync/rsyncd.motd
transfer logging = no
log format = %t %a %m %f %b
syslog facility = local3
timeout = 300

[gentoo-x86-portage]
#this entry is for compatibility
path = /opt/gentoo-rsync/portage
comment = Gentoo Linux Portage tree

[gentoo-portage]
#modern versions of portage use this entry
path = /usr/portage
comment = Gentoo Linux Portage tree mirror
exclude = distfiles

[gentoo-packages]
#For distributing Portage packages (distfiles) to internal clients
path = /usr/portage/distfiles
comment = Gentoo Linux Packages mirror


Can somebody please tell me what I'm doing wrong or what needs to be adjusted?

Thanks!
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Tue Jan 10, 2006 6:40 pm    Post subject: Reply with quote

From the client machine, what does the command:

Code:
$ rsync rsync://192.168.1.4


output?
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
bigdave1
Tux's lil' helper
Tux's lil' helper


Joined: 03 May 2005
Posts: 106
Location: Raleigh, NC

PostPosted: Thu Jan 12, 2006 12:09 am    Post subject: Reply with quote

Well, I figured out my problem, I was making changes to the /etc/rsync/rsyncd.conf file instead of the /etc/rsyncd.conf file. However, I now have a new problem. What I would like to happen when I emerge something (ie. emerge gdesklets-core), have my portage server download the ebuild for that package all and the dependencies if its not there and cache it in its portage directory, and then serve it to the client that requested it. However, thats not what is happening. When I do a 'emerge gdesklets-core' here's what I get:

Code:

linuxclient etc # emerge gdesklets-core
Calculating dependencies ...done!
>>> emerge (1 of 5) dev-python/pyorbit-2.0.1 to /
>>> Downloading http://distfiles.gentoo.org/distfiles/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading http://distro.ibiblio.org/pub/Linux/distributions/gentoo/distfiles/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading ftp://ftp.sunet.se/pub/X11/GNOME/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading ftp://archive.progeny.com/GNOME/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading ftp://ftp.gnome.org/pub/gnome/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading http://ftp.gnome.org/pub/gnome/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading ftp://ftp.no.gnome.org/pub/GNOME/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
>>> Downloading ftp://ftp.gnome.org/pub/gnome/2.0.0/sources/pyorbit/2.0/pyorbit-2.0.1.tar.bz2
link_stat "pyorbit-2.0.1.tar.bz2" (in gentoo-packages) failed: No such file or directory
client: nothing to do: perhaps you need to specify some filenames or the --recursive option?
rsync error: some files could not be transferred (code 23) at main.c(653)
!!! Couldn't download pyorbit-2.0.1.tar.bz2. Aborting.


As you can see, its not even attempting to connect to my portage server. Here is what my /etc/rsyncd.conf file looks like:

Code:

# /etc/rsyncd.conf
# $Header: /var/cvsroot/gentoo-x86/net-misc/rsync/files/rsyncd.conf,v 1.6 2005/$

# Minimal configuration file for rsync daemon
# See rsync(1) and rsyncd.conf(5) man pages for help

# This line is required by the /etc/init.d/rsyncd script
pid file = /var/run/rsyncd.pid
use chroot = yes
read only = yes

# Simple example for enabling your own local rsync server
[gentoo-portage]
        path = /usr/portage
        comment = Gentoo Linux Portage tree
        exclude = /distfiles /packages

[gentoo-packages]
#For distributing Portage packages (distfiles) to internal clients
path = /usr/portage/distfiles
comment = Gentoo Linux Packages mirror


And here is what my make.conf file looks like on my client:

Code:

# These settings were set by the catalyst build script that automatically built$
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-O2 -pipe -march=athlon-xp"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"
USE="~x86 gtk qt gnome gdm alsa cdr nptl cups foomaticdb ppds dga xvid cdparano$
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
SYNC="rsync://copenhagen/gentoo-portage"
FETCHCOMMAND="rsync rsync://copenhagen/gentoo-packages/\${FILE} ${DISTDIR}"


I have the hostname of my portage server set to copenhagen which I have the IP defined in my /etc/hosts to point to 192.168.1.4.

And here is what my make.conf file looks like on my portage server:

Code:

# These settings were set by the catalyst build script that automatically built$
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-Os -march=pentium3 -pipe"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j3"
USE="-X mmx sse cvs"


Like I mentioned before, what I would like to have happen is if I try to emerge something that is not cached on the portage server, have the portage server download the ebuild for that package and all the dependencies and cache it locally, then serve it to the client that requested it for the emerge. Any ideas what I need to change to make that happen with my setup? Right now, when I do an 'emerge sync' on my client, that works fine. Just not emerge <package>...

Thanks!
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Thu Jan 12, 2006 2:20 am    Post subject: Reply with quote

Code:
FETCHCOMMAND="rsync rsync://copenhagen/gentoo-packages/\${FILE} ${DISTDIR}"


That is your problem right there. You need some mechanism for the client to notify the server what to fetch. Some solutions make use of squid, but squid usually don't cache overly large files (yes, it actually has an option to not cache files bigger than x bytes - argument: it will expell too many other pages from the cache). Enters torpage. To the best of my knowledge the configuration for torpage is well documented, and can be obtained from http://www.kroon.co.za/torpage.php - it was developed to exactly what you are now attemting.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
bigdave1
Tux's lil' helper
Tux's lil' helper


Joined: 03 May 2005
Posts: 106
Location: Raleigh, NC

PostPosted: Thu Jan 12, 2006 12:54 pm    Post subject: Reply with quote

Cool.. Now have an even better question. Now that I've downloaded the ebuild for torpage, where do I save it and how do I get it to emerge? :)

Thanks!
Back to top
View user's profile Send private message
jkroon
Tux's lil' helper
Tux's lil' helper


Joined: 15 Oct 2003
Posts: 108
Location: South Africa

PostPosted: Thu Jan 12, 2006 1:50 pm    Post subject: Reply with quote

Code:
 #mkdir -p /usr/local/portage/sys-apps/torpage
# echo 'PORTDIR_OVERLAY="/usr/local/portage"' >> /etc/make.conf
# echo 'sys-apps/torpage ~x86' >> /etc/portage/package.keywords
# mv /path/to/torpage-0.2.3.ebuild /usr/local/portage/sys-apps/torpage/
# emerge -f torpage
# ebuild /usr/local/portage/sys-apps/torpage/torpage-0.2.3.ebuild digest
# emerge -av torpage


If you want me to break that down just shout.
_________________
There are 10 kinds of people in the world,
those who understand binary and who don't
Back to top
View user's profile Send private message
bigdave1
Tux's lil' helper
Tux's lil' helper


Joined: 03 May 2005
Posts: 106
Location: Raleigh, NC

PostPosted: Fri Jan 13, 2006 2:10 am    Post subject: Reply with quote

Ok, this may sound like a stupid question, but do I need to install this on the server or the client or both? According to the readme on the link you provided, it sounds like both. I just want to make sure so I get this right.

Thanks!
Back to top
View user's profile Send private message
bigdave1
Tux's lil' helper
Tux's lil' helper


Joined: 03 May 2005
Posts: 106
Location: Raleigh, NC

PostPosted: Fri Jan 13, 2006 2:51 am    Post subject: Reply with quote

Ok, well I installed torpage onto the server and client anyway. So, disregard my previous post. However, when I try to emerge something it doesn't pull it from my portage server. And my portage server doesn't go out and retrieve it. Here's what I get when I try to emerge gdesklets-core:

Code:

linuxclient portage # emerge gdesklets-core
Calculating dependencies ...done!
>>> emerge (1 of 5) dev-python/pyorbit-2.0.1 to /
>>> Downloading torpage://copenhagen/distfiles/pyorbit-2.0.1.tar.bz2
USAGE:  /usr/sbin/torpage_fetch URI DISTDIR
    URI - a valid uri, probably prefixed with torpage://
        DISTDIR - the destination directory, must exist.
 is not a directory!
>>> Downloading http://distfiles.gentoo.org/distfiles/pyorbit-2.0.1.tar.bz2
--21:55:05--  http://distfiles.gentoo.org/distfiles/pyorbit-2.0.1.tar.bz2
           => `/usr/portage/distfiles/pyorbit-2.0.1.tar.bz2'
Resolving distfiles.gentoo.org... 64.50.238.52, 216.165.129.135, 156.56.247.195
Connecting to distfiles.gentoo.org[64.50.238.52]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 242,959 [application/x-tar]

100%[====================================>] 242,959      126.07K/s             



As you can see it gives an error when trying to grab it from copenhagen, my portage server. Here's what the /etc/make.conf file looks like on my client:

Code:

# These settings were set by the catalyst build script that automatically built this stage
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-O2 -pipe -march=athlon-xp"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"
USE="~x86 gtk qt gnome gdm alsa cdr nptl cups foomaticdb ppds dga xvid cdparanoia esd 3dnow 3dnowext acpi avi doc gif jack javascript jpeg mp3 mpeg opengl perl php png quicktime samba spell spl tcpd usb wmf wxwindows xmms win32codecs"
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
SYNC="rsync://copenhagen/gentoo-portage"
FETCHCOMMAND_TORPAGE="/usr/sbin/torpage_fetch \#{URI} \${DISTDIR}"
RESUMECOMMAND_TORPAGE="${FETCHCOMMAND_TORPAGE}"
PORTDIR_OVERLAY="/usr/portage"


And here's what /etc/make.conf file looks like now on my server:

Code:

# These settings were set by the catalyst build script that automatically built$
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-Os -march=pentium3 -pipe"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j3"
USE="-X mmx sse cvs"
PORTDIR_OVERLAY="/usr/portage"


I have started the torpage service using "/etc/init.d/torpage start" and added it to the default runlevel. I'm not sure where to go from this point. Any ideas what I need to do?

Thanks!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 5 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum