HOWTO:Download Cache for your LAN-Http-Replicator (ver 3.0)

JohnHerdy · n00b Joined: 11 Jul 2002 Posts: 61

Ok did some more research, it seems that the new version of portage doesn't respect my .wgetrc-settings anymore. After adding the following lines to /etc/make.conf 2.0.51-clients work as well;

http_proxy="http://YourMirrorHere.com:8080"
RESUMECOMMAND=" /usr/bin/wget -t 5 --passive-ftp \${URI} -O \${DISTDIR}/\${FILE}"

With 2.0.50 this wasn't neccesary the proxy-settings where read from .wgetrc. Hmmm, strange...

kmarasco

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

kmarasco

kmarasco · Posted: Thu Oct 28, 2004 10:24 am Post subject: Alternative Resume Syntax

Thanks for your help flybynite. Since I experienced the bug with both ftp and http, I tried to find an alternative syntax for the resume command. I think that the following syntax should have the same effect, but does not require that the file name variable be passed in order to replace a partial file. It is as follows:

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

matbintang · n00b Joined: 12 Aug 2003 Posts: 22 Location: Canada

flybynite!

You Da Man!!

_________________
$>cd /pub
$>more beer

kmarasco · Posted: Fri Oct 29, 2004 5:09 am Post subject:

Thanks again for all of your help. I commend you for your excellent work and ongoing support effort :!:

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

Thanks ( smiles proudly)

I just hope you still feel that way when I say you need to change portage_util.writemsg back to portage.writemsg !!

Or download repcacheman ver 3.2 at http://www.updatedlinux.com/replicator/portagefix/repcacheman

BlinkEye · Posted: Sat Oct 30, 2004 10:14 am Post subject:

i do have a problem: repcacheman runs every 5 minutes but even though it should have nothing to do (i.e. i didn't download any packages and /usr/portage/distfiles/ is empy) it eats a lot of my precious memory (my server is an oldie):

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

The release of portage 2.0.51 has caused a number of problems.

These problems haven't affected how well http-replicator itself runs, only with the support script repcacheman.

I have updated ebuilds with changes to support portage 2.0.51. I need help testing:

http://www.updatedlinux.com/replicator/http-replicator-flybynite-1.6a.tar.bz2

This release contains 5 versions:
* net-misc/http-replicator :
[ ] 2.0-r2 (0) OVERLAY
[ ] 2.0-r3 (0) OVERLAY
[ ] 2.1_rc3 (0) OVERLAY
[ ] 2.1_rc3-r1 (0) OVERLAY
[ I] 2.1 (0) OVERLAY

2.0-r2 and 2.1_rc3 DEPEND on portage 2.0.50 and won't work with the current stable portage 2.0.51 I left these in only for the 1 or 2 gentooers who don't like to upgrade :-)

2.0-r3 , 2.1_rc3_r1 and 2.1 will work with any current portage version. 2.1 primarily adds external proxy support and improved logging.

2.1 is the latest and is stable, but the changes aren't fully documented in the howto yet.

Follow the howto at the start of this thread for all versions. Then If you upgraded to 2.1 -look at these changes:

**Note The config file location has changed in 2.1. Edit /etc/conf.d/http-replicator to add your external proxy and check other defaults.

**Note repcacheman has changed in 2.1. If you changed the default cache dir or the default user, you must call repcacheman with those options.

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

I had a question about repcacheman and what it does and how often it should be run.

Repcacheman only needs to run after emerge is run on the server ( except emerge sync)

Here is some more info on repcacheman:

meowsqueak · Posted: Mon Nov 01, 2004 4:33 am Post subject:

I upgraded to 2.1 but I'm getting this problem on one of my LANs. The following spews out about 20 times:

meowsqueak · Posted: Mon Nov 01, 2004 8:12 pm Post subject:

Ok, I did some reading into the 301 response:

http://www.checkupdown.com/status/E301.html

According to the output, wget first tries to download http://gentoo.oregonstate.edu/distfiles/fontconfig-2.2.3.tar.gz. The proxy request is sent and the response is a 301 error (presumably from the original server?). The Location specifies http://gentoo.osuosl.org/distfiles/fontconfig-2.2.3.tar.gz as the alternative URL, and wget follows this. But when it tries to download this, it gets another 301 with the location set to the exact same URL and the process repeats until the limit (20) is hit. So, is this a problem with this server in particular, or is it a problem with the http-replicator proxy? I suspect the server is fine because a manual wget of the osuosl.org file works fine (no followed location). Does http-replicator cache server responses or something?

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

First, go ahead and update the link to http://gentoo.osuosl.org/ - since http://gentoo.oregonstate.edu/ is outdated anyway. I've updated the howto.

The response isn't cached by http-replicator- might be cached by your ISP or somewhere else.

Then if you still have this problem, upgrade to http-replicator 2.1, set debug in the config and post the logs...

meowsqueak · Posted: Wed Nov 03, 2004 8:26 pm Post subject:

Hmmm, I wonder if my squid proxy is caching something - here's the log:

flybynite · l33t Joined: 06 Dec 2002 Posts: 620

meowsqueak · Posted: Thu Nov 04, 2004 9:30 pm Post subject:

Maxwell · Tux's lil' helper Joined: 10 Dec 2003 Posts: 97

Well, i suppose i must be doing something stupid, but i need some help.

I installed http_replicator as i should, but as i have an http proxy port=8080, i switched replicator's port to 5000. But it looks like http_replicator isn't used by my notebook, in my internal lan. the value "http_proxy" in make.conf is the ip and door of the server, correct?

Both my /usr/portage/distfiles and my /var/log/http-replicator are empty. The log file also doesn't report anything when i emerge a file at the server.

Suggestions?
_________________
Freedom works. Use it!
Linux, by Gentoo

meowsqueak · Posted: Mon Nov 08, 2004 9:50 pm Post subject: Re: Problems using it