Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO:Download Cache for your LAN-Http-Replicator (ver 3.0)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 22, 23, 24  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
melinux
n00b
n00b


Joined: 25 May 2006
Posts: 58
Location: Malta

PostPosted: Sat Feb 14, 2009 5:28 pm    Post subject: Reply with quote

flybynite wrote:
I seem to remember fixing this a while ago, I guess it might be changing again? What version of replicator and portage are you running?


portage version 2.1.6.4

and http-replicator version 3.0-r1 (~x86)

Thanks,
melinux
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 618

PostPosted: Fri Feb 20, 2009 1:18 am    Post subject: Reply with quote

melinux wrote:

portage version 2.1.6.4


Portage switched from portage_manifest to portage.manifest in version 2.2-r6 or greater and repcacheman is fixed for that version. Now the change seems to have been backported to at least your portage version 2.1.6.4

Could you check to see if the current fix works for you to confirm this?

Just cp the file /usr/portage/net-proxy/http-replicator/files/http-replicator-3.0-repcacheman-0.44-r1 somewhere and run it instead of the system version.

Here is some code if you need it.


Code:

mkdir /root/repcachemantest
cd /root/repcachemantest
cp  /usr/portage/net-proxy/http-replicator/files/http-replicator-3.0-repcacheman-0.44-r1 .
./http-replicator-3.0-repcacheman-0.44-r1
Back to top
View user's profile Send private message
jongeek
Tux's lil' helper
Tux's lil' helper


Joined: 13 Jul 2007
Posts: 135
Location: The Humid, Festering Swamps of Florida

PostPosted: Fri Mar 13, 2009 5:32 pm    Post subject: same thing here Reply with quote

flybynite wrote:

Could you check to see if the current fix works for you to confirm this?

Just cp the file /usr/portage/net-proxy/http-replicator/files/http-replicator-3.0-repcacheman-0.44-r1 somewhere and run it instead of the system version.

I was having the same warnings with portage 2.1.6.7. I ran the repcacheman script as you described, and then it did not display the warnings.
Back to top
View user's profile Send private message
melinux
n00b
n00b


Joined: 25 May 2006
Posts: 58
Location: Malta

PostPosted: Fri Mar 13, 2009 6:34 pm    Post subject: Re: same thing here Reply with quote

jongeek wrote:
flybynite wrote:

Could you check to see if the current fix works for you to confirm this?

Just cp the file /usr/portage/net-proxy/http-replicator/files/http-replicator-3.0-repcacheman-0.44-r1 somewhere and run it instead of the system version.

I was having the same warnings with portage 2.1.6.7. I ran the repcacheman script as you described, and then it did not display the warnings.

At present I can't test that pc as its not working (hardware - overheating problems) right now...
I installed the current http-replicator and I have no problems as yet, on another machine. Thanks anyway.
Back to top
View user's profile Send private message
Onkl
n00b
n00b


Joined: 01 Dec 2009
Posts: 2

PostPosted: Tue Dec 01, 2009 9:24 am    Post subject: Reply with quote

I have set up http-replicator to also serve as PORTAGE_BINHOST. Works as advertised.

However, when I use

Code:

quickpkg --include-config=y <package>


to make a package with my adapted settings to my LAN, the package gets permissions of 600, which http-replicator will not serve. Changing the permissions to 644 solves this, but it is a hassle that I forget to do most of the time.

Is there a way to make http-replicator also serve those files, or make quickpkg to set the correct permissions?
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 618

PostPosted: Fri Dec 04, 2009 3:09 am    Post subject: Reply with quote

Onkl wrote:
Is there a way to make http-replicator also serve those files, or make quickpkg to set the correct permissions?


Code:
h2 ~ # quickpkg --help

  --umask=UMASK         umask used during package creation (default is 0077)


Quickpkg's default umask of 0077 denies all users except root from even reading the file. I can't explain why it does that. Replicator can't change or work around such drastic file permissions.

There must be many others who feel that default umask is too drastic because Quickpkg does give you the option to change the umask to a more normal 022.

Try

Code:

h2 ~ # quickpkg --umask=022  nano
h2 ~ # ls -l /var/tmp/packages/All/nano*
-rw-r--r-- 1 root root 202145 2009-12-03 20:53 /var/tmp/packages/All/nano-2.1.10.tbz2


Now replicator can read the file and serve it to others.

You can make this easy by putting
Code:
alias quickpkg='quickpkg --umask=022'
in your ~/.bashrc
Back to top
View user's profile Send private message
Onkl
n00b
n00b


Joined: 01 Dec 2009
Posts: 2

PostPosted: Fri Dec 04, 2009 8:46 am    Post subject: Reply with quote

Thanks a lot.

And since an "emerge -b" builds the package with 644 it is indeed strange that quickpkg is so restrictive.
Back to top
View user's profile Send private message
sam_i_am
Tux's lil' helper
Tux's lil' helper


Joined: 19 Sep 2003
Posts: 124

PostPosted: Sun Jun 20, 2010 10:31 pm    Post subject: service doesn't start on one host Reply with quote

Hi,

Thanks for this great program. I'm using at home and at work without any issues except:

On one host (x86), the http-replicator doesn't start. It says "failed to start service". Log says "HttpReplicator started", but it seems to die. I enabled debug option, but no other error messages. However, on a different host (x86_64) on the same network, it works with the exact same config parameters.

Any tips on how to go about finding out why?

Other than the being x86, the failing host also runs apache on port 80 and 443, but port 8080 is clear.

Sam
_________________
Sam
Back to top
View user's profile Send private message
knight77
n00b
n00b


Joined: 29 Jun 2009
Posts: 25

PostPosted: Thu Sep 30, 2010 1:49 pm    Post subject: Re: service doesn't start on one host Reply with quote

Hi there.

The problem is still there (http-replicator-3.0-r2 stable in portage x86).

I filed a bug on it: http://bugs.gentoo.org/show_bug.cgi?id=339079

There is also another thread here about this problem, workaround included: http://forums.gentoo.org/viewtopic-t-787761.html
Back to top
View user's profile Send private message
Havin_it
Veteran
Veteran


Joined: 17 Jul 2005
Posts: 1076
Location: Edinburgh, UK

PostPosted: Wed Mar 13, 2013 3:48 pm    Post subject: Reply with quote

Thought this might be worth a mention, even though it's a bit non-standard...

I've used http-replicator for years, and I've just realised I wasn't doing so in the prescribed way. I've never even been aware of repcacheman until today, in fact. I never liked the idea of the distfiles ending up duplicated on the server, so I do things this way:

  1. Set http-replicator's cache dir to the same as my $DISTDIR
  2. Local clients use the proxy, server machine does not


The upshot is that when clients fetch a distfile, it goes in (or is already in, and gets served from) the cache via http-replicator; when the server fetches, the files are saved in the same dir and http-replicator will serve these for local clients as well, even though it didn't fetch them personally.

This seems to have worked fine for me over the years, though I'd welcome any drawbacks to this approach that y'all veterans might be able to identify. Hope it helps someone.
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2439

PostPosted: Wed Mar 13, 2013 4:59 pm    Post subject: Reply with quote

I had http-replicator fail to come up a week or two ago, also. The problem was that the lockfile is at /var/log/http-replicator/http-replicator.pid, and the portage user didn't have write access.

A while back Linux adopted a tmpfs-based /run directory, and /var/run became a symlink to /run. When it was in /var/run it was persistent, and the emerge process could chown /var/run/http-replicator to portage, and it would stick. Now that /run is tmpfs, it's recreated on every boot, and such things don't stick. The fix would be to do a chown against /run/http-replicator in the initscript, prior to changing to the portage userid.

The problem I have is that this doesn't universally fail, so I don't understand what's going on. A few days ago that system lost power, and when I powered it back up, I made a mental note about http-replicator, since I hadn't actually changed the initscript. Then I forgot. Upon seeing this thread, I checked that system and http-replicator is running, even though it doesn't seem that it should be. Moreover, now the lockfile is /run/http-replicator.pid, owned by root, instead of /run/http-replicator/http-replicator.pid, owned by portage.

On another system where I'm running http-replicator it's still on the deeper path. Looking into the initscripts, the locations I see for the lockfile are correct, yet they're all 3.0-r3 - I don't quite get it.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
khayyam
Advocate
Advocate


Joined: 07 Jun 2012
Posts: 2185

PostPosted: Wed Mar 13, 2013 6:04 pm    Post subject: Reply with quote

depontius wrote:
A while back Linux adopted a tmpfs-based /run directory, and /var/run became a symlink to /run. When it was in /var/run it was persistent, and the emerge process could chown /var/run/http-replicator to portage, and it would stick. Now that /run is tmpfs, it's recreated on every boot, and such things don't stick. The fix would be to do a chown against /run/http-replicator in the initscript, prior to changing to the portage userid.

depontius ... when this migration happend openrc intoduced an implimentation of systemd's tmpfiles.d. This implimentation is 100% compatable with the above linked manpage, though openrc doesn't create the configuration dir /etc/tmpfiles.d on install.

Code:
# equery files =sys-apps/openrc-0.11.8 |grep tmpfiles
/etc/conf.d/tmpfiles
/etc/init.d/tmpfiles.setup
/lib/rc/sh/tmpfiles.sh
/usr/share/openrc/runlevels/boot/tmpfiles.setup

So, using the above, and setting tempfiles.setup in the runlevel, you should be able have /run/http-replicator created with the correct ownership, etc.

HTH & best ... khay
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2439

PostPosted: Wed Mar 13, 2013 6:16 pm    Post subject: Reply with quote

Thanks for the info. I'd seen stuff on /etc/tmpfiles.d as a systemd thing, but didn't realize that OpenRC had it, as well.

What bothers me more is that right now is that I have 2 systems running http-replicator-3.0-r3, both x86. One has /run/http-replicator.pid and the other has /run/http-replicator/http-replicator.pid - in the initscript.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
Havin_it
Veteran
Veteran


Joined: 17 Jul 2005
Posts: 1076
Location: Edinburgh, UK

PostPosted: Wed Mar 13, 2013 10:06 pm    Post subject: Reply with quote

Just wondering, is it possible/supported to run multiple instances of http-replicator? I've got another caching-proxying task it might be useful for, but I want to use a different cache dir for it.

Possible?
Back to top
View user's profile Send private message
mbar
Veteran
Veteran


Joined: 19 Jan 2005
Posts: 1952
Location: Poland

PostPosted: Mon Mar 18, 2013 2:17 pm    Post subject: Reply with quote

Has anybody the same problem as me (~amd64)? http-replicator-4.0_alpha2

Code:
/etc/init.d/http-replicator start
 * Caching service dependencies ...                                                                                                                                                     [ ok ]
 * Starting Http-Replicator ...
Traceback (most recent call last):
  File "/usr/bin/http-replicator", line 4, in <module>
    import Params, Request, Response, fiber, weakref
ImportError: No module named Params
 * start-stop-daemon: failed to start `/usr/bin/http-replicator'
 * Failed to start Http-Replicator                                                                                                                                                      [ !! ]
 * ERROR: http-replicator failed to start
Back to top
View user's profile Send private message
ToeiRei
Veteran
Veteran


Joined: 03 Jan 2005
Posts: 1155
Location: Austria

PostPosted: Tue Mar 19, 2013 7:46 am    Post subject: Reply with quote

same problem here. Trying to track it down.

Update 1:
Looks like a Namespace / Path problem to me as removing 'Params' did make it complain about the next missing module

Solved by downgrading to net-proxy/http-replicator-3.0-r3
_________________
Blog | btrfs | Please stand by - The mailer daemon is busy burning your messages in hell...
Back to top
View user's profile Send private message
TomWij
Developer
Developer


Joined: 04 Jul 2012
Posts: 1546

PostPosted: Wed Jun 05, 2013 1:33 pm    Post subject: Reply with quote

Please report bugs at Bugzilla, we don't keep track of bugs on the forum. Thanks.

Quote:
+ 05 Jun 2013; Tom Wijsman <TomWij@gentoo.org>
+ +http-replicator-4.0_alpha2-r1.ebuild:
+ Revision bump, added missing Python modules. Fixes bug #472122 reported by
+ Russell Knighton.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 22, 23, 24
Page 24 of 24

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum