Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO:Download Cache for your LAN-Http-Replicator (ver 3.0)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 20, 21, 22, 23, 24  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
hielvc
Advocate
Advocate


Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca

PostPosted: Fri Dec 22, 2006 7:49 pm    Post subject: Reply with quote

RichieB and any others who useing a chroot you have to point to your chroots packages directory not the servers package dirtectory unless thats where you are copying your packages to. My chroot is /home/gentoo and my packages dir for it is /home/gentoo/usr/packages ( dont ask me why I put it there ) so in my /etc/conf.d/http-replicator I have DAEMON_OPTS="$DAEMON_OPTS --alias /home/gentoo/usr/packages/All:All" and it works fine. That is when I can get portage to use it :evil:
_________________
An A-Z Index of the Linux BASH command line
Back to top
View user's profile Send private message
naim
n00b
n00b


Joined: 07 Jun 2005
Posts: 22

PostPosted: Sat Feb 10, 2007 12:33 pm    Post subject: [Errno 28] No space left on device Reply with quote

i keep getting this message, i checked, the partition i assigned to /var/cache/http-replicator is 50% full, with 5 Gb free.
/dev/sda7 11G 5.1G 5.2G 50% /var/cache/http-replicator
i switched to verbose logging and i found the following in the logs::

Quote:
Traceback (most recent call last):
File "/usr/lib/python2.4/asyncore.py", line 69, in read
obj.handle_read_event()
File "/usr/lib/python2.4/asyncore.py", line 391, in handle_read_event
self.handle_read()
File "/usr/bin/http-replicator", line 160, in handle_read
self.data.seek(0) # move to beginning
IOError: [Errno 28] No space left on device

and on client side ::

Quote:
Calculating dependencies... done!

>>> Emerging (1 of 4) dev-perl/Archive-Zip-1.16 to /
>>> Downloading 'http://192.168.20.251:8083/distfiles/Archive-Zip-1.16.tar.gz'
--14:17:52-- http://192.168.20.251:8083/distfiles/Archive-Zip-1.16.tar.gz
=> `/usr/portage/distfiles/Archive-Zip-1.16.tar.gz'
Connecting to 192.168.20.251:8083... connected.
Proxy request sent, awaiting response... No data received.
Retrying.


My /etc/conf.d/httpreplicator.conf

Quote:

GENERAL_OPTS="--dir /var/cache/http-replicator"
GENERAL_OPTS="$GENERAL_OPTS --user portage"
DAEMON_OPTS="$GENERAL_OPTS"
DAEMON_OPTS="$DAEMON_OPTS --external proxy.network.company:8080"
DAEMON_OPTS="$DAEMON_OPTS --alias /usr/portage/packages/All:All"
DAEMON_OPTS="$DAEMON_OPTS --log /var/log/http-replicator.log"
DAEMON_OPTS="$DAEMON_OPTS --debug"
DAEMON_OPTS="$DAEMON_OPTS --ip 172.16.36.*"
DAEMON_OPTS="$DAEMON_OPTS --ip 192.168.20.*"
DAEMON_OPTS="$DAEMON_OPTS --port 8083"



Any ideas ??
Back to top
View user's profile Send private message
bunkacid
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2005
Posts: 111
Location: Laguna Hills, CA

PostPosted: Sat Feb 10, 2007 4:49 pm    Post subject: Re: [Errno 28] No space left on device Reply with quote

naim wrote:
i keep getting this message, i checked, the partition i assigned to /var/cache/http-replicator is 50% full, with 5 Gb free.
/dev/sda7 11G 5.1G 5.2G 50% /var/cache/http-replicator
i switched to verbose logging and i found the following in the logs::

Any ideas ??


You've run out of space somewhere that http-replicator is outputting data/logs/information/etc to.
What are the results of;
Code:
df -m
Back to top
View user's profile Send private message
naim
n00b
n00b


Joined: 07 Jun 2005
Posts: 22

PostPosted: Mon Feb 12, 2007 2:54 pm    Post subject: Reply with quote

oops, root partition full, i was looking the wrong way, thanks
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Sun Mar 04, 2007 7:15 am    Post subject: Reply with quote

I have been using this for a while but all of a sudden I get this:

Code:
root@smoker / # repcacheman


Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/

Comparing directories....
Done!

Deleting duplicate file(s) in /usr/portage/distfiles/
http-replicator_3.0.tar.gz
Done!

New files in DISTDIR:
seamonkey-1.1-patches-0.1.tar.bz2
mozilla-launcher-1.56.bz2
seamonkey-1.1.1.source.tar.bz2

Checking authenticity and integrity of new files...
Searching for ebuilds...
Done!

Found 22919 ebuilds.

Extracting the checksums....
Done!

Verifying checksum's....
/usr/portage/distfiles/seamonkey-1.1-patches-0.1.tar.bz2
Traceback (most recent call last):
  File "/usr/bin/repcacheman.py", line 203, in ?
    if t["MD5"]:
KeyError: 'MD5'
root@smoker / #


I tried reemerging it but I still get the same error. What is this? Is this me or something else?

Thanks

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Sun Mar 04, 2007 7:56 am    Post subject: Reply with quote

dalek wrote:

/usr/portage/distfiles/seamonkey-1.1-patches-0.1.tar.bz2
KeyError: 'MD5'
[/code]


This type of error is probably caused by an error in the portage data. Wait a while, re-sync and see if it goes away.
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Sun Mar 04, 2007 6:30 pm    Post subject: Reply with quote

Hmmm, I should have mentioned this has been going on for a week or better. I have synced three or four times since this started.

I do seem to recall that portage and gentoolkit was upgraded recently. Could that have something to do with this? I may go back a version of gentoolkit and try it. I think that was the last one upgraded and was about the time this started.

Open to any ideas though. I really need to fix this so I can update my other rigs.

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Mon Mar 05, 2007 5:48 am    Post subject: Reply with quote

Well, I went back to a older version of portage and gentoolkit and still no joy. I also synced again.

Any more ideas? I need some. :wink:

:D :D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Tue Mar 06, 2007 7:57 am    Post subject: Reply with quote

OK. This fixed it:

Code:
mv -v /usr/portage/distfiles/seamonkey-1.1-patches-0.1.tar.bz2 /root/


That will work until I need that patch again. Does nobody have a clue what is up with that? Is the patch corrupt or something?

Thanks

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
bunkacid
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2005
Posts: 111
Location: Laguna Hills, CA

PostPosted: Tue Mar 06, 2007 11:58 pm    Post subject: Reply with quote

I believe some of the portage Devs are making changes to the Manifest files which could explain the MD5 error.
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Wed Mar 07, 2007 2:51 am    Post subject: Reply with quote

Thanks for the info. I'm surprised no one else has posted they are getting the same error though. Maybe it will sort itself out pretty soon.

:D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Wed Mar 07, 2007 4:41 am    Post subject: Reply with quote

dalek wrote:
I'm surprised no one else has posted they are getting the same error though.


repcacheman uses portage functions, which means this error could be specific to your portage system or setup.

1. Does this file belong to a local portage package? A package that you made yourself or copied from somewhere or maybe even from a portage overlay?

2. Since you upgraded portage lately, according to http://www.gentoo.org/proj/en/portage/doc/common-problems.xml , it could be caused by a old version of portage or maybe an incomplete upgrade of portage:

Quote:

Receiving
portage.db["/"]["porttree"].dbapi.auxdb[porttree_root][cat].clear()
KeyError: 'app-dicts'
while updating portage cache

* Bugreport: 100444
* Caused by: old portage version
* Solution: update portage and run emerge --sync again


I know you don't have this exact error, but I believe the KeyError seems to be the important part and is the error you reported :-)
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Wed Mar 07, 2007 5:37 am    Post subject: Reply with quote

I went back to a older version of portage, just to see if it was a change made in portage that just kicked in. Same thing, even after doing a sync again. I have since went back to the new version of portage. I also did the same thing with gentoolkit just in case.

I read the link you posted, since the fix was to reemerge portage, I would hope that downgrading and upgrading would have taken care of that. The key word is hope though. :wink:

The file did come from emergeing Seamonkey. It was a recent upgrade. I don't even know how to do a overlay. After moving the file as mentioned above, I have not had the error since, even after syncing.

Still open to ideas though. I can move the file back if needed.

Thanks

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
bunkacid
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2005
Posts: 111
Location: Laguna Hills, CA

PostPosted: Wed Mar 07, 2007 5:54 am    Post subject: Reply with quote

Have you tried removing the offending file?
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Wed Mar 07, 2007 6:13 am    Post subject: Reply with quote

Yup, that was how I got rid of the error. I have ran into corrupt files before though and it would just delete them or not copy them over and go on. On this one, it made the CPU go to about 100%, sit there for a really loooong time and puke out that error. It has never took that long to run repcacheman before though. It just seemed that this file triggered something nasty.

:D :D :D :D :D

edit: forgot the error part. :oops:
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Wed Mar 07, 2007 5:27 pm    Post subject: Reply with quote

Hmmm, seems I tried to duplicate the error and I can't even get the same file.

Did the dev's update the ebuild last night or are you using an old version of the patches so you haven't emerge sync'd??

I only have patches 1.1.1 ??

Quote:

repcacheman


Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /var/tmp/distfiles/

Comparing directories....
Done!

Deleting duplicate file(s) in /var/tmp/distfiles/
(snip)
seamonkey-1.1.1-patches-0.1.tar.bz2

Done!

SUMMARY:
Found 113 duplicate file(s).
Deleted 113 dupe(s).


Done!
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Wed Mar 07, 2007 7:18 pm    Post subject: Reply with quote

You may be on to something. I did a emerge -f seamonkey and it redownloaded the patch, it worked fine when I ran repcacheman. I would have to guess that the old one is corrupt but how come portage used it to install Seamonkey and not notice that I wonder?

That is weird. I guess that is why it happened to me. :lol: :lol:

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
enkryptz
n00b
n00b


Joined: 10 Mar 2007
Posts: 1

PostPosted: Sun Mar 11, 2007 2:33 pm    Post subject: Reply with quote

Can anyone hook me up with the latest sources for http replicator?
The webpage seems to be inactive since a long time, there is some news about a 3.1 development version,
but I can only find 3.0 which is quite old.
Any public CVS or SVN repository available where I can follow the development?
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Sun Mar 11, 2007 10:51 pm    Post subject: Reply with quote

Try here:

http://cudlug.cudenver.edu/gentoo/distfiles/

and just look for the http-replicator tarball. It's long list but it's in there.

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Sat Mar 17, 2007 2:57 pm    Post subject: Reply with quote

Regarding dalek's 'MD5' error while running repcacheman -- you're not the only one! I *have* been having that exact problem, but it isn't with the seamonkey file, it's with all kinds of different files. At first, I just did what I usually do with an offending file in distfiles - delete it, but the problem seems pretty rampant right now (at least, with the set of stuff I've gotten lately in distfiles), so I'm looking at it as a more serious problem.

I haven't done any rebuilds of portage or gentoolkit. That is, as an attempt to fix the problem, I did get the latest portage and gentoolkit updates, but I think the MD5 problem was happening before that anyway. I've also done multiple syncs and 'emerge -uDav world' over the last week.

Here's hoping it goes away in the near future. I'm not sure what to try that might help.
Back to top
View user's profile Send private message
hielvc
Advocate
Advocate


Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca

PostPosted: Sun Mar 18, 2007 1:01 am    Post subject: Reply with quote

Same here. I have had 2 offeneding packages. The second one was
Quote:
Verifying checksum's....
/usr/portage/distfiles/eix-0.9.1.tar.bz2
Traceback (most recent call last):
File "/usr/bin/repcacheman.py", line 203, in ?
if t["MD5"]:
KeyError: 'MD5'

_________________
An A-Z Index of the Linux BASH command line
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA

PostPosted: Sun Mar 18, 2007 2:37 am    Post subject: Reply with quote

This is sort of funny. I keyworded and unmasked that eix package and did a fetch for it. I then ran repcacheman. I got a error but not for the eix package. I got this:

Code:
Deleting duplicate file(s) in /usr/portage/distfiles/
eix-0.9.1.tar.bz2
java-config-wrapper-0.12.tar.bz2
javatoolkit-0.2.0.tar.bz2
java-config-1.3.7.tar.bz2
java-config-2.0.31.tar.bz2
Done!

New files in DISTDIR:
j2sdk-1_4_2-doc.zip
jdk-1_5_0-doc-r1.zip

Checking authenticity and integrity of new files...
Searching for ebuilds...
Done!

Found 22902 ebuilds.

Extracting the checksums....
Done!

Verifying checksum's....
/usr/portage/distfiles/j2sdk-1_4_2-doc.zip
Traceback (most recent call last):
  File "/usr/bin/repcacheman.py", line 203, in ?
    if t["MD5"]:
KeyError: 'MD5'
root@smoker / #           


So mine was fine with eix where your wasn't. I synced last night to. Do you get the same error for j2sdk?

I'm hoping this will help them figure out if it is a local thing specific to us or if it is something else.

:D :D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Sun Mar 18, 2007 3:27 pm    Post subject: Reply with quote

Another bit of info: I just ran repcacheman on one of my other servers (which I hadn't done in awhile - maybe a week or two) and it plowed through everything in distfiles just fine. The interesting piece is that this server is actually the one that feeds packages to all of my LAN machines -- so it had the same set of files in distfiles that caused the 'MD5' error when running repcacheman on the other machine. I wonder if that indicates that perhaps the problem might originate when http-replicator serves package files. The two machines on which I have http-replicator running are as near identical as I can get (that's actually the purpose - one is a backup/test for the other), with the exception that the one (with no 'MD5' errors) sucks packages down from the internet, and the other (with 'MD5' errors) sucks packages down from the first machine.

Too busy right now to run targeted experiments to see if I can narrow down the circumstances surrounding the problem, but if I get a chance, I'll report back here.
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Mon Mar 19, 2007 4:27 am    Post subject: Reply with quote

enkryptz wrote:
Can anyone hook me up with the latest sources for http replicator?


The current ebuild is the latest sources.

http://gertjan.freezope.org/replicator/ is the last known web page for Gertjan. He authored http-replicator specifically for debian and is not a gentoo user.

I worked with him to improve the code and make it gentoo friendly. I am the author of repcacheman and other gentoo specific code such as the ebuild and config's.

Gertjan, despite my best effort, wanted to completely rewrite the code from scratch rather than refactor and add much needed features. As far as I know, he never completed any updates.

Are you interested in hacking the code? Do you need some specific update?
Back to top
View user's profile Send private message
flybynite
l33t
l33t


Joined: 06 Dec 2002
Posts: 620

PostPosted: Mon Mar 19, 2007 6:19 am    Post subject: Reply with quote

dalek,dahoste,hielvc,

Congrats, you've found a BUG!

It seems there is a change in portage working its way into the portage tree somehow related to GLEP 44. It changes the way portage stores the checksums of files.

This wouldn't cause any problems by itself since repcacheman only calls the portage database functions, but it seems MD5's are either no longer going to be used or portage dev's are not bothering to add them to the new manifest format anymore.

I'll see what I can find out about the change in portage.

Thanks again!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 20, 21, 22, 23, 24  Next
Page 21 of 24

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum