View previous topic :: View next topic |
Author |
Message |
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Sun Aug 29, 2004 6:35 pm Post subject: |
|
|
tomaw wrote: | Works very nicely here too. Is the server capable of taking the load of people testing this out? |
We will see the server is directly connected to the DFN - therefore it *can* take some load . However, the *public* beta phase just began - so there is no experience how well it handles many users yet.
bp |
|
Back to top |
|
|
PrakashP Veteran
Joined: 27 Oct 2003 Posts: 1249 Location: C.C.A.A., Germania
|
Posted: Sun Aug 29, 2004 6:43 pm Post subject: |
|
|
Thanks for your comment. I set GENTOO_MIRRORS="" and report back how it goes now.
Last edited by PrakashP on Sun Aug 29, 2004 8:14 pm; edited 1 time in total |
|
Back to top |
|
|
ruth Retired Dev
Joined: 07 Sep 2003 Posts: 640 Location: M / AN / BY / GER
|
Posted: Sun Aug 29, 2004 7:35 pm Post subject: |
|
|
hi,
@blackpenguin:
thank you...
works like a charm and saves me a lot of bandwidth...
thank you, again...
@devs:
please - make this an official gentoo feature...
regards
rootshell _________________ "The compiler has tried twice to abort and cannot do so; therefore, compilation will now terminate."
-- IBM PL/I (F) error manual |
|
Back to top |
|
|
NoUseForAName Tux's lil' helper
Joined: 22 Feb 2003 Posts: 84 Location: Sweden
|
Posted: Sun Aug 29, 2004 8:26 pm Post subject: |
|
|
Ok, found some more packages that didn't work.
* ruby when upgraded from ruby-1.8.1.tar.gz to ruby-1.8.2.preview2.tar.gz
* xinelib when upgraded from
xine-lib-1-rc3a.tar.gz to xine-lib-1-rc5.tar.gz
* xvid when upgraded from xvidcore-1.0.0-beta3.tar.bz2 to xvidcore-1.0.1.tar.bz2
This patch should fix all those problems:
Code: |
--- getdelta.sh 2004-08-29 19:54:40.000000000 +0200
+++ /usr/local/bin/getdelta.sh 2004-08-29 22:12:40.025669552 +0200
@@ -215,7 +215,7 @@
# and probably it will fail sometimes.
#
# we just ignore any occurence of
-# "pre","rc","[0-9],"_","-","." in the filenames and test
+# "preview","pre","rc","beta","[0-9],"_","-","." in the filenames and test
# if they are the same (or VERY similar (differ only in 1 char)).
# to reduce the files to check, we only check files
# that have the same first 2 letters in name
@@ -240,7 +240,7 @@
-e "s/src7/srcGGG/g" <<< $NEW_FILE)
fi
- mask=$(sed -e "s/[0-9]//g" -e "s/pre//g" -e "s/rc//g" -e "s/[\._-]//g" -e "s/\+//g" -e "s/ //g" <<< "${MASK_FILENAME}")
+ mask=$(sed -e "s/[0-9]//g" -e "s/preview//g" -e "s/pre//g" -e "s/rc//g" -e "s/beta//g" -e "s/[\._-]//g" -e "s/\+//g" -e "s/ //g" <<< "${MASK_FILENAME}")
let len1=$(wc -c <<< $mask)-1
@@ -259,7 +259,7 @@
-e "s/src6/srcFFF/g" \
-e "s/src7/srcGGG/g" <<< $name)
fi
- mask2=$(sed -e "s/[0-9]//g" -e "s/pre//g" -e "s/rc//g" -e "s/[\._-]//g" -e "s/\+//g" -e "s/ //g" <<< "${MASK_FILENAME}")
+ mask2=$(sed -e "s/[0-9]//g" -e "s/preview//g" -e "s/pre//g" -e "s/rc//g" -e "s/beta//g" -e "s/[\._-]//g" -e "s/\+//g" -e "s/ //g" <<< "${MASK_FILENAME}")
# add any file, that results in the same mask or differ not more than one letter
@@ -267,9 +267,9 @@
if [ $len1 -gt $len2 ]
then
max=${len1}
- let min=${len2}-1
+ let min=${len2}
else
- let min=${len1}-1
+ let min=${len1}
max=${len2}
fi
let df=${max}-${min}
|
|
|
Back to top |
|
|
NoUseForAName Tux's lil' helper
Joined: 22 Feb 2003 Posts: 84 Location: Sweden
|
Posted: Sun Aug 29, 2004 8:35 pm Post subject: |
|
|
Changing the default of fetch to add resuming of files would probably be nice too, like:
Code: | FETCH="/usr/bin/wget -c -t 1 --passive-ftp" |
|
|
Back to top |
|
|
NoUseForAName Tux's lil' helper
Joined: 22 Feb 2003 Posts: 84 Location: Sweden
|
Posted: Sun Aug 29, 2004 9:30 pm Post subject: |
|
|
Ok, and another fix
The line:
Code: | ${VERBOSITY} && echo -e "$\n{RED}The server could not build the dtu-file for ${NEW_FILE}${NORMAL}\n"| tee -a $LOGFILE |
should be changed to:
Code: | ${VERBOSITY} && echo -e "${RED}The server could not build the dtu-file for ${NEW_FILE}${NORMAL}\n"| tee -a $LOGFILE |
(removing the \n so it actually gets printed in red and not with a newline...) |
|
Back to top |
|
|
Stefan1801 n00b
Joined: 20 Aug 2004 Posts: 50 Location: Marienmünster
|
Posted: Sun Aug 29, 2004 9:55 pm Post subject: |
|
|
how can i get those stats, like "...originial size=xxmb, saved= xx%" ?
thanks. _________________ well, i'm drowning in time to a desperate beat
---
Router
Workstation |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Mon Aug 30, 2004 4:19 pm Post subject: |
|
|
Stefan1801 wrote: | how can i get those stats, like "...originial size=xxmb, saved= xx%" ? |
Just have a look at http://linux01.gwdg.de/~nlissne/
bp |
|
Back to top |
|
|
pleusicles n00b
Joined: 26 May 2004 Posts: 44
|
Posted: Mon Aug 30, 2004 5:44 pm Post subject: |
|
|
First of all, thanks for this util, it's wonderful!
I just found a problem regarding the search for previously dl'ed distfiles (yes, I know it's tricky as stated in the script...):
I tried to upgrade mplayer, which includes downloading the following files:
font-arial-iso-8859-1.tar.bz2 and
font-arial-iso-8859-2.tar.bz2
Of course, 1 and 2 are interpreted as version instead of being part of the name... The server builds the dtu, it's downloaded, but then the checksums don't match.
After a few tries, the whole emerge aborts instead of downloading the original file.
Just the end of the story:
Code: |
>>> Downloading http://ftp3.mplayerhq.hu/MPlayer//releases/fonts/font-arial-iso-8859-1.tar.bz2
Checking if a Thirdparty-Mirror is used ...
Chosen Mirror is mplayer
Searching for a previosly downloaded file in /store/distfiles
We have following candidates to choose from
font-arial-iso-8859-2.tar.bz2
The best of all is ... font-arial-iso-8859-2.tar.bz2
Trying to download font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu
--19:23:15-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=font-arial-iso-8859-2.tar.bz2&want=font-arial-iso-8859-1.tar.bz2&version=0.5.2&mirror=mplayer&path=/releases/fonts/
=> `deltup.php?have=font-arial-iso-8859-2.tar.bz2&want=font-arial-iso-8859-1.tar.bz2&version=0.5.2&mirror=mplayer&path=%2Freleases%2Ffonts%2F'
Resolving linux01.gwdg.de... 134.76.13.21
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://linux01.gwdg.de/~nlissne/deltas/font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu [following]
--19:23:16-- http://linux01.gwdg.de/%7Enlissne/deltas/font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu
=> `font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu'
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 113,958 [application/x-bzip2]
100%[=============================================================>] 113,958 72.69K/s
19:23:17 (72.52 KB/s) - `font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu' saved [113958/113958]
GOT font-arial-iso-8859-2.tar.bz2-font-arial-iso-8859-1.tar.bz2.dtu
Yes, we got the dtu-file now let's build font-arial-iso-8859-1.tar.bz2...
font-arial-iso-8859-2.tar.bz2 -> font-arial-iso-8859-1.tar.bz2: OK
cleaning up
!!! Fetched file: font-arial-iso-8859-1.tar.bz2 MD5 FAILED! Removing corrupt distfile...
!!! Couldn't download font-arial-iso-8859-1.tar.bz2. Aborting.
!!! Fetch for /usr/portage/media-video/mplayer/mplayer-1.0_pre4-r7.ebuild failed, continuing...
!!! Some fetch errors were encountered. Please see above for details.
|
|
|
Back to top |
|
|
GurliGebis Retired Dev
Joined: 08 Aug 2002 Posts: 509
|
Posted: Mon Aug 30, 2004 8:26 pm Post subject: |
|
|
hehe, it seems to have a problem selecting the right file to diff from:
Code: | Searching for a previosly downloaded file in /usr/portage/distfiles
We have following candidates to choose from
libxslt-1.1.8.tar.bz2
metalog-0.8_pre20031130.tar.gz
ncurses-5.4.tar.gz
net-snmp-5.1.1.tar.gz
openvpn-1.6.0.tar.gz
rc-scripts-1.5.2.tar.bz2
slocate-2.7.tar.gz
texinfo-4.7.tar.bz2
vim-6.3-lang.tar.gz
The best of all is ... vim-6.3-lang.tar.gz
Trying to download vim-6.3-lang.tar.gz-libxslt-1.1.9.tar.bz2.dtu |
_________________ Queen Rocks. |
|
Back to top |
|
|
TPC Tux's lil' helper
Joined: 16 Sep 2003 Posts: 135 Location: Sweden
|
Posted: Tue Aug 31, 2004 6:40 am Post subject: |
|
|
This needs to be fixed for slotted packages..
I had version 2.x of a slotted package, but then I tried to emerge a program that depended on 1.x, so it tried to get the 2.x to 1.x dtu (I aborted and downloaded manuall).
If it should do this or not, I'm not to decide (because I have no idea if this saves space or not), but it shouldn't delete the other version if you have set it to delete old files when new are created, for obvious reasons. |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Tue Aug 31, 2004 11:04 am Post subject: |
|
|
GurliGebis wrote: | hehe, it seems to have a problem selecting the right file to diff from:
Code: | Searching for a previosly downloaded file in /usr/portage/distfiles
We have following candidates to choose from
libxslt-1.1.8.tar.bz2
metalog-0.8_pre20031130.tar.gz
ncurses-5.4.tar.gz
net-snmp-5.1.1.tar.gz
openvpn-1.6.0.tar.gz
rc-scripts-1.5.2.tar.bz2
slocate-2.7.tar.gz
texinfo-4.7.tar.bz2
vim-6.3-lang.tar.gz
The best of all is ... vim-6.3-lang.tar.gz
Trying to download vim-6.3-lang.tar.gz-libxslt-1.1.9.tar.bz2.dtu |
|
What? Which version of getdelta.sh are you using?
Is that patched? I know the alg. used to find the candidates sometimes fails but this example should never happen. What getdelta.sh does is:
1. remove any numbers and some special chars from the names
2. compare the results - and allow a maximum of 1 char differ
This point obviously failed in your example.
3. compare the candidates given by (2.) with exact filename (special-chars and numbers included) -- Again this failed in your example.
I've never seen that behaviour before (anyway the server would block a request for vim-6.3-lang.tar.gz-libxslt-1.1.9.tar.bz2.dtu as obviously INVALID.
I have seen false positives like trying to update libmcrypt from an existing libgcrypt, but *your* example does not match the rules above.
So you must have changed your getdeltas.sh manually - did you?
bp |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Tue Aug 31, 2004 11:14 am Post subject: |
|
|
pleusicles wrote: | First of all, thanks for this util, it's wonderful!
I just found a problem regarding the search for previously dl'ed distfiles (yes, I know it's tricky as stated in the script...):
I tried to upgrade mplayer, which includes downloading the following files:
font-arial-iso-8859-1.tar.bz2 and
font-arial-iso-8859-2.tar.bz2
|
Although this looks a bit funny - it's not wrong at all - these two files are similar enough to make a dtu - and actually you've saved 51.47% downloadsize.
And, NO, the MD5SUM is exactly that of the file the server got when created the patch. Deltup ensures this by including the md5sum of both - the source and the destination file in the dtu itself. And since you got an "OK" anything went right - according to deltup.
So why did you get an MD5SUM error then - probably the file has changed on mplayerhq.com - check bugs.gentoo.org if other users who do not use getdelta.sh have the same problem - I guess the MD5SUM stored in portage tree does not match the file from the mplayer's site anymore.
The only problem with the files above is:
If you would use getdelta.sh with REMOVE_OLD=true it would delete the "old" file -- and you would fail later since the file is missing. The only way to prevent this at the moment is to touch the files in your $DISTDIR. If they exist (even with 0 bytes) portage uses RESUMECOMMAND insteadt of FETCHCOMMAND - so it does not use getdelta.sh for the file and anything will be ok. (anyway, according to your output you did not use REMOVE_OLD, so this workaround is addressed to people who do)
bp |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Tue Aug 31, 2004 11:21 am Post subject: |
|
|
TPC wrote: | This needs to be fixed for slotted packages..
I had version 2.x of a slotted package, but then I tried to emerge a program that depended on 1.x, so it tried to get the 2.x to 1.x dtu (I aborted and downloaded manuall). |
Downgrades are ok with deltup, too. Often enough it still saves downloadsize, as long as there are enough parts of the source-code that did not change.
However you do not want these files deleted, and the only way to prevent this at the moment is to touch the wanted filename in your distfiles directory. When portage finds the touched file it wont use getdelta.sh, but use your RESUMECOMMAND instead.
This is a dirty workaround, I know. I will include a feature with a user-defined exception-list where users can put in filenames they dont want to be deleted by the REMOVE_OLD feature. But this is a future-tune -- so use the the workaround until then.
Thanks.
bp |
|
Back to top |
|
|
MaxDamage l33t
Joined: 03 Jan 2004 Posts: 650 Location: Oviedo, Spain
|
Posted: Tue Aug 31, 2004 12:47 pm Post subject: |
|
|
It's working very well here. Only tried three emerges, time will say if it works well 100%. _________________ La PDA de tungsteno |
|
Back to top |
|
|
GurliGebis Retired Dev
Joined: 08 Aug 2002 Posts: 509
|
Posted: Tue Aug 31, 2004 2:55 pm Post subject: |
|
|
Haven't changed my getdelta.sh .
Used the quied step by step, so I have no clue why this happends, but I'll update getdelta.sh when the next version comes out. _________________ Queen Rocks. |
|
Back to top |
|
|
pleusicles n00b
Joined: 26 May 2004 Posts: 44
|
Posted: Tue Aug 31, 2004 8:03 pm Post subject: |
|
|
Quote: | Although this looks a bit funny - it's not wrong at all - these two files are similar enough to make a dtu - and actually you've saved 51.47% downloadsize. |
Yes, but the script downloaded the dtu 5 times
blackpenguin wrote: |
So why did you get an MD5SUM error then - probably the file has changed on mplayerhq.com - check bugs.gentoo.org if other users who do not use getdelta.sh have the same problem - I guess the MD5SUM stored in portage tree does not match the file from the mplayer's site anymore.
|
I checked it; md5sum seems to be correct in portage. BTW, the files are downloaded correctly and verified if getdelta.sh is disabled. However, the problem is reproducable when using getdelta.sh
It would be interesting to know if others who DO use getdelta.sh have the same problem; so, if anyone is interested, please
Code: | emerge -f =media-video/mplayer-1.0_pre4-r7 |
... with no font-arial-iso-8859-*.tar.bz2 in DISTFILES.
Thanks in advance,
Pleusicles |
|
Back to top |
|
|
tomaw Guru
Joined: 26 Mar 2003 Posts: 429 Location: UK
|
Posted: Tue Aug 31, 2004 8:13 pm Post subject: |
|
|
Mine works fine here fetching mplayer, here's the output:
Code: |
tom ~ # rm /usr/portage-distfiles/font-arial-iso-8859-*
tom ~ # vi /etc/make.conf
tom ~ # emerge -f =media-video/mplayer-1.0_pre4-r7 --oneshot
Calculating dependencies ...done!
>>> emerge (1 of 1) media-video/mplayer-1.0_pre4-r7 to /
>>> Previously fetched file: Blue-1.4.tar.bz2 MD5 ;-)
>>> Downloading ftp:///ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage-distfiles
No old version of the requested file found.
The dtu could not be fetched, downloading full file from original URL
ftp:///ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2: Empty host.
>>> Downloading ftp://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage-distfiles
No old version of the requested file found.
The dtu could not be fetched, downloading full file from original URL
--21:10:37-- ftp://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2
=> `font-arial-iso-8859-1.tar.bz2'
Resolving mirrors.sec.informatik.tu-darmstadt.de... 130.83.164.194
Connecting to mirrors.sec.informatik.tu-darmstadt.de[130.83.164.194]:21... failed: Connection refused.
>>> Downloading http://ftp.snt.utwente.nl/pub/os/linux/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2
Checking if a Thirdparty-Mirror is used ...
Chosen Mirror is gentoo
Searching for a previosly downloaded file in /usr/portage-distfiles
No old version of the requested file found.
The dtu could not be fetched, downloading full file from original URL
--21:10:39-- http://ftp.snt.utwente.nl/pub/os/linux/gentoo/distfiles/font-arial-iso-8859-1.tar.bz2
=> `font-arial-iso-8859-1.tar.bz2'
Resolving ftp.snt.utwente.nl... 130.89.175.1, 2001:610:1908:8000::175:1
Connecting to ftp.snt.utwente.nl[130.89.175.1]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 234,242 [application/x-tar]
100%[==========================================================================================================================>] 234,242 171.74K/s
21:10:42 (171.18 KB/s) - `font-arial-iso-8859-1.tar.bz2' saved [234242/234242]
>>> font-arial-iso-8859-1.tar.bz2 MD5 ;-)
>>> Previously fetched file: svgalib_helper-1.9.17-mplayer.tar.bz2 MD5 ;-)
>>> Downloading ftp:///ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage-distfiles
We have following candidates to choose from
MPlayer-1.0pre5.tar.bz2
The best of all is ... MPlayer-1.0pre5.tar.bz2
Trying to download MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.dtu
--21:10:42-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2
=> `deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2'
Resolving linux01.gwdg.de... 134.76.13.21
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://linux01.gwdg.de/~nlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed [following]
--21:10:42-- http://linux01.gwdg.de/%7Enlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
=> `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed'
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [application/x-bzip2]
100%[==========================================================================================================================>] 45 --.--K/s
21:10:42 (439.45 KB/s) - `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed' saved [45/45]
GOT MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
$
{RED}The server could not build the dtu-file for MPlayer-1.0pre4.tar.bz2
reason:
sorry, could not get MPlayer-1.0pre5.tar.bz2
The dtu could not be fetched, downloading full file from original URL
ftp:///ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2: Empty host.
>>> Downloading ftp://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage-distfiles
We have following candidates to choose from
MPlayer-1.0pre5.tar.bz2
The best of all is ... MPlayer-1.0pre5.tar.bz2
Trying to download MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.dtu
--21:10:46-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2
=> `deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2'
Resolving linux01.gwdg.de... 134.76.13.21
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://linux01.gwdg.de/~nlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed [following]
--21:10:46-- http://linux01.gwdg.de/%7Enlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
=> `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed'
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [application/x-bzip2]
100%[==========================================================================================================================>] 45 --.--K/s
21:10:46 (439.45 KB/s) - `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed' saved [45/45]
GOT MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
$
{RED}The server could not build the dtu-file for MPlayer-1.0pre4.tar.bz2
reason:
sorry, could not get MPlayer-1.0pre5.tar.bz2
The dtu could not be fetched, downloading full file from original URL
--21:10:46-- ftp://mirrors.sec.informatik.tu-darmstadt.de/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2
=> `MPlayer-1.0pre4.tar.bz2'
Resolving mirrors.sec.informatik.tu-darmstadt.de... 130.83.164.194
Connecting to mirrors.sec.informatik.tu-darmstadt.de[130.83.164.194]:21... failed: Connection refused.
>>> Downloading http://ftp.snt.utwente.nl/pub/os/linux/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2
Checking if a Thirdparty-Mirror is used ...
Chosen Mirror is gentoo
Searching for a previosly downloaded file in /usr/portage-distfiles
We have following candidates to choose from
MPlayer-1.0pre5.tar.bz2
The best of all is ... MPlayer-1.0pre5.tar.bz2
Trying to download MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.dtu
--21:10:47-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2&mirror=gentoo&path=/
=> `deltup.php?have=MPlayer-1.0pre5.tar.bz2&want=MPlayer-1.0pre4.tar.bz2&version=0.5.2&mirror=gentoo&path=%2F'
Resolving linux01.gwdg.de... 134.76.13.21
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://linux01.gwdg.de/~nlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed [following]
--21:10:47-- http://linux01.gwdg.de/%7Enlissne/deltas/MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
=> `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed'
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 45 [application/x-bzip2]
100%[==========================================================================================================================>] 45 --.--K/s
21:10:47 (439.45 KB/s) - `MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed' saved [45/45]
GOT MPlayer-1.0pre5.tar.bz2-MPlayer-1.0pre4.tar.bz2.failed
$
{RED}The server could not build the dtu-file for MPlayer-1.0pre4.tar.bz2
reason:
sorry, could not get MPlayer-1.0pre5.tar.bz2
The dtu could not be fetched, downloading full file from original URL
--21:10:47-- http://ftp.snt.utwente.nl/pub/os/linux/gentoo/distfiles/MPlayer-1.0pre4.tar.bz2
=> `MPlayer-1.0pre4.tar.bz2'
Resolving ftp.snt.utwente.nl... 130.89.175.1, 2001:610:1908:8000::175:1
Connecting to ftp.snt.utwente.nl[130.89.175.1]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4,913,390 [application/x-tar]
100%[==========================================================================================================================>] 4,913,390 123.39K/s ETA 00:00
21:11:17 (162.69 KB/s) - `MPlayer-1.0pre4.tar.bz2' saved [4913390/4913390]
>>> MPlayer-1.0pre4.tar.bz2 MD5 ;-)
>>> Downloading ftp:///ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/distfiles/font-arial-iso-8859-2.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage-distfiles
We have following candidates to choose from
font-arial-iso-8859-1.tar.bz2
The best of all is ... font-arial-iso-8859-1.tar.bz2
Trying to download font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu
|
_________________ Tom Wesley |
|
Back to top |
|
|
pleusicles n00b
Joined: 26 May 2004 Posts: 44
|
Posted: Tue Aug 31, 2004 9:52 pm Post subject: |
|
|
tomaw: thanks for trying. The only difference I see is that - for some reason unknown to me - in your case font-arial-iso-8859-1.tar.bz2 is downloaded prior to -2.tar.bz2, while on my machine the reverse is true.
I tried replicate your case by
1. fetching mplayer with getdelta.sh disabled; both font files were downloaded successfully.
2. Then I deleted -2.tar.bz2 only,
3. then enabled getdelta.sh and did an emerge -f again:
Code: |
>>> emerge (1 of 1) media-video/mplayer-1.0_pre4-r7 to /
>>> Previously fetched file: svgalib_helper-1.9.17-mplayer.tar.bz2 MD5 ;-)
>>> Downloading http://gentoo.osuosl.org/distfiles/font-arial-iso-8859-2.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /store/distfiles
We have following candidates to choose from
font-arial-iso-8859-1.tar.bz2
The best of all is ... font-arial-iso-8859-1.tar.bz2
Trying to download font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu
--23:22:20-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=font-arial-iso-8859-1.tar.bz2&want=font-arial-iso-8859-2.tar.bz2&version=0.5.2
=> `deltup.php?have=font-arial-iso-8859-1.tar.bz2&want=font-arial-iso-8859-2.tar.bz2&version=0.5.2'
Resolving linux01.gwdg.de... 134.76.13.21
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://linux01.gwdg.de/~nlissne/deltas/font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu [following]
--23:22:20-- http://linux01.gwdg.de/%7Enlissne/deltas/font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu
=> `font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu'
Connecting to linux01.gwdg.de[134.76.13.21]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104,815 [application/x-bzip2]
100%[=============================================================>] 104,815 89.16K/s
23:22:21 (88.90 KB/s) - `font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu' saved [104815/104815]
GOT font-arial-iso-8859-1.tar.bz2-font-arial-iso-8859-2.tar.bz2.dtu
Yes, we got the dtu-file now let's build font-arial-iso-8859-2.tar.bz2...
font-arial-iso-8859-1.tar.bz2 -> font-arial-iso-8859-2.tar.bz2: previous package is corrupt
cleaning up
The dtu could not be fetched, downloading full file from original URL
--23:22:22-- http://gentoo.osuosl.org/distfiles/font-arial-iso-8859-2.tar.bz2
=> `font-arial-iso-8859-2.tar.bz2'
Resolving gentoo.osuosl.org... 140.211.166.134
Connecting to gentoo.osuosl.org[140.211.166.134]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 222,208 [application/x-tar]
100%[=============================================================>] 222,208 64.52K/s ETA 00:00
23:22:26 (64.37 KB/s) - `font-arial-iso-8859-2.tar.bz2' saved [222208/222208]
>>> font-arial-iso-8859-2.tar.bz2 MD5 ;-)
>>> Previously fetched file: font-arial-iso-8859-1.tar.bz2 MD5 ;-)
>>> Previously fetched file: mplayer-1.0_pre4-mga-kernel2.6.patch.tar.bz2 MD5 ;-)
>>> Previously fetched file: MPlayer-1.0pre4.tar.bz2 MD5 ;-)
>>> md5 src_uri ;-) MPlayer-1.0pre4.tar.bz2
>>> md5 src_uri ;-) font-arial-iso-8859-1.tar.bz2
>>> md5 src_uri ;-) font-arial-iso-8859-2.tar.bz2
>>> md5 src_uri ;-) mplayer-1.0_pre4-mga-kernel2.6.patch.tar.bz2
>>> md5 src_uri ;-) svgalib_helper-1.9.17-mplayer.tar.bz2
|
It seems that portage accepted the md5sum of -1.tar.bz2, but deltup found the same file to be corrupt
("previous package is corrupt"). Interesting... |
|
Back to top |
|
|
mikenerone n00b
Joined: 11 Feb 2004 Posts: 22 Location: San Antonio, TX
|
Posted: Wed Sep 01, 2004 6:43 am Post subject: |
|
|
blackpenguin wrote: | riksta wrote: | forgive me if i am wrong, but why not just use RSYNC for this? |
Uhm.... what? I am not sure if I catch your idea. - If you mean to unpack the local old version to somewhere and then rsync with the wanted version - well - that cannot be done with a downloadwrapper since you will fail to get the same MD5SUM again when compressing back to an archive again. |
I think riksta may have meant the same thing that occurred to me (but I, too, am not sure this would work the way I think): After finding a best candidate for a binary diff, couldn't portage (or a FETCHCOMMAND) simply copy that candidate to the new filename in distfiles, then "update" that file via rsync from a distfile mirror? |
|
Back to top |
|
|
PrakashP Veteran
Joined: 27 Oct 2003 Posts: 1249 Location: C.C.A.A., Germania
|
Posted: Wed Sep 01, 2004 7:13 am Post subject: |
|
|
Grrr, I still have the problem that the server doesn't find files to generate the dtu.
In my case it was with imlib2-1.1.1 to imlib2-1.1.2 and with ut2004.
Code: |
Calculating dependencies ...done!
>>> emerge (1 of 1) games-fps/ut2004-3270 to /
>>> Downloading http://mirror1.icculus.org/ut2004/ut2004-lnxpatch3270-1.tar.bz2
Checking if a Thirdparty-Mirror is used ...
No third-party mirror.
Searching for a previosly downloaded file in /usr/portage/distfiles
We have following candidates to choose from
ut2004-lnxpatch3236-1.tar.bz2
The best of all is ... ut2004-lnxpatch3236-1.tar.bz2
Trying to download ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.dtu
--09:09:51-- http://linux01.gwdg.de/%7Enlissne/deltup.php?have=ut2004-lnxpatch3236-1.tar.bz2&want=ut2004-lnxpatch3270-1.tar.bz2&version=0.5.2
=> `deltup.php?have=ut2004-lnxpatch3236-1.tar.bz2&want=ut2004-lnxpatch3270-1.tar.bz2&version=0.5.2'
Auflösen des Hostnamen »linux01.gwdg.de«.... 134.76.13.21
Verbindungsaufbau zu linux01.gwdg.de[134.76.13.21]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: http://linux01.gwdg.de/~nlissne/deltas/ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.failed[folge]
--09:09:51-- http://linux01.gwdg.de/%7Enlissne/deltas/ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.failed
=> `ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.failed'
Verbindungsaufbau zu linux01.gwdg.de[134.76.13.21]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 51 [application/x-bzip2]
100%[==================================================================>] 51 --.--K/s
09:09:51 (498.05 KB/s) - »ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.failed« gespeichert [51/51]
GOT ut2004-lnxpatch3236-1.tar.bz2-ut2004-lnxpatch3270-1.tar.bz2.failed
The server could not build the dtu-file for ut2004-lnxpatch3270-1.tar.bz2
reason:
sorry, could not get ut2004-lnxpatch3236-1.tar.bz2
The dtu could not be fetched, downloading full file from original URL
--09:09:51-- http://mirror1.icculus.org/ut2004/ut2004-lnxpatch3270-1.tar.bz2
=> `ut2004-lnxpatch3270-1.tar.bz2'
Auflösen des Hostnamen »mirror1.icculus.org«.... 216.239.132.108
Verbindungsaufbau zu mirror1.icculus.org[216.239.132.108]:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 19,664,589 [application/octet-stream]
0% [ ] 20,112 26.36K/s
|
I checked it and the older version of ut2004 is downladable at that location, so I wonder why the server isn't able to do so.
BTW, I set GENTOO_MIRRORS="". |
|
Back to top |
|
|
Nominus n00b
Joined: 15 Feb 2004 Posts: 17
|
Posted: Wed Sep 01, 2004 8:49 pm Post subject: |
|
|
I have noticed that detlup producing algoritm isn't enouth efficient as it could be, for example today the server failed to produce size-efficient delta for 5 files, and i had to download xdeltas from http://gd.tuwien.ac.at/opsys/linux/sorcerer/xdeltas/ which are still size-efficient with same files. |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Wed Sep 01, 2004 11:12 pm Post subject: |
|
|
Nominus wrote: | I have noticed that detlup producing algoritm isn't enouth efficient as it could be, for example today the server failed to produce size-efficient delta for 5 files, and i had to download xdeltas from http://gd.tuwien.ac.at/opsys/linux/sorcerer/xdeltas/ which are still size-efficient with same files. |
You should know that deltup is just a wrapper for xdelta-files - actually the difference is done by xdelta. Therefore it's hard to believe what you say. (but it's ok, if you just wanted to give another source of delta-files - we never can have enough of good links)
You should also see, that deltup is md5-aware - that means you do not only get a file with the same archive-content - it's bit-identical - which it is not with xdelta only - at least not when used with *.gz files. But we need a bit-identical file to pass the MD5 check of portage (and actually to be sure the file haven't changed on the way)
However, feel *free* to use another source of deltas - just do what you feel what you need to do.
bp |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Thu Sep 02, 2004 12:12 am Post subject: |
|
|
PrakashKC wrote: | Grrr, I still have the problem that the server doesn't find files to generate the dtu.
In my case it was with imlib2-1.1.1 to imlib2-1.1.2 and with ut2004.
...
I checked it and the older version of ut2004 is downladable at that location, so I wonder why the server isn't able to do so.
BTW, I set GENTOO_MIRRORS="". |
about imlib2-1.1.1 to imlib2-1.1.2 - I wonder what you mean, since there is a dtu for that on the server,
about ut2004 - as you might have seen in your output, it says "No third party mirror" - what that means is it didn't find any mirror-group for your URL - and the server tries to download it from a list of known gentoo-mirrors then. But they do not mirror ut2004 -- you cannot download arbitrary files via the deltup-server.
getdelta.sh tries to find the correct mirror-group (therefore it makes sense to set GENTOO_MIRRORS="") and gives the server a clue where to find the wanted file with that. ut2004 is not available on any of the known mirrors - neither the gentoo mirrors, nor any of the third-party-mirrors.
So that's the reason why the server could not find the file (the original URL is not passed to the server - and it would not make sense to do so)
bp |
|
Back to top |
|
|
blackpenguin n00b
Joined: 09 Mar 2004 Posts: 43 Location: Germany
|
Posted: Thu Sep 02, 2004 12:38 am Post subject: |
|
|
pleusicles wrote: |
It seems that portage accepted the md5sum of -1.tar.bz2, but deltup found the same file to be corrupt
("previous package is corrupt"). Interesting... |
Neither portage nor deltup is wrong in that case. As you can check on bugs.gentoo.org - the font-files on mplayerhq change every few months without any obvious reason - therefore the md5sum often changes.
"previous package is corrupt" means, the file the server got from mplayerhq is not the one you had in your distfiles already, but you have the one, that the package-maintainer has, too.
I dont know why the mplayer-team change these files from time to time (this might be just an untar and tar again) - but they do - it happens again and again that the portage-md5sum does not match especially for these 2 files. Not much we can do about this. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|