| View previous topic :: View next topic |
| Author |
Message |
Cintra
Advocate
Joined: 03 Apr 2004
Posts: 2111
Location: Norway
|
 Posted: Wed Apr 06, 2005 7:33 am Post subject: |
 |
|
No, no cleaners.. this is what I have in distfiles
| Code: | -rw-r--r-- 1 root portage 59 Mar 28 03:47 portage-20050327.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Mar 29 03:47 portage-20050328.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Mar 30 03:45 portage-20050329.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Mar 31 03:46 portage-20050330.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Apr 1 03:45 portage-20050331.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Apr 2 03:46 portage-20050401.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Apr 3 03:45 portage-20050402.tar.bz2.md5sum
-rw-r--r-- 1 root portage 59 Apr 4 03:47 portage-20050403.tar.bz2.md5sum
-rw-r--r-- 1 root portage 19081117 Apr 5 03:45 portage-20050404.tar.bz2
-rw-r--r-- 1 root portage 59 Apr 5 03:45 portage-20050404.tar.bz2.md5sum
-rw-r--r-- 1 root portage 19072373 Apr 6 07:08 portage-20050405.tar.bz2
-rw-r--r-- 1 root portage 59 Apr 6 07:06 portage-20050405.tar.bz2.md5sum
|
mvh
edit: I've not run the latest script yet, as I've been upgrading my kernel..
Just a mo' - I've just emerged logrotate, at the end of which I get
| Code: | >>> Regenerating /etc/ld.so.cache...
* Caching service dependencies ... [ ok ]
>>> app-admin/logrotate-3.6.5-r1 merged.
>>> Recording app-admin/logrotate in "world" favorites file...
*** Adjusting ccache permissions for portage user...
>>> clean: No packages selected for removal.
>>> Auto-cleaning packages ...
>>> No outdated packages were found on your system.
* GNU info directory index is up-to-date.
|
so perhaps I do have auto-cleaning without realizing it..?
Now where on earth would that be
_________________
"I am not bound to please thee with my answers" W.S. |
|
| Back to top |
|
 |
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Wed Apr 06, 2005 7:54 am Post subject: |
|
|
| Cintra wrote: | No, no cleaners.. this is what I have in distfiles
| Code: | ..snip...
-rw-r--r-- 1 root portage 19081117 Apr 5 03:45 portage-20050404.tar.bz2
-rw-r--r-- 1 root portage 59 Apr 5 03:45 portage-20050404.tar.bz2.md5sum
-rw-r--r-- 1 root portage 19072373 Apr 6 07:08 portage-20050405.tar.bz2
-rw-r--r-- 1 root portage 59 Apr 6 07:06 portage-20050405.tar.bz2.md5sum
|
mvh
| Code: | >>> clean: No packages selected for removal.
|
so perhaps I do have auto-cleaning without realizing it..?
Now where on earth would that be |
That's pkg cleansing (removing old pkgs that aren't slotted and N versions installed), not distfile cleansing 
Aside from that, you have two snapshots available for usage. If it doesn't pull the 05-06 patch, kindly give a yell, or pop your head into irc.freenode.net, channel #gentoo-portage, and look for me (same nick as I use here).
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
Cintra
Advocate
Joined: 03 Apr 2004
Posts: 2111
Location: Norway
|
| Posted: Wed Apr 06, 2005 5:01 pm Post subject: |
|
|
Ran an update using the latest script and all works fine now
Nothing downloaded..
| Code: | /root/emerge-delta-webrsync
Looking for available base versions for a delta
fetching patches
failed fetching snapshot-20050405-20050406.patch.bz2.md5sum
no patches found? up to date? syncing
Fetching most recent snapshot
Attempting to fetch file dated: 20050406
--- No md5sum present on the mirror. (Not yet available.)
Attempting to fetch file dated: 20050405
portage-20050405.tar.bz2: OK
=== snapshot portage-20050405.tar.bz2 is correct, using it
Syncing local tree...
building file list ...
115307 files to consider |
will try it again in the morning
mvh
_________________
"I am not bound to please thee with my answers" W.S. |
|
| Back to top |
|
|
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Thu Apr 07, 2005 3:13 am Post subject: |
|
|
| Cintra wrote: | Nothing downloaded..
| Code: | failed fetching snapshot-20050405-20050406.patch.bz2.md5sum
no patches found? up to date? syncing
Fetching most recent snapshot
Attempting to fetch file dated: 20050406
--- No md5sum present on the mirror. (Not yet available.)
Attempting to fetch file dated: 20050405
portage-20050405.tar.bz2: OK
=== snapshot portage-20050405.tar.bz2 is correct, using it
Syncing local tree... |
will try it again in the morning
mvh |
Would be wise; the snapshots are a day behind (max), so 0405 is avail on 0406. The attempting to fetch file dated: 20050405 is demonstrationg of that
I should rework the logic so it explicitly states it, although at this point, I'm hesitant to fool with emerge-delta-webrsync for a few days. It's working, mainly.
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
Cintra
Advocate
Joined: 03 Apr 2004
Posts: 2111
Location: Norway
|
| Posted: Thu Apr 07, 2005 7:06 am Post subject: |
|
|
This mornings results.. another long dl
7.4.2005 07:35 | Code: |
/root/emerge-delta-webrsync
Looking for available base versions for a delta
fetching patches
failed fetching snapshot-20050405-20050406.patch.bz2.md5sum
no patches found? up to date? syncing
Fetching most recent snapshot
Attempting to fetch file dated: 20050407
--- No md5sum present on the mirror. (Not yet available.)
Attempting to fetch file dated: 20050406
portage-20050406.tar.bz2: OK
Syncing local tree...
building file list ...
115348 files to consider
./
app-accessibility/speech-tools/
app-accessibility/speech-tools/ChangeLog
... |
can this be a mirror question?
Mine are:
| Code: | GENTOO_MIRRORS="http://mirror.gentoo.no/ ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo ftp://ftp.du.se/pub/os/gentoo"
SYNC="rsync://rsync.gentoo.no/gentoo-portage/" |
Mvh
_________________
"I am not bound to please thee with my answers" W.S. |
|
| Back to top |
|
|
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Thu Apr 07, 2005 8:15 am Post subject: |
|
|
You have a nack for digging out the catches of this script it seems...
patches are uploaded about an hour after snapshots are released; beyond that, the script *was* attempting to fallback to full fetch, rather then a direct sync of what it has. Corrected that, and re-uploaded.
Corrected a rare segfault in diffball w/ N patches also, and pushed 0.6.2 into the tree...
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
Cintra
Advocate
Joined: 03 Apr 2004
Posts: 2111
Location: Norway
|
| Posted: Thu Apr 07, 2005 8:40 am Post subject: |
|
|
| ferringb wrote: | | You have a nack for digging out the catches of this script it seems... |
Yes, it just seems to happen by osmosis 
Mvh
Edit: do I need diffball..?
_________________
"I am not bound to please thee with my answers" W.S. |
|
| Back to top |
|
|
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Thu Apr 07, 2005 8:58 am Post subject: |
|
|
| Cintra wrote: | | Edit: do I need diffball..? |
Err... yeah, it is the app that does the reconstruction
I should add a check for that, and fall back to full fetch if it's missing thinking about it...
This also would explain why full fetching always occurs for you
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
Cintra
Advocate
Joined: 03 Apr 2004
Posts: 2111
Location: Norway
|
| Posted: Thu Apr 07, 2005 10:04 am Post subject: |
|
|
| ferringb wrote: | | Cintra wrote: | | Edit: do I need diffball..? |
Err... yeah, it is the app that does the reconstruction
I should add a check for that, and fall back to full fetch if it's missing thinking about it...
This also would explain why full fetching always occurs for you |
Have just run another before emerging diffball, and it looks better
| Code: | # !426
/root/emerge-delta-webrsync
Looking for available base versions for a delta
fetching patches
failed fetching snapshot-20050406-20050407.patch.bz2.md5sum
no patches found? up to date? syncing
Syncing local tree...
building file list ...
115348 files to consider
Number of files: 115348
Number of files transferred: 0
Total file size: 90576173 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 2712961
Total bytes written: 2713014
Total bytes read: 20
wrote 2713014 bytes read 20 bytes 15114.40 bytes/sec
total size is 90576173 speedup is 33.39
cleaning up
transferring metadata/cache
skipping sync
>>> Updating Portage cache 100%
|
Anyway, I'll emerge diffball now.. this is going to be some script!
Edit: | Code: | 8.4.2005 11:41
# /root/emerge-delta-webrsync
Looking for available base versions for a delta
fetching patches
snapshot-20050406-20050407.patch.bz2: OK
failed fetching snapshot-20050407-20050408.patch.bz2.md5sum
patch_fh size=85294
patch_type=7
disabling bufferless, patch_count(1) != 1
verbosity level(1)
src_fh size=19100760
reconstruction return=0, commands=13668
result was 13668 commands
versions size is 177397760
applied 1 patches
reordering commands? 1
reconstructing target file based off of dcbuff commands...
collapsing
processing src 0: 9127 commands.
processing src 1: 4541 commands.
reconstruction completed successfully
re-compressing
portage-20050407.tar:
beginning update to the tree
Syncing local tree... |
Looks good to me...
_________________
"I am not bound to please thee with my answers" W.S. |
|
| Back to top |
|
|
ferringb
Developer
Joined: 03 Apr 2003
Posts: 355
Location: USA
|
| Posted: Sat Apr 09, 2005 3:26 pm Post subject: delta-webrsync update |
|
|
Switched the generation format, patches are small enough bdelta is edging out.
Should work transparently, if you see any funkyness give a yell please.
_________________
I don't want to be buried in a pet cemetery. ~Ramones |
|
| Back to top |
|
|
dirtyepic
Developer
Joined: 22 Oct 2004
Posts: 1614
Location: sk.ca
|
| Posted: Sat Apr 09, 2005 8:09 pm Post subject: Re: Do not download EBUILDS |
|
|
| gravis wrote: | | Legoguy wrote: | | BastianBalthazarBux wrote: | | gravis wrote: |
Download a big archive is a good idea, but not for the whole tree.
/usr/portage isn't necessary, we only need the cache. Emerge should download ebuilds on demand. |
This is a great idea IMHO, most of times when you upgrade a package downloads are needed so download 40 kb of patches and ebuild is light.
However this is a rather global change, someone know what this implies ? |
Yes. It implies breaking the entire install for someone who wants to do it without a network connection. Although in that case I guess an entire portage tree could be included on the livecd.. actually I think it already is.
Also, in the case of someone not just installing but using the system offline, they're pretty much doomed. Not that that isn't already the case... but, heh.
Great idea - but it's only plausible for people that can connect to the internet. Which is mostly all of us. |
How do you manage an installation without network today ? This is exactly the same problem : You must have some packages to populate the /usr/portage/distfiles/ dir. We can do the same with a stage (3?) version of the portage tree. ebuilds on cd would be used to install the minimum system. |
ever own a laptop? ever accidently unmerge dhcphd? ever had your internet connection go down? etc. etc.
_________________
by design, by neglect
for a fact or just for effect |
|
| Back to top |
|
|
tuxp3
n00b
Joined: 28 May 2004
Posts: 61
|
| Posted: Sat Apr 09, 2005 8:45 pm Post subject: Re: Do not download EBUILDS |
|
|
| dirtyepic wrote: | | gravis wrote: | | Legoguy wrote: | | BastianBalthazarBux wrote: | | gravis wrote: |
Download a big archive is a good idea, but not for the whole tree.
/usr/portage isn't necessary, we only need the cache. Emerge should download ebuilds on demand. |
This is a great idea IMHO, most of times when you upgrade a package downloads are needed so download 40 kb of patches and ebuild is light.
However this is a rather global change, someone know what this implies ? |
Yes. It implies breaking the entire install for someone who wants to do it without a network connection. Although in that case I guess an entire portage tree could be included on the livecd.. actually I think it already is.
Also, in the case of someone not just installing but using the system offline, they're pretty much doomed. Not that that isn't already the case... but, heh.
Great idea - but it's only plausible for people that can connect to the internet. Which is mostly all of us. |
How do you manage an installation without network today ? This is exactly the same problem : You must have some packages to populate the /usr/portage/distfiles/ dir. We can do the same with a stage (3?) version of the portage tree. ebuilds on cd would be used to install the minimum system. |
ever own a laptop? ever accidently unmerge dhcphd? ever had your internet connection go down? etc. etc. |
well, im not sure how well this works, but to get internet u could always temporarly set to static, and then get the files u need to reemerge dhcpcd (u have a typo)
also, for some of us, we keep .tbz2s of everything so fixing that problem is as simple as "emerge -avk dhcpcd".. the only time it doesnt work is when i unmerged gcc 3.3 before emergiing libstdc++-v3 , but thats me being insane.. normal people shouldnt have that happen.. also what does a laptop have to do with anything..
Thanks, Tux |
|
| Back to top |
|
|
Cinquero
Apprentice
Joined: 24 Jun 2004
Posts: 249
|
| Posted: Sun Sep 17, 2006 3:55 pm Post subject: |
|
|
| Well, it is pretty clear why this is faster than rsync: using rsync on a single file is always faster than running it over the whole portage tree. An even better solution would probably be to put portage onto a ext2 file system inside a file mounted via loop device: that way, modifications to portage would be more incremental-like (no rebuilding of existing strcutures). Clients could then mount it read-only.... |
|
| Back to top |
|
|
|
Unsupported Software
