Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Shake your files for their own good
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Tue Aug 15, 2006 9:59 pm    Post subject: Reply with quote

Sorry, I wasn't at home :?

Quote:
It should roll right over /dev and /proc just fine. Just don't run it over any libs that you need to run... ESPECIALLY libc. In fact, just avoid doing it on a running system. It's safer :)
Are you sure that your problems resulted from a Shake bug ? smartctl and memtest didn't repported anything ?

I'm about to mark the new version of shake (0.23) as a stable one (no longer alpha), and proposed it for inclusion in Portage, so it's important for me that it works flawlessly (old bugs are fixed btw)

If it still had a bug, I am really confused :oops: . I'll do all that I can to repair systems broke by Shake, if there is such. Just drop me a mail at the address given in "shake --help".

That said, those who wants to use the latest version can do it through the ebuild available there (Changelog). It should be safe to run it while the system is used (except maybe by nfs, due to locks issues).

Quote:
I understand that it would be best to run it on a non-system partition, but are there specific reasons /not/ to run it on my active system partitions?
Some files used by Shake (like parts of the glibc) cannot be shocked while the system run.

Quote:
Will it properly skip /dev/ and /proc/, since they're not real filesystems
Yes, but it's greately advised to run shake with "--one-filesystem" to also avoid fat/ntfs partition. "--one-filesystem" will be the default for Shake 0.24

Last edited by 98111 on Fri Aug 18, 2006 10:56 pm; edited 1 time in total
Back to top
View user's profile Send private message
BlueFusion
Guru
Guru


Joined: 08 Mar 2006
Posts: 320

PostPosted: Wed Aug 16, 2006 1:03 am    Post subject: Reply with quote

I ran over a running system and it went fine. The only problems were with the files in use, and that was just saying it could not move them. When running chroot'd from a LiveCD, all but like 3 or 4 files could not be shook (bash, shake, and a few libs). No issues with killing the syste, however.
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Wed Aug 16, 2006 4:28 am    Post subject: Reply with quote

Possibly you could have it load needed files to swap before running, so that you could run it on an active partition to keep it from going nuts over a file being in use.
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
Janne Pikkarainen
Veteran
Veteran


Joined: 29 Jul 2003
Posts: 1143
Location: Helsinki, Finland

PostPosted: Wed Aug 16, 2006 8:36 am    Post subject: Reply with quote

This is a "thank you" message. :D

Most of the time I'm a Gentoo user, but my laptop is running Ubuntu Dapper. I just tried to shake its HD with shake v0.23 and cool, it works! I took a brave route and shaked *everything* (ok, just with sudo shake --verbose / without fancy flags, but still). Since HD of this poor thing has every now and then been practically full and I test all kind of stuff with it all the time, and this system has even been upgraded from Hoary to Breezy to Dapper, filesystem (ext3) had became a bit cluttered.

Previously during bootup HD light would flash and HD was seeking a lot, at least it sounded like that. After running shake a reboot was a lot smoother: light was practically lit and background seeking noise was almost gone. I don't know about the boot time though, it's always been pretty snappy.

e17 and all the applications I've loaded so far (xterm, Firefox, Thunderbird) seemed to load faster and with lot less seeking, too. Yay!
_________________
Yes, I'm the man. Now it's your turn to decide if I meant "Yes, I'm the male." or "Yes, I am the Unix Manual Page.".
Back to top
View user's profile Send private message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3490

PostPosted: Wed Aug 16, 2006 8:48 am    Post subject: Reply with quote

Has somebody test it on a (dm-) encrypted partition? I think it's safe (because it is transparency), but I want not to try it if there are risks (for the important data on the encrypted partition).
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Wed Aug 16, 2006 2:45 pm    Post subject: Reply with quote

Dralnu wrote:
Possibly you could have it load needed files to swap before running, so that you could run it on an active partition to keep it from going nuts over a file being in use.
File used by Shake are locked when it run, so it won't broke them, at least :P
Linking static should do the trick. I don't think that's really usefull but I added a shake-static target to the future 0.24 Makefile :wink: .

Janne Pikkarainen wrote:
Previously during bootup HD light would flash and HD was seeking a lot, at least it sounded like that. After running shake a reboot was a lot smoother: light was practically lit and background seeking noise was almost gone. I don't know about the boot time though, it's always been pretty snappy.

e17 and all the applications I've loaded so far (xterm, Firefox, Thunderbird) seemed to load faster and with lot less seeking, too. Yay!
Thanks :D

slick wrote:
Has somebody test it on a (dm-) encrypted partition? I think it's safe (because it is transparency), but I want not to try it if there are risks (for the important data on the encrypted partition).
It should be the same for Shake, but I did not tried. Please tell me if you do.
Back to top
View user's profile Send private message
Diredicker
n00b
n00b


Joined: 26 Apr 2006
Posts: 31
Location: NL

PostPosted: Sat Aug 19, 2006 1:19 pm    Post subject: Reply with quote

I tried to emerge shake but it wont compile, getting this error :(
Anyone know what to do?

Code:
executive.c: In function `fcopy':
executive.c:64: error: passing arg 2 of `ioctl' with different width due to prototype
executive.c:146: error: passing arg 3 of `write' with different width due to prototype
executive.c: In function `shake_reg':
executive.c:232: error: passing arg 3 of `fcopy' with different width due to prototype
executive.c:246: error: passing arg 3 of `fcopy' with different width due to prototype
executive.c: In function `list_dir':
executive.c:346: error: passing arg 2 of `qsort' with different width due to prototype
executive.c: In function `list_stdin':
executive.c:405: error: passing arg 2 of `qsort' with different width due to prototype
judge.c: In function `judge_list':
judge.c:245: error: passing arg 1 of `abs' with different width due to prototype
judge.c:247: error: passing arg 1 of `abs' with different width due to prototype
judge.c:252: error: passing arg 1 of `abs' with different width due to prototype
make: *** [executive.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [judge.o] Error 1

!!! ERROR: sys-apps/shake-0.23 failed.
Call stack:
  ebuild.sh, line 1539:   Called dyn_compile
  ebuild.sh, line 939:   Called src_compile
  ebuild.sh, line 609:   Called die

!!! emake failed
!!! If you need support, post the topmost build error, and the call stack if relevant.

!!! This ebuild is from an overlay: '/usr/local/overlays'
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Sat Aug 19, 2006 7:09 pm    Post subject: Reply with quote

Diredicker wrote:
I tried to emerge shake but it wont compile, getting this error :(
Anyone know what to do?

Code:
executive.c: In function `fcopy':
executive.c:64: error: passing arg 2 of `ioctl' with different width due to prototype
executive.c:146: error: passing arg 3 of `write' with different width due to prototype
[...]
That bug again... sorry. It was an abusive use of -Wconversion .
Shake 0.24 (tarball, Changelog, ebuild) fix this problem, add "static" useflag and makes "--one-file-system" the default.
Please confirm me that you are on a non-x86_32 system, and that it fixed the bug.
Back to top
View user's profile Send private message
zxy
Veteran
Veteran


Joined: 06 Jan 2006
Posts: 1160
Location: in bed in front of the computer

PostPosted: Sun Aug 20, 2006 1:28 pm    Post subject: Reply with quote

I would like to try shake.

I have a list of files that I precache while booting. Is it possible to shake just these files. Will they be put together on the disk? Then boot should be done in a flash :D, I hope.

What is the command line for this. I have all files listed in a file. Each file in it's own line.
_________________
Nature does not hurry, yet everything is accomplished.
Lao Tzu
Back to top
View user's profile Send private message
zxy
Veteran
Veteran


Joined: 06 Jan 2006
Posts: 1160
Location: in bed in front of the computer

PostPosted: Sun Aug 20, 2006 2:27 pm    Post subject: Reply with quote

I just tried shake.

1. user_xattr makes my reiser4 partitions not being able to mount
2. reiserfs partitions boot ok with user_xattr
3. I tried shake -pvv on a file that is on a reiserfs partition. No results.
I checked first with -pvv
Code:
shake -pvv /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2
IDEAL   START   END     FRAGC   CRUMBC  AGE     SHOCKED NAME
0       1058804 1077680 6       0       0       0       /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2

after without -pvv
Code:

localhost # shake  /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2
localhost # shake -pvv /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2
IDEAL   START   END     FRAGC   CRUMBC  AGE     SHOCKED NAME
0       1058804 1077680 6       0       0       0       /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2

Why are there still 6 fragments?

----EDIT -----
I had to be on the same partition when starting shake.This way it works.
But it still lets 17 fragments on other 236K file. It does nothing even if I run shake many times. :(
_________________
Nature does not hurry, yet everything is accomplished.
Lao Tzu
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Sun Aug 20, 2006 3:08 pm    Post subject: Reply with quote

zxy wrote:
Code:

localhost # shake  /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2
localhost # shake -pvv /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2
IDEAL   START   END     FRAGC   CRUMBC  AGE     SHOCKED NAME
0       1058804 1077680 6       0       0       0       /var/cache/http-replicator/apache-ant-1.6.5-src.tar.bz2

Why are there still 6 fragments?

----EDIT -----
I had to be on the same partition when starting shake.This way it works.
But it still lets 17 fragments on other 236K file. It does nothing even if I run shake many times. :(
As you can see, SHOCKED=0 . This is because the file is new (AGE=0 days), and therefore they would be little or no benefits shaking it (as a fragmented new file generally mean a fragmented free space).
If you want to force it to be shocked, call
Code:
shake --old=0 FILE
and files older or as old than 0 day, that is all files, will be shocked. But the results may be negative.
For more information about advanced use of shake, please refer to http://vleu.net and to the manpage.

EDIT: If you have a list of file to be shocked, you can do
Code:
cat list | shake
to tell shake that you want them near each others, or
Code:
shake `cat list`
to tell it that they are independent.
But IMHO you won't see a great improvement between {prefetch} and {prefetch + shake}, as prefetch will reduce seeking to the lowest possible.
Back to top
View user's profile Send private message
zxy
Veteran
Veteran


Joined: 06 Jan 2006
Posts: 1160
Location: in bed in front of the computer

PostPosted: Sun Aug 20, 2006 3:39 pm    Post subject: Reply with quote

When I use reiser4 partition which cannot be mounted with xattr, I get a lot of negative numbers for start or end in the output. Is this ok?
_________________
Nature does not hurry, yet everything is accomplished.
Lao Tzu
Back to top
View user's profile Send private message
indianiec
Tux's lil' helper
Tux's lil' helper


Joined: 01 Feb 2005
Posts: 97
Location: Krakow

PostPosted: Sun Aug 20, 2006 5:51 pm    Post subject: Reply with quote

You can also check this stuff from Con Kolivas.
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Sun Aug 20, 2006 7:27 pm    Post subject: Reply with quote

zxy wrote:
When I use reiser4 partition which cannot be mounted with xattr, I get a lot of negative numbers for start or end in the output. Is this ok?
No definitively. I would like to be sure that it is not a bug in reiserfs4. Could you download http://vleu.net/shake/fb_r4.c, compile it with
Code:
gcc fb_r4.c -o fb_r4
and run it as root on a file for which ShaKe detect the start at a negative value ?
If it return a negative value, I will contact the reiser team. In both case, I would like the output of "shake -pvv" on this file.
Back to top
View user's profile Send private message
zxy
Veteran
Veteran


Joined: 06 Jan 2006
Posts: 1160
Location: in bed in front of the computer

PostPosted: Sun Aug 20, 2006 7:49 pm    Post subject: Reply with quote

@indianiec
I know for his defrag.sh, but it doesn't put files together on the disk. I want this because of faster boot.

@98111
Here you have the outputs
Code:
localhost shake # ./fb_r4 /usr/portage/distfiles/Sunken.tar.gz
FIBMAP=-22, FIGETBSZ=4096

localhost shake # shake -pvv /usr/portage/distfiles/Sunken.tar.gz
IDEAL   START   END     FRAGC   CRUMBC  AGE     SHOCKED NAME
0       -22     -22     1       0       1       0       /usr/portage/distfiles/Sunken.tar.gz


Here is also the output from fsck.reiser4:
Code:
localhost shake # fsck.reiser4 /dev/md7
*******************************************************************
This is an EXPERIMENTAL version of fsck.reiser4. Read README first.
*******************************************************************

Fscking the /dev/md7 block device.
Will check the consistency of the Reiser4 SuperBlock.
Will check the consistency of the Reiser4 FileSystem.
Continue?
(Yes/No): yes
***** fsck.reiser4 started at Mon Aug 21 22:56:27 2006
Reiser4 fs was detected on /dev/md7.
Master super block (16):
magic:          ReIsEr4
blksize:        4096
format:         0x0 (format40)
uuid:           ddccfb51-f64d-44c5-9e3e-a86ea0d243f1
label:          <none>

Format super block (17):
plugin:         format40
description:    Disk-format for reiser4.
magic:          ReIsEr40FoRmAt
flushes:        0
mkfs id:        0x4437aaf2
blocks:         1469888
free blocks:    1433177
root block:     36675
tail policy:    0x2 (smart)
next oid:       0x1233b
file count:     1378
tree height:    3
key policy:     LARGE


CHECKING STORAGE TREE
        Read nodes 741
        Nodes left in the tree 741
                Leaves of them 726, Twigs of them 14
        Time interval: Mon Aug 21 22:56:27 2006 - Mon Aug 21 22:56:28 2006
CHECKING EXTENT REGIONS.
        Read twigs 14
        Time interval: Mon Aug 21 22:56:28 2006 - Mon Aug 21 22:56:28 2006
CHECKING SEMANTIC TREE
        Found 1378 objects.
        Time interval: Mon Aug 21 22:56:28 2006 - Mon Aug 21 22:56:28 2006
***** fsck.reiser4 finished at Mon Aug 21 22:56:28 2006
Closing fs...done

FS is consistent.

_________________
Nature does not hurry, yet everything is accomplished.
Lao Tzu
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Sun Aug 20, 2006 9:43 pm    Post subject: Reply with quote

zxy wrote:
Here you have the outputs
Code:
localhost shake # ./fb_r4 /usr/portage/distfiles/Sunken.tar.gz
FIBMAP=-22, FIGETBSZ=4096

localhost shake # shake -pvv /usr/portage/distfiles/Sunken.tar.gz
IDEAL   START   END     FRAGC   CRUMBC  AGE     SHOCKED NAME
0       -22     -22     1       0       1       0       /usr/portage/distfiles/Sunken.tar.gz
For me it's a bug in Reiserfs4, as FIBMAP should never return negative values.
I'll tell them when I'll be sure (I don't have the time to look to that before tomorrow), but I'm afraid to have to tell you that you should not use shake on reiserfs4 before the problem is fixed.
Back to top
View user's profile Send private message
Diredicker
n00b
n00b


Joined: 26 Apr 2006
Posts: 31
Location: NL

PostPosted: Mon Aug 21, 2006 7:43 am    Post subject: Reply with quote

98111 wrote:
Diredicker wrote:
I tried to emerge shake but it wont compile, getting this error :(
Anyone know what to do?

Code:
executive.c: In function `fcopy':
executive.c:64: error: passing arg 2 of `ioctl' with different width due to prototype
executive.c:146: error: passing arg 3 of `write' with different width due to prototype
[...]
That bug again... sorry. It was an abusive use of -Wconversion .
Shake 0.24 (tarball, Changelog, ebuild) fix this problem, add "static" useflag and makes "--one-file-system" the default.
Please confirm me that you are on a non-x86_32 system, and that it fixed the bug.


Yes i forgot to mention i'm using an AMD64 based system, and it emerged just fine now :lol:
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Wed Aug 30, 2006 8:14 pm    Post subject: Reply with quote

zxy wrote:
@98111
Here you have the outputs
Code:
localhost shake # ./fb_r4 /usr/portage/distfiles/Sunken.tar.gz
FIBMAP=-22, FIGETBSZ=4096
It was a bug in reiserfs4. Gurus from namesys just fixed it there.
The latest shake (0.27: tarball, Changelog, ebuild) handle the bug, but you should wait for a new version of reiserfs4.

By the way the project is now registered at Savannah (http://savannah.nongnu.org/projects/shake/). There you'll find a bugzilla, CVS and all that :wink:.
Back to top
View user's profile Send private message
zxy
Veteran
Veteran


Joined: 06 Jan 2006
Posts: 1160
Location: in bed in front of the computer

PostPosted: Fri Sep 01, 2006 4:54 pm    Post subject: Reply with quote

thanks
I'm looking forward to new reiser4
_________________
Nature does not hurry, yet everything is accomplished.
Lao Tzu
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1305
Location: Mississippi USA

PostPosted: Tue Sep 05, 2006 8:30 am    Post subject: Reply with quote

OK. I saw where someone emerged this, where is it?

Code:
root@smoker / # emerge -s shake
Searching...
[ Results for search key : shake ]
[ Applications found : 0 ]


root@smoker / # equery list -p shake
[ Searching for package 'shake' in all categories among: ]
 * installed packages
 * Portage tree (/usr/portage)
root@smoker / #   


Just curious.

:D :D :D :D :D
_________________
My rig: Gigabyte GA-770T-USB3 mobo AMD Phenom II X4 955 3.2GHz ZALMAN CNPS10X Performa CPU cooler
G.SKILL 16GB DDR3 PC3 12800 Memory Nvidia GT-220 video card LG W2253 Monitor
WD1600AAJS & WD2502ABYS hard drives
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
yabbadabbadont
Advocate
Advocate


Joined: 14 Mar 2003
Posts: 4791
Location: 2 exits past crazy

PostPosted: Tue Sep 05, 2006 8:43 am    Post subject: Reply with quote

dalek wrote:
OK. I saw where someone emerged this, where is it?

Code:
root@smoker / # emerge -s shake
Searching...
[ Results for search key : shake ]
[ Applications found : 0 ]


root@smoker / # equery list -p shake
[ Searching for package 'shake' in all categories among: ]
 * installed packages
 * Portage tree (/usr/portage)
root@smoker / #   


Just curious.

:D :D :D :D :D

You will have to download the ebuild from http://savannah.nongnu.org/projects/shake/ and setup a portage overlay to use it.
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1305
Location: Mississippi USA

PostPosted: Tue Sep 05, 2006 12:04 pm    Post subject: Reply with quote

yabbadabbadont wrote:

You will have to download the ebuild from http://savannah.nongnu.org/projects/shake/ and setup a portage overlay to use it.


Oh. Are there plans to put it in portage?

:D :D :D :D :D
_________________
My rig: Gigabyte GA-770T-USB3 mobo AMD Phenom II X4 955 3.2GHz ZALMAN CNPS10X Performa CPU cooler
G.SKILL 16GB DDR3 PC3 12800 Memory Nvidia GT-220 video card LG W2253 Monitor
WD1600AAJS & WD2502ABYS hard drives
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
Bigun
Veteran
Veteran


Joined: 21 Sep 2003
Posts: 1974

PostPosted: Tue Sep 05, 2006 12:52 pm    Post subject: Reply with quote

Would it be better to run shake when the system is booted? Or boot from a live cd, chroot in, and run shake?

*bookmarked*
Back to top
View user's profile Send private message
BlueFusion
Guru
Guru


Joined: 08 Mar 2006
Posts: 320

PostPosted: Tue Sep 05, 2006 5:23 pm    Post subject: Reply with quote

I ran it from both a lice OS and a LiveCD. The problem with running it from a live OS is that alot of files are in use and can't be shaken (shook?) by the program. Running from a LiveCD, only 2 or 3 files were in use (shake and some libs).
Back to top
View user's profile Send private message
Bigun
Veteran
Veteran


Joined: 21 Sep 2003
Posts: 1974

PostPosted: Wed Sep 06, 2006 8:53 pm    Post subject: Reply with quote

Has someone tested this on an AMD64 system yet?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 4 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum