Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Shake your files for their own good
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
mrsaccess
n00b
n00b


Joined: 03 Sep 2005
Posts: 27
Location: Greece

PostPosted: Thu Feb 01, 2007 10:49 am    Post subject: Reply with quote

Hello, congratulations for this amazing piece of software you wrote!

I used it yesterday with very good results! The speedup was really big in apps that their libraries aren't usually loaded in my ram, ie wine and gtk apps. In kde apps (kde is my desktop so it's libraries are always in the ram) the speedup was small but noticeable. Also I noticed a nice speedup in apps that use the filesystem as a cache, for example digiKam which creates thumbnails of your photos in order to load faster.

Unfortunately I ran into a small -for my occasion- problem and I want to share it in case anyone else comes across it.

Before running shake I closed the X server and switched to a virtual terminal. From there I used shake to defrag my /:
Code:
$ shake /


My fstab line for / (and most of my partitions):
Code:
/dev/sda3               /               reiserfs        defaults,noatime,notail,user_xattr      0 1

Although noatime is used it worked well.

At some point of the process (I guess 1 to 2 hours after it started) the disk activity stopped entirely and my computer wouldn't accept any input except Alt+Fnumber to switch virtual terminals. Also it didn't respond to ssh connection attempts (but it responded to ping).
After a fair amount of waiting and since there wasn't any disk activity for many minutes I reset my computer.

Upon boot I had a kernel panic because the file /lib/libc.so.6, which is part of glibc, was corrupted. I guess that is what "froze" the pc in first place.
Luckily for me I had another machine with gentoo so I copied it's libc.so.6 to my machine and everything went fine. Till now I haven't noticed any other problems (emerging deep update world right now).
Unfortunately if someone hasn't another gentoo installation near will be in a difficult position as with a corrupted glibc you can't even chroot to your system from a livecd in order to fix things.

Hope my comments will help a bit. :D
_________________
Hardware: The parts of your pc you can kick.
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Thu Feb 01, 2007 11:46 am    Post subject: Reply with quote

You're right. And the problem is serious.
I'm really sorry about damaged systems. Hopefully, it should'nt occur on all libc, but that do not excuse me.

The problem is that writing to /lib/libc-2.4.so crash the system. Even if you respect the correct order (take a write lock, then shake, then release)(imho the write lock should be refused).
Difficulty is that, in order to ensure I spot the bug, I wrote a short code wich done that :P . So my computer no longer boot.

If I remind well, older version of ld.so aquired a lock() on open libs. I don't know if the new behaviour is a bug or a feature, but shake need a workaround.
I have some ideas about it, but I first need to find a glibc guru. I must go now, but will do the update this evening (arround 19hour GMT).

I lack english words to ask you to forgive me. Thanks for your clear bugreport which allowed me to spot the problem. Also I hadn't the time to check for grammar and spelling... as you see :?
Shame on me
Back to top
View user's profile Send private message
mrsaccess
n00b
n00b


Joined: 03 Sep 2005
Posts: 27
Location: Greece

PostPosted: Thu Feb 01, 2007 12:28 pm    Post subject: Reply with quote

I really appreciate your fast response to the problem report. :D
Don't worry for the "damage", no harm done! Besides I'm too happy with the result to care about libc. :P

PS. English isn't my native language so I have to worry about the mistakes I make instead noticing other people mistakes. :D
_________________
Hardware: The parts of your pc you can kick.
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Thu Feb 01, 2007 9:41 pm    Post subject: Reply with quote

Version 0.29 (tarball, Changelog, ebuild) is available an include a quick and dirty workaround.
Will investigate more and make a 0.30 release ASAP.
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Thu Feb 01, 2007 10:27 pm    Post subject: Reply with quote

Something to think about would be to require the target fs to be unmounted (i.e. run it from a LiveCD). I would think there would be more problems running on a mounted FS then not, so that might be worth looking into.
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
getchoo
n00b
n00b


Joined: 19 Sep 2006
Posts: 38
Location: Alberta, Canada

PostPosted: Thu Feb 08, 2007 5:48 am    Post subject: Reply with quote

I'd like to try this program, but it's giving me the following error:
Code:

tux shake-0.29 # ls
COPYING    GPL.txt   distrib  executive.c  executive.o  judge.h  linux.c  main.c  msg.h
Changelog  Makefile  doc      executive.h  judge.c      judge.o  linux.h  msg.c   unattr.c
tux shake-0.29 # make
gcc -std=gnu99 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_XOPEN_SOURCE=600 -O2 -D_POSIX_C_SOURCE=200112L -Wall -pedantic-errors  -Wcast-align -Wpointer-arith -Wbad-function-cast -DVERSION=\"0.29\" -DNDEBUG -c linux.c -o linux.o
linux.c:23:43: error: attr/attributes.h: No such file or directory
linux.c: In function 'set_ptime':
linux.c:42: warning: implicit declaration of function 'attr_setf'
linux.c:43: error: 'ATTR_DONTFOLLOW' undeclared (first use in this function)
linux.c:43: error: (Each undeclared identifier is reported only once
linux.c:43: error: for each function it appears in.)
linux.c: In function 'get_ptime':
linux.c:55: warning: implicit declaration of function 'attr_getf'
linux.c:55: error: 'ATTR_DONTFOLLOW' undeclared (first use in this function)
make: *** [linux.o] Error 1


8O
Back to top
View user's profile Send private message
98111
n00b
n00b


Joined: 23 Feb 2003
Posts: 41
Location: Lyon, France (aka un_brice )

PostPosted: Thu Feb 08, 2007 2:31 pm    Post subject: Reply with quote

getchoo wrote:
I'd like to try this program, but it's giving me the following error:
Code:
linux.c:23:43: error: attr/attributes.h: No such file or directory

8O
Please install sys-apps/attr . You can also use the ebuild.
Back to top
View user's profile Send private message
Jkay
Apprentice
Apprentice


Joined: 31 Dec 2002
Posts: 214
Location: thatched-roof cottages

PostPosted: Sun Feb 25, 2007 6:23 am    Post subject: Reply with quote

Hi. I haven't used Shake since last summer, so I grabbed 0.29 and shook:

/home
/var/db/pkg
/var/cache/edb
/usr/portage

Am I missing any other directory which Portage makes a mess in?

I only ran Shake in non-system dirs, for now ... until things settle, and we're non-alpha!! Just getting
Code:

emerge --sync && emerge -uavDN world

to be a lot faster is good enough for now!!

Thanks again for the great work.

EDIT - I should mention that I have /var and /usr and /home and / all on separate hard drives or partitions.
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Sun Feb 25, 2007 8:41 am    Post subject: Reply with quote

Jkay wrote:
Hi. I haven't used Shake since last summer, so I grabbed 0.29 and shook:

/home
/var/db/pkg
/var/cache/edb
/usr/portage

Am I missing any other directory which Portage makes a mess in?

I only ran Shake in non-system dirs, for now ... until things settle, and we're non-alpha!! Just getting
Code:

emerge --sync && emerge -uavDN world

to be a lot faster is good enough for now!!

Thanks again for the great work.

EDIT - I should mention that I have /var and /usr and /home and / all on separate hard drives or partitions.


I think /var/tmp and /tmp get messed up real good at times.
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
psyqil
Advocate
Advocate


Joined: 26 May 2003
Posts: 2767

PostPosted: Sun Feb 25, 2007 5:58 pm    Post subject: Reply with quote

Dralnu wrote:
I think /var/tmp and /tmp get messed up real good at times.
I think those should be deleted rather than 'shaken'...
Back to top
View user's profile Send private message
Dralnu
Veteran
Veteran


Joined: 24 May 2006
Posts: 1919

PostPosted: Sun Feb 25, 2007 6:44 pm    Post subject: Reply with quote

psyqil wrote:
Dralnu wrote:
I think /var/tmp and /tmp get messed up real good at times.
I think those should be deleted rather than 'shaken'...


True, but he asked what he missed that portage makes a mess with. If you don't reboot all the time (like some of us), it could make a very minor diff :)
_________________
The day Microsoft makes a product that doesn't suck, is the day they make a vacuum cleaner.
Back to top
View user's profile Send private message
sobers_2002
Veteran
Veteran


Joined: 16 Mar 2004
Posts: 1128

PostPosted: Tue Mar 06, 2007 9:18 pm    Post subject: Reply with quote

Dralnu wrote:
Something to think about would be to require the target fs to be unmounted (i.e. run it from a LiveCD). I would think there would be more problems running on a mounted FS then not, so that might be worth looking into.


that should be the case, but the problem reported by mrsaccess happened even when I ran it via livecd :cry: :cry: :cry: I tried it with v0.28 on reiser4. I am not sure about the performance increase because I was too afraid to experiment after that corruption :P :P. If it's fixed in v0.29 I'll try it again and report back.

and it really does help to have another box of gentoo around :D
_________________
Pdict - dockable dictionary client for linux
FREE97WIN: Use this code on Dreamhost and you get $97 off !!
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Wed Mar 28, 2007 1:06 am    Post subject: Reply with quote

This looks interesting. I am looking forward to a stable version.
Back to top
View user's profile Send private message
mudrii
l33t
l33t


Joined: 26 Jun 2003
Posts: 789
Location: Singapore

PostPosted: Fri Jul 13, 2007 4:15 am    Post subject: Reply with quote

Hei

Big Thx for grate pice of soft
Just waiting for 0.30
_________________
www.gentoo.ro
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1553
Location: U.S.A.

PostPosted: Thu Jul 19, 2007 1:47 am    Post subject: Reply with quote

So what's the status? Have you "submitted it for inclusion in gentoo"?
Back to top
View user's profile Send private message
AssociateX
Tux's lil' helper
Tux's lil' helper


Joined: 19 Feb 2004
Posts: 134
Location: North Dakota

PostPosted: Sat Sep 15, 2007 2:53 pm    Post subject: Reply with quote

Trying it now... yippy, this is fun.

Is there a way to make the columns line up?

So the stuff for the following are really under them: IDEAL START END FRAGC CRUMBC AGE SHOCKED NAME

Thanks a ton!
_________________
AssociateX
Gentoo rocks!
Back to top
View user's profile Send private message
tabascoz
n00b
n00b


Joined: 14 May 2003
Posts: 39
Location: Rio de Janeiro

PostPosted: Sat Sep 15, 2007 11:48 pm    Post subject: Shake and reiser4 2.6.22 Reply with quote

Hi!

First, thanks for this great program !

I have been using shake with no problems in reiser4 until 2.6.22 series.

Shake gives errors like that in all files:

Code:
shake: /mnt/incoming//firmware/licencas/00059e80d0a1-v5_0-8186.dat: FIBMAP failed: Invalid argument


I am using reiser4 + criptocompress + ragnarok-2.6.22-r1

i think this is a reiser4-related bug, in others partitions on the same system with ext3 i have no problems.

It can be fixed in a new version?

Thanks!
Back to top
View user's profile Send private message
hitachi
Guru
Guru


Joined: 20 Feb 2006
Posts: 466

PostPosted: Thu Oct 30, 2008 11:36 am    Post subject: your file system needs you! Reply with quote

And please remember: your file system needs you!

Like if you are using something on torrent, lets say downloading the debian bluray installation iso you might end up with one iso file separated in 79425794 pieces. Consider using cp the file from your scratch/torrents/ to where you want the file for future use instead of mv. Well I know you need to have the space on the disk an then to remove the original file but it might be worth it.
_________________
Athlon 64 X2 3800+
Micro-ATX-Mainboard in einer OEM-Version von Asus mit Nvidias nForce4 (A8NE-FM/S).
PEG-Grafikkarte von Leadtek mit Nvidia GeForce 6600
3 GB RAM
350-Watt-Netzteil (von Fortron/Source)
4 x 500GB S-ATA HDD (raid1 und raid5)
Back to top
View user's profile Send private message
albright
Advocate
Advocate


Joined: 16 Nov 2003
Posts: 2060
Location: Near Toronto

PostPosted: Fri Oct 31, 2008 2:13 am    Post subject: Reply with quote

sorry here's a dumb question: how do I tell the extent of
fragmentation of a whole directory or partition?

(I know that fsck shows this but can I run this on a live
mounted filesystem?)
_________________
.... there is nothing - absolutely nothing - half so much worth
doing as simply messing about with Linux ...
(apologies to Kenneth Graeme)
Back to top
View user's profile Send private message
a.b.
Apprentice
Apprentice


Joined: 23 Mar 2008
Posts: 218
Location: Anus Mundi, Germany

PostPosted: Fri Oct 31, 2008 2:16 am    Post subject: Reply with quote

albright wrote:

(I know that fsck shows this but can I run this on a live
mounted filesystem?)

Code:
fsck -n /dev/foo
Back to top
View user's profile Send private message
albright
Advocate
Advocate


Joined: 16 Nov 2003
Posts: 2060
Location: Near Toronto

PostPosted: Fri Oct 31, 2008 2:45 am    Post subject: Reply with quote

Thanks for not just saying RTFM (which would have
been fair enough I must admit)
_________________
.... there is nothing - absolutely nothing - half so much worth
doing as simply messing about with Linux ...
(apologies to Kenneth Graeme)
Back to top
View user's profile Send private message
albright
Advocate
Advocate


Joined: 16 Nov 2003
Posts: 2060
Location: Near Toronto

PostPosted: Fri Oct 31, 2008 3:30 pm    Post subject: Reply with quote

this is getting off topic, but ...

that -n option works on ide drive but on sata there is
no report of fragmentation:

Code:
wes64@gothmog ~ $ fsck -n -C /dev/sda5
fsck 1.41.2 (02-Oct-2008)
e2fsck 1.41.2 (02-Oct-2008)
Warning!  /dev/sda5 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/dev/sda5: clean, 473522/1531072 files, 2397328/3058366 blocks


Also, everytime I run fsck -n on my ide notebook, it reports file system
errors (even after running fsck for real at bootup). I'm wondering if there
is a serious problem with this drive, or what.

Like this:

Code:
wesfuj@thorin ~ $ fsck -n -C /dev/hda4
fsck 1.41.2 (02-Oct-2008)
e2fsck 1.41.2 (02-Oct-2008)
Warning!  /dev/hda4 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/dev/hda4 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (3551097, counted=3551093).
Fix? no
Free inodes count wrong for group #679 (15354, counted=15357).
Fix? no
Free inodes count wrong (13278975, counted=13278964).
Fix? no
/dev/hda4: ********** WARNING: Filesystem still has errors **********
/dev/hda4: 631041/13910016 files (9.1% non-contiguous), 24257418/27808515 blocks

_________________
.... there is nothing - absolutely nothing - half so much worth
doing as simply messing about with Linux ...
(apologies to Kenneth Graeme)
Back to top
View user's profile Send private message
SlashBeast
Moderator
Moderator


Joined: 23 May 2006
Posts: 2799

PostPosted: Fri Oct 31, 2008 3:56 pm    Post subject: Reply with quote

I tried to use this soft but..
Code:
slashbeast@mizore build % ./shake ~/foo
./shake: /home/slashbeast/foo: FIBMAP failed: Operation not permitted

Any idea?
Back to top
View user's profile Send private message
gimpel
Advocate
Advocate


Joined: 15 Oct 2004
Posts: 2720
Location: Munich, Bavaria

PostPosted: Fri Oct 31, 2008 4:17 pm    Post subject: Reply with quote

SlashBeast wrote:
I tried to use this soft but..
Code:
slashbeast@mizore build % ./shake ~/foo
./shake: /home/slashbeast/foo: FIBMAP failed: Operation not permitted

Any idea?

http://vleu.net/shake/ wrote:
How to use it ?
[...]

As root, call shake my_dir
;)
_________________
http://proaudio.tuxfamily.org/wiki - pro-audio software overlay
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6
Page 6 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum