Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
New filesystem: Btrfs!
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 24, 25, 26  
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Mon May 24, 2010 2:51 am    Post subject: Reply with quote

Dont Panic wrote:
It would have been nice to have caught btrfs "in the act" of doing whatever it did.

It sounds like that by the time you found out that directory had issues, the time had already past for gaining insight as to why btrfs was behaving this way.

As to your general question about making sure your data is still there, I've thought about it before also, and don't have a good answer (especially when it comes to big, space-hungry stuff like movies and music). It's nice to have an older back-up around in addition to your fresher backups, because it's easy to make an un-noticed mistake in deleting important data and then propagate that mistake through your backups.
Not sure about "catching in the act" part. How can a user catch it? Only the checks it does internally can catch that!

Yeah, the second part about propagating the deletion is important. Its handled well with snapshots. But its a slippery slope, because your oldest snapshot will keep holding all your future deletes and will eat space which you think you have deleted.
Back to top
View user's profile Send private message
aoeuaoue
n00b
n00b


Joined: 26 Feb 2010
Posts: 31

PostPosted: Wed May 26, 2010 3:14 am    Post subject: First incompatabilty issue Reply with quote

hi,

BTRFS managed to render ALL my kvm images useless! Don't ask me how but it did. Linux OSes installs would crap out with input/output error. The images would segfault durnig boot, m$$ 7 iteration got stuck indefinately at the m$$ logo.

I went insane trying to figure this out. I compiled qemu-kvm every which way possible, tried running kvm with all option combinations known to man, etc. All my attempts where to no avail. However liveCDs DID work, the problem was the direct interaction between the image and OS.

BTRFS is squarely at fault and I strongly believe the compress option to be the main culprit.

I found out by trying to create a new image with qemu-img create -f qcow2 this.img 5G which returns the following message: qemu-img: Error while formatting.

Don't ask me why but I had the suspicion that BTRFS was the problem but I went onto create a new image on a XFS partition and it all went swell. Then I move that partition over to the BTRFS partition and installed m$$$ 7 just fine. I thing this is a serious bug.

Another issue is with CCACHE and -j 8 or any high number. With some earlier kernel BTRFS would simply segfault or send the kernel panicking. Later on emerge would get stuck as soon gcc touched CCACHE it would stop and become unkillable. In dmesg log it'd get flooded with something about 'transid checksum...' error, or along these lines. The last one the system was still working but unable to emerge anything. I fixed that problem by not mounting CCACHE partition and disabling it /etc/make.conf.

Currently I have about 3/4 of my system BRFS'fied without problems at all, well except that weird CCACHE thingy. Also I am running with BTRFS' RAID0 option. Yes I know about the riskyness but I am doing just for the lolz with three 1TB drives (about to add another one).

I have a question though, anyone know how does the thread_pool=4 work? I got a 4 core so I add this to fstab:

Code:
/dev/sdb                        /mnt/drive   btrfs   noatime,compress,thread_pool=4,subvol=.multilib 0 1


thanks!
Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Wed May 26, 2010 4:15 pm    Post subject: Reply with quote

What kernel were you using?
Back to top
View user's profile Send private message
regomodo
Guru
Guru


Joined: 25 Mar 2008
Posts: 445

PostPosted: Wed May 26, 2010 5:56 pm    Post subject: Reply with quote

skellr wrote:
Yeah it is. :) It seems I have the same issue when it's in the sbin directory. Try moving it into bin/.

:x sbin/ has the same perms as bin/

edit: uid=0 gid=0 should be good enough.
Code:
/ # id
uid=0 gid=0 groups=0,1,2,3,4,6,10,11,26,27


Moved it over and despite tab-complete catching the binary

Code:
funtoo-pc src # chroot initramfs/ /bin/busybox sh
/ # btrfsctl
sh: btrfsctl: not found
/ #


Some other testing

Code:

/ # ./init
Trying to run btrfsctl
./init: line 9: /bin/btrfsctl: not found
./init: line 9: rescue_shell: not found
mount: mounting /dev/sde7 on /mnt/root failed: Device or resource busy
switch_root: bad newroot /mnt/root
/ # cat init
#!/bin/busybox sh

# Mount the /proc and /sys filesystems.
mount -vt proc none /proc
mount -vt sysfs none /sys

# Btrfs stuff
echo "Trying to run btrfsctl"
/bin/btrfsctl -A /dev/sda /dev/sdb || rescue_shell

# Mount the root filesystem.
mount -o ro /dev/sde7 /mnt/root

# Clean up.
umount /proc
umount /sys

# Boot the real thing.
exec switch_root /mnt/root /sbin/init

rescue_shell() {
    echo "Something went wrong. Dropping you to a shell."
    busybox --install -s
    exec /bin/sh
}

/ #
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Wed May 26, 2010 8:29 pm    Post subject: Reply with quote

Is /bin/btrfsctl a static binary? If not, why not? I have a feeling that its dynamic and its not loading a dependent lib. busybox is crappy that way...it will always say 'not found', although it means to say 'I am dumb, I can't find the dependent libuuid.so.2, but because I am dumb, I can't tell you that!'. I spent countless hours around that and finally found that one of the dependent libs was not there.

Now, my custom programs are always statically built before they go into initramfs.
Back to top
View user's profile Send private message
regomodo
Guru
Guru


Joined: 25 Mar 2008
Posts: 445

PostPosted: Wed May 26, 2010 8:43 pm    Post subject: Reply with quote

devsk wrote:
Is /bin/btrfsctl a static binary? If not, why not? I have a feeling that its dynamic and its not loading a dependent lib. busybox is crappy that way...it will always say 'not found', although it means to say 'I am dumb, I can't find the dependent libuuid.so.2, but because I am dumb, I can't tell you that!'. I spent countless hours around that and finally found that one of the dependent libs was not there.

Now, my custom programs are always statically built before they go into initramfs.


That's what I thought but I'm not sure how I would go about doing that. From using ldd I can see I have the necessary libraries but i'm unsure (because of all the symlinked libraries) if i've copied a symlink or not.

Code:

funtoo-pc initramfs # ls -la lib/
total 1617
drwxr-xr-x  2 root root     216 May 21 21:15 .
drwxr-xr-x 11 root root     288 May 26 18:54 ..
-rwxr-xr-x  1 root root  123160 Apr 30 21:00 ld-linux-x86-64.so.2
-rwxr-xr-x  1 root root  121292 Apr 30 21:03 ld-linux.so.2
-rwxr-xr-x  1 root root 1383600 Apr 28 22:23 libc.so.6
lrwxrwxrwx  1 root root      12 Nov 15  2009 libdl.so.2 -> libdl-2.9.so
-rwxr-xr-x  1 root root   18832 Apr 28 22:23 libuuid.so.1
funtoo-pc initramfs # ldd /sbin/btrfsctl
        linux-vdso.so.1 =>  (0x00007fff65bff000)
        libuuid.so.1 => /lib/libuuid.so.1 (0x00007f0eb70d0000)
        libc.so.6 => /lib/libc.so.6 (0x00007f0eb6d7a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0eb72d5000)
funtoo-pc initramfs #
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Wed May 26, 2010 9:28 pm    Post subject: Reply with quote

Why not build it static? 'ebuild `equery which btrfs-progs` install` and then just add -static to the link of btrfsctl and rebuild and copy over.
Back to top
View user's profile Send private message
regomodo
Guru
Guru


Joined: 25 Mar 2008
Posts: 445

PostPosted: Thu May 27, 2010 6:05 pm    Post subject: Reply with quote

devsk wrote:
Why not build it static? 'ebuild `equery which btrfs-progs` install` and then just add -static to the link of btrfsctl and rebuild and copy over.


Cheers for the info but:

Code:

funtoo-pc src # ebuild `equery which btrfs-progs` install
Unavailable repository 'gentoo' referenced by masters entry in '/usr/local/portage/layman/science/metadata/layout.conf'
Unavailable repository 'gentoo' referenced by masters entry in '/usr/local/portage/layman/science/metadata/layout.conf'
>>> Existing ${T}/environment for 'btrfs-progs-0.19' will be sourced. Run
>>> 'clean' to start with a fresh environment.
 * btrfs-progs-0.19.tar.bz2 RMD160 SHA1 SHA256 size ;-) ...                                                                   [ ok ]
 * checking ebuild checksums ;-) ...                                                                                          [ ok ]
 * checking auxfile checksums ;-) ...                                                                                         [ ok ]
 * checking miscfile checksums ;-) ...                                                                                        [ ok ]
 * checking btrfs-progs-0.19.tar.bz2 ;-) ...                                                                                  [ ok ]
 * CPV:  sys-fs/btrfs-progs-0.19
 * REPO: funtoo
 * USE:  acl amd64 elibc_glibc kernel_linux multilib userland_GNU
>>> Checking btrfs-progs-0.19.tar.bz2's mtime...
>>> WORKDIR is up-to-date, keeping...
>>> It appears that 'btrfs-progs-0.19' is already compiled; skipping.
>>> Remove '/var/tmp/portage/sys-fs/btrfs-progs-0.19/.compiled' to force compilation.
>>> Test phase [not enabled]: sys-fs/btrfs-progs-0.19
>>> It appears that 'btrfs-progs-0.19' is already installed; skipping.
>>> Remove '/var/tmp/portage/sys-fs/btrfs-progs-0.19/.installed' to force install.


 * QA Notice: Pre-stripped files found:
 * /sbin/btrfs-show
 * /sbin/btrfs-vol
 * /sbin/btrfsctl
 * /sbin/btrfsck
 * /sbin/btrfstune
 * /sbin/btrfs-image
 * /sbin/mkbtrfs
 * /sbin/btrfs-convert
ecompressdir: bzip2 -9 /usr/share/man
funtoo-pc src # cp -a /var/tmp/portage/sys-fs/btrfs-progs-0.19/image/sbin/btrfsctl /usr/src/initramfs/bin/btrfsctl-static
funtoo-pc src # chroot initramfs/ /bin/busybox sh
/ # /bin/btrfsctl-static
sh: /bin/btrfsctl-static: not found
/ #


The binary is still not found.

It's been almost 6months without being able to directly being able to boot my desktop without having to manually mount. I look forward to the day I can.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Thu May 27, 2010 9:00 pm    Post subject: Reply with quote

I am sorry, but you forgot the "-static" part. What does
Code:
ldd /var/tmp/portage/sys-fs/btrfs-progs-0.19/image/sbin/btrfsctl
return?

After you do ebuild `equery which btrfs-progs` install, you need to go to the work area and rebuild btrfsctl with -static parameter passed to the 'gcc'. Like this :

Code:
13:59:34 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# l btrfsck
-rwxr-xr-x 1 portage portage 187681 May 27 13:59 btrfsck*

13:59:39 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# ldd btrfsck
        linux-gate.so.1 =>  (0xb7829000)
        libuuid.so.1 => /lib/libuuid.so.1 (0xb7802000)
        libc.so.6 => /lib/libc.so.6 (0xb76bd000)
        /lib/ld-linux.so.2 (0xb782a000)

13:59:43 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# \rm btrfsck

13:59:50 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# make -n btrfsck
gcc -g -Werror -Os -o btrfsck btrfsck.o ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o root-tree.o dir-item.o file-item.o inode-item.o inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o volumes.o utils.o  -luuid

13:59:53 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# gcc -static -g -Werror -Os -o btrfsck btrfsck.o ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o root-tree.o dir-item.o file-item.o inode-item.o inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o volumes.o utils.o  -luuid

14:00:01 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# ldd btrfsck
        not a dynamic executable

14:00:09 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# l btrfsck
-rwxr-xr-x 1 root root 773746 May 27 14:00 btrfsck*
Now, just copy this to your initramfs.
Back to top
View user's profile Send private message
regomodo
Guru
Guru


Joined: 25 Mar 2008
Posts: 445

PostPosted: Fri May 28, 2010 2:22 pm    Post subject: Reply with quote

devsk wrote:
I am sorry, but you forgot the "-static" part. What does
Code:
ldd /var/tmp/portage/sys-fs/btrfs-progs-0.19/image/sbin/btrfsctl
return?

That returned the same as above.
devsk wrote:

After you do ebuild `equery which btrfs-progs` install, you need to go to the work area and rebuild btrfsctl with -static parameter passed to the 'gcc'. Like this :

Code:
13:59:34 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# l btrfsck
-rwxr-xr-x 1 portage portage 187681 May 27 13:59 btrfsck*

13:59:39 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# ldd btrfsck
        linux-gate.so.1 =>  (0xb7829000)
        libuuid.so.1 => /lib/libuuid.so.1 (0xb7802000)
        libc.so.6 => /lib/libc.so.6 (0xb76bd000)
        /lib/ld-linux.so.2 (0xb782a000)

13:59:43 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# \rm btrfsck

13:59:50 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# make -n btrfsck
gcc -g -Werror -Os -o btrfsck btrfsck.o ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o root-tree.o dir-item.o file-item.o inode-item.o inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o volumes.o utils.o  -luuid

13:59:53 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# gcc -static -g -Werror -Os -o btrfsck btrfsck.o ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o root-tree.o dir-item.o file-item.o inode-item.o inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o volumes.o utils.o  -luuid

14:00:01 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# ldd btrfsck
        not a dynamic executable

14:00:09 root@localhost /var/tmp/portage/sys-fs/btrfs-progs-0.19/work/btrfs-progs-0.19
# l btrfsck
-rwxr-xr-x 1 root root 773746 May 27 14:00 btrfsck*
Now, just copy this to your initramfs.


:D Many thanks! That works fine for btrfsctl. Finally a non-manual boot!
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Fri May 28, 2010 5:53 pm    Post subject: Reply with quote

Alright! Awesome! Good to know that u are now able to boot without manual intervention....:-D
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 20053

PostPosted: Thu Aug 05, 2010 8:42 pm    Post subject: Reply with quote

Continued with New filesystem: Btrfs! (cont.).
_________________
Quis separabit? Quo animo?
Back to top
View user's profile Send private message
Display posts from previous:   
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page Previous  1, 2, 3 ... 24, 25, 26
Page 26 of 26

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum