Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo on ZFS
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 04, 2012 6:04 am    Post subject: Gentoo on ZFS Reply with quote

I am a new Gentoo developer. I am working on ZFS support in Gentoo Linux, among a few other things. I am the person responsible for the sys-fs/zfs and sys-kernel/spl ebuilds in the tree. I am also the person who wrote ZFS support in genkernel. I have been working hard on ZFS support for the better part of the year and that is starting to bear fruit.

In particular, support for Linux swap on ZFS zvols, kernel preemption and Gentoo Hardened will soon enter the tree in the form of a snapshot. It is possible to boot off ZFS using GRUB2, but that is limited to single drive pools and mirrored setups. I am working on a port of FreeBSD's bootloader to address that. I intend to release documentation detailing how to install Gentoo Linux on ZFS when it is ready.

Does anyone have any questions?


Last edited by ryao on Mon Jun 04, 2012 7:04 am; edited 1 time in total
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Mon Jun 04, 2012 6:26 am    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
Does anyone have any questions?

Not really
But thanks to address, many !
So...
Thank you !
_________________
Back to top
View user's profile Send private message
phajdan.jr
Retired Dev
Retired Dev


Joined: 23 Mar 2006
Posts: 1777
Location: Poland

PostPosted: Mon Jun 04, 2012 11:14 am    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
Does anyone have any questions?


Is ZFS on Linux fast and stable? I'd be interested in using the features, but I'm just not sure how "hacky" is that.
_________________
http://phajdan-jr.blogspot.com/
Back to top
View user's profile Send private message
peje
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2003
Posts: 100

PostPosted: Mon Jun 04, 2012 11:38 am    Post subject: Reply with quote

In my case it's stabler and faster than in FreeBSD so I would say its fast and stable.
cu peje
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 04, 2012 11:43 am    Post subject: Re: Gentoo on ZFS Reply with quote

phajdan.jr wrote:
ryao wrote:
Does anyone have any questions?


Is ZFS on Linux fast and stable? I'd be interested in using the features, but I'm just not sure how "hacky" is that.


Qualitatively speaking, I see a performance improvement whenever I rebuild a system using it. My desktop, which has 8GB of RAM, is more responsive. I have not done any serious quantitative performance analysis, but I do have some figures. I have a server with 6x 5400RPM 2TB disks that was using md raid 6, lvm and ext4. Sequential write performance would never exceed 20MB/sec, but with ZFS raidz2, it is able to achieve 286MB/sec sequential write performance when using dd to write 4GB of zeroes (without compression). The code is not performance optimized, but all indicators are promising.

ZFS has the ARC page replacement algorithm, which is likely responsible for the improvement in my desktop's interactive response. ZFS also has the ability to use SSDs as L2ARC and SLOG devices, which should enable it to outscale other RAID/LVM/FS combinations. Using a L2ARC device purely benefits read IOPS by effectively expanding the space that the ARC algorithm has. Using a SLOG device makes random write performance match sequential write performance by requiring that all changes be written to it and then written to the actual disks asynchronously. Sequential write performance is equal to that of the SLOG device. ZFS organizes disks into vdevs and does striping across vdevs, so the maximum write performance is theoretically equal to the sum of the SLOG devices across all of the vdevs.

As for stability, people have had good experiences with the code, but I discovered numerous possible deadlocks involving direct reclaim when trying to make Linux swap on zvols work. Many of those deadlocks could even occur on systems that did not do swap on ZFS. So far, my GIT contains fixes for all but one of those bugs.

The remaining bug is a regression caused by the use of PF_MEMALLOC in sys-kernel/spl to workaround the kernel deadlock described in bug #416685. The developers at LLNL discovered it close to two years ago, talked to the kernel virtual memory subsystem maintainer and came to an agreement that it was a bug. They then discovered the PF_MEMALLOC workaround and no attempts have been made to fix the deadlock since then, although I want to address it soon. Using PF_MEMALLOC prevented a deadlock where memory allocations during direct reclaim could trigger direct reclaim inside the upstream Linux kernel code. It also has the side effect of allowing pages to be allocated from ZONE_DMA. Under heavy memory pressure, this will cause ZONE_DMA to become exhausted, causing a deadlock.

The deadlock caused by the remaining PF_MEMALLOC instance is a rare issue. Nearly all of the people on freenode that use Gentoo on ZFS have never encountered it and I do not think it should stop people from using ZFS. With that said, you will likley want to use code from my GIT. It is on Github, including an overlay that provides ebuilds that pull from it:

https://github.com/ryao/zfs-overlay

I plan to put a snapshot into the portage tree after I solve the last PF_MEMALLOC issue and implement Linux 3.4.0 support. I am uncertain how I will get a patch for the issue in the linux upstream sources into them, although I intend to push to get it into gentoo-sources when it is ready.
Back to top
View user's profile Send private message
EatMeerkats
Apprentice
Apprentice


Joined: 15 Mar 2006
Posts: 234

PostPosted: Mon Jun 04, 2012 4:38 pm    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
Does anyone have any questions?

I've been running your overlay for a while now with no problems, but a few days ago when I re-ran genkernel again, I ended up with a system that won't boot. I don't remember the exact details, but it hangs when trying to mount the root filesystem. It was working before with the exact same kernel, but one thing that I noticed is that mount now also passes the "zfsutil" option, which it previously didn't. Have there been any recent changes to the genkernel support that could have caused this? For now, I've downgraded to a slightly older kernel that has a working initrd.

Also, when do you plan to add kernel 3.4 support? IIRC, the main repository already has the relevant patches. Thanks!
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Tue Jun 05, 2012 10:49 am    Post subject: Re: Gentoo on ZFS Reply with quote

EatMeerkats wrote:
ryao wrote:
Does anyone have any questions?

I've been running your overlay for a while now with no problems, but a few days ago when I re-ran genkernel again, I ended up with a system that won't boot. I don't remember the exact details, but it hangs when trying to mount the root filesystem. It was working before with the exact same kernel, but one thing that I noticed is that mount now also passes the "zfsutil" option, which it previously didn't. Have there been any recent changes to the genkernel support that could have caused this? For now, I've downgraded to a slightly older kernel that has a working initrd.


The zfsutil is something that I had failed to pass originally. It should not cause this. Which kernel sources do you use, what is your kernel version, which version of genkernel do you use and what command did you execute to generate the initramfs? Is it possible that you could have failed to rebuild the kernel modules against the updated kernel?

Note that I have a genkernel improvement planned that will eliminate the need to explicitly rebuild out-of-tree kernel modules by including it as part of the process, but for now, it is necessary to do --callback='module-rebuild rebuild' whenever you rebuild your initramfs.

EatMeerkats wrote:
Also, when do you plan to add kernel 3.4 support? IIRC, the main repository already has the relevant patches. Thanks!


Unfortunately, those patches do not meet upstream's coding standards. I will likely work on Linux 3.4 support after I resolve the last PF_MEMALLOC issue, assuming someone else does not beat me to it.
Back to top
View user's profile Send private message
EatMeerkats
Apprentice
Apprentice


Joined: 15 Mar 2006
Posts: 234

PostPosted: Tue Jun 05, 2012 12:51 pm    Post subject: Reply with quote

I'm running vanilla-sources-3.3.6 and genkernel-3.4.34, with the command: genkernel all --no-clean --no-mountboot --zfs --bootloader=grub --callback="module-rebuild rebuild" (using Grub + ext4 boot partition instead of Grub2)
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Tue Jun 05, 2012 10:47 pm    Post subject: Reply with quote

EatMeerkats wrote:
I'm running vanilla-sources-3.3.6 and genkernel-3.4.34, with the command: genkernel all --no-clean --no-mountboot --zfs --bootloader=grub --callback="module-rebuild rebuild" (using Grub + ext4 boot partition instead of Grub2)


Is my overlay up to date? There was a regression in my overlay's zfs-9999.ebuild that prevented sys-fs/zfs from being added to the module database, so `module-rebuild rebuild` would not rebuild it. You can check if this is happening on your system by looking for sys-fs/zfs in `module-rebuild list`. Run `layman -S && emerge --oneshot sys-kernel/spl sys-fs/zfs` to correct it.
Back to top
View user's profile Send private message
EatMeerkats
Apprentice
Apprentice


Joined: 15 Mar 2006
Posts: 234

PostPosted: Tue Jun 05, 2012 10:55 pm    Post subject: Reply with quote

Yes, I had noticed that when I first switched to ZFS root, but I've since updated the overlay and it's definitely being rebuilt now.
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Wed Jun 06, 2012 4:20 am    Post subject: Reply with quote

EatMeerkats wrote:
Yes, I had noticed that when I first switched to ZFS root, but I've since updated the overlay and it's definitely being rebuilt now.


I was able to reproduce this on my system. It seemed that I made a mistake when introducing Gentoo Hardened support into ZFS. I have reverted that patch for now. You should be able to run genkernel and these issues will go away.

With that said, I am going to make a snapshot in a day or two. That way people will not need to use 9999 ebuilds anymore.
Back to top
View user's profile Send private message
crtpoms
n00b
n00b


Joined: 06 Jun 2012
Posts: 1

PostPosted: Wed Jun 06, 2012 4:06 pm    Post subject: zfs stops at openrc. Reply with quote

Tried installing gentoo using your guide but after reboot it would get stuck on openrc. Not sure what the issue is. My approach was to have a rpool on a single ssd drive and a dpool raidz on 4 sata drives for /home and /usr/portage on dpool.
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Wed Jun 06, 2012 4:30 pm    Post subject: Re: zfs stops at openrc. Reply with quote

crtpoms wrote:
Tried installing gentoo using your guide but after reboot it would get stuck on openrc. Not sure what the issue is. My approach was to have a rpool on a single ssd drive and a dpool raidz on 4 sata drives for /home and /usr/portage on dpool.


Would you elaborate on the point at which it got stuck? Also, there are two reboots in my guide. Is this after the first reboot (needed to refresh partition tables) or the second reboot (needed to start the system)?
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Wed Jun 06, 2012 5:23 pm    Post subject: Re: Gentoo on ZFS Reply with quote

EatMeerkats wrote:
Also, when do you plan to add kernel 3.4 support? IIRC, the main repository already has the relevant patches. Thanks!


I have written new patches that should meet upstream's coding standards. They are in my development overlay and I have opened pull requests with upstream:

https://github.com/zfsonlinux/spl/pull/124
https://github.com/zfsonlinux/zfs/pull/776
Back to top
View user's profile Send private message
EatMeerkats
Apprentice
Apprentice


Joined: 15 Mar 2006
Posts: 234

PostPosted: Thu Jun 07, 2012 3:14 pm    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
EatMeerkats wrote:
Also, when do you plan to add kernel 3.4 support? IIRC, the main repository already has the relevant patches. Thanks!


I have written new patches that should meet upstream's coding standards. They are in my development overlay and I have opened pull requests with upstream:

https://github.com/zfsonlinux/spl/pull/124
https://github.com/zfsonlinux/zfs/pull/776

Great, thanks! Just upgraded to 3.4.1 and everything seems to be working well.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Jun 08, 2012 12:18 pm    Post subject: Re: Gentoo on ZFS Reply with quote

EatMeerkats wrote:
ryao wrote:
EatMeerkats wrote:
Also, when do you plan to add kernel 3.4 support? IIRC, the main repository already has the relevant patches. Thanks!


I have written new patches that should meet upstream's coding standards. They are in my development overlay and I have opened pull requests with upstream:

https://github.com/zfsonlinux/spl/pull/124
https://github.com/zfsonlinux/zfs/pull/776

Great, thanks! Just upgraded to 3.4.1 and everything seems to be working well.


++

way better than before :)

unfortunately videos still can't be viewed or streamed properly (e.g. youtube) and inputs are still interrupted

local videos (via smplayer2) can't be even viewed - probably due to pulseaudio getting interrupted & mplayer not liking being interrupted

are the fixes you mentioned in the other thread (https://forums.gentoo.org/viewtopic-p-7054854.html#7054854) already included in your overlay ? [edit3: seems they are already included]


if yes, I'll have to zfs alone - for now :(

(currently giving it a short test on a backup-drive)


speed and throughput is nice but this lag is killing me (it has significantly improved but using it on the desktop & multi-tasking is still not possible that way; it could
also be an issue with data-writing on ~amd64 and the core architecture)


might the reason be my kernel config ? (close to no debug optinos activated) or desktop environment ?

I'm running compiz-fusion (0.8.6), compositing with xf86-video-ati on xfce


anyone else having the same issues ?

this is running on 3.4.0 kernel with full preemption


edit:

fortunately now it only happens while writing - when it's not writing that doesn't happen

before it afaik happened all the time when the kernel module was loaded

external S-ATA drive with NCQ disable (queue_depth set to "1")


edit2:

watching videos with alsa is also affected, btw
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Fri Jun 08, 2012 3:36 pm    Post subject: Re: Gentoo on ZFS Reply with quote

kernelOfTruth wrote:
unfortunately videos still can't be viewed or streamed properly (e.g. youtube) and inputs are still interrupted

local videos (via smplayer2) can't be even viewed - probably due to pulseaudio getting interrupted & mplayer not liking being interrupted


What is is your system hardware and would you describe steps to reproduce this in more detail?
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Jun 08, 2012 3:58 pm    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
kernelOfTruth wrote:
unfortunately videos still can't be viewed or streamed properly (e.g. youtube) and inputs are still interrupted

local videos (via smplayer2) can't be even viewed - probably due to pulseaudio getting interrupted & mplayer not liking being interrupted


What is is your system hardware and would you describe steps to reproduce this in more detail?


Motherboard: Bulldog II (Intel P55 Express Chipset); Packard Bell custom µATX board
Processor: Intel Core i7 860
RAM: 8 GB
Intel ICH 10: harddrives I observed this (afaik) are: WD2002FAEX & HD203WI; running ahci (via bios enabled);

i/o schedulers: CFQ & BFQ (3.3 kernel); BFS & CFS (with no preemption on 3.3), CFS on 3.4 & CONFIG_TREE_PREEMPT_RCU=y, CONFIG_PREEMPT_RCU=y, CONFIG_PREEMPT=y

~amd64 Gentoo, vanilla kernel 3.4.0

kernel config: http://pastebin.com/Sr9HNV7A (valid 1 month)

pulseaudio: 2.0


what else do you need ?


thanks ! :)


edit:

the zpool is in a cryptsetup partition so it's not directly interacting with the harddrive


edit2:

the same applies to all other partitions (system, swap, /home)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 18, 2012 2:18 am    Post subject: Re: Gentoo on ZFS Reply with quote

Unfortunatley, I did not spot your reply until now. What do you have on your pool? Does this problem occur merely from the module being loaded or is your media player software interacting with ZFS in some way?

Note that the best way to get in touch with me is in #zfsonlinux on freenode.
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Wed Jun 27, 2012 8:52 pm    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
Does anyone have any questions?


I tried to run the tests, that were posted on phoronix with my 120GB Intel SSD: http://openbenchmarking.org/result/1206267-SU-ZFSLINUXB51

But gave up on FS-Mark, this runs for ages...
Flexible IO Tester = ~112s
FS-Mark (1000 files 1MB) = ~25files/s
The FS-Mark test with 5000 files wasn't finished with the first test after ~30min

This option:
Code:
#ARC tuning - You can configure the RAM that ARC uses (untested with genkernel)
echo options zfs zfs_arc_max=512M > /etc/modprobe.d/zfs.conf

Is this the maximum memory used by the filesystem altogether or only the 'cache'? I noticed up to ~12GB of 16GB RAM used during the benchmarks and assume zfs had the lion share.

Any tuning guides welcome...

//edit: I think those settings for the module are totally ignored, because the module is loaded before /etc becomes available? I'll later try some kernel command line.

//edit: grmpf:
https://github.com/zfsonlinux/zfs/issues/698

//edit: only possible to allocate arc max size during module loading, so I try later 'modprobe -r' :twisted:
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Thu Jun 28, 2012 9:01 pm    Post subject: Re: Gentoo on ZFS Reply with quote

disi wrote:
ryao wrote:
Does anyone have any questions?


I tried to run the tests, that were posted on phoronix with my 120GB Intel SSD: http://openbenchmarking.org/result/1206267-SU-ZFSLINUXB51

But gave up on FS-Mark, this runs for ages...
Flexible IO Tester = ~112s
FS-Mark (1000 files 1MB) = ~25files/s
The FS-Mark test with 5000 files wasn't finished with the first test after ~30min


If I had to guess, I would suspect that this is a mix of an improper ashift value and high flash utilization, in either actual space usage or the SSD's dirty pages. I suggest emailing the mailing list for further advice:

http://zfsonlinux.org/lists.html

disi wrote:
//edit: I think those settings for the module are totally ignored, because the module is loaded before /etc becomes available? I'll later try some kernel command line.


That is correct. It is a known issue that I have not had time to fix.
Back to top
View user's profile Send private message
tclover
Guru
Guru


Joined: 10 Apr 2011
Posts: 516

PostPosted: Sat Jun 30, 2012 12:11 pm    Post subject: some experiment(s) [with kerel 3.4.3] Reply with quote

I've just merged zfs the other day and was trying to experiments a few things here and there. I'd consider adding support to sys-boot/mkinitramfs-ll (ref.: sig). However, I'm struggling(?) on compression property:
zfs list:
NAME              USED  AVAIL  REFER  MOUNTPOINT
upl              1004M  32.0G   136K  none
upl/home          272K  32.0G   136K  /pool/home
upl/home/root     136K  32.0G   136K  /pool/root
upl/root          160K  32.0G   160K  /pool
upl/var          1003M  32.0G   160K  /pool/var
upl/var/layman    102M  32.0G   102M  /pool/var/lib/layman
upl/var/portage   900M  32.0G   900M  /pool/var/portage

[/var/]portage tree size is ~300MB and ~60MB for a squashded copy (aufs+squashfs[gzip]); [/var/lib/]layman size is ~36MB and ~15MB (aufs+squashfs[gzip]). And here I got almost x3 size with property compression=gzip-9 zfs (WTF).
zfs get compressratio upl/var/portag:
NAME             PROPERTY       VALUE  SOURCE
upl/var/portage  compressratio  1.34x  -

zfs get compressratio upl/var/layman:
NAME            PROPERTY       VALUE  SOURCE
upl/var/layman  compressratio  1.52x  -

I created that pool with something like:
Code:
cryptsetup luksFormat /dev/sda4 -caes-xts-plain:sha256 -s256
cryptsetup luksOpen /dev/sda4 zpv
zpool create -f -o ashift=12 -O normalization=formD -m none -R /pool upl /dev/mapper/zpv
zfs create -o mountpoint=none upl/root
zfs create -o mountpoint=/ upl/root
zfs create -o mountpoint=/home upl/home
zfs create -o mountpoint=/root upl/home/root
zfs create -o mountpoint=/var upl/var
zfs create -o mountpoint=/var/portage    -o compression=gzip-9 upl/var/portage
zfs create -o mountpoint=/var/lib/layman -o compression=gzip-9 upl/var/layman

cp -a /var/lib/layman /pool/var/lib
cp -a /var/portage /pool/var/

I did not yet read the whole man pages of zpool(8) and zfs(8) to be able to get what is going on or what could be going wrong atm.
So what could be the cause of getting x3 size with compression=gzip[-9]?
_________________
home/:mkinitramfs-ll/:supervision/:e-gtk-theme/:overlay/
Back to top
View user's profile Send private message
yngwin
Retired Dev
Retired Dev


Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China

PostPosted: Sun Jul 01, 2012 5:10 am    Post subject: Reply with quote

On the Gentoo Wiki there is a request for a Gentoo on ZFS article. That would be a good place to get all the relevant documentation in one place. If you guys could get that started, that would be grand... I am available for help on editing / wikifying (among other places in the #gentoo-wiki channel).
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Jul 03, 2012 2:12 pm    Post subject: Re: Gentoo on ZFS Reply with quote

ryao wrote:
Unfortunatley, I did not spot your reply until now. What do you have on your pool? Does this problem occur merely from the module being loaded or is your media player software interacting with ZFS in some way?

Note that the best way to get in touch with me is in #zfsonlinux on freenode.


no worries :)

I just had some more time to experiment with ZFS

sure, I'll see when I find some time to hop on IRC



[edit3:]

gzip compression played a role in combination with CFS scheduler

(http://don.blogs.smugmug.com/tag/lzjb/) <-- confirms that gzip compression pegs the cpu fully and that's also my observation


so I switched to lzjb compression - I don't need best compression ratio on this new harddrive but it's nevertheless a nice bonus and lzjb even seems to raise performance and the load on the cpu is way lower

lowering compression helped with the issue but the main issue is still when ZFS is writing to the harddrive


I'm not sure how much of a role using cryptsetup/luks plays here since I don't want to try writing all of my data into unencrypted space



the optimizations for encryption are selected in kernel config

zcat /proc/config.gz | grep TWO
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_NETWORK_SECMARK=y
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_TWOFISH_COMMON=y
CONFIG_CRYPTO_TWOFISH_X86_64=y
CONFIG_CRYPTO_TWOFISH_X86_64_3WAY=y


zcat /proc/config.gz | grep AES
# CONFIG_SND_MAESTRO3 is not set
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_X86_64=y
CONFIG_CRYPTO_AES_NI_INTEL=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m

no sandybridge here


zcat /proc/config.gz | grep PCR
CONFIG_CRYPTO_PCRYPT=y


[edit3, end.]





if anyone of you guys wants to try out ZFS on a desktop, e.g. as a backup drive or even home

give Con Kolivas ck-patchset with BFS Scheduler (423) on 3.4 kernel a try,

make sure you patch it up to version 424 and add Chen's O(1) tweak: http://phoronix.com/forums/showthread.php?71658-RIFS-ES-Linux-Kernel-Scheduler-Released&p=272499#post272499







related to my issue:

the above mentioned setup with BFS + the additional tweaks sound - even with 2 streams - has no stutters

but videos every 1-2 seconds have some stutters - sound continues unaffected with the new cpu scheduler


fortunately this now is only when ZFS is writing data on the partition (compared to zfs-fuse and newer zfs releases afaik starting after 22 or 23)

merely loading the kernel module doesn't seem to affect anything


another issue is that while playing video - entering text also is affected: e.g. entering "still viewing has jerky behavior ...."

e.g. the l in "still" or any other random letter is simply ignored - as if there was a big lock in the kernel occuring blocking everything else during the write of data



to recap:

on default kernel scheduler (CFS):

1) sound playback with pulseaudio is jerky - this becomes especially obvious with 2 sound streams through pulseaudio and gzip compression activated (partly scheduling issue with CFS) [is improved with BFS + O(1) patch]

2) scrolling in browser with mouse middle button (usb mouse) is jerky [is improved with BFS + O(1) patch]

3) watching (HD) videos on youtube stutters a lot - any form of video, actually

4) entering text and other everyday productivity tasks are affected in the sense that letters are simply ignored/omitted or e.g. other keys of the keyboard for the time while this "lock" occurs the computer is "blind" for input from outside [is improved with BFS + O(1) patch]

5) streaming audio/video content is more affected than playing videos or sound locally

6) this luckily (?) only happens when ZFS is writing data, when ZFS is idling the kernel module doesn't affect workflow


hope that helps


edit:

made some changes - video playback is especially affected

the issue can be mitigated by using alternative cpu schedulers + tweaks


edit2:

when running test on http://speedtest.net/

the needle slightly goes down every 1-2 seconds matching with the above mentioned behavior

this shows that this seems to affect the kernel as a whole ...


edit3:

added info above with cryptsetup/luks and compression

CFS/the cpu scheduler seems to play an important part

the major part is when writing to the disk

not sure how much encryption and non-direct writing to the disk plays in creating this issue
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Sun Jul 08, 2012 2:14 pm    Post subject: Re: Gentoo on ZFS Reply with quote

kernelOfTruth wrote:
I'm not sure how much of a role using cryptsetup/luks plays here since I don't want to try writing all of my data into unencrypted space


Are these advanced format hard drives. It could be that you correctly used ashift=12, ZFS is writing 4096-byte blocks while cryptsetup/luks does not handle that well.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next
Page 1 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum