Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo on ZFS
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Apr 29, 2015 12:08 am    Post subject: Reply with quote

0.6.4.1 is highly recommended - especially due to the following commit:

https://github.com/zfsonlinux/zfs/commit/7af63221dc6a1dbd3ce7ebf88136b0169341a4f8

Allocate zfs_znode_cache on the Linux slab wrote:

The Linux slab, in general, performs better than the SPl slab in cases
where a lot of objects are allocated and fragmentation is likely present.

This patch fixes pathologically bad behavior in cases where the ARC is
filled with mostly metadata and a user program needs to allocate and
dirty enough memory which would require an insignificant amount of the
ARC to be reclaimed.

If zfs_znode_cache is on the SPL slab, the system may spin for a very
long time trying to reclaim sufficient memory. If it is on the Linux
slab, the behavior has been observed to be much more predictible; the
memory is reclaimed more efficiently.


(stability & responsiveness, reliability under load)


https://github.com/zfsonlinux/zfs/commit/e548a597d0820fecf89137c0aa7add8ad4aa340c

Mark additional functions as PF_FSTRANS wrote:

Prevent deadlocks by disabling direct reclaim during all NFS, xattr,
ctldir, and super function calls. This is related to 40d06e3.



+

https://github.com/zfsonlinux/zfs/commit/320e58d5a61fb4264acfb1782b2a337063ae13fa

Extend PF_FSTRANS critical regions wrote:

Additional testing has shown that the region covered by PF_FSTRANS
needs to be extended to cover the zpl_xattr_security_init() and
init_acl() functions. The zpl_mark_dirty() function can also recurse
and therefore must always be protected.



the latter 2 out of the three are mostly related to stability & locking (deadlocks, livelocks, etc.),

which can occur under certain workloads or after some runtime


the branch is:

https://github.com/zfsonlinux/zfs/commits/zfs-0.6.4-release


so - no, it's not much ("too") behind

but it addresses some sore points
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
fearedbliss
Retired Dev
Retired Dev


Joined: 13 Jul 2012
Posts: 4
Location: Brooklyn, NY

PostPosted: Wed Apr 29, 2015 2:47 am    Post subject: Reply with quote

WWWW wrote:
thank you for the work.

Is the 0.6.4.1 too far behind?

Should I try 0.6.4.1 from bliss overlay?

I went through all the trouble setting up layman and such. Now I can choose official 0.6.4 or non-official 0.6.4.1.

There seems to be important fixes:

Code:

-  Fixed io-spare.sh script for ZED.
-  Fixed multiple deadlocks which might occur when reclaiming memory.
-  Fixed excessive CPU usage for meta data heavy workloads when reclaiming the ARC.


Deadlocking with memory seems like a delicate issue.

thanks again.

ps.: oh packaging, that looks like an easy task, is it possible to help with packaging? How to start helping?


It is ready when it is ready. Asking/demanding for updates when this is completely a volunteer project won't help anyone. Also ryao shouldn't need to explain himself on why he hasn't pushed the builds.

My overlay is highly volatile btw. Most of the stuff I push there is pretty much for myself (I just make it public), don't blame me if stuff breaks or vanishes.

@kernelOfTruth

I see you are maintaining some zfs srm stuff. I made this a year ago and it automates the process:
https://github.com/fearedbliss/sysresccd-moddat

You can also find the latest isos here (with mirrors): http://xyinn.org/doku.php
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Wed Apr 29, 2015 12:32 pm    Post subject: Reply with quote

WWWW wrote:
is it possible to help with packaging? How to start helping?

Start by bumping ebuilds in your local overlay. This simply means copying them to a newer-versioned filename, and then seeing how the build goes, then editing the ebuild to make it work for you.

Then you post it to fellow users, usually via bugzilla; quite often you don't need to edit anything at all and can simply state that copying the old ebuild to the new name works.

That's how everyone else is doing it, and where all those overlays come from.

You should be discussing with fellow users, preferably in IRC or on the forums, especially when you're starting out.
#gentoo-dev-help on IRC: chat.freenode.net and #bash for the implementation language.
For build-system help, eg make, autoconf, automake, cmake et al, the best place is ##workingset though there is a #cmake as well.

Another way to help out easily, is to file stabilisation bugs for packages you follow, provided they fit the criteria outlined.
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Wed Apr 29, 2015 1:09 pm    Post subject: Reply with quote

WWWW wrote:
thank you for the work.

Is the 0.6.4.1 too far behind?

Should I try 0.6.4.1 from bliss overlay?

I went through all the trouble setting up layman and such. Now I can choose official 0.6.4 or non-official 0.6.4.1.

There seems to be important fixes:

Code:

-  Fixed io-spare.sh script for ZED.
-  Fixed multiple deadlocks which might occur when reclaiming memory.
-  Fixed excessive CPU usage for meta data heavy workloads when reclaiming the ARC.


Deadlocking with memory seems like a delicate issue.

thanks again.

ps.: oh packaging, that looks like an easy task, is it possible to help with packaging? How to start helping?


They are useful fixes and they are a continuation of the kmem-rework that I started last summer. However, there are a few things to consider.

  1. The dependencies for sys-kernel/spl and sys-fs/zfs-kmod were not designed for the interim versioning scheme that upstream decided to use. It was originally just for Fedora/EPEL packaging fixes, but it was expanded recently to include important fixes. If we go with them, I will need to take time to redesign the dependencies to handle it and I don't have that for a few days. This is why I went with 0.6.4 rather than 0.6.4.1.
  2. There are multiple deadlocks that I am tracking, some of which do not yet have fixes in HEAD and some of them are arguably more severe. My time is limited enough and the deadlock fixes in 0.6.4.1 are rare enough that I would prefer to batch them with the other deadlock fixes.


As for helping out, I regret to say that ZoL packaging is not quite as straightforward as it appears. Doing proper QA on core components such as filesystem drivers to ensure that they work is very important. Consequently, I have developed a few rules for myself when doing ZoL packaging:


  1. Do proper regression testing to ensure that as many foreseeable problems as possible are avoided, including but not limited to recurrences of past mistakes.
  2. Never commit after 9pm (EST for me) because errors are more likely at that time.
  3. Never commit if unsure about QA.


The rationale behind these priorities requires some explanation. I have been doing the ZoL packaging for over 3 years and I have had made my share of mistakes during that time. I have also seen mistakes made by upstream by both myself and other upstream contributors. Each time there is a mistake by myself or by upstream, we both make our own efforts to prevent them from recurring. The best method is the development of automated unit tests that encapsulate knowledge gained from past mistakes, but writing them requires at least 3 times more time than testing by hand requires because you need to be certain that they test what you intended to test and that you design them in such a way that changes in the code do not cause false positives/negatives. The upstream tests are geared primarily toward the environment at LLNL where the project lead is employed as part of a team responsible for the software stack on the Sequoia supercomputer. Distribution-specific concerns (especially backports for key fixes) are not covered quite as well. This requires me to take time to develop my own automated tests, especially for things that are Gentoo-specific. Until I do, I will need to run tests by hand. This has been by far the largest time requirement for the ZoL packaging. While I have made some progress toward that in my spare time, I am not quite there yet. Once we have them, it will become a negligible drain on time and could be done by anyone when the tests are properly automated, but that would also eliminate the circumstances that lead to your offer of assistance.

That said, I would love assistance in developing automated tests. If you are interested in assisting in the development of those, please contact me in #zfsonlinux on the freenode IRC network. That is by far the most efficient way to reach me at the moment.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Apr 30, 2015 1:14 pm    Post subject: Reply with quote

FearedBliss wrote:


It is ready when it is ready. Asking/demanding for updates when this is completely a volunteer project won't help anyone. Also ryao shouldn't need to explain himself on why he hasn't pushed the builds.

My overlay is highly volatile btw. Most of the stuff I push there is pretty much for myself (I just make it public), don't blame me if stuff breaks or vanishes.



Acknowledged,

so I hope you don't mind if I forked your repo immediately & uploaded your scripts to have a non-moving target :wink:

FearedBliss wrote:


@kernelOfTruth

I see you are maintaining some zfs srm stuff. I made this a year ago and it automates the process:
https://github.com/fearedbliss/sysresccd-moddat

You can also find the latest isos here (with mirrors): http://xyinn.org/doku.php


Thank you very much ! :D

that should definitely make things easier,

I'll take a deeper look in my free time
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Fri May 01, 2015 2:20 pm    Post subject: Reply with quote

kernelOfTruth wrote:
FearedBliss wrote:


It is ready when it is ready. Asking/demanding for updates when this is completely a volunteer project won't help anyone. Also ryao shouldn't need to explain himself on why he hasn't pushed the builds.

My overlay is highly volatile btw. Most of the stuff I push there is pretty much for myself (I just make it public), don't blame me if stuff breaks or vanishes.



Acknowledged,

so I hope you don't mind if I forked your repo immediately & uploaded your scripts to have a non-moving target :wink:


If someone doesn't want others to be able to fork their software, they shouldn't make it OSS. I know of very few people whose idea of OSS is "look, but don't touch". While I am not a lawyer, I believe all software under an OSI-compliant license is licensed in such a way that forking is legal, which includes all of the OpenZFS/ZFSOnLinux code, all Gentoo-specific code, all OSS and "F/OSS" as per the FSF (which I believe to be a subset of OSS). Consequently, how others feel about it does not restrict you from taking it and doing your own things. If necessary, I have access to lawyers who I could ask, but I do not consider this to be a matter that requires confirmation.

Also, while I might make some comments with regard to the need for rigorous testing to minimize the potential for regressions, I do not intend to stop anyone from taking the code and doing what they want with it. I am a strong believer in the idea that we should share code without trying to force our own individual visions of how it would be used onto others. The 9999 ZFS ebuilds in the tree should facilitate that, with the caveat that less QA is done, especially if pointed at a private development branch. If you want to do this, you can put something along the lines of the following into make.conf:

Code:
#spl_LIVE_BRANCH=master
spl_LIVE_REPO='/home/richard/devel/spl'
#zfs_kmod_LIVE_BRANCH=master
zfs_kmod_LIVE_REPO='/home/richard/devel/zfs'
#zfs_LIVE_BRANCH=master
zfs_LIVE_REPO='/home/richard/devel/zfs'


The 9999 ebuilds will honor those settings. Change the paths/urls to valid git repositories and you can build from any valid repository. You can even specify a branch (or even a precise commit via ${PKGNAME}_LIVE_COMMIT if I recall correctly). I use this myself when developing changes. My workstation's packages are often built in this manner. I used to use epatch_user for testing changes, but this was much more convenient, especially after I combined it with 9p-virtio such that I could build on the host for the guest without having to deal with mounting block devices or synchronization issues:

https://wiki.gentoo.org/wiki/ZFSOnLinux_Development_Guide#Gentoo_with_a_9p-virtio_rootfs

If anyone does do this, keep in mind NOT to fetch code from unencrypted remote repositories. Doing that leaves you open to the insertion of malicious code into the kernel through a MITM attack on the fetch itself. I am not aware of any instances of this attack in the wild, but it is something that merits attention. It is also why I update my local tree used for updates via emerge-webrsync with PGP verification. In the interest of full disclosure, the tree that I use for Gentoo commits is updated via a SSH-encrypted tunnel authenticated by a 2048-bit RSA keypair that was generated with sane entropy.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat May 02, 2015 1:16 pm    Post subject: Reply with quote

ryao wrote:
kernelOfTruth wrote:
FearedBliss wrote:


It is ready when it is ready. Asking/demanding for updates when this is completely a volunteer project won't help anyone. Also ryao shouldn't need to explain himself on why he hasn't pushed the builds.

My overlay is highly volatile btw. Most of the stuff I push there is pretty much for myself (I just make it public), don't blame me if stuff breaks or vanishes.



Acknowledged,

so I hope you don't mind if I forked your repo immediately & uploaded your scripts to have a non-moving target :wink:


If someone doesn't want others to be able to fork their software, they shouldn't make it OSS. I know of very few people whose idea of OSS is "look, but don't touch". While I am not a lawyer, I believe all software under an OSI-compliant license is licensed in such a way that forking is legal, which includes all of the OpenZFS/ZFSOnLinux code, all Gentoo-specific code, all OSS and "F/OSS" as per the FSF (which I believe to be a subset of OSS). Consequently, how others feel about it does not restrict you from taking it and doing your own things. If necessary, I have access to lawyers who I could ask, but I do not consider this to be a matter that requires confirmation.



Thank you very much for that !

Yes, that "look, but don't touch" was or still is very prominent in some opensource projects (over at #XDA a few projects can't even be called opensource anymore since some devs add changes and hide their work [e.g. kernels in binary form] behind a paywall and never publishing the source changes) which is sad - since the goal shall be the (public) improvement of the code as a common gain in quality of work.

Also since the code is published under the MPL 2.0 it should be fine - so no need of lawyers in that case :)


ryao wrote:


Also, while I might make some comments with regard to the need for rigorous testing to minimize the potential for regressions, I do not intend to stop anyone from taking the code and doing what they want with it. I am a strong believer in the idea that we should share code without trying to force our own individual visions of how it would be used onto others. The 9999 ZFS ebuilds in the tree should facilitate that, with the caveat that less QA is done, especially if pointed at a private development branch. If you want to do this, you can put something along the lines of the following into make.conf:

Code:
#spl_LIVE_BRANCH=master
spl_LIVE_REPO='/home/richard/devel/spl'
#zfs_kmod_LIVE_BRANCH=master
zfs_kmod_LIVE_REPO='/home/richard/devel/zfs'
#zfs_LIVE_BRANCH=master
zfs_LIVE_REPO='/home/richard/devel/zfs'


The 9999 ebuilds will honor those settings. Change the paths/urls to valid git repositories and you can build from any valid repository. You can even specify a branch (or even a precise commit via ${PKGNAME}_LIVE_COMMIT if I recall correctly). I use this myself when developing changes. My workstation's packages are often built in this manner. I used to use epatch_user for testing changes, but this was much more convenient, especially after I combined it with 9p-virtio such that I could build on the host for the guest without having to deal with mounting block devices or synchronization issues:

https://wiki.gentoo.org/wiki/ZFSOnLinux_Development_Guide#Gentoo_with_a_9p-virtio_rootfs

If anyone does do this, keep in mind NOT to fetch code from unencrypted remote repositories. Doing that leaves you open to the insertion of malicious code into the kernel through a MITM attack on the fetch itself. I am not aware of any instances of this attack in the wild, but it is something that merits attention. It is also why I update my local tree used for updates via emerge-webrsync with PGP verification. In the interest of full disclosure, the tree that I use for Gentoo commits is updated via a SSH-encrypted tunnel authenticated by a 2048-bit RSA keypair that was generated with sane entropy.


And using and/or modifying your live ebuilds worked wonderfully !

well, obviously I followed a way much too complicated approach if I understood your lines correctly:

I first uploaded the git branch to github and from there let it fetch by portage

does that mean that e.g.

Code:
spl_LIVE_REPO='/home/richard/devel/spl'


makes you autonomous from github - so that the repo is fetched from your /home ?

didn't know that was possible 8O

Excellent !


ryao wrote:


If anyone does do this, keep in mind NOT to fetch code from unencrypted remote repositories. Doing that leaves you open to the insertion of malicious code into the kernel through a MITM attack on the fetch itself. I am not aware of any instances of this attack in the wild, but it is something that merits attention. It is also why I update my local tree used for updates via emerge-webrsync with PGP verification. In the interest of full disclosure, the tree that I use for Gentoo commits is updated via a SSH-encrypted tunnel authenticated by a 2048-bit RSA keypair that was generated with sane entropy.



It appears that I need to take a deeper look into this,

I'm already syncing my portage tree mainly via webrsync but one can never be too cautious
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Sun May 03, 2015 8:25 pm    Post subject: Reply with quote

kernelOfTruth wrote:
ryao wrote:


Also, while I might make some comments with regard to the need for rigorous testing to minimize the potential for regressions, I do not intend to stop anyone from taking the code and doing what they want with it. I am a strong believer in the idea that we should share code without trying to force our own individual visions of how it would be used onto others. The 9999 ZFS ebuilds in the tree should facilitate that, with the caveat that less QA is done, especially if pointed at a private development branch. If you want to do this, you can put something along the lines of the following into make.conf:

Code:
#spl_LIVE_BRANCH=master
spl_LIVE_REPO='/home/richard/devel/spl'
#zfs_kmod_LIVE_BRANCH=master
zfs_kmod_LIVE_REPO='/home/richard/devel/zfs'
#zfs_LIVE_BRANCH=master
zfs_LIVE_REPO='/home/richard/devel/zfs'


The 9999 ebuilds will honor those settings. Change the paths/urls to valid git repositories and you can build from any valid repository. You can even specify a branch (or even a precise commit via ${PKGNAME}_LIVE_COMMIT if I recall correctly). I use this myself when developing changes. My workstation's packages are often built in this manner. I used to use epatch_user for testing changes, but this was much more convenient, especially after I combined it with 9p-virtio such that I could build on the host for the guest without having to deal with mounting block devices or synchronization issues:

https://wiki.gentoo.org/wiki/ZFSOnLinux_Development_Guide#Gentoo_with_a_9p-virtio_rootfs

If anyone does do this, keep in mind NOT to fetch code from unencrypted remote repositories. Doing that leaves you open to the insertion of malicious code into the kernel through a MITM attack on the fetch itself. I am not aware of any instances of this attack in the wild, but it is something that merits attention. It is also why I update my local tree used for updates via emerge-webrsync with PGP verification. In the interest of full disclosure, the tree that I use for Gentoo commits is updated via a SSH-encrypted tunnel authenticated by a 2048-bit RSA keypair that was generated with sane entropy.


And using and/or modifying your live ebuilds worked wonderfully !

well, obviously I followed a way much too complicated approach if I understood your lines correctly:

I first uploaded the git branch to github and from there let it fetch by portage

does that mean that e.g.

Code:
spl_LIVE_REPO='/home/richard/devel/spl'


makes you autonomous from github - so that the repo is fetched from your /home ?

didn't know that was possible 8O

Excellent !


You would want to do something like:

Code:
mkdir ~/devel
git clone https://github.com/zfsonlinux/spl.git ~/devel/spl


Then do whatever changes you want there, commit them to a branch and tell the ebuild to look there through either portage's make.conf (like I did) or an environment variable. Once you do that, there is no need for a network connection to use the live ebuilds.
Back to top
View user's profile Send private message
fearedbliss
Retired Dev
Retired Dev


Joined: 13 Jul 2012
Posts: 4
Location: Brooklyn, NY

PostPosted: Sun May 03, 2015 11:28 pm    Post subject: Reply with quote

kernelOfTruth wrote:
FearedBliss wrote:


It is ready when it is ready. Asking/demanding for updates when this is completely a volunteer project won't help anyone. Also ryao shouldn't need to explain himself on why he hasn't pushed the builds.

My overlay is highly volatile btw. Most of the stuff I push there is pretty much for myself (I just make it public), don't blame me if stuff breaks or vanishes.




I agree with what ryao said about OSS development. I published my stuff because I want people to use it. Feel free to fork and even burn the overlay on cds LOL.
Back to top
View user's profile Send private message
Manuel.G
n00b
n00b


Joined: 07 Jan 2015
Posts: 8

PostPosted: Thu May 14, 2015 6:26 pm    Post subject: Reply with quote

Yamakuzure wrote:
I have just upgraded and have a question about that ewarn:
Code:
 * Messages for package sys-fs/zfs-kmod-0.6.4:

 * This version of ZFSOnLinux includes support for new feature flags
 * that are incompatible with ZFSOnLinux 0.6.3 and GRUB2 support for
 * /boot with the new feature flags is not yet available.
 * Do *NOT* upgrade root pools to use the new feature flags.
 * Any new pools will be created with the new feature flags by default
 * and will not be compatible with older versions of ZFSOnLinux. To
 * create a newpool that is backward compatible, use
 * zpool create -o version=28 ...
 * Then explicitly enable older features. Note that the LZ4 feature has
 * been upgraded to support metadata compression and has not been
 * tested against the older GRUB2 code base. GRUB2 support will be
 * updated as soon as the GRUB2 developers and Open ZFS community write
 * GRUB2 patchese that pass mutual review.


I've got a question regarding the ewarn as well: My complete / is on a ZFS pool, including /boot. That pool was created with 0.6.3-r1. After upgrading to the latest kernel and ZoL 0.6.4, the corresponding initramfs gives me an error when it tries to import my root pool. ("no dataset found")
As far as I understand the warning, it should mount without problems, I just should not upgrade the pool to the latest version before grub2 has been updated for that. Am I misunderstanding and I currently can't upgrade to 0.6.4 if I need to boot from ZFS? I wouldn't mind that, 0.6.3 is running perfectly fine for me, I'm just asking as I'm not 100% sure if I understand the ewarn correctly. If I'm understanding it right, the problem must be somewhere else in my new initramfs/kernel, I guess.

Thanks in advance!
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Fri May 15, 2015 8:46 am    Post subject: Reply with quote

Manuel.G wrote:
Yamakuzure wrote:
I have just upgraded and have a question about that ewarn:
Code:
 * Messages for package sys-fs/zfs-kmod-0.6.4:

 * This version of ZFSOnLinux includes support for new feature flags
 * that are incompatible with ZFSOnLinux 0.6.3 and GRUB2 support for
 * /boot with the new feature flags is not yet available.
 * Do *NOT* upgrade root pools to use the new feature flags.
 * Any new pools will be created with the new feature flags by default
 * and will not be compatible with older versions of ZFSOnLinux. To
 * create a newpool that is backward compatible, use
 * zpool create -o version=28 ...
 * Then explicitly enable older features. Note that the LZ4 feature has
 * been upgraded to support metadata compression and has not been
 * tested against the older GRUB2 code base. GRUB2 support will be
 * updated as soon as the GRUB2 developers and Open ZFS community write
 * GRUB2 patchese that pass mutual review.


I've got a question regarding the ewarn as well: My complete / is on a ZFS pool, including /boot. That pool was created with 0.6.3-r1. After upgrading to the latest kernel and ZoL 0.6.4, the corresponding initramfs gives me an error when it tries to import my root pool. ("no dataset found")
As far as I understand the warning, it should mount without problems, I just should not upgrade the pool to the latest version before grub2 has been updated for that. Am I misunderstanding and I currently can't upgrade to 0.6.4 if I need to boot from ZFS? I wouldn't mind that, 0.6.3 is running perfectly fine for me, I'm just asking as I'm not 100% sure if I understand the ewarn correctly. If I'm understanding it right, the problem must be somewhere else in my new initramfs/kernel, I guess.

Thanks in advance!
Weird. As long as you do not upgrade your pool, it is fully compatible with zfs-0.6.3 and remains so. I did upgrade to sys-kernel/genkernel-next-60 before creating my new initramfs, though.

And I can confirm that if your root is on zfs, but your /boot is not, you can happily upgrade.

So just put your /boot on a tiny ext2 partition and you should be fine.

(Here is a blog describing how to shrink zfs pools.)
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
Manuel.G
n00b
n00b


Joined: 07 Jan 2015
Posts: 8

PostPosted: Fri May 15, 2015 10:05 am    Post subject: Reply with quote

Thanks for the reply! But shouldn't my current setup, with /boot being in my root pool, still work? The ewarn says "GRUB2 support for /boot with the new feature flags is not yet available", but my /boot is inside an old 0.6.3 pool, so it is not "/boot with the new feature flags". That's why I'm asking – the way the ewarn states it, I think it should just work. I only want to clarify this before starting to debug other things. Should I have misunderstood the ewarn, I'd rather wait for the GRUB2 update than fiddle with my root pool and /boot, as my current setup works just fine. :)
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Tue May 19, 2015 6:31 am    Post subject: Reply with quote

Manuel.G wrote:
Thanks for the reply! But shouldn't my current setup, with /boot being in my root pool, still work? The ewarn says "GRUB2 support for /boot with the new feature flags is not yet available", but my /boot is inside an old 0.6.3 pool, so it is not "/boot with the new feature flags". That's why I'm asking – the way the ewarn states it, I think it should just work. I only want to clarify this before starting to debug other things. Should I have misunderstood the ewarn, I'd rather wait for the GRUB2 update than fiddle with my root pool and /boot, as my current setup works just fine. :)
That's how I understood the ewarn, too. As long as you do not upgrade the pool, grub should be happily keep booting.
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Jun 13, 2015 7:39 pm    Post subject: Reply with quote

@ryao:

thank you

@FearedBliss:

thanks !




@ryao:

perhaps you or the others can explain where all the memory goes:


I ran a stage4 backup job shortly after the system started up

this saves everything on the root partition [Btrfs]

(excluding /home [ZFS], /usr/portage [Btrfs])


Code:
time (ionice -c3 schedtool -D -e tar -cp / -X /root/scripts/stage4.excl | ionice -c3 schedtool -D -e 7z a -si -t7z -m0=lzma2 -mx=7 -mfb=64 -md=64m -ms=on -mmt=8 /home/system/stage4-amd64-xeon-backup-197.t7z)





space consumption of / (root)

Quote:
35G 15G 17G 47% /


Size Used Avail Use%



Worth mentioning is that there's also a 20 GB zram drive in RAM which was empty:

Code:
/dev/zram0                                                                                                      20G  191M   20G   1% /var/tmp




zfs.conf:

Code:
options zfs zfs_arc_meta_limit=6442450944

#(6 GB)

options zfs zfs_arc_min=0x100000000

#(4 GB)

options zfs zfs_arc_max=0x200000000

#(8 GB)




Do the caches of the ARC + ARC meta add up or is the ARC meta included in the ARC (logically it should, right ?)


The strange thing is:

cat /proc/meminfo

Quote:
Slab: 1292764 kB
SReclaimable: 883696 kB
SUnreclaim: 409068 kB



cat /proc/spl/kstat/zfs/arcstats

Quote:
p 4 6041380352
c 4 8589934592
c_min 4 4294967296
c_max 4 8589934592
size 4 6401223376
hdr_size 4 25559456
data_size 4 6218892288
meta_size 4 113704448
other_size 4 38928912





Total RAM available is 32 GiB


When 75% of the backup job is done:

cat /proc/buddyinfo
Code:
Node 0, zone      DMA      2      2      2      0      3      2      0      0      1      1      3
Node 0, zone    DMA32   2244   1526   3282   1883   1155    643    411    277    153     79     86
Node 0, zone   Normal  17170   4968  18074  14592   8752   4878   3132   1961   1161    680    512



7z consumes 4 GB when it's running



When the backup job is finished and manually running compaction:

cat /proc/buddyinfo
Code:
Node 0, zone      DMA      2      2      2      0      3      2      0      0      1      1      3
Node 0, zone    DMA32   1490    914    347     53    123     74     37     18      7     48    163
Node 0, zone   Normal  12498   6803   2723    230    601    416    219     72     21    293   1066





Using the nvidia (proprietary) or the nouveau driver makes no difference


right now Btrfs consumes around 650 MB

Quote:
631257 631256 99% 0.95K 19129 33 612128K btrfs_inode
198441 191889 96% 0.08K 3891 51 15564K btrfs_extent_state



Running a

Code:
echo 1 > /proc/sys/vm/drop_caches


doesn't really make much of a difference

Quote:
Slab: 1167000 kB
SReclaimable: 756032 kB
SUnreclaim: 410968 kB


echo 3 > /proc/sys/vm/drop_caches

regularly leads to a loop, hang, etc. which does NOT finish and also does NOT help (at least the last time I tried - so I'm leaving that out)






Slub merges are enabled (NO slub_nomerge),

using slub_nomerge did not really make a difference in this behavior



All this points to heavy memory fragmentation right ?

Not many high order allocations are possible anymore


Around 2-2.5 GiB of RAM are used by apps (okular, evince, firefox, wine, xfce, konqueror)



Running ZFS with the ABD or the Lock contention fix patchstack did not make any difference



Last thing I could think of is disabling zram entirely (previously I had a swap-partition in RAM),

now only /var/tmp in RAM with zram

and see whether that makes a change - which I doubt



Tl;dr

is it possible that ZFS leads to such grave memory fragmentation ?

how to reverse or limit it ?


The only thing right now to reset it is to export the zpool


There must be a better way :?



edit:


it seems dropping caches and running compaction, then giving it some time - and dropping caches and running compaction again - helps somewhat:


Quote:
cat /proc/buddyinfo
Node 0, zone DMA 2 2 2 0 3 2 0 0 1 1 3
Node 0, zone DMA32 4850 3202 1724 780 391 277 199 141 108 85 432
Node 0, zone Normal 27725 18747 10470 5225 1982 935 402 143 66 151 4237


that should be more or less okay



Anyway - I filed an issue over at ZFS' github issue-tracker
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Jun 19, 2015 1:36 am    Post subject: Reply with quote

ZFS lower latency settings

Code:
echo 128 > /sys/module/zfs/parameters/zfs_vdev_max_active
# default zfs_vdev_max_active = 1000

echo 3 > /sys/module/zfs/parameters/zfs_vdev_async_write_max_active
# default zfs_vdev_async_write_max_active = 10
echo 5 > /sys/module/zfs/parameters/zfs_vdev_sync_write_max_active
# default zfs_vdev_sync_write_max_active = 10
echo 3 > /sys/module/zfs/parameters/zfs_vdev_async_read_max_active
# default zfs_vdev_async_read_max_active = 3
echo 10 > /sys/module/zfs/parameters/zfs_vdev_sync_read_max_active
# default zfs_vdev_sync_read_max_active = 10

echo 5 > /sys/module/zfs/parameters/zfs_vdev_sync_write_min_active
# default zfs_vdev_sync_write_min_active = 10
echo 1 > /sys/module/zfs/parameters/zfs_vdev_async_read_min_active
# default zfs_vdev_async_read_min_active = 1
echo 5 > /sys/module/zfs/parameters/zfs_vdev_sync_read_min_active
# default zfs_vdev_sync_read_min_active = 10
echo 1 > /sys/module/zfs/parameters/zfs_vdev_async_write_min_active
# default zfs_vdev_async_write_min_active = 1

echo 2 > /sys/module/zfs/parameters/zfs_txg_timeout
# default zfs_txg_timeout = 5



With the default settings I'm getting latencies up to 3000 ms+ when running rsync or copying big stuff (measured with latencytop and the CFS cpu scheduler)


With those settings the latencies are down to 200-300 ms (so far)


For giggles you could try out setting all of the above to 1

except zfs_vdev_max_active

which might work even better :wink:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Fri Jun 19, 2015 7:09 am    Post subject: Reply with quote

Alright, I have put your commands into a starter file in /etc/local.d. Let's see what it does.

Good find! Thank you very much!

Maybe I will try out bfq again. I have switched to no-op yesterday, because bfq started to make my system unusable again.
Before the latest eudev and udev-init-scripts update, bfq worked like a charm. Of course I had set /sys/module/zfs/parameters/zfs_vdev_scheduler to "bfq", too. But it no longer works whenever I have a Windows 7 VM open, which I must have all day for my companies project managing. We use Inloox, so no chance without a modern MS Outlook. :-(
Whenever the VM is running, clicking anywhere in a regular window outside the VM causes VMware to go up to 300% CPU usage (without actually doing anything) and the system hangs for 3-5 seconds. Even if the click just moves the cursor in a text editor.
With No-Op, all is well. Not as snappy as with bfq, but well enough.
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
MTZ
Tux's lil' helper
Tux's lil' helper


Joined: 13 Jul 2002
Posts: 90
Location: Germany - near FFM

PostPosted: Sat Jun 27, 2015 6:09 am    Post subject: Reply with quote

Next version got released:

Quote:

Version 0.6.4.2 of the zfs-0.6.4-release branch is now available. This
branch receives low-risk changes and important bug fixes made since the
last major release. New packages based on this release should make their
way into downstream package repositories in the near future.

For a list of changes in this release see below or refer to the release page
at https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4.2

* Fix panic due to corrupt nvlist when running utilities zfsonlinux/zfs#3335
* Fix hard lockup due to infinite loop in zfs_zget() zfsonlinux/zfs#3349
* Fix panic on unmount due to iput taskq zfsonlinux/zfs#3281
* Improve metadata shrinker performance on pre-3.1 kernels zfsonlinux/zfs#3501
* Linux 4.1 compat: use read_iter() / write_iter()
* Linux 3.12 compat: NUMA-aware per-superblock shrinker
* Fix spurious hung task watchdog stack traces zfsonlinux/zfs#3402
* Fix module loading in zfs import systemd service zfsonlinux/zfs#3440
* Fix intermittent libzfs_init() failure to open /dev/zfs zfsonlinux/zfs#2556
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Jun 27, 2015 10:16 am    Post subject: Reply with quote

If you are heavily using the L2ARC (so that it'll get written "over capacity" several times over)


the counter for size calculation might overflow and that would lead to the infamous

16.0E result in zpool list

and performance could come to a crawling halt


There's already fixes in latest master

https://github.com/zfsonlinux/zfs/commit/fa720217b9765303aaa882a9ccdf70c185acc53d
https://github.com/zfsonlinux/zfs/commit/d962d5dad9dae20dec096abe73f22a4c813199fd
https://github.com/zfsonlinux/zfs/commit/ef56b0780c80ebb0b1e637b8b8c79530a8ab3201


with stability reasons in mind these haven't been included in this release yet



Especially https://github.com/zfsonlinux/zfs/commit/ef56b0780c80ebb0b1e637b8b8c79530a8ab3201 got some wider testing already related to

https://github.com/zfsonlinux/zfs/issues/3400 , https://github.com/zfsonlinux/zfs/pull/3216 and other use-cases


the other 2 patches (https://github.com/zfsonlinux/zfs/commit/d962d5dad9dae20dec096abe73f22a4c813199fd - Illumos 5701) aren't tested that much yet


The buildbots have currently a limitation: they don't or only little are capable to test the L2ARC

probably therefore that caution



Feel free to add it, if needed

and give it a good testing.

It is advised to have a separated backup when attempting to do so for some time



Enjoy !
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Jun 27, 2015 10:19 pm    Post subject: Reply with quote

If you're tinkering with ARC values - the following hex, byte and value conversion list might be helpful:


Quote:
Here is a list of hex conversions for common sizes : bytes memory pages
20Gb=0x500000000=21474836480 5242880 16Gb=0x400000000=17179869184 4194304 12Gb=0x300000000=12884901888 3145728 10Gb =0x280000000=10737418240 2621440 8Gb =0x200000000=8589934592 2097152 7Gb 0x1C0000000=7516192768 1835008 6Gb =0x180000000 = 6442450944 1572864 5Gb=0x140000000 =5368709120 1310720 4Gb=0x100000000= 4294967296 1048576 3.75Gb=0xF0000000 =4026531840 983040 3Gb=0xC0000000= 3221225472 786432 2.75Gb=0xB0000000= 2952790016 720896 2.5Gb =0xA0000000= 2684354560 655360 2Gb = 0x80000000 =2147483648 524288 1.92GB =0x7B03A000=2063835136 503866 1.75Gb= 0x70000000 =1879048192 458752 1.5Gb=0x60000000= 1610612736 393216 1Gb = 0X40000000 = 1073741824 262144 944Mb =0x3B031000 = 990056448 241713 940Mb=0XEC00000000=985661440 240640 880Mb= 0x37000000=922746880 225280 768Mb= 0X30000000=805606368 196681 ( 3/4 of 1 gb quad) 512Mb=0x20000000 = 536870912 131072 383Mb=0x17F00000=401604608 98048 256Mb=0x10000000 = 268435456 65536 128Mb=0x8000000 = 134217728 32768 80Mb =0x5000000 =83886080 20480 64MB= 0X4000000 = 67108864 16384 16Mb =0x1000000=16777216 4096 8MB=0x800000=8388608 2048


origin:

http://support.hp.com/si-en/document/c00936430
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sun Jun 28, 2015 5:06 pm    Post subject: Reply with quote

Blimey KoT that's a load of hassle; just use printf and shell arithmetic (bash has bitwise operators):
Code:
$ printx $((1 << 12))
1000
$ printd $((1 << 12))
4096
$ printd $((1 << 20))
1048576
$ printx $((1 << 20))
100000
from my ~/.bashrc:
Code:
# scripting
alias print="printf '%s\n'"
alias printl="printf '%s\n'"
# handy for code at cli
alias printx="printf '%X\n'"
alias printd="printf '%d\n'"
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Jul 21, 2015 8:42 am    Post subject: Reply with quote

@steveL:

Thanks mate !


It appears I need to learn more of the basics :roll:


Very useful :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1221

PostPosted: Wed Jul 29, 2015 12:26 pm    Post subject: Reply with quote

any info on 0.6.4 -> 0.6.4.2 changes ?

[edit]
I just have tried 0.6.4.2 on gentoo-sources 3.14.47 (which works great with 0.6.3 for a long time) and after boot I had no pools visible.
modules were there:
Code:
zfs                  1825152  0
zunicode              315324  1 zfs
zavl                    3753  1 zfs
zcommon                30124  1 zfs
znvpair                37112  2 zfs,zcommon
spl                    46914  3 zfs,zcommon,znvpair

and service was started, but no pools.

Code:
/etc/init.d/zfs status
no pools available

no pools available


I had to revert to 0.6.3 and after reboot everything was fine.
the only thing that I notices after this short 0.6.4.2 test, is that now in 0.6.3, when I make zpool status, I get:

Code:
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.


any clue?
[/edit]
_________________
gentoo user


Last edited by tnt on Wed Jul 29, 2015 5:01 pm; edited 1 time in total
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Jul 29, 2015 4:58 pm    Post subject: Reply with quote

tnt wrote:
any info on 0.6.4 -> 0.6.4.2 changes ?


http://zfsonlinux.org/

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4.2

Quote:

Bug Fixes

Fix panic due to corrupt nvlist when running utilities zfsonlinux/zfs#3335
Fix hard lockup due to infinite loop in zfs_zget() zfsonlinux/zfs#3349
Fix panic on unmount due to iput taskq zfsonlinux/zfs#3281
Improve metadata shrinker performance on pre-3.1 kernels zfsonlinux/zfs#3501
Linux 4.1 compat: use read_iter() / write_iter()
Linux 3.12 compat: NUMA-aware per-superblock shrinker
Fix spurious hung task watchdog stack traces zfsonlinux/zfs#3402
Fix module loading in zfs import systemd service zfsonlinux/zfs#3440
Fix intermittent libzfs_init() failure to open /dev/zfs zfsonlinux/zfs#2556




https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4.1

Quote:
Bug Fixes

Fixed io-spare.sh script for ZED.
Fixed multiple deadlocks which might occur when reclaiming memory.
Fixed excessive CPU usage for meta data heavy workloads when reclaiming the ARC

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
mrbassie
l33t
l33t


Joined: 31 May 2013
Posts: 771
Location: over here

PostPosted: Thu Jul 30, 2015 7:05 am    Post subject: Reply with quote

tnt wrote:


Code:
/etc/init.d/zfs status
no pools available

no pools available


I had to revert to 0.6.3 and after reboot everything was fine.
[/edit]


That happened to me when I upgraded from 0.6.2-r4 to 0.6.3. I very cleverly thought 'Hmm, maybe I need to do zpool upgrade' and had to rebuild the pool and restore a stage 4 (/ on zfs) because I then couldn't downgrade.

More recently I went from 0.6.2-r4 to r5 to 0.6.3 to 0.6.4 to .1 then .2. Everything went fine.

Sorry that I don't know enough to help you troubleshoot but at least you're aware you're not the only one who's encountered this.
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1221

PostPosted: Thu Jul 30, 2015 7:31 am    Post subject: Reply with quote

thx either way, mrbassie.

at least I saw no pools in 0.6.4.2, so I had no chance to "zpool upgrade" and kill my 40TB :)
_________________
gentoo user
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 6 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum