Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
ZFSonLinux, SystemRescueCD up-to-date kernel-modules (4.3.0)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Sep 05, 2014 3:59 pm    Post subject: ZFSonLinux, SystemRescueCD up-to-date kernel-modules (4.3.0) Reply with quote

Hi guys,


in case you're running a rather "bleeding-edge" state of the ZFS zpools on your system ("feature flags"),
trying to gain access to it in a case of emergency (e.g. system isn't able to run, has been b0rked, filesystem corruption, etc.)
could be quite difficult.

My little new project might be of help to you:

https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD

Quote:
target build architecture of the modules was march=core2

so it should run on the most hardware, if you run into issues please report, will have to re-compile all stuff, then



in a nutshell all you need is:

nutshell wrote:
* an usbstick running SystemRescueCD 4.3.0, access to internet
* adding those modules to the usbstick, rebooting
* and you should have access to your pool :)



I hope that will be as helpful to you as it is to me

Enjoy !
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Sep 16, 2014 9:45 pm    Post subject: Reply with quote

2nd release is out

of the updated modules (still for 4.3.0)



https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/commits/ZFS-for-SysRescCD-4.3.0-r2


Changelog wrote:

still on SystemRescueCD 4.3.0
sys-kernel/spl at f9bde4f74be5e5030571415d9c869400dff3eda6
sys-fs/zfs-kmod & sys-fs/zfs at 2d501583430e54be39c6aed033fc6f841e469f23
added 2484 patchset (zvol rework)
added 2351 patchset (sha256 x86_64 optimization v2; SSSE3, AVX, AVX2)
added 2672 patchset (Illumos #3525 persistent l2arc)
kernel-modules compiled with march=nocona, should work on older 64bit boxes now

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
Yamakuzure
Veteran
Veteran


Joined: 21 Jun 2006
Posts: 1447
Location: Bardowick, Germany

PostPosted: Wed Sep 17, 2014 8:45 am    Post subject: Reply with quote

Is this the same as the Funtoo sysresccd with ZFS or did you add more to it? How do they differ apart from the version used? (Funtoo still uses 4.2.0)
_________________
systemd - The biggest fallacies
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2949
Location: The Peanut Gallery

PostPosted: Wed Sep 17, 2014 3:21 pm    Post subject: Reply with quote

Nice work, koT: keep it up :-)
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Sep 17, 2014 5:57 pm    Post subject: Reply with quote

Yamakuzure wrote:
Is this the same as the Funtoo sysresccd with ZFS or did you add more to it? How do they differ apart from the version used? (Funtoo still uses 4.2.0)


Funtoo's SysRescCD is whole on it's own


in principle yes: unfortunately I couldn't find the bliss-isomaker since the creator totally nuked the repo with it from github's existance :( , bliss-initramfs 1.8.1 luckily still was on github's history :o


so it's the up-to-date kernel modules from yesterday - without the integration into the iso

the modules are bleeding edge in contrast to Funtoo's stable (release) modules and contain additional patches

this should make deduplication, usage of the L2ARC and heavy zvol usage more bearable

I've had some issues with zvol-creation and 2484 in the past so YMMV



otherwise I would have probably used an bleeding-edge 3.16 kernel with latest btrfs integration-branch code (which really seems to behave tons more reasonable than in the past)



steveL wrote:
Nice work, koT: keep it up :-)


thanks mate :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
nitro322
Guru
Guru


Joined: 24 Jul 2002
Posts: 553
Location: USA

PostPosted: Fri Oct 03, 2014 3:59 am    Post subject: Reply with quote

Thanks, kernelOfTruth. I'm in the middle of testing out ZFS for a new home server right now and this was one of my areas of concern. I was able to successfully import/mount, update, and then export/unmount my ZFS filesystem from within syscresccd after adding your kernel modules to the image. Much appreciated.
_________________
http://www.legroom.net/
Back to top
View user's profile Send private message
e3k
Apprentice
Apprentice


Joined: 01 Oct 2007
Posts: 200
Location: Slovakia

PostPosted: Thu Oct 09, 2014 8:36 pm    Post subject: Reply with quote

i used the readme provided by systemrescuecd to add your srms but when i run zpool i get a message that zfs.ko was not found. i am building my own rescue cd for the first time do you have more instructions for me than just in the nutshell?
_________________
---__(o)__---
  | |
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Oct 09, 2014 9:45 pm    Post subject: Reply with quote

nitro322 wrote:
Thanks, kernelOfTruth. I'm in the middle of testing out ZFS for a new home server right now and this was one of my areas of concern. I was able to successfully import/mount, update, and then export/unmount my ZFS filesystem from within syscresccd after adding your kernel modules to the image. Much appreciated.


My pleasure !

Glad it was of use to you :D

e3k wrote:
i used the readme provided by systemrescuecd to add your srms but when i run zpool i get a message that zfs.ko was not found. i am building my own rescue cd for the first time do you have more instructions for me than just in the nutshell?


which readme are you referring to ?

the one in my repo or the following one: http://www.sysresccd.org/Modules ?

anyway - posting the Howto here:

HOWTO wrote:

-----------------------------------------------------------------------------------------------------------------------
- when using SystemRescueCD 4.3.0 on an USB stick - everything needed to do is
- follow the instructions from http://www.sysresccd.org/Sysresccd-manual-en_How_to_install_SystemRescueCd_on_an_USB-stick
- have the usbstick mounted (mount -t vfat /dev/sdf1 /mnt/usbstick)
- then copy over the zfs-core and zfs-kmod SRM files to the root of the usbstick(zfs-core-3.10.45-std430-amd64.srm,
zfs-core-3.14.9-alt430-amd64.srm, zfs-kmod-3.10.45-std430-amd64.srm, zfs-kmod-3.14.9-alt430-amd64.srm)
- make sure that the usbstick is bootable, otherwise rewrite syslinux to the device (e.g. syslinux /dev/sdf1 && sync)
- ...
- during boot choose the 64bit versions (both default and alternative are supported), pick your keymap configuration
- after boot has finished run 'depmod'
- then a simple 'modprobe zfs' should be enough
- check state and version of zfs with 'zpool upgrade -v'
- Enjoy !



hope that helps in some way

if anything's missing don't hesitate to ask and please write what steps you already did or where it fails
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
e3k
Apprentice
Apprentice


Joined: 01 Oct 2007
Posts: 200
Location: Slovakia

PostPosted: Fri Oct 10, 2014 8:31 am    Post subject: Reply with quote

thank you it is working now:
mounting the sysrescuecd.iso -o loop
running the usb_inst.sh from the mount dir root
copying your srms
rebooting
depmod
modprobe zfs
_________________
---__(o)__---
  | |
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Oct 10, 2014 11:46 pm    Post subject: Reply with quote

awesome ! :)


updated forum signature & github to point to latest branch
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2769
Location: Pittsburgh, PA, USA

PostPosted: Thu Nov 13, 2014 5:57 pm    Post subject: Reply with quote

@kernelOfTruth

Thanks a lot. This greatly helped me make my own sysrescue iso. I needed that because I could not use your latest release on an old Athlon 64 X2 system and the funtoo version does not support git zfs..

Also I needed an ISO because I use YUMI to make a bootable USB stick with several different ISOs installed.
http://www.pendrivelinux.com/yumi-multiboot-usb-creator/
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Nov 15, 2014 6:18 pm    Post subject: Reply with quote

@drescherjm:

my pleasure :)


could you please post the CFLAGS/CXXFLAGS you used to get it to work on your CPU ?

for now I compiled the testing-modules for the 3.14* kernel with

Code:
-march=nocona -mtune=generic


but I'm not sure whether it would run on an Athlon 64 X2 (even though it supports SSE3)


edit:

https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/commit/ca50865edb6186ed3b1f9b05a1c0964a338b7a73
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5738
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Nov 20, 2014 7:58 pm    Post subject: Reply with quote

Quote:
testing modules with

CFLAGS="-O2 -march=nocona -mtune=generic -pipe"
CXXFLAGS="${CFLAGS}"

spl- & zfs-tree states are from upstream, NOW (11/20/2014; 20.11.2014)

Since lots of patches from the last testing modules have been merged meanwhile into upstream
and since 2129 (the major ARC changes) and the spl patch don't apply cleanly anymore - due to further evaluation work
they're left out for now;
as requested another round of test modules - now also for 3.10.45 kernel

additional SPL-patches:
* NONE this time

additional ZFS-patches:
* 10.10.2014_Illumos-4950 files sometimes can't be removed from a full filesystem_2784
* 14.11.2014_Illumos_5244 zio pipeline callers should explicitly invoke next stage_2828
* 14.11.2014_Store common Linux xattrs as native SA with xattr-sa_2809
* 19.11.2014_Don't read space maps during import for readonly pools_2909

This should make the modules especially helpful during troubleshooting and if you're affected by
one of the bugs in those additional patches

Keep in mind though that without the ARC changes there might be higher chance of possible OOM issues
(if your workflow causes it or has caused it in the past)

Cheers !



Commit: https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/commit/729895d94074b9be9a90f1a43367b2544fc05101

Branch: https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/commits/ZFS-for-SysRescCD-4.3.0-r2-testing02

lots of the additional patches since testing01 have been merged upstream meanwhile,

both the modules for the 3.14* and 3.10* kernel have been rebuilt

https://github.com/zfsonlinux/zfs/pull/2129 is left out this time (lower write & read rates, higher probability of OOM issues if your workflow or setup is prone to trigger it)

besides that it should (hopefully) work without issues

only compile-tested for now - haven't had any time to test them yet (just came home)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2769
Location: Pittsburgh, PA, USA

PostPosted: Tue Nov 25, 2014 1:17 am    Post subject: Reply with quote

Sorry. I was out of the country last week so no access to the amd server. It looks like from the git hub feed that you have created amd modules. I'll look at these when I get a chance..
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum