Hardware RAID... kernel panic

Message

FcukThisGame · Post by **FcukThisGame** » Thu Oct 06, 2011 10:46 pm

I'm new to installing Gentoo on hardware RAID... I'm using the newest version to date, amd64, on a hardened profile, with hardened-sources. /dev/sda* recognizes all of the partitions on the device. I applied new filesystems to them and started building, assuming the hardware RAID was invisible to the OS. Finish installing, reboot, KP. Great.

Boot back off the livecd, mount, chroot, add support for the RAID device (Symbios Logic SAS1068 PCI-X Fusion-MPT SAS), reboot, still KP. Repeat a few times, and we're at the present.

I did some googling about hw raid, it said to check /dev/mapper from the livecd. I have control only, which I believe indicates that the RAID volume wasn't identified by the livecd. modprobe shows me that it's using mptsas kernel module. The drive is recognized and functions normally as /dev/sda, but the system can't seem to identify /dev/sda at boot.

/dev/mapper contents, lspci -k, and .config below:

Code: Select all

livecd ~ # ls /dev/mapper
control
livecd ~ # lspci -k
00:00.0 Host bridge: Intel Corporation E7230/3000/3010 Memory Controller Hub
        Subsystem: Dell PowerEdge SC440
00:01.0 PCI bridge: Intel Corporation E7230/3000/3010 PCI Express Root Port
        Kernel driver in use: pcieport
00:1c.0 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 1 (rev 01)
        Kernel driver in use: pcieport
00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev 01)
        Kernel driver in use: pcieport
00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 01)
        Kernel driver in use: pcieport
00:1d.0 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #1 (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci-hcd
00:1d.1 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #2 (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci-hcd
00:1d.2 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #3 (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci-hcd
00:1d.3 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #4 (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci-hcd
00:1d.7 USB Controller: Intel Corporation N10/ICH 7 Family USB2 EHCI Controller (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: ehci_hcd
        Kernel modules: ehci-hcd
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: ata_piix
        Kernel modules: ata_generic, pata_acpi, ata_piix
00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE Controller (rev 01)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: ata_piix
        Kernel modules: ata_generic, pata_acpi, ata_piix
00:1f.3 SMBus: Intel Corporation N10/ICH 7 Family SMBus Controller (rev 01)
        Subsystem: Dell PowerEdge SC440
01:00.0 PCI bridge: Intel Corporation 6702PXH PCI Express-to-PCI Bridge A (rev 09)
02:08.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS (rev 01)
        Subsystem: Dell SAS 5/iR Adapter RAID Controller
        Kernel driver in use: mptsas
        Kernel modules: mptsas
05:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754 Gigabit Ethernet PCI Express (rev 02)
        Subsystem: Dell PowerEdge SC440
        Kernel driver in use: tg3
        Kernel modules: tg3
06:04.0 Ethernet controller: D-Link System Inc DGE-530T Gigabit Ethernet Adapter (rev 11) (rev 11)
        Subsystem: D-Link System Inc DGE-530T Gigabit Ethernet Adapter (rev 11)
        Kernel driver in use: skge
        Kernel modules: skge
06:07.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
        Subsystem: Dell PowerEdge SC440

Kernel config here

Can anyone point me in the right direction?

Thanks!

DaggyStyle · Post by **DaggyStyle** » Fri Oct 07, 2011 5:04 am

what exact kp are you getting?

krinn · Post by **krinn** » Fri Oct 07, 2011 2:30 pm

Yes, the disks for the kernel will appears as sda and /dev/mapper will remain empty as it's not for hardward raid but software volume
Depending on your driver it should be more informative in the kernel dmesg
Look for a sample : http://forums.gentoo.org/viewtopic-p-63 ... ml#6350807

And if your sign is correct, you have more than just 1 controller, and more than 1 disk attach to them, so you are just assuming like many users that linux should call your disk /dev/sda because linux call it /dev/sda when you use the livecd : it's a mistake.
Look here for explain & way to fix that easy : http://forums.gentoo.org/viewtopic-p-68 ... ml#6824250

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 2:49 pm

Thanks for the reply.

No this isn't my personal desktop, it's a Dell PowerEdge SC440 server. According to Dell the part number for the RAID controller is UN939, which is listed as "PERC 5/ir SAS PCI-e"

lspci -k tells me:

Code: Select all

02:08.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS (rev 01)
        Subsystem: Dell SAS 5/iR Adapter RAID Controller
        Kernel driver in use: mptsas
        Kernel modules: mptsas

When i look for 'raid' in dmesg I see this:

Code: Select all

Command line: root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/image.squashfs cdroot initrd=gentoo.igz vga=791 BOOT_IMAGE=gentoo dodmraid
Kernel command line: root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/image.squashfs cdroot initrd=gentoo.igz vga=791 BOOT_IMAGE=gentoo dodmraid
mptsas: ioc0: attaching raid volume, channel 1, id 0
Adaptec aacraid driver 1.1-7[28000]-ms
megaraid cmm: 2.20.2.7 (Release Date: Sun Jul 16 00:01:03 EST 2006)
megaraid: 2.20.5.1 (Release Date: Thu Nov 16 15:32:35 EST 2006)
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
raid6: int64x1   1867 MB/s
raid6: int64x2   1984 MB/s
raid6: int64x4   1735 MB/s
raid6: int64x8   1289 MB/s
raid6: sse2x1    3202 MB/s
raid6: sse2x2    4498 MB/s
raid6: sse2x4    6644 MB/s
raid6: using algorithm sse2x4 (6644 MB/s)
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: raid10 personality registered for level 10

Searching for 'scsi' seems to generate output closer to the sample you linked to:

Code: Select all

livecd / # dmesg | grep scsi
iscsi: registered transport (tcp)
scsi0 : ata_piix
scsi1 : ata_piix
scsi2 : ata_piix
scsi3 : ata_piix
scsi 0:0:0:0: CD-ROM            TSSTcorp CDRW/DVD TSH492B TB06 PQ: 0 ANSI: 5
sr0: scsi3-mmc drive: 24x/48x writer cd/rw xa/form2 cdda tray
sr 0:0:0:0: Attached scsi CD-ROM sr0
sr 0:0:0:0: Attached scsi generic sg0 type 5
scsi: <fdomain> Detection failed (no card)
scsi4 : ioc0: LSISAS1068 B0, FwRev=000a3300h, Ports=1, MaxQ=286, IRQ=16
scsi 4:0:0:0: Direct-Access     ATA      WDC WD2500YS-18S 6C07 PQ: 0 ANSI: 5
scsi 4:0:0:0: Attached scsi generic sg1 type 0
scsi 4:0:1:0: Direct-Access     ATA      WDC WD2500YS-18S 6C07 PQ: 0 ANSI: 5
scsi 4:0:1:0: Attached scsi generic sg2 type 0
scsi 4:1:0:0: Direct-Access     Dell     VIRTUAL DISK     1028 PQ: 0 ANSI: 5
sd 4:1:0:0: Attached scsi generic sg3 type 0

Am I correct to assume that I need to get /dev/mapper to show a volume before I can (re)start the installation?
If I get that to work (or don't need to) can I use the data on /dev/sda (I built the entire system on there...) or do I need to start over?

krinn · Post by **krinn** » Fri Oct 07, 2011 3:17 pm

No you're incorrect assuming /dev/mapper should have any values : this is logical volume manager, it's not for hardware raid as i already told you, so except if you build software raid over an hardware raid, this will remain empty.

All the "raid6" and "md" messages you see in your dmesg are just because you have enable software raid, unrelated to hardware itself, but no harm to have it.

Because of your short output, canno't really see sda disk status, at least it show your raid array is handle and assign as sg3

Post your actual kernel error when you boot like DaggyStyle told you (in a too short version in my taste), let's expand his quote because kp isn't killer potatoes :

what exact kernel panic are you getting?

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 3:34 pm

krinn wrote:the disks for the kernel will appears as sda and /dev/mapper will remain empty as it's not for hardward raid but software volume

Derp. You most definitely did tell me that. My apologies for failure to read.

Kernel Panic error

FWIW, /dev/sda1 is my boot partition, /dev/sda2 is root.

EDIT: also relevant, grub.conf:

Code: Select all

default 0
timeout 5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Gentoo Linux 2.6.39-hardened-r8
root (hd0,0)
kernel /boot/kernel-2.6.39-hardened-r8 root=/dev/sda2

krinn · Post by **krinn** » Fri Oct 07, 2011 4:00 pm

the driver sr is load only and your cdrom appears alone

You have something missing :
- lacking support for scsi disk in kernel (sd)
- lacking support for you disk controller (mptsas)
- support for disk controller is build as module and not include in kernel (mptsas should be Y and not M)

Eheh, and if you think your kernel have the 3, then maybe you've build it, but like many users do, fail to actually install it (there's many ways to fail doing such a simple task : including not mounting boot and kernel is now copy to /boot in your root partition instead of your boot partition, bad target name when copying...)

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 4:11 pm

SCSI:

Code: Select all

(chroot) livecd linux # cat .config | grep SCSI
# SCSI device support
CONFIG_SCSI_MOD=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y
# SCSI support type (disk, tape, CD-ROM)
# CONFIG_SCSI_MULTI_LUN is not set
CONFIG_SCSI_CONSTANTS=y
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m
# SCSI Transports
CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
CONFIG_SCSI_SAS_ATTRS=y
CONFIG_SCSI_SAS_LIBSAS=y
# CONFIG_SCSI_SAS_ATA is not set
CONFIG_SCSI_SAS_HOST_SMP=y
# CONFIG_SCSI_SRP_ATTRS is not set
CONFIG_SCSI_LOWLEVEL=y
# CONFIG_ISCSI_TCP is not set
# CONFIG_ISCSI_BOOT_SYSFS is not set
# CONFIG_SCSI_CXGB3_ISCSI is not set
# CONFIG_SCSI_CXGB4_ISCSI is not set
# CONFIG_SCSI_BNX2_ISCSI is not set
# CONFIG_SCSI_BNX2X_FCOE is not set
# CONFIG_BE2ISCSI is not set
# CONFIG_SCSI_HPSA is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_3W_SAS is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_AIC94XX is not set
# CONFIG_SCSI_MVSAS is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_ARCMSR is not set
CONFIG_SCSI_MPT2SAS=y
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS_LOGGING=y
# CONFIG_SCSI_HPTIOP is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_VMWARE_PVSCSI is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_STEX is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_QLA_ISCSI is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set
# CONFIG_SCSI_PMCRAID is not set
# CONFIG_SCSI_PM8001 is not set
# CONFIG_SCSI_SRP is not set
# CONFIG_SCSI_BFA_FC is not set
# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
# CONFIG_ISCSI_IBFT_FIND is not set

MPTSAS:

Code: Select all

(chroot) livecd linux # cat .config | grep SAS
CONFIG_SCSI_SAS_ATTRS=y
CONFIG_SCSI_SAS_LIBSAS=y
# CONFIG_SCSI_SAS_ATA is not set
CONFIG_SCSI_SAS_HOST_SMP=y
# CONFIG_SCSI_3W_SAS is not set
# CONFIG_SCSI_MVSAS is not set
# CONFIG_MEGARAID_SAS is not set
CONFIG_SCSI_MPT2SAS=y
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS_LOGGING=y

let me recompile, make sure /boot is mounted, and re-copy... I'll keep you posted.

Thanks for your help so far!

krinn · Post by **krinn** » Fri Oct 07, 2011 4:27 pm

Code: Select all

grep BLK_DEV_SD .config
CONFIG_BLK_DEV_SD=y

scsi disk option doesn't have any SCSI string in it

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 4:37 pm

Whoops.

Code: Select all

(chroot) livecd linux # grep BLK_DEV_SD .config
CONFIG_BLK_DEV_SD=y

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 5:02 pm

I recompiled, verified that /boot was mounted, copied the kernel over, updated grub.conf, and rebooted, same error.

Okay, I just checked the output... for some reason it built an x86 kernel. This would explain why it's not updating as it should as i've been copying the 64-bit kernel. I'm chrooting back in now as we speak... why would it build a 32-bit kernel? my CHOST is the default 64-bit one.

EDIT: /usr/src/linux/arch/x86_64/boot/bzImage is a symlink to /usr/src/linux/arch/x86/boot/bzImage. Is that normal???

krinn · Post by **krinn** » Fri Oct 07, 2011 5:57 pm

for the symlink, i don't really know, i suppose it's normal, i don't expect kernel to build itself something weird
you can check you your kernel with "file pathtokernel" and lookout it's a 64bits or not

for your kernel boot problem, you still don't see sd answer ? no partitions are shown? and also check the line just after "kernel panic" you should see your kernel version, and it should tell you :
swapper not tainted kernelversion #number

#number is total number of compilation you've made to build that kernel, as your previous pic show #3 at least if you build and install a newer one, that number should be higher.

Feel free to repost a pic if you prefer

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 6:09 pm

According to the IRC guys that's normal. x86_64 and i386 were merged into x86.

Code: Select all

(chroot) livecd linux # file /boot/kernel-2.6.39-hardened-r8.2
/boot/kernel-2.6.39-hardened-r8.2: Linux kernel x86 boot executable bzImage, version 2.6.39-hardened-r8 (root@livecd) #5 SMP Fri Oct 7 11:48:11 EST , RO-rootFS, root_dev 0x802, swap_dev 0x4, Normal VGA

I think that also reflects the arch merge.

I've enabled a few more SCSI/SAS/RAID options in the kernel... rebooting now.

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 6:12 pm

Newest kernel boot, same KP. This time it says #7.

FcukThisGame · Post by **FcukThisGame** » Fri Oct 07, 2011 6:40 pm

DERP. Missing menuconfig->device drivers: [] Fusion MPT Device Support. Compiling now. hopefully this should be my last kernel compile for a while.