proliant 360 cmptble hw raid controller VS softraid [solved]

Message

Mgiese · Post by **Mgiese** » Thu Jul 30, 2020 11:14 am

Hi guys,

i have had several issues using a hp pro liant dl360e server with raid configuration.

i preferably want a hardware raid controller. so that if one harddrive fails, i can see the red light on the respective drive.

the integrated raid controller gives me issues. i can setup a logical volume in bios, but this device is never been detected by any linux (tried ubuntu,knoppix so far)

my questions is now, does somebody have any intel on a hardware raid controller which is sufficiently detected in linux with mentioned boot sticks/(gentoo in the end) ?

i also had a look into using a softraid in gentoo, there i have the following questions :

how can i detect if a drive fails ? is there a possibility that if a harddrive fails the LED on the harddrive in question is turning red too ?

the next thing i dont understand : if using softraid my harddrives will still be detected as /dev/sda and /dev/sdb, then i issue the follwing command :

Code: Select all

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1

for the boot partition

and of course

Code: Select all

mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3

for the root partition

my question is now, if i need kernel support for softraid how can i later on boot from /dev/md0 (which will not be detected by the systems to boot from at this point) ??

thanks for any help on this. as i said the easiest thing would probably be to be a compatible hardraid controller..!?

i hope you can give me some insights

thanks a lot in advance

Fitzcarraldo · Post by **Fitzcarraldo** » Thu Jul 30, 2020 3:06 pm

Regarding detecting a failed drive when using software RAID, when nothing is wrong with my RAID 1 arrays I see '[UU]' in /proc/mdstat as follows:

Code: Select all

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb3[1] sda3[0]
      201195392 blocks super 1.2 [2/2] [UU]
      
md0 : active raid1 sdb2[1] sda2[0]
      16768896 blocks super 1.2 [2/2] [UU]
      
md2 : active raid1 sda4[0] sdb4[1]
      2712028992 blocks super 1.2 [2/2] [UU]
      
md3 : active raid1 sde2[1] sdc2[0]
      2930132800 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>

If I were to see '[_U]' or '[U_]' then one of the drives in my RAID 1 pairs would have failed. To let me know automatically, I created a cron job that runs every ten minutes on my server and will send me an e-mail -- which I will normally receive on my phone -- to inform one of the drives in a RAID pair has failed:

Code: Select all

$ cat checkRAID.sh
#!/bin/bash
if [[ `grep "\[_U\]" /proc/mdstat` ]] || [[ `grep "\[U_\]" /proc/mdstat` ]]; then
    cat /home/fitzcarraldo/raid-email.txt /proc/mdstat | /usr/sbin/sendmail -i -t
fi

Code: Select all

$ sudo crontab -l | grep RAID
*/10 * * * * /home/fitzcarraldo/checkRAID.sh

I prefer to do it that way.

See https://raid.wiki.kernel.org/index.php/ ... our_system for some more tips.

Although some people have /boot in a RAID, have you considered putting /boot on a non-RAID partition (possibly on a separate SSD, USB pen drive or microSD Card) to avoid the hassle? I prefer to have /boot on a microSD Card (or USB stick) in my server. I also have a backup on a spare microSD card.

mike155 · Post by **mike155** » Thu Jul 30, 2020 3:15 pm

Does the hp pro liant dl360e have a hardware RAID controller? Which one? What does lspci show?

Or is it a fakeRAID controller?

Post by **NeddySeagoon** » Thu Jul 30, 2020 3:52 pm

Mgiese,

The mdadm service will email you with events that happen to your software raid set.

What raid level do you want?

misphit · Post by **misphit** » Fri Jul 31, 2020 12:41 am

Mgiese,

i preferably want a hardware raid controller. so that if one harddrive fails, i can see the red light on the respective drive.

the integrated raid controller gives me issues. i can setup a logical volume in bios, but this device is never been detected by any linux (tried ubuntu,knoppix so far)

You should most definitely use the build in HP Smart Array Controller.
This is professional hardware, perfectly supported by Gentoo. Why not use it?

However, before it can be used, it should first be set up.
When the server boots, press F5. This should drop you in the Smart Storage Administrator, a buildin HP linux system,
which wil give you a nice graphical utility to set up your physical and logical disks.
Be warned however that when you are done and press "save" the utility will wipe the partitiontable of the disks involved.
Only use empty disks.

i also had a look into using a softraid in gentoo, there i have the following questions :

Please do not use software raid on a Proliant.
The Smart array controller is perfectly supported by linux.

Take you time. These are fantastic machines to work with.
Why don't you try to boot with a Gentoo Admin disk instead of this Ubuntu stuff?
Ilo even lets you boot this thing with an .iso on your computer and the server in a datacentre somewhere.

HTH
Misphit

Mgiese · Post by **Mgiese** » Fri Jul 31, 2020 8:40 am

misphit wrote:Mgiese,

i preferably want a hardware raid controller. so that if one harddrive fails, i can see the red light on the respective drive.

the integrated raid controller gives me issues. i can setup a logical volume in bios, but this device is never been detected by any linux (tried ubuntu,knoppix so far)
You should most definitely use the build in HP Smart Array Controller.
This is professional hardware, perfectly supported by Gentoo. Why not use it?

However, before it can be used, it should first be set up.
When the server boots, press F5. This should drop you in the Smart Storage Administrator, a buildin HP linux system,
which wil give you a nice graphical utility to set up your physical and logical disks.
Be warned however that when you are done and press "save" the utility will wipe the partitiontable of the disks involved.
Only use empty disks.

i also had a look into using a softraid in gentoo, there i have the following questions :
Please do not use software raid on a Proliant.
The Smart array controller is perfectly supported by linux.

Take you time. These are fantastic machines to work with.
Why don't you try to boot with a Gentoo Admin disk instead of this Ubuntu stuff?
Ilo even lets you boot this thing with an .iso on your computer and the server in a datacentre somewhere.

HTH
Misphit

as mentioned when using the internal raid controller(i think it was a 420) and setting up a logical device, i could never find a /dev/sd* .. so i was at a loss. i used the F5 integrated bios to set up the raid storage.... maybe you could tell me the name under /dev/ of my logical raid1 volume ?? that would be a huge help, if it really is supported in linux

regards

edit : i used this ubuntu stuff, cause i setup lots of machines for noobs with ubuntu. i also tried knoppix which is my first choice when it comes to a live linux or a gentoo install. lots of pro tools are integrated in knoppix and i have been setting up dozens of gentoo machines (since 2005) with it. but also there i could not find my logical drive in /dev/something*

Mgiese · Post by **Mgiese** » Fri Jul 31, 2020 8:41 am

NeddySeagoon wrote:Mgiese,

The mdadm service will email you with events that happen to your software raid set.

What raid level do you want?

simple mirror, raid1

Fitzcarraldo · Post by **Fitzcarraldo** » Fri Jul 31, 2020 10:56 am

I don't know about the HP P420 RAID controller, but my server has the HP B120i RAID controller, which requires the hpasm driver and is really 'fake RAID'. When I installed my server back in 2016, the only Linux driver available was for Red Hat and SUSE Enterprise, but I don't know what the situation is now. There used to be a hpasm driver for earlier versions of Ubuntu, but that was discontinued several years ago, so I use software RAID (mdadm). I disabled the B120i and enabled SATA AHCI support in the BIOS (I pressed F9 at boot to open the BIOS menu and changed 'System Options/SATA Controller Options' to 'Enable SATA AHCI Support' and I disabled 'HPSmart Array Controller' in 'PCI Device Enable/Disable').

My server has four 3TB WD Red HDDs with four RAID 1 arrays and has been running 24/7 with software RAID since Q1 of 2016 without any problems. I've had a couple of unplanned shutdowns to the server while I was away from home on work trips when the APC UPS battery died prematurely (uncontrolled shutdown) and another time when a power cut lasted several hours and the UPS ran out of battery power and shutdown the server (controlled shutdown), but mdadm handled those fine. Just had to do a fsck on boot to be sure everything was OK.

By the way, I have just noticed the following message on the Linux RAID Wiki which might be of interest to WD Red users:

https://raid.wiki.kernel.org/index.php/Linux_Raid

WARNING !

Do NOT buy new (2019 on) WD Red drives for your array !!! For the reason, read the section on Timeout Mismatch. Desktop drives are also increasingly using shingled drives and these are completely unsuitable for raid.

Edit: It's now official - ALL WD Reds are SMR, and completely unsuitable for RAID. They have moved their CMR drives into the Red Plus line, which are allegedly guaranteed to be CMR.

Fortunately I bought the Reds for my server in 2016. When I eventually have to replace them, I'm glad I stumbled across the above warning first.

Anon-E-moose · Post by **Anon-E-moose** » Fri Jul 31, 2020 11:01 am

I've swapped out of using the wd reds, and are using the seagate ironwolf nas series instead

Fitzcarraldo · Post by **Fitzcarraldo** » Fri Jul 31, 2020 11:04 am

<off-topic>
Anon-E-moose, How are you finding the Seagates? I'm a bit wary of Seagate drives after a 3 months old 1TB Seagate HDD went up in smoke (literally) on me a few years ago.
</off-topic>

Post by **NeddySeagoon** » Fri Jul 31, 2020 4:37 pm

Mgiese,

In raid1, there is nothing for a hardware raid controller to do.

Different raid controllers use different proprietary on disk data layouts. You will need to keep validated backups, or a spare raid controller in case your hardware raid controller fails.
That's true of hardware or fake raid.

With an mdadm raid set, you connected the drives to a different computer and unless the failure damaged the drives, it just works.

Anon-E-moose · Post by **Anon-E-moose** » Fri Jul 31, 2020 4:46 pm

Fitzcarraldo wrote:<off-topic>
Anon-E-moose, How are you finding the Seagates? I'm a bit wary of Seagate drives after a 3 months old 1TB Seagate HDD went up in smoke (literally) on me a few years ago.
</off-topic>

I have had problems with seagate, but that was many years ago, and I stayed away from them for years, but the ironwolf nas have good reviews, and I've not had any problems with them, but they've only been running for a few months now (24x7).

misphit · Post by **misphit** » Sat Aug 01, 2020 2:28 am

Mgiese,

as mentioned when using the internal raid controller(i think it was a 420) and setting up a logical device, i could never find a /dev/sd* .. so i was at a loss. i used the F5 integrated bios to set up the raid storage.... maybe you could tell me the name under /dev/ of my logical raid1 volume ?? that would be a huge help, if it really is supported in linux

I also thought you had a 420, however, when i looked it up the 360e has a B120i controller, not a P420i.
I have to apologize here, because although the P420 is well supported by linux, the B120 which you have is not.

It is a fakeraid controller, and needs a proprietary driver.

This is the one that Fitzcarraldo speaks about.
you best follow his advice.

If you do want to use HW raid, you should get yourself a P420 raid controller.
It is still available, new, refurbished or used. (HP number is 631671-B21)

That one is well supported in the kernel, by

Code: Select all

CONFIG_SCSI_HPSA, CONFIG_ATA_GENERIC, CONFIG_ATA_PIIX, CONFIG_ATA_SFF,

It then is invisible to the OS, so the first two disks in raid 1 shows up as 1 disk /dev/sda.

HTH,
Misphit

Post by **NeddySeagoon** » Sat Aug 01, 2020 2:24 pm

misphit,

fakeraid is software raid.
fakeraid is proprietary and performed by the firmware. Software raid is open source and performed by the kernel.

For raid1, there are no redundancy syndromes to calculate. Its just write the data out once per mirror.
That data is already in RAM, so its just additional DMA setups.
There is no point in buying a raid controller for that.

Mgiese · Post by **Mgiese** » Mon Aug 03, 2020 1:26 pm

misphit wrote:Mgiese,

as mentioned when using the internal raid controller(i think it was a 420) and setting up a logical device, i could never find a /dev/sd* .. so i was at a loss. i used the F5 integrated bios to set up the raid storage.... maybe you could tell me the name under /dev/ of my logical raid1 volume ?? that would be a huge help, if it really is supported in linux
I also thought you had a 420, however, when i looked it up the 360e has a B120i controller, not a P420i.
I have to apologize here, because although the P420 is well supported by linux, the B120 which you have is not.

It is a fakeraid controller, and needs a proprietary driver.

This is the one that Fitzcarraldo speaks about.
you best follow his advice.

If you do want to use HW raid, you should get yourself a P420 raid controller.
It is still available, new, refurbished or used. (HP number is 631671-B21)

That one is well supported in the kernel, by
Code: Select all
CONFIG_SCSI_HPSA, CONFIG_ATA_GENERIC, CONFIG_ATA_PIIX, CONFIG_ATA_SFF, 
It then is invisible to the OS, so the first two disks in raid 1 shows up as 1 disk /dev/sda.

HTH,
Misphit

thanks for that information : i checked ,which raid hardware controller is on my hp proliant dl360p. it is the b320i raid controller. The raid controller bios is setting up my 2 drivers, and creates one logical drive (2*hdd raid 1).

i also setup the b320i as boot controller. the raid controller is showing the /dev/sda as my raid1 logical drive.

unfortually, gentoo live dvd(usb) and knoppix live dvd(usb) dont detected my raid1 logical drive. the boot usb drive always uses /dev/sda (knoppix and gentoo)

honestly i dont understand this. using a hardware raid controller should not be so difficult as it is atm

very disappointing

ACU reports B320 controller with 1 logical drive on /dev/sda. But no linux version detects that drive ;( i need help please

Post by **NeddySeagoon** » Mon Aug 03, 2020 1:41 pm

Mgiese,

Its a fakeraid controller, (software in the BIOS), not hardware raid. it just looks like hardware raid to the operating system, hence the fake in fakeraid.

You can try sys-fs/dmraid, which is the Linux fakeraid driver but it may not know your metadata layout.
The alternative is the proprietary driver from HP, which probably won't install with a modern kernel.

https://www.abmx.com/blog/difference-ha ... e-fakeraid is a good summary of your options, leaving out Hardware RAID.

Fitzcarraldo · Post by **Fitzcarraldo** » Tue Aug 04, 2020 2:33 am

Mgiese,

B320i controllers in ProLiant 'e' series servers are 'fake RAID'. You need an HP driver for that controller. HP only provides drivers for Red Hat Enterprise Linux and SUSE LINUX Enterprise Server. You can download the HP driver for those two distributions from the HP Enterprise site:

https://support.hpe.com/hpsc/swd/public ... bb4efce88a

https://support.hpe.com/hpsc/swd/public ... 9a05c6ccaf

You can use the RPM package in CentOS instead of RHEL, if you want.

If you want to use that controller with Gentoo, use software RAID (sys-fs/mdadm).

Mgiese · Post by **Mgiese** » Thu May 20, 2021 8:44 pm

thanks for all the intel. got myself a gen9 proliant 360 with 440 raid controller. now i can mirror a hdd using the raid and the have access to it without loading a driver (/dev/sda).

thanks a lot !