Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How do you automount multipath'ed filesystems at boot?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Thu Jan 10, 2008 10:23 pm    Post subject: How do you automount multipath'ed filesystems at boot? Reply with quote

Hi.

I've successfully setup multipath with two QLA23xx fiber channel cards. Now I'd like the filesystem on the device nodes in /dev/mapper/<node names> to mount automatically after /etc/init.d/multipathd has stared and created these nodes.

I thought I could use fstab unfortunately fstab is used and parsed *before* multipathd is started - i.e. the device nodes are created by multipathd from runlevel "boot" after automounting using fstab takes place. Is there a preferred way to mount filesystems on nodes created with multipathd? Udev rules? Local init scripts? whatever?

Thanks for any suggestion.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
planet-admin
Apprentice
Apprentice


Joined: 27 Mar 2004
Posts: 213
Location: Boise, ID

PostPosted: Wed Feb 20, 2008 6:39 am    Post subject: Reply with quote

I was wondering if you could share the details of how you set up the multipath and the config files? I'm about to be involved in a project involving multipath and qla2xxx-based cards, with a disk array with dual port dual controllers.

Thanks,
Michael
_________________
Michael S. Moody
Sr. Systems Engineer
Global Systems Consulting
Web: http://www.GlobalSystemsConsulting.com
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Wed Feb 20, 2008 10:42 am    Post subject: Reply with quote

My hardest time was to find the worldwide names that are involved in multipath.conf :-D . They're given by scsi_id. You can list the unique ID of you disks with the following command, provided all your disks appear under /dev as sd*:
Code:
for DSK in /dev/sd?; do echo "${DSK}: $(scsi_id -g -u -s /block/${DSK##*/}) ($(scsi_id -g -u -s /block/${DSK##*/} | wc -m))"; done | sort -uk 2

The first column shows the disk device name, the second column the scsi ID/worldwide name and the third column the length of the second term. After that /etc/multipath.conf was trivial:
/etc/multipath.conf:
blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][[0-9]*]"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
        devnode "^sda[0-9]*"
}
multipaths {
        multipath {
                wwid            3600a0b800013e6b2000003424781cfea
                alias           fcs0p
                path_grouping_policy    failover
                path_checker    tur
        }
        multipath {
                wwid            3600a0b800013e6b2000003434781d00e
                alias           fcs1p
                path_grouping_policy    failover
                path_checker    tur
        }
        multipath {
                wwid            3600a0b800013e706000003234781d0c3
                alias           fcs2p
                path_grouping_policy    failover
                path_checker    tur
        }
        multipath {
                wwid            3600a0b800013e70600000322477dfa3d
                alias           fcd0p
                path_grouping_policy    failover
                path_checker    tur
        }
}

Exclude your local, system disk (/dev/sda) and create an alias, which appears under /dev/mapper for all your logical units of your SAN. Note I also created a custom UDEV rule - which is based upon /etc/udev/rules.d/65-multipath.rules - to have all these device nodes under /dev/mapper symlinked from /dev.
/etc/udev/rules.d/custom-multipath.rules:
ACTION=="add", KERNEL=="dm-*", SUBSYSTEM=="block", SYMLINK+="$env{DM_NAME}"

Here I am at the moment. I'm still busy trying an initramfs with genkernel to launch multipathd before /etc/fstab mounts the filesystems. (Forget about mkinitrd and nash for it lacks firmware loading support.)

I used gentoo-sources-2.6.23* and I've tested multipath with an OpenVZ (2.6.18 based) kernel. I'm considering the test version (~ARCH) of openvz-sources (>=2.6.22).

I searched a lot and found this page, which summarizes all I did, basically. I also read this document to understand how multipath can be implemented under Linux. Recent kernels include multipath support through device mapper. That's what I used.

Hope it helps.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Sun Mar 09, 2008 11:58 am    Post subject: Reply with quote

I've finally written a script which creates an initial RAM drive. It combines RedHat's mkinitrd and some of Gentoo's genkernel features - after lots of attempts and digging mkinitrd, I've come to the conclusion that it can't be applied to Gentoo for there are too many differences that are inherent to both distros.

So I've tried building an initramfs with genkernel but it turned out the initramfs doesn't load firmware - although it's designed to. Note I know why and I'll probably file a bug. The main reason is that /linuxrc mounts sysfs after loading modules. However firmware loading requires sysfs to be mounted before loading modules.

In the meantime I found genkernel scripts overly complex for such a simple task - compared against mkinitrd; genkernel supports many cases, i.e. booting from a LiveCD, NFS, LVM, which makes it quite complex. Too bad you have to change the generated RAMFS to tune it to fit with multipath/device mapper. OTOH genkernel scripts are neat and very well written, which makes them quite clear to understand.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!


Last edited by VinzC on Mon Mar 10, 2008 10:11 pm; edited 2 times in total
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Mon Mar 10, 2008 9:45 pm    Post subject: Reply with quote

Here's the script that creates the initramfs using cpio.
/usr/local/sbin/mkinitrd-ng/mkinitrd.sh:
#!/bin/sh

# Create an initial ramdisk image. The script was first intended
# for use with multipath on an IBM xSeries server equipped with
# QLogic (qla2xxx) fiber channel. The ramdisk uses busybox for
# firmware loading.
#
# Created 2008 - VinzC (v_cadet@yahoo.com)

# Flags
DEBUG=0
VERBOSE=
OVERWRITE=
LVM=
MPATH=
MODUPDATE=0

# Variables
THISAPP=${0##*/}
VERSION="0.0.35"
KERNEL=`uname -r`
TMP_DIR="/tmp/mkinitrd/$$"
INIT_SCRIPT=/usr/local/share/mkinitrd-ng/init.sh

# Desired components
MODULES=()
FIRMWARES=()

#
### Local functions
#

# Show arguments and usage
function help() {
   cat << EOF
$THISAPP - version $VERSION
Create an initial RAM filesystem for Gentoo Linux.

Help coming...
EOF
   exit 0
}

# Echoes a message in verbose mode
function verbose() {
   [[ -n "$VERBOSE" ]] && echo "$@"
}

# Display an error message and exit
# $1      exit code
# ${2..n}   message string(s)
function error_exit() {
   local x=$1
   shift; echo $*
   exit ${x}
}

# Copy (install) one file/directory at a time, dereferencing
# $1 (s)   source file (full path name)
# $2 (s)   destination file (full path name)
function instn() {
   # Don't copy if destination exists
   [[ -f $2 ]] && return

   verbose " - $1 -> $2"
   cp -pL $1 $2 2>/dev/null
}

# Copy (install) one file/directory at a time, preserving symlinks
# $1 (s)   source file (full path name)
# $2 (s)   destination file (full path name)
function inst() {
   # Don't copy if destination exists
   [[ -f $2 ]] && return

   verbose " - $1 -> $2"
   cp -pd $1 $2 2>/dev/null

   # Detect whether link or regular file!
   if [[ -L $1 ]]; then
      local LNK=$(readlink $1)
      inst ${1%/*}/$LNK ${2%/*}/$LNK
   fi
}

# Copy (install) one module at a time (including its dependencies)
# into the destination directory.
# $1   module source file (full path name)
function instmodule() {
   local modules_dir=/lib/modules/${KERNEL}
   local module_file=${modules_dir}/${1##*/}
   local depmod=${modules_dir}/modules.dep

   # Don't install more than once
   grep -q "^$module_file" ${TMP_DIR}$depmod 2>/dev/null && return 0

   # Copy given module (full path) to the destination directory
   if [[ -x /usr/bin/strip ]]; then
      strip -g ${1} -o ${TMP_DIR}${module_file}
   else
      cp -pd ${1} ${TMP_DIR}${module_file}
   fi

   # Update module dependencies"
   # TODO: do it manually or support current kernel only...
   depmod -b $TMP_DIR $KERNEL
   verbose " - ${1##*/}"

   # Also copy its dependencies...
   local module
   local deplist=$(sed -n "s/^\/lib\/modules\/[^ ]\+${1##*/}://p" $depmod)
   for module in $deplist; do
      #[[ -f $module ]] || continue
      instmodule $module
   done
}

# Retrieve the list of files in a package
function getpkgfiles() {
   for PKG in $*; do
      equery -q f $PKG
   done
}

# Copy (install) binary files from a package.
# $1   list of packages to select
# $2   list of binary files to select
# $3   binary object type (lib|bin, defaults to bin)
function instpkgbin() {
   [[ -n $1 ]] && [[ -n $2 ]] || return 1
   getpkgfiles $1 | sed -e "/${3:-bin}\//!d" -e "/${2// /\|}/!d" | \
   while read BIN; do
      echo $BIN
      ldd $BIN | grep -o '\/[^ ]\+'
   done | sort -u | \
   while read BIN; do
      inst $BIN ${TMP_DIR}${BIN}
   done
}

# Parse arguments
while [[ -n "$1" ]]; do
   case $1 in
      -h|--help)
         help ;;

      -k|--kernel*)
         [[ $1 == "--kernel="* ]] || shift
         KERNEL="${1#--kernel=}" ;;

      -s|--init*)
         [[ $1 == "--init="* ]] || shift
         INIT_SCRIPT="${1#--init=}" ;;

      -m|--module*)
         [[ $1 == "--module="* ]] || shift
         MODULES=( ${MODULES[*]} "${1#--module=}" ) ;;

      -b|--firmware*)
         [[ $1 == "--firmware="* ]] || shift
         FIRMWARES=( ${FIRMWARES[*]} "${1#--firmware=}" ) ;;

      -v|--verbose)
         VERBOSE=-v ;;

      --lvm|--enable-lvm)
         MODULES=( ${MODULES[*]} "dm-mod" )
         LVM=1 ;;

      --multipath|--enable-multipath)
         MODULES=( ${MODULES[*]} "dm-round-robin" )
         MPATH=1 ;;

      -u|--update-modules)
         MODUPDATE=1 ;;

      -f|--force|--overwrite)
         OVERWRITE=-f ;;

      -*|--*)
         error_exit 1 "Invalid option $1. See $THISAPP --help." ;;

      *)
         INIT_IMAGE="$1" ;;
   esac
   shift
done

# Variables depending on parameters
INIT_IMAGE=/boot/initramfs-${KERNEL}.igz

# Sanity checks
[[ -f "$INIT_SCRIPT" ]] || error_exit 1 "Init script $INIT_SCRIPT was not found."

# Must mount boot filesystem (if possible) to check
# the ramdrive image file
mount /boot/ 2>/dev/null && verbose "Mounted /boot"
[[ -f "$INIT_IMAGE" ]] && [[ -z "$OVERWRITE" ]] && error_exit 1 "Image file exists. Use -f to overwrite."

#
### Start ###
#
verbose "Building ${TMP_DIR} directory tree..."
verbose "Using $INIT_SCRIPT ($KERNEL)"

# Create directory tree
for DIR in bin dev etc lib/firmware lib/modules/${KERNEL} proc sys; do
   mkdir -p ${TMP_DIR}/${DIR}
done
(cd ${TMP_DIR}; ln -s bin sbin)

# Create mandatory device nodes
mknod ${TMP_DIR}/dev/console c 5 1
mknod ${TMP_DIR}/dev/null c 1 3
mknod ${TMP_DIR}/dev/tty1 c 4 1

chmod 600 ${TMP_DIR}/dev/console
chmod 600 ${TMP_DIR}/dev/tty1

# Copy initial script
cp ${INIT_SCRIPT} ${TMP_DIR}/init
chmod +x ${TMP_DIR}/init
(cd ${TMP_DIR}/sbin; ln -s ../init init)

# Copy busybox executable and make symlinks
instpkgbin busybox busybox
(cd ${TMP_DIR}/bin; for BIN in '[' ash cut echo mdev sh uname; do ln -s busybox $BIN; done)

# Compression library is required
instpkgbin zlib "^\/lib\/libz\.so" lib

# Copy module files. Loop through module list to make sure every file exists.
[[ -f /lib/modules/${KERNEL}/modules.dep ]] || \
   error_exit 2 "No module dependencies found. Exiting."


# Store module list file
MOD_LIST=${TMP_DIR}/etc/modules.list

verbose "Copying modules: ${MODULES[*]}"
[[ $MODUPDATE -eq 1 ]] && update-modules -f

for module_name in ${MODULES[*]}; do
   # Find module real name
   module=$(find /lib/modules/${KERNEL}/ -name ${module_name}.ko -o -name ${module_name}.o | head -n 1)
   if [[ -z "$module" ]]; then
      echo "Warning: $module_name was not found."
      continue
   fi

   # Add module to the list
   instmodule $module && echo "$module_name" >> $MOD_LIST
done

# Add module parameters and aliases from /etc/modprobe.conf
verbose "Configuring modules"
grep -Ff $MOD_LIST /etc/modprobe.conf | sed -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*$/d' \
   -e "s#/lib/modules/[^ ]\+/#/lib/modules/${KERNEL}/#g" > ${TMP_DIR}/etc/modprobe.conf

# Clean empty module files
for FILE in ${TMP_DIR}/lib/modules/${KERNEL}/modules.*; do
   sed -i '/^[[:space:]]*\(#\|$\)/d' $FILE
   [[ -s $FILE ]] || rm ${VERBOSE} $FILE
done


# Copy firmwares (don't preserve links)
[[ -n ${FIRMWARES[*]} ]] && verbose "Copying firmwares"
for FIRMWARE in ${FIRMWARES[*]}; do
   instn /lib/firmware/$FIRMWARE ${TMP_DIR}/lib/firmware/${FIRMWARE}
done


# Install other packages (LVM aso)
if [[ -n "$LVM" ]]; then
   if [[ -f /etc/lvm/lvm.conf ]]; then
      verbose "Enabling LVM..."
      instpkgbin "device-mapper lvm2" "dmsetup lv.* pv.* vg.*"
      inst /etc/lvm/lvm.conf ${TMP_DIR}/etc/lvm/lvm.conf
   else
      echo "Warning: lvm binaries are not installed."
   fi
fi


# Install multipath (relies on dm-mod)
if [[ -n "$MPATH" ]]; then
   if [[ -f /etc/multipath.conf ]]; then
      verbose "Enabling multipath..."
      instpkgbin "device-mapper multipath-tools" "dmsetup mpath multipath$ kpartx"
      inst /etc/multipath.conf ${TMP_DIR}/etc/multipath.conf
      # scsi_id must be copied apart for it's a symlink to /lib/udev/scsi_id
      instn $(which scsi_id) ${TMP_DIR}/sbin/scsi_id
   else
      echo "Warning: multipath-tools are not installed."
   fi
fi


# Create and compress initramfs image
verbose "Compressing image ${INIT_IMAGE}..."
(cd ${TMP_DIR}; find * | cpio --quiet -H newc -o) | gzip -9 > $INIT_IMAGE && \
echo "Done." && \
[[ $DEBUG -ne 0 ]] || rm -r /tmp/mkinitrd


# Flush disk data and exit
sync

if [[ -n "$MPATH" ]]; then
   echo -e "\nPlease pass argument 'mpath' to the kernel to enable multipath support."
elif [[ -n "$LVM" ]]; then
   echo -e "\nPlease pass argument 'lvm' to the kernel to enable LVM support."
fi

Here's the init script.
/usr/local/sbin/mkinitrd-ng/linuxrc:
#!/bin/sh

# Colors
NORMAL="\033[0m"
WARN="\033[33;1m"
BAD="\033[31;1m"
BOLD="\033[1m"
GOOD="\033[32;1m"

# Initialize system variables
QUIET='1'
CONSOLE=/dev/console
SCAN_DELAY='1'
SYSROOT=''
MNTDIR='/newroot'

KV=`uname -r`
KMAJOR=`echo $KV | cut -f1 -d.`
KMINOR=`echo $KV | cut -f2 -d.`
KVER="${KMAJOR}.${KMINOR}"

LVM='0'
MPATH='0'

# Local functions
quiet_kmsg() {
   # if QUIET is set make the kernel less chatty
   [ -n "$QUIET" ] && echo '0' > /proc/sys/kernel/printk
}

verbose_kmsg() {
   # if QUIET is set make the kernel less chatty
   [ -n "$QUIET" ] && echo '6' > /proc/sys/kernel/printk
}

scan_delay() {
   # Sleep a specific number of seconds if SCAN_DELAY is set otherwise only sleep
   # 1 second
   if [ -n "${SCAN_DELAY}" ]
   then
      sleep ${SCAN_DELAY}
   else
      sleep 1
   fi
}

# msg functions arguments
# $1 string
# $2 hide flag
good_msg() {
   msg_string=$1
   msg_string="${msg_string:-...}"
   [ "$2" != 1 ] && echo -e "${GOOD}>>${NORMAL}${BOLD} ${msg_string} ${NORMAL}"
}

bad_msg() {
   msg_string=$1
   msg_string="${msg_string:-...}"
   [ "$2" != 1 ] && echo -e "${BAD}!!${NORMAL}${BOLD} ${msg_string} ${NORMAL}"
}

warn_msg() {
   msg_string=$1
   msg_string="${msg_string:-...}"
   [ "$2" != 1 ] && echo -e "${WARN}**${NORMAL}${BOLD} ${msg_string} ${NORMAL}"
}

strlen() {
   if [ -z "$1" ]
   then
      echo "usage: strlen <variable_name>"
      die
   fi
   eval echo "\${#${1}}"
}

parse_opt() {
   case "$1" in
      *\=*)
         local key_name="`echo "$1" | cut -f1 -d=`"
         local key_len=`strlen key_name`
         local value_start=$((key_len+2))
         echo "$1" | cut -c ${value_start}-
      ;;
   esac
}

mount_sysfs() {
   mount -t sysfs /sys /sys >/dev/null 2>&1
   ret=$?
   [ "$ret" -eq '0' ] || bad_msg "Failed to mount /sys!"
}

run_shell() {
   /bin/ash
}

detect_sbp2_devices() {
   # http://www.linux1394.org/sbp2.php

   # /proc
   # /proc/scsi/sbp2/0, /proc/scsi/sbp2/1, etc.
   #
   # You may manually add/remove SBP-2 devices via the procfs with:
   # add-single-device <h> <b> <t> <l> or remove-single-device <h> <b> <t> <l>,
   # where:
   #
   # <h> = host (starting at zero for first SCSI adapter)
   # <b> = bus (normally zero)
   # <t> = target (starting at zero for first SBP-2 device)
   # <l> - lun (normally zero)
   # e.g. To manually add/detect a new SBP-2 device
   # echo "scsi add-single-device 0 0 0 0" > /proc/scsi/scsi
   # e.g. To manually remove a SBP-2 device after it's been unplugged
   # echo "scsi remove-single-device 0 0 0 0" > /proc/scsi/scsi
   # e.g. To check to see which SBP-2/SCSI devices are currently registered
   # cat /proc/scsi/scsi

   [ -e /proc/scsi/scsi ] && echo 'scsi add-single-device 0 0 0 0' > /proc/scsi/scsi
}

setup_busybox() {
   mount -t proc proc /proc >/dev/null 2>&1
   mount -o remount,rw / >/dev/null 2>&1

   # Set up symlinks
   /bin/busybox --install -s

   if [ -e /bin/lvm ]
   then
      ln -s /bin/lvm /bin/vgscan
      ln -s /bin/lvm /bin/vgchange
   fi
}

setup_hotplug() {
   echo /sbin/mdev > /proc/sys/kernel/hotplug
}

start_dev_mgr() {
   cd /sys
   good_msg 'Activating mdev'
   mdev -s
   cd /
}

load_modules() {
   if [ -d "/lib/modules/${KV}" ] && [ -f /etc/modules.list ]
   then
      # Load appropriate kernel modules
      good_msg 'Loading modules'
      local module
      local module_list=`cat /etc/modules.list`
      for module in $module_list; do
         echo "   -> ${module}..."
         modprobe $module
      done
   fi
}

scan_volumes() {
   #good_msg 'Checking if volumes need to be started...'

   # Here, we check for /dev/device-mapper, and if it exists, we setup a
   # a symlink, which should hopefully fix bug #142775 and bug #147015
   if [ -e /dev/device-mapper ] && [ ! -e /dev/mapper/control ]
   then
      mkdir -p /dev/mapper
      ln -sf /dev/device-mapper /dev/mapper/control
   fi

   # Check LVM
   if [ "${LVM}" = '1' ]
   then
      if [ -e '/bin/vgscan' -a -e '/bin/vgchange' ]
      then
         good_msg "Scanning for Volume Groups"
         /bin/vgscan --ignorelockingfailure --mknodes 2>/dev/null
         sleep 2
         good_msg "Activating Volume Groups"
         /bin/vgchange -ay --ignorelockingfailure 2>/dev/null
      else
         bad_msg "vgscan or vgchange not found: skipping LVM volume group activation!"
      fi
   fi

   # Check multipath
   if [ "${MPATH}" = '1' ]
   then
      good_msg "Scanning multipath devices"
      /sbin/multipath -v3
      scan_delay
      /sbin/dmsetup ls --target multipath --exec "/sbin/kpartx -a"
      # Show established multipath nodes (redundant)
      # /sbin/dmsetup ls
   fi
}


###############################################################################
# Start
###############################################################################

# Clean input/output
exec >${CONSOLE} <${CONSOLE} 2>&1

if [ "$$" != '1' ]
then
   echo '/linuxrc has to be run as the init process as the one'
   echo 'with a PID of 1. Try adding init="/linuxrc" to the'
   echo 'kernel command line or running "exec /linuxrc".'
   exit 1
fi

setup_busybox

quiet_kmsg

CMDLINE="`cat /proc/cmdline`"

# Scan CMDLINE for any specified sysroot= or cdroot arguments
for parm in ${CMDLINE}
do
   case "${parm}" in
      root\=*|sysroot\=*)
         SYSROOT=`parse_opt "${parm}"` ;;

      scandelay\=*)
         SCAN_DELAY=`parse_opt "${parm}"` ;;

      # Volume manager options
      lvm)
         LVM='1' ;;

      mpath)
         MPATH='1' ;;

      # Redirect output to a specific tty
      CONSOLE\=*\|console\=*)
         CONSOLE=`parse_opt "${parm}"`
         exec >${CONSOLE} <${CONSOLE} 2>&1
      ;;
   esac
done

# Serial bus protocol
detect_sbp2_devices

# Setup hotplugging for firmware loading
setup_hotplug

# Mount sysfs. MUST be active before loading modules.
# (Firmware loading relies on sysfs.)
mount_sysfs

# Load modules (use insmod <full path>)
load_modules

# Delay if needed for USB hardware
scan_delay

# Start device manager
start_dev_mgr

# Go verbose...
verbose_kmsg

# Scan volumes
scan_volumes

mkdir -p "${MNTDIR}"
CHROOT="${MNTDIR}"

# Check for a root block device
if [ ! -b "${SYSROOT}" ]
then
   bad_msg "Block device ${SYSROOT} is not a valid root device! Failing back to the shell..."
   SYSROOT=""
   run_shell
fi

good_msg "Mounting $SYSROOT on $MNTDIR ..."
mount -o ro ${SYSROOT} ${MNTDIR}

# Also check /dev/tty1 ?
if [ ! -e "${CHROOT}/dev/console" ] || [ ! -e "${CHROOT}/dev/null" ]
then
   echo -ne "${BAD}>>${NORMAL}${BOLD} init: Error - /dev is broken!${NORMAL}"
else
   echo -ne "${GOOD}>>${NORMAL}${BOLD} init: Booting (initramfs)${NORMAL}"
fi

cat /proc/mounts > /etc/mtab

cd "${CHROOT}"
mkdir "${CHROOT}/proc" "${CHROOT}/sys" 2>/dev/null
echo -ne "${BOLD}.${NORMAL}"
   umount /sys || echo '*: Failed to unmount the initrd /sys!'
   umount /proc || echo '*: Failed to unmount the initrd /proc!'
echo -e "${BOLD}.${NORMAL}"

# Conditions:
# - / must be tmpfs or ramfs
# - /init must exist and be a regular file (NOT a symlink to /bin/init)
# - the currently running process must have PID 1
# (Best run with "exec ..."
exec switch_root -c "/dev/console" "${CHROOT}" /sbin/init

echo 'A fatal error has probably occured since /sbin/init did not'
echo 'boot correctly. Trying to open a shell...'
echo
exec /bin/sh

Command line example:
Code:
mkinitrd -v -s /usr/local/share/mkinitrd-ng/linuxrc --module=qla2xxx --firmware=ql2300_fw.bin -k 2.6.18-028stab53 --multipath -f

The script creates an init ramdrive, which name is based on the current kernel (or the selected kernel if passed as an argument). I traditionnally symlink the resulting image with /boot/initrd:
Code:
cd /boot
ln -s initramfs-$(uname -r).igz initrd

Busybox only needs the initrd= argument. You can keep kernel argument root=/dev/... to designate the root filesystem. Additional (custom) arguments are lvm and mpath. Here's an example using syslinux|extlinux.
/boot/extlinux.conf:
default openvz
...
label openvz
    kernel openvz
    append root=/dev/sda2 vga=0x317 CONSOLE=/dev/tty1 softlevel=vz initrd=/initrd mpath

There are only a limited amount of sanity checks. The latest version of busybox is required. If you want LVM and/or multipath you must install sys-fs/lvm2 and/or sys-fs/multipath-tools. Both will bring sys-fs/device-mapper. I've tested the script with multipath only.

The creation script uses package names to retrieve actual file names and pulls down dependencies (i.e. dynamic libraries) as well. Hence you don't need a static version of busybox nor of any other package. I've got a basic initramfs of about 3.5 MB with multipath modules and tools.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
dulrich
n00b
n00b


Joined: 23 Apr 2005
Posts: 46

PostPosted: Thu Nov 20, 2008 9:58 pm    Post subject: Reply with quote

Hi VinzC,

I have made some modification to genkernel to add support for multipath in initramfs. The patch is here :
https://forums.gentoo.org/viewtopic-t-714801.html

Thank you for your post which was for me very usefull.

I hope this could be add in the genkernel source !
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Fri Nov 21, 2008 9:07 am    Post subject: Reply with quote

Thanks a whole lot for your patch, really. I didn't want to go genkernel in the first place but later on I realized I had to tweak my scripts because initrd do not work with recent kernels. So your genkernel patch comes handy for it's the only way I found to build working init RAM filesystems :D.

:thumbsup:
Thanks again.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Wed Feb 18, 2009 12:09 pm    Post subject: Reply with quote

Quote:
My hardest time was to find the worldwide names that are involved in multipath.conf :-D . They're given by scsi_id. "


I still got problems with this stage. I can use scsi_id in RHEL and it works like you say, but scsi_id on Gentoo doesn't have the same options. I got:

Code:
equery belongs scsi_id
[ Searching for file(s) scsi_id in *... ]
sys-fs/udev-124-r1 (/lib64/udev/scsi_id)
sys-fs/udev-124-r1 (/sbin/scsi_id -> ../lib64/udev/scsi_id)


Even though I can read wwid using scsi_id /dev/sdd (this gives wwid onlyt when you do echo options=--whitelisted
>> /etc/scsi_id.config), I cannot force multipathd to do so. Even if I put getuid_callout "/sbin/scsi_id %n" in /etc/multipath.conf file, I still get:

Code:
Feb 18 12:52:46 | --------start up--------
Feb 18 12:52:46 | read /etc/multipath.conf
/lib/udev/scsi_id: invalid option -- 's'
Feb 18 12:52:46 | error calling out /lib/udev/scsi_id -g -u -s /block/sdf
/lib/udev/scsi_id: invalid option -- 's'
Feb 18 12:52:46 | error calling out /lib/udev/scsi_id -g -u -s /block/sde
/lib/udev/scsi_id: invalid option -- 's'
Feb 18 12:52:46 | error calling out /lib/udev/scsi_id -g -u -s /block/sdd
/lib/udev/scsi_id: invalid option -- 's'
Feb 18 12:52:46 | error calling out /lib/udev/scsi_id -g -u -s /block/sdc
Feb 18 12:52:46 | path checkers start up


If I do not blacklist local disks, I can see that multipath uses the command from config file to them....

So here I stuck. Any ideas?
Back to top
View user's profile Send private message
IlGab
Guru
Guru


Joined: 26 Nov 2004
Posts: 486

PostPosted: Wed Feb 18, 2009 2:06 pm    Post subject: Reply with quote

I change the original bin scsi_id into scsi_id.bin and I create a bash script called scsi_id that change the -s with -d and /block in /dev.
It works great :)

Can someone explain the difference between multipat and mdadm multipath options ?
Can I use mdadm with fiber channel storage device ?
Back to top
View user's profile Send private message
Januszzz
Guru
Guru


Joined: 04 Feb 2006
Posts: 367
Location: Opole, Poland

PostPosted: Wed Feb 18, 2009 11:20 pm    Post subject: Reply with quote

Quote:
I change the original bin scsi_id into scsi_id.bin and I create a bash script called scsi_id that change the -s with -d and /block in /dev.
It works great :)


Cool :-) I couldn't resist on opening new topic and here is my solution:

https://forums.gentoo.org/viewtopic-t-738467-highlight-.html

So it was sed through source files and simple substitution... I haven't check if thats right in code, but multipathing works.... I don't like bugs, I'm admin only but this one looks as simple as stick... damn bugs :-)
Back to top
View user's profile Send private message
moot.xk
n00b
n00b


Joined: 28 Feb 2009
Posts: 1

PostPosted: Sat Feb 28, 2009 7:10 pm    Post subject: Reply with quote

I don't have a gentoo box to check with at the moment, however ...

On a RHEL system, you can add the _netdev option to a mountpoint in fstab. This causes the filesystem to be skipped during the initial boot phase. Later on, the netfs script in /etc/init.d will try to mount up all the entries that had _netdev. As long as multipathd is started beforehand, the netfs script should mount up your mpath device.
Back to top
View user's profile Send private message
TJNII
l33t
l33t


Joined: 09 Nov 2003
Posts: 637
Location: for(;;);

PostPosted: Wed Mar 09, 2011 8:10 pm    Post subject: Reply with quote

So is the solution in this thread to use a initrd to enable mounting at boot? Seems like overkill to me, but I assume this was not the sole reason that was done.

Unless someone has a cleaner option I'm going to make a custom init script that mounts the multipath'd volumes, and make the daemons that use those filesystems depend on it.
_________________
Maintaining documentation on upgrades is apparently not within the power of Gentoo
Beware when you emerge -u world!
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Wed Mar 09, 2011 9:16 pm    Post subject: Reply with quote

TJNII wrote:
So is the solution in this thread to use a initrd to enable mounting at boot? Seems like overkill to me, [...]


It's not really overkill. I came to the conclusion an initrd was required for the sole reason of fstab. That file is processed to mount all the filesystems and that occurs before any service starts.

In the default configuration, multipathd is a service that is of course run *after* fstab has been processed. And that service is responsible for providing the system with new filesystems and device nodes (e.g. from a SAN). I wanted all my filesystems to be mounted with fstab so I had to make all my disk nodes available beforehand.

The only way is an initrd. That was my conclusion three years ago :D .
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
TJNII
l33t
l33t


Joined: 09 Nov 2003
Posts: 637
Location: for(;;);

PostPosted: Wed Mar 09, 2011 9:48 pm    Post subject: Reply with quote

Fair enough. I don't feel like traveling down that road, so I've gone this way instead:

/etc/init.d/mpmount:
Code:
#!/sbin/runscript
# mpmount: Short script to mount volumes on multipathed devices after multipathd starts
# 3/9/11 TJNII

depend() {
        need checkfs multipathd
}

start() {
        # mount multipathed filesystems
        ebegin "Mounting local filesystems on multipathed devices"

        grep mapper /etc/fstab | sed -e 's|[^ \t]\+[ \t]\+\([^ \t]\+\).*|\1|' | while read tgt; do
                mount $tgt
                if [ "$?" != 0 ]; then
                        eend "$tgt failed to mount"
                fi
        done

        eend

}


I've added the noauto flag to my multipathed volumes to that localmount doesn't balk. All my fstab entries for these use the /dev/mapper/ nodes. This greps them out of the fstab, and then calls mount on their mountpoint. I would expect that it honors all fstab options, but my needs are simple so I haven't verified that. I'll just make my daemons that use the multipath module depend on this. (My root is not multipathed.)
_________________
Maintaining documentation on upgrades is apparently not within the power of Gentoo
Beware when you emerge -u world!
Back to top
View user's profile Send private message
VinzC
Watchman
Watchman


Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood

PostPosted: Wed Mar 09, 2011 10:20 pm    Post subject: Reply with quote

That's also one possibility but the main reason I didn't implemented it was there are two «services» that do the same thing (fstab and this script). As for the initrd way, genkernel now fully supports it. I just didn't have the opportunity to test recently it though.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum