can't boot with new kernel 2.6.39-gentoo

[m27315]

Hi,

Ran across a new "feature" that I had not seen before. On an up to date system, I tried to upgrade to the latest kernel, from 2.6.38-gentoo-r5 to 2.6.39-gentoo.

I copied the old .config to the the new dir, ran "make oldconfig", picked reasonable options, etc.:

Code: Select all

# /usr/src/linux-2.6.38-gentoo-r5/.config /usr/src/linux-2.6.39-gentoo
# cd /usr/src
# ln -snf linux-2.6.39-gentoo linux
# cd linux
# make oldconfig
... <choose reasonable options> ...
# make -j4
# make modules_intall
# update-modules
# mount /boot
# cp -L arch/x86_64/boot/bzImage /boot/kernel-2.6.39
# cp .config /boot/config-2.6.39
# cp System.map /boot/System.map-2.6.39
# vi /boot/grub/grub.conf ## make necessary changes

Anyway, now the screen just locks up after rebooting. Grub echoes back the start-up lines, and tries to hand off to the kernel, but then everything locks up. As far as I can tell the kernel never initializes and openRC does not even have a chance to start.

BTW, I have additionally tried saying no to all new options in the "make oldconfig" process, and that did not help.

Any suggestions?

Thanks!

Trevor

...

For reference, here's my GRUB entry:

Code: Select all

title Gentoo Linux 2.6.39-gentoo
root (hd0,1)
kernel /boot/kernel-2.6.39-gentoo root=LABEL=VPRIME_ROOT splash=silent,fadein,theme:livecd-2010.0 video=uvesafb:1280x720-32,mtrr:3,ywrap quiet console=tty1 fbcon=scrollback:256K rootfstype=ext4
initrd /boot/fbsplash-label-initramfs.cpio.gz

All of this is identical to my working 2.6.39-r5 kernel, except the kernel location.

[bjlockie]

I don't know why I don't need an "initrd" line but I don't.

[hdias]

Hi,

I have the same problem after upgrade to 2.6.39 kernel.
Please take a look at this bug: http://bugs.gentoo.org/show_bug.cgi?id=368109

The solution to boot with 2.6.39 kernel:

Add this option to the grub.conf: video=vesafb:off

Code: Select all

title=Gentoo Linux (2.6.39-gentoo)
        root (hd0,0)
        kernel (hd0,0)/kernel-genkernel-x86_64-2.6.39-gentoo ro root=LABEL=/ video=vesafb:off splash=silent,theme:natural_gentoo console=tty1 quiet
        initrd (hd0,0)/initramfs-genkernel-x86_64-2.6.39-gentoo

Hi,

Ran across a new "feature" that I had not seen before. On an up to date system, I tried to upgrade to the latest kernel, from 2.6.38-gentoo-r5 to 2.6.39-gentoo.

I copied the old .config to the the new dir, ran "make oldconfig", picked reasonable options, etc.:

Code: Select all

# /usr/src/linux-2.6.38-gentoo-r5/.config /usr/src/linux-2.6.39-gentoo
# cd /usr/src
# ln -snf linux-2.6.39-gentoo linux
# cd linux
# make oldconfig
... <choose reasonable options> ...
# make -j4
# make modules_intall
# update-modules
# mount /boot
# cp -L arch/x86_64/boot/bzImage /boot/kernel-2.6.39
# cp .config /boot/config-2.6.39
# cp System.map /boot/System.map-2.6.39
# vi /boot/grub/grub.conf ## make necessary changes

Anyway, now the screen just locks up after rebooting. Grub echoes back the start-up lines, and tries to hand off to the kernel, but then everything locks up. As far as I can tell the kernel never initializes and openRC does not even have a chance to start.

BTW, I have additionally tried saying no to all new options in the "make oldconfig" process, and that did not help.

Any suggestions?

Thanks!

Trevor

...

For reference, here's my GRUB entry:

Code: Select all

title Gentoo Linux 2.6.39-gentoo
root (hd0,1)
kernel /boot/kernel-2.6.39-gentoo root=LABEL=VPRIME_ROOT splash=silent,fadein,theme:livecd-2010.0 video=uvesafb:1280x720-32,mtrr:3,ywrap quiet console=tty1 fbcon=scrollback:256K rootfstype=ext4
initrd /boot/fbsplash-label-initramfs.cpio.gz

All of this is identical to my working 2.6.39-r5 kernel, except the kernel location.

[m27315]

I don't know why I don't need an "initrd" line but I don't.

I use an initramfs to load the splash screen and to do HDD label-based lookup. It allows me to find the root partition by label (root=LABEL=VPRIME_ROOT) instead of by bus number (root=/dev/sda1). I went to this model, because the drive lettering kept being thrown off depending on whether or not I had an eSATA drive plugged in. Most of the other distros also use this approach (or UUID labeling), because it helps to stabilize your config in regards to drive letters. Here's some more info:

http://en.gentoo-wiki.com/wiki/Initramfs
http://en.gentoo-wiki.com/wiki/Fbsplash

For reference, here's my initramfs start-up script (/usr/src/initramfs/init):

Code: Select all

#!/bin/busybox sh
rescue_shell() {
    echo "Something went wrong. Dropping you to a shell."
    busybox --install -s
    exec /bin/sh
}
mini_udev() {
    echo /sbin/mdev > /proc/sys/kernel/hotplug
    mdev -s
}
uuidlabel_root() {
    for cmd in $(cat /proc/cmdline) ; do
        case $cmd in
        root=*)
            type=$(echo $cmd | cut -d= -f2)
            if [ $type == "LABEL" ] || [ $type == "UUID" ] ; then
                uuid=$(echo $cmd | cut -d= -f3)
                mount -o ro $(findfs "$type"="$uuid") /mnt/root
            else
                mount -o ro $(echo $cmd | cut -d= -f2) /mnt/root
            fi
            ;;
        esac
    done
}
# Mount the /proc and /sys filesystems.
mount -t proc none /proc
mount -t sysfs none /sys
# Launch mdev (mini_udev)
mini_udev
# Do your stuff here.
#echo "This script mounts rootfs and boots it up, nothing more!"
# Mount the root filesystem.
#mount -o ro /dev/sda1 /mnt/root
uuidlabel_root || rescue_shell
# Clean up.
umount /proc
umount /sys
# Boot the real thing.
exec switch_root /mnt/root /sbin/init

I've used this for years, and it works fine on all my previous kernels, 2.6.38-r5 notably.

Oh, and here's my 2.6.39 kernel config: http://pastebin.com/8r4c0GS0

Thanks!

Trevor

[m27315]

I have the same problem after upgrade to 2.6.39 kernel.
Please take a look at this bug: http://bugs.gentoo.org/show_bug.cgi?id=368109

The solution to boot with 2.6.39 kernel:

Add this option to the grub.conf: video=vesafb:off

Code: Select all

title=Gentoo Linux (2.6.39-gentoo)
        root (hd0,0)
        kernel (hd0,0)/kernel-genkernel-x86_64-2.6.39-gentoo ro root=LABEL=/ video=vesafb:off splash=silent,theme:natural_gentoo console=tty1 quiet
        initrd (hd0,0)/initramfs-genkernel-x86_64-2.6.39-gentoo

Thanks for the great tip! Unfortunately, it did not entirely help me. I had to additionally disable my initrd (initramfs). Once I did that, I lost the splash screen and disk-label functionality, so I had to revert to Grub's most basic form, which for me is:

Code: Select all

title Gentoo Linux 2.6.39-gentoo
root (hd0,1)
kernel /boot/kernel-2.6.39-gentoo ro root=/dev/sdb3

This works for me.

Apparently, I've lost some functionality that was built into my initramfs image. Any ideas?

Thanks!

Trevor

[NeddySeagoon]

m27315,

Try rebuilding your initrd. Its mostly portable across kernel versions but every now and again it breaks.
The fbcondecor patch is in gentoo-sources-2.6.39, so its not that.

[m27315]

Try rebuilding your initrd. Its mostly portable across kernel versions but every now and again it breaks.

Thanks for the tip, NeddySeagoon! Sadly, that did not help. Here's what I did, in case I should have done something else:

Code: Select all

# cp -a /bin/busybox /usr/src/initramfs/bin/busybox
# rm /boot/fbsplash-label-initramfs.cpio.gz
# cd /usr/src/initramfs
# find . -print0 | cpio --null -ov --format=newc | gzip -9 > /boot/fbsplash-label-initramfs.cpio.gz

BTW, I left the "static" flag on my version of busybox:

Code: Select all

# emerge -pv busybox
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-apps/busybox-1.17.4  USE="ipv6 mdev pam static -debug -make-symlinks -savedconfig (-selinux)" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

Could there be something in my init script that is not compatible with 2.6.39? Maybe mdev doesn't work for 2.6.39? Maybe someone could post a working init script that supports HDD labels? Or, maybe I have some bad kernel config?

Thanks!

Trevor

[NeddySeagoon]

m27315,

You need +static on everything in the initrd. There are no libs nor any dynamic linker there.
I suppose that busybox is all you need, what about mdev ?

I use a static /dev in my initrd, when I have to have an initrd.

[sipingal]

convert the cpio-initrd-file to loopback device:

Code: Select all

# cd /boot
# dd if=/dev/zero of=initrd_loop bs=1000k count=1
# mke2fs -F -m0 initrd_loop
# mount -t ext2 -o loop initrd_loop /mnt
# cd /mnt/
# zcat /boot/<original initrd file> | cpio -idmv
# cd -
# umount /mnt/
# gzip -9 initrd_loop

then use initrd_loop.gz, but not work for me:(, I'm using build-in-kernel initramfs_data.cpio(it does not support loop device).

Code: Select all

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/etc/splash/initramfs_data-d620.cpio"

[mziab]

I also got hit by this after upgrading to 2.6.39. I initially attributed the hang to some breakage in nouveau/dri.

Further testing revealed two things:
1) The crash happens with nvidia-drivers + uvesafb, as well as with nouveau + KMS.
2) Removing initrd line in grub.conf allows me to boot. The splash starts a bit later, but it works.

Since I could reproduce the bug regardless of which framebuffer was used, it seems like the breakage is in fbcondecor itself. For the record, I did regenerate the initramfs using splash_geninitramfs, but it didn't change anything. It seems something goes awry when fbcondecor is started during early boot, but not later. I hope someone figures out what's wrong soon. For now I'll remove that initrd line.

[marco1475]

I have the same issue and I can confirm that it has something to do with fbcondecor being started that early in the boot process. All of this was (and is) working for any kernel version < 2.6.39:

I am using binary nvidia-drivers and uvesafb.ko in a custom initramfs to get a nice splash screen early in the boot process and to check which GPU is active (my netbook has two, on-board Intel and a dedicated nVidia graphics card that can be toggled by a hardware switch). It's a very simple script I am using:

Code: Select all

#!/bin/busybox sh

# Mount proc and sys
mount -t proc none /proc
mount -t sysfs none /sys

# Mount the root file system
mount /dev/sda3 /root

# Detect GPU
VIDEO=$( lspci | grep -c "nVidia" )
if [[ "${VIDEO}" == 1 ]]; then
	echo "Using NVIDIA video."

	# [creating a bunch of symbolic links]

	# Start uvesafb in 800x600
	sleep 1
	modprobe uvesafb mode_option=800x600-32 scroll=ywrap
else
	echo "Using Intel video."

	# [unimportant code]
fi

# Clean up
umount /proc
umount /sys

# Boot the real thing
exec switch_root /root /sbin/init

During booting with kernel 2.6.39 the boot process hangs when uvesafb is called to create the color frame buffer:

Code: Select all

uvesafb: NVIDIA Corportation, G98 Board - 05666e90, Chip Rev   , 0 EM: NVIDIA, VBE v3.0
uvesafb: protected mode interface info at c000:cd80
uvesafb: pmi: set display start = c00ccde3, set palette = c00cce3e
uvesafb: pmi: ports = 3b4 3b5 3ba 3c0 3c1 3c4 3c5 3c6 3c7 3c8 3c9 3cc 3ce 3cf 3d0 3d1 3d2 3d3 3d4 3d5 3da
uvesafb: VBIOS/hardware doesn't support DDC transfers
uvesafb: no monitor limits have been set, default refresh rate will be used
uvesafb: scrolling: ywrap using protected mode interface, yres_virtual=1536
uvesafb: Console: switching to colour frame buffer device 100x37

Under normal circumstances the screen switches to the color frame buffer using my theme and the boot process continues, the next line being something like "fbcondecor: console using color-depth 24". But with kernel 2.6.39 it just hangs here until it says:

Code: Select all

kworker/u:0 used greatest stack depth: 5840 bytes left

I checked but it didn't look like fbcondecor has been updated in quite some time. I wonder what is different about the 2.6.39 kernel that could cause this. Any ideas?

Thanks,

[m27315]

Following the parallel bug issue: http://bugs.gentoo.org/show_bug.cgi?id=368109#c5

I also disabled my framebuffer console decorations in the kernel and was then able to boot-up using both splash and my initrd image. Here's my working GRUB entry for reference:

Code: Select all

title Gentoo Linux 2.6.39-gentoo SPLASH
root (hd0,1)
kernel /boot/kernel-2.6.39-gentoo root=LABEL=VPRIME_ROOT splash=silent,fadein,theme:livecd-2010.0 video=vesafb:off quiet console=tty1 rootfstype=ext4
initrd /boot/fbsplash-label-initramfs.cpio.gz

It ain't pretty. I'm missing the console decorations, which is really nice for console work (i.e., tighter font), but at least I'm moving again AND I have initrd support.

However, I'm leaving the thread as UNSOLVED, until the exact issues is tracked down in the framebuffer and console decorations and ... whatever else might be going on. Workaround is not the same as SOLVED in my book.

@NeddySeagoon: I believe static busybox comes with mdev support, if the appropriate USE flags are enabled (+static +mdev).

Thanks to all for the help, tips, and input. Hopefully, someone can find a real fix now.

[WendyB]

I have the same problem.
I'm using vesafb, commenting the initrd line makes the computer boot with framebuffer, but without decorations
If you need the initrd for other things you might rebuild it without the splash things
latest vanilla sources work fine

The fbcondecor service works without a problem so I still have consoledecoration, only later in the bootprocess

[mziab]

For the record, I've sifted through the framebuffer-related commits between 2.6.39-rc7 and 2.6.39-final. Those are the relevant changes, in chronological order:

fbcon: add lifetime refcount to opened frame buffers
fbmem: make read/write/ioctl use the frame buffer at open time
fbmem: fix remove_conflicting_framebuffers races
Further fbcon sanity checking

I've yet to try reverting them, but it seems like a possible interim solution.

UPDATE: Yes, reverting in reverse order works. Alternatively, one could copy over drivers/video/fbmem.c and include/linux/fb.h from 2.6.39-rc7.

[Hu]

UPDATE: Yes, reverting in reverse order works. Alternative, one could copy over drivers/video/fbmem.c and include/linux/fb.h from 2.6.39-rc7.

Do you need to revert all of the cited commits to get this working or only some of the later ones? Either way, it would be worth a bug report to find why these commits conflict with the previously working decorations. Based on the comment from WendyB that "latest vanilla sources works fine", it sounds like this is a problem with the fbcondecor patch interacting badly with the upstream code. Could someone who uses framebuffer decorations confirm or refute this?

[Otamay]

2.6.39-rc7 works fine with decorations.

However, when fbcondecor-2.6.38 patch is applied to 2.6.39 vanilla sources, there is a 48 lines offset in drivers/video/fbmem.c, and an 1 line offset in include/linux/fb.h

So I think maybe the problem could be in fbmem.c . Not too helpful, but..

[mziab]

Do you need to revert all of the cited commits to get this working or only some of the later ones?

Upon further inspection it seems the breakage comes from the third commit (fbmem: fix remove_conflicting_framebuffers races). The splash worked until I re-applied the third patch. So you'll have to revert the last two commits, since they're interrelated.

[Pilsner]

I've had the same trouble with 2.6.39-gentoo-r3.
Also of a sudden new kernel did not boot and my grub.conf was simple as that, no decorations, no initrd:

Code: Select all

kernel /vmlinuz-2.6.39-gentoo-r3 root=/dev/sda6 video=1024x768

It booted only when I removed the "video=1024x768" part. But now the console is pretty ugly since only left upper part is utilized. Xorg works fine.

[Tony0945]

I've also been fighting this for a few days.

Code: Select all

title=Gentoo Linux (2.6.39-gentoo-r3friday)
root (hd1,0)
kernel /kernel-genkernel-x86_64-2.6.39-gentoo-r3friday  root=/dev/ram0 init=/linuxrc ramdisk=8192 real_root=/dev/sdb3 udev
initrd /initramfs-genkernel-x86_64-2.6.39-gentoo-r3friday vga=ask

I have a Gigabyte 880G based board with the on-board radeon video disabled and am running an EVGA 8400GS video card. I replaced the on-board video because Windows kept blue-screening in the ati driver. Linux was fine. With the nvidia card, I had to add the nouveau driver to get X working, but with 2.6.39, I keep "green screening" on boot. It dies just as udev starts.

Rather than manually copying the config, I build using

Code: Select all

genkernel --config-file=/etc/kernels/my_kernel_config --menuconfig all

The only thing I did in menuconfig was add the kernel tag "friday".

I'll try disabling the frame buffer, but this was supposed to be corrected in 2.6.39-r3. If not, why was it marked stable? I always run a stable amd64 system unless a package I need has NO stable version. Stable isn't supposed to blow up on boot.

[Tony0945]

Disabling frame buffer did not help. I see that my basement server is running 2.6.39-r3 with no problem. Hardware differences are that that system has onboard nvidia Geforce only and it is running an Athlon64 X2 (Newcastle?) and this system has the ati radeon onboard video plus an nvdia pcie card while running on an Phenom II X6. I'll try removing the radeon module.

My previous post said the boot died while loading udev, that is incorrect. It dies when loading mdev.

[Hu]

I am not familiar with a failure mode that results in a "green screen." Could you elaborate on what you mean by this? I have heard of failures where the screen goes black and then into powersave when the kernel tries to switch to high resolution mode.

[NeddySeagoon]

Tony0945,

Lets start at the beginning. Please posy your lspci output and put the broken kernel.config on a pastebin as it won't fit in a post.
How do you make your initrd ?

[Tony0945]

I call it the "green screen of death" cf BSOD. The display freezes, disk activity stops and the background changes from black to a sick green, looking rather like an algae filled swamp. The text is still there in green-tinged white.

lspci output:

• 00:00.0 Host bridge: Advanced Micro Devices [AMD] RS780 Host Bridge Alternate
  00:02.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (ext gfx port 0)
  00:09.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 4)
  00:0a.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 5)
  00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [IDE mode] (rev 40)
  00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
  00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
  00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
  00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
  00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 42)
  00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller (rev 40)
  00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
  00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller (rev 40)
  00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40)
  00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2 Controller
  00:15.0 PCI bridge: ATI Technologies Inc Device 43a0
  00:15.1 PCI bridge: ATI Technologies Inc Device 43a1
  00:16.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
  00:16.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
  00:18.0 Host bridge: Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] HyperTransport Configuration
  00:18.1 Host bridge: Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Address Map
  00:18.2 Host bridge: Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] DRAM Controller
  00:18.3 Host bridge: Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Miscellaneous Control
  00:18.4 Host bridge: Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Link Control
  01:00.0 VGA compatible controller: nVidia Corporation Device 10c3 (rev a2)
  01:00.1 Audio device: nVidia Corporation High Definition Audio Controller (rev a1)
  02:00.0 USB Controller: NEC Corporation Device 0194 (rev 04)
  03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
  04:06.0 Multimedia video controller: Conexant Systems, Inc. CX23418 Single-Chip MPEG-2 Encoder with Integrated Analog Video/Broadcast Audio Decoder
  04:07.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
  05:00.0 SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)
  05:00.1 IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)

Code: Select all

Casti tony # pastebin /etc/kernels/*39*custom2
Use of uninitialized value in lc at /usr/bin/pastebin line 227.
http://pastebin.com/6JZgSYy3
Casti tony # 

Did that actually do something?

I point out that this config works fine in 2.6.38, which where I'm posting from.

Code: Select all

Casti tony # uname -a
Linux Casti 2.6.38-gentoo-r6custom2 #1 SMP PREEMPT Thu Jul 14 17:43:01 CDT 2011 x86_64 AMD Phenom(tm) II X6 1090T Processor AuthenticAMD GNU/Linux

[Tony0945]

I forgot. You also asked how I made the kernel.

I made it by running

Code: Select all

genkernel --menuconfig --kernel-config=/etc/kernels/kernel-config-x86_64-2.6.38-gentoo-r6custom2 all

I like to use genkernel because it automates the tedium of copying bzimage and editing grub.conf. But I like to customize the config, to remove support for hardware I'll never have and add things I want like support for the Hauppauge HVR-1600 that you see in the lspci. I also had to edit the config to add the nouveau driver.

Notice that the input was the config for 2.6.38-r6 which works, but what I posted was the config kernel-config-x86_64-2.6.39-gentoo-r3custom2 which genkernel had output as a result of the succesful compile.