| View previous topic :: View next topic |
| Author |
Message |
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
 Posted: Fri Aug 18, 2017 12:14 pm Post subject: Kernel just stops? |
 |
|
I have a strange issue I need help solving. I am installing Gentoo onto an old system with a Pentium D (64bit, dual-core) CPU and 2GB of RAM to function as a printing and scanning server. The thing is, it just freezes when booting. No kernel panic, no logging, nothing. It just plain stops after freeing the unused kernel memory. Below is an image of the location it freezes at.
Screenshot of crash/freeze
Where do I begin? I have checked fstab, my kernel config, everything. Below is what I believe is relevant to this issue. Ask if you need anything else.
Kernel Configuration
Output of "lspci -knn"
Output of "cat /proc/cpuinfo"
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
 |
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Fri Aug 18, 2017 1:37 pm Post subject: |
|
|
Sounds to me like agetty is borked or the init system never launches it.
I presume you ran lspci etc. on a CD based Linux. Or are you able to ssh in?
What init system and version? |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Fri Aug 18, 2017 3:58 pm Post subject: |
|
|
I use System Rescue CD 4.90 to do my Gentoo installs. Not sure what init versions and such it has. I can get into the chroot and use it just fine via this method, but the kernel never boots. This is a fresh Gentoo install done last week, so it has whatever versions are out currently unless the init and/or agetty has updated since then.
*UPDATE*
I have a tiny bit of initramfs stuff to detect the BTRFS RAID1 array. This works fine on many other Gentoo boxes.
| Code: |
CONFIG_INITRAMFS_SOURCE="/usr/src/initramfs/initramfs_list"
|
/usr/src/initramfs/initramfs_list:
| Code: |
# directory structure
dir /proc 755 0 0
dir /usr 755 0 0
dir /bin 755 0 0
dir /sys 755 0 0
dir /var 755 0 0
dir /lib64 755 0 0
dir /sbin 755 0 0
dir /mnt 755 0 0
dir /mnt/root 755 0 0
dir /mnt/boot 755 0 0
dir /etc 755 0 0
dir /root 700 0 0
dir /dev 755 0 0
# busybox
file /bin/busybox /bin/busybox 755 0 0
#
# fsck deps
#
file /sbin/fsck /sbin/fsck 755 0 0
file /lib64/libmount.so.1 /lib64/libmount.so.1 755 0 0
file /lib64/libblkid.so.1 /lib64/libblkid.so.1 755 0 0
file /lib64/libc.so.6 /lib64/libc.so.6 755 0 0
file /lib64/libuuid.so.1 /lib64/libuuid.so.1 755 0 0
file /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 755 0 0
#
# fsck.ext3 and added deps
#
file /sbin/fsck.ext3 /sbin/fsck.ext3 755 0 0
file /lib64/libext2fs.so.2 /lib64/libext2fs.so.2 755 0 0
file /lib64/libcom_err.so.2 /lib64/libcom_err.so.2 755 0 0
file /lib64/libe2p.so.2 /lib64/libe2p.so.2 755 0 0
file /lib64/libpthread.so.0 /lib64/libpthread.so.0 755 0 0
#
# btrfs utils and added deps
#
file /sbin/btrfs /sbin/btrfs 755 0 0
file /sbin/btrfs-convert /sbin/btrfs-convert 755 0 0
file /sbin/btrfs-debug-tree /sbin/btrfs-debug-tree 755 0 0
file /sbin/btrfs-find-root /sbin/btrfs-find-root 755 0 0
file /sbin/btrfs-image /sbin/btrfs-image 755 0 0
file /sbin/btrfs-map-logical /sbin/btrfs-map-logical 755 0 0
#file /sbin/btrfs-show-super /sbin/btrfs-show-super 755 0 0
file /sbin/btrfs-zero-log /sbin/btrfs-zero-log 755 0 0
file /sbin/btrfsck /sbin/btrfsck 755 0 0
file /sbin/btrfstune /sbin/btrfstune 755 0 0
file /sbin/mkfs.btrfs /sbin/mkfs.btrfs 755 0 0
file /lib64/libz.so.1 /lib64/libz.so.1 755 0 0
file /lib64/liblzo2.so /usr/lib64/liblzo2.so 755 0 0
#
# Nano editor and added deps
#
file /sbin/nano /usr/bin/nano 755 0 0
file /lib64/libncursesw.so.6 /lib64/libncursesw.so.6 755 0 0
file /lib64/libdl.so.2 /lib64/libdl.so.2 755 0 0
#
# mknod and added deps
#
file /sbin/mknod /bin/mknod 755 0 0
#
# more viewer
#
file /sbin/more /bin/more 755 0 0
#
# init script
#
file /init /usr/src/initramfs/init 755 0 0
#
# fstab
#
file /etc/fstab /usr/src/initramfs/fstab 644 0 0
|
/usr/src/initramfs/init:
| Code: |
#!/bin/busybox sh
rescue_shell() {
echo "$@"
echo "Something went wrong. Dropping you to a shell."
busybox --install -s
exec /bin/sh
}
mount_root() {
echo "scanning for btrfs filesystems.... will take about 5-10 seconds"
/sbin/btrfs device scan
echo "mounting /mnt/root"
mount /mnt/root
}
# temporarily mount proc and sys
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev
# disable kernel messages from popping onto the screen
echo 0 > /proc/sys/kernel/printk
# clear the screen
clear
# mounting rootfs on /mnt/root
mount_root || rescue_shell "Error with uuidlabel_root"
echo "All done. Switching to real root."
# clean up. The init process will remount proc sys and dev later
umount /proc
umount /sys
umount /dev
# switch to the real root and execute init
exec switch_root /mnt/root /sbin/init
|
/usr/src/initramfs/fstab:
| Code: |
/dev/sda2 /mnt/root btrfs device=/dev/sda2,device=/dev/sdb2,defaults,subvol=@root 0 0
/dev/sda2 /mnt/boot btrfs device=/dev/sda2,device=/dev/sdb2,defaults,subvol=@boot 0 0
/dev/sda2 /mnt/root/usr/src btrfs device=/dev/sda2,device=/dev/sdb2,defaults,subvol=@source 0 0
/dev/sda2 /mnt/root/home btrfs device=/dev/sda2,device=/dev/sdb2,defaults,subvol=@home 0 0
|
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
NTU
Apprentice
Joined: 17 Jul 2015
Posts: 187
|
| Posted: Sat Aug 19, 2017 2:57 am Post subject: |
|
|
Is this a custom kernel configuration? What options do you have set, can you post the config? Have you been able to boot that same kernel config on the same system before? You should always be able to get some debug info if you configured the kernel hacking section properly. Early printk, compile with frame pointers, compile with debug info, and make sure you have symbols turned on (KALLSYMS) Perhaps also enable the NMI selftest and locking tests (spinlock and mutex)
I have 3 different kernels installed on my system and bounce between them depending on what I'm doing. I do everything from Coreboot development to RTAI kernel hacking. These are some snippets from my test/debug kernel config:
| Code: | #
# printk and dmesg options
#
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=8
CONFIG_BOOT_PRINTK_DELAY=y
#
# Compile-time checks and compiler options
#
CONFIG_DEBUG_INFO=y
CONFIG_READABLE_ASM=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_PAGE_OWNER=y
CONFIG_DEBUG_FS=y
CONFIG_HEADERS_CHECK=y
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_SECTION_MISMATCH_WARN_ONLY=y
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
CONFIG_STACK_VALIDATION=y
CONFIG_DEBUG_KERNEL=y
#
# Memory Debugging
#
CONFIG_PAGE_EXTENSION=y
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT=y
CONFIG_PAGE_POISONING=y
# CONFIG_PAGE_POISONING_NO_SANITY is not set
CONFIG_PAGE_POISONING_ZERO=y
# CONFIG_DEBUG_OBJECTS is not set
CONFIG_SLUB_DEBUG_ON=y
# CONFIG_SLUB_STATS is not set
CONFIG_HAVE_DEBUG_KMEMLEAK=y
# CONFIG_DEBUG_KMEMLEAK is not set
# CONFIG_DEBUG_STACK_USAGE is not set
CONFIG_DEBUG_VM=y
# CONFIG_DEBUG_VM_VMACACHE is not set
# CONFIG_DEBUG_VM_RB is not set
# CONFIG_DEBUG_VM_PGFLAGS is not set
CONFIG_DEBUG_VIRTUAL=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_DEBUG_PER_CPU_MAPS=y
CONFIG_HAVE_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_SHIRQ=y
#
# Debug Lockups and Hangs
#
# CONFIG_LOCKUP_DETECTOR is not set (for you, probably turn this on)
# CONFIG_DETECT_HUNG_TASK is not set (and turn this on to)
# CONFIG_WQ_WATCHDOG is not set
CONFIG_PANIC_ON_OOPS=y
CONFIG_PANIC_ON_OOPS_VALUE=1
CONFIG_PANIC_TIMEOUT=10
CONFIG_SCHED_DEBUG=y
CONFIG_SCHED_INFO=y
CONFIG_SCHEDSTATS=y
CONFIG_SCHED_STACK_END_CHECK=y
CONFIG_DEBUG_TIMEKEEPING=y
#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
# CONFIG_DEBUG_WW_MUTEX_SLOWPATH is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_ATOMIC_SLEEP is not set
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
# CONFIG_LOCK_TORTURE_TEST is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_PI_LIST=y
CONFIG_DEBUG_SG=y
CONFIG_DEBUG_NOTIFIERS=y
CONFIG_DEBUG_CREDENTIALS=y
#
# RCU Debugging
#
# CONFIG_PROVE_RCU is not set
CONFIG_SPARSE_RCU_POINTER=y
# CONFIG_TORTURE_TEST is not set
# CONFIG_RCU_PERF_TEST is not set
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=21
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_EQS_DEBUG=y
# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_NOTIFIER_ERROR_INJECTION is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_FENTRY=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
#
# Runtime Testing
#
CONFIG_TEST_LIST_SORT=y
CONFIG_ATOMIC64_SELFTEST=y
CONFIG_TEST_RHASHTABLE=y
CONFIG_TEST_HASH=y
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_TEST_STATIC_KEYS=m
CONFIG_STRICT_DEVMEM=y
CONFIG_IO_STRICT_DEVMEM=y
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
CONFIG_X86_PTDUMP_CORE=y
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA_TEST=y
CONFIG_DEBUG_WX=y
CONFIG_DEBUG_SET_MODULE_RONX=y
# CONFIG_DEBUG_NX_TEST is not set
CONFIG_DOUBLEFAULT=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
# CONFIG_IO_DELAY_0X80 is not set (set this)
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
CONFIG_IO_DELAY_NONE=y (this will change to 0X80)
CONFIG_DEFAULT_IO_DELAY_TYPE=3 (this will change to 0)
# CONFIG_DEBUG_BOOT_PARAMS is not set
CONFIG_CPA_DEBUG=y
CONFIG_OPTIMIZE_INLINING=y
CONFIG_DEBUG_ENTRY=y
CONFIG_DEBUG_NMI_SELFTEST=y
CONFIG_X86_DEBUG_FPU=y |
Additional options:
| Code: | #
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_ELFCORE=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_SCRIPT=y
# CONFIG_HAVE_AOUT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_COREDUMP=y
#
# Generic Driver Options
#
CONFIG_ALLOW_DEV_COREDUMP=y |
Hope this helps! |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Sat Aug 19, 2017 8:06 pm Post subject: |
|
|
My kernel configuration is linked in the first post. It is the second link. It is the complete kernel configuration, pasted straight from .config in the kernel source directory.
Still no luck getting this to boot. I am supposed to have this in Monday and things are looking bad. If I had a log I'd know what to look for but still no go here. Is there anything else I can provide to help diagnose this issue?
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Sat Aug 19, 2017 8:22 pm Post subject: |
|
|
There seems to be no evidence that the kernel unpacked the initramfs or that the init script mounted your root filesystem.
Did you see any evidence scroll by during the boot? |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Sun Aug 20, 2017 1:45 pm Post subject: |
|
|
I do not use an initramfs. That is to say, I do not build one using something like genkernel.
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sun Aug 20, 2017 4:19 pm Post subject: |
|
|
I just encountered this on one of my machines.
I changed from the no longer in the tree radeon-ucode to linux-firmware.
Then this happened. Look at your firmware. |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Sun Aug 20, 2017 4:50 pm Post subject: |
|
|
| The_Great_Sephiroth wrote: | | I do not use an initramfs. That is to say, I do not build one using something like genkernel. |
You have the internal initramfs from CONFIG_INITRAMFS_SOURCE.
I was trying to determine if it stops before or after your init script. It looks like it stops before your init script. Maybe Tony0945 is correct. |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Sun Aug 20, 2017 7:49 pm Post subject: |
|
|
No firmware needed for this system. Intel integrated graphics, no ATI/AMD/nVidia. I have no clue how to tell if my script ran since there is no log. I believe it does stop before, but not sure why.
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Sun Aug 20, 2017 8:52 pm Post subject: |
|
|
I rebuilt the kernel from scratch but will not know if it worked until Monday when I am on-site. I removed the old kernel .config, did a "make menuconfig", and went through the entire process again. The current configuration is linked below.
Current kernel configuration
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Sun Aug 20, 2017 10:10 pm Post subject: |
|
|
| I noticed the fstab uses sda2 but the screenshot only shows sda1. Not sure if that is the problem. |
|
| Back to top |
|
|
The_Great_Sephiroth
Veteran
Joined: 03 Oct 2014
Posts: 1602
Location: Fayetteville, NC, USA
|
| Posted: Mon Aug 21, 2017 9:43 pm Post subject: |
|
|
Let's call this, "bad kernel". I booted System Rescue CD, chrooted, synced, installed new kernel, copied the config, did "make oldconfig", answered a few new questions, built it, and it booted. Copied the configuration back to 4.9.34, built, no boot. No clue what the heck is up with that. I am good now, and actually just upgraded to 4.12.5 on the domain controller. Thanks for your help!
_________________
Ever picture systemd as what runs "The Borg"? |
|
| Back to top |
|
|
|
Kernel & Hardware
