Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Out-of-memory kernel panic at boot after glibc upgrade
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on Sparc
View previous topic :: View next topic  
Author Message
bender86
Guru
Guru


Joined: 18 Mar 2005
Posts: 464

PostPosted: Fri Apr 14, 2017 7:17 pm    Post subject: Out-of-memory kernel panic at boot after glibc upgrade Reply with quote

Hello,
I have a problem with my SPARC machine (Netra X1).
I'm fairly sure the problem was caused by a glibc upgrade (2.22-r4 -> 2.23-r3). That is, I upgraded every other package, I rebooted and upgraded glibc. The last thing I could see (I was using a tmux session) was the message "safely unmerging existing instance" (or whatever text emerge prints in this case). At that point I could not issue any command in the tmux session, and neither open a shell over SSH. I suspect there was a memory leak which saturated the swapfile.
Then I hard rebooted.
At this point, after the kernel is loaded, I get a out-of-memory kernel panic. At first it was something related to "loadkeys", so I rebooted with "init=/bin/sh" and removed "keymaps" from the boot runlevel. Now I have a similar problem with mount. Attached is the complete log from when I power on the machine (lines like --> [NNN lines] mean that the previous line was repeated NNN times).

Any idea on what could have been the problem? I had a similar issue few years ago when after a glibc upgrade every binary dynamically linked to it failed and had to throw away my installation and start from scratch, but this time if I boot with "init=/bin/sh" I can run commands.

If I manually mount my filesystems and unmount /dev and mount /proc I can run emerge. However, I then get an out-of-memory kernel panic.

I have a cross compiler and emerge, so I could create a new installation, but I'm not sure if I'll be able to merge the packages.


Code:
lom>poweron
lom>
LOM event: +0h42m7s host power on
ü
Sun Netra X1 (UltraSPARC-IIe 400MHz), No Keyboard
OpenBoot 4.0, 512 MB memory installed, Serial #50660900.
Ethernet address 0:3:ba:5:6:24, Host ID: 83050624.



Initializing Memory -

Boot device: disk  File and args:
SILO Version 1.4.14_git20120819_p1
\
Select which kernel boot:
  * linux       : default (4.4.39)
    maintenance : maintenance (4.4.39)
    emergency   : /bin/sh (4.4.39)
    old         : maintenance (4.1.15-r1)

boot:
Allocated 64 Megs of memory at 0x40000000 for kernel
|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\Loaded kernel version 4.4.39

PROMLIB: Sun IEEE Boot Prom 'OBP 4.0.14 2001/11/07 17:16'
PROMLIB: Root node compatible: sun4u
Linux version 4.4.39-gentoo (root@netra) (gcc version 4.9.3 (Gentoo 4.9.3 p1.5, pie-0.6.4) ) #1 Sun Jan 1 15:57:23 CET 2017
bootconsole [earlyprom0] enabled
ARCH: SUN4U
Ethernet address: 00:03:ba:05:06:24
MM: PAGE_OFFSET is 0xfffff80000000000 (max_phys_bits == 40)
MM: VMALLOC [0x0000000100000000 --> 0x0000060000000000]
MM: VMEMMAP [0x0000060000000000 --> 0x00000c0000000000]
Kernel: Using 2 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /pci@1f,0/isa@7/serial@0,3f8
PROM: Built device tree with 71105 bytes of memory.
Top of RAM: 0x67ec8000, Total RAM: 0x1fe36000
Memory hole size: 1152MB
Allocated 16384 bytes for kernel page tables.
Zone ranges:
  Normal   [mem 0x0000000000000000-0x0000000067ec7fff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x0000000007ffffff]
  node   0: [mem 0x0000000020000000-0x0000000027ffffff]
  node   0: [mem 0x0000000040000000-0x0000000047ffffff]
  node   0: [mem 0x0000000060000000-0x0000000066ffdfff]
  node   0: [mem 0x0000000067000000-0x0000000067dfffff]
  node   0: [mem 0x0000000067e80000-0x0000000067eaffff]
  node   0: [mem 0x0000000067ec0000-0x0000000067ec7fff]
Initmem setup node 0 [mem 0x0000000000000000-0x0000000067ec7fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,mul32,div32,v8plus]
CPU CAPS: [vis]
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 64796
Kernel command line: root=/dev/sda1 ro softlevel=default
PID hash table entries: 2048 (order: 1, 16384 bytes)
Dentry cache hash table entries: 65536 (order: 6, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 5, 262144 bytes)
Sorting __ex_table...
Memory: 499680K/522456K available (3507K kernel code, 199K rwdata, 888K rodata, 144K init, 518K bss, 22776K reserved, 0K cma-reserved)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
NR_IRQS:2048 nr_irqs:2048 1
clocksource: hbtick: mask: 0xffffffffffffffff max_cycles: 0x148020aa9, max_idle_ns: 440795202069 ns
clocksource: mult[b400012e] shift[24]
clockevent: mult[16c16bf] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled
bootconsole [earlyprom0] disabled
PROMLIB: Sun IEEE Boot Prom 'OBP 4.0.14 2001/11/07 17:16'
PROMLIB: Root node compatible: sun4u
Linux version 4.4.39-gentoo (root@netra) (gcc version 4.9.3 (Gentoo 4.9.3 p1.5, pie-0.6.4) ) #1 Sun Jan 1 15:57:23 CET 2017
bootconsole [earlyprom0] enabled
ARCH: SUN4U
Ethernet address: 00:03:ba:05:06:24
MM: PAGE_OFFSET is 0xfffff80000000000 (max_phys_bits == 40)
MM: VMALLOC [0x0000000100000000 --> 0x0000060000000000]
MM: VMEMMAP [0x0000060000000000 --> 0x00000c0000000000]
Kernel: Using 2 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /pci@1f,0/isa@7/serial@0,3f8
PROM: Built device tree with 71105 bytes of memory.
Top of RAM: 0x67ec8000, Total RAM: 0x1fe36000
Memory hole size: 1152MB
Allocated 16384 bytes for kernel page tables.
Zone ranges:
  Normal   [mem 0x0000000000000000-0x0000000067ec7fff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x0000000007ffffff]
  node   0: [mem 0x0000000020000000-0x0000000027ffffff]
  node   0: [mem 0x0000000040000000-0x0000000047ffffff]
  node   0: [mem 0x0000000060000000-0x0000000066ffdfff]
  node   0: [mem 0x0000000067000000-0x0000000067dfffff]
  node   0: [mem 0x0000000067e80000-0x0000000067eaffff]
  node   0: [mem 0x0000000067ec0000-0x0000000067ec7fff]
Initmem setup node 0 [mem 0x0000000000000000-0x0000000067ec7fff]
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,mul32,div32,v8plus]
CPU CAPS: [vis]
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 64796
Kernel command line: root=/dev/sda1 ro softlevel=default
PID hash table entries: 2048 (order: 1, 16384 bytes)
Dentry cache hash table entries: 65536 (order: 6, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 5, 262144 bytes)
Sorting __ex_table...
Memory: 499680K/522456K available (3507K kernel code, 199K rwdata, 888K rodata, 144K init, 518K bss, 22776K reserved, 0K cma-reserved)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
NR_IRQS:2048 nr_irqs:2048 1
clocksource: hbtick: mask: 0xffffffffffffffff max_cycles: 0x148020aa9, max_idle_ns: 440795202069 ns
clocksource: mult[b400012e] shift[24]
clockevent: mult[16c16bf] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled
bootconsole [earlyprom0] disabled
Calibrating delay using timer specific routine.. 11.13 BogoMIPS (lpj=55655)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 8192 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 8192 bytes)
devtmpfs: initialized
Performance events: No support for PMU type 'ultra12'
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
NET: Registered protocol family 16
/pci@1f,0: PCI IO[1fe02000000] MEM[1ff00000000] MEM64[1ff00000000]
/pci@1f,0: SABRE PCI Bus Module ver[0:0]
PCI: Scanning PBM /pci@1f,0
sabre f0069d08: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x1fe02000000-0x1fe02ffffff] (bus address [0x0000-0xffffff])
pci_bus 0000:00: root bus resource [mem 0x1ff00000000-0x1ffffffffff] (bus address [0x00000000-0xffffffff])
pci_bus 0000:00: root bus resource [mem 0x1ff00000000-0x1ffffffffff] (bus address [0x00000000-0xffffffff])
pci_bus 0000:00: root bus resource [bus 00]
pci 0000:00:03.0: quirk: [io  0x1fe02002000-0x1fe0200203f] claimed by ali7101 ACPI
pci 0000:00:03.0: quirk: [io  0x1fe02004000-0x1fe0200401f] claimed by ali7101 SMB
pci 0000:00:05.0: can't claim BAR 6 [mem 0x1ff00080000-0x1ff000bffff]: address conflict with Video RAM area [??? 0x1ff000a0000-0x1ff000bffff flags 0x80000000]
vgaarb: loaded
SCSI subsystem initialized
/pci@1f,0/isa@7/rtc@0,70: RTC regs at 0x1fe02000070
clocksource: Switched to clocksource hbtick
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 32768 bytes)
TCP bind hash table entries: 4096 (order: 2, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 0, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 0, 8192 bytes)
NET: Registered protocol family 1
pci 0000:00:07.0: Activating ISA DMA hang workarounds
power: Control reg at 1fe02002000
futex hash table entries: 256 (order: -1, 6144 bytes)
Initialise system trusted keyring
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 1024 (order 0, 8192 bytes)
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler cfq registered (default)
f0076dcc: ttyS0 at MMIO 0x1fe020003f8 (irq = 7, base_baud = 115387) is a 16550A
Console: ttyS0 (SU)
console [ttyS0] enabled
f0078ab8: ttyS1 at MMIO 0x1fe020002e8 (irq = 7, base_baud = 115387) is a 16550A
scsi host0: pata_ali
scsi host1: pata_ali
ata1: PATA max UDMA/66 cmd 0x1fe02010200 ctl 0x1fe02010218 bmdma 0x1fe02010220 irq 12
ata2: PATA max UDMA/66 cmd 0x1fe02010210 ctl 0x1fe02010208 bmdma 0x1fe02010228 irq 12
mousedev: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos rtc_cmos: no alarms, 114 bytes nvram
NET: Registered protocol family 17
Loading compiled-in X.509 certificates
Kernel unaligned access at TPC[634634] mpi_read_buffer+0xd4/0x120
Loaded X.509 cert 'Build time autogenerated kernel key: 42ebc4053e8333df398250e8135e9626ea63a026'
ata1.00: ATA-5: ST38410A, 3.03, max UDMA/66
ata1.00: 16841664 sectors, multi 0: LBA
rtc_cmos rtc_cmos: setting system clock to 2017-04-14 18:27:19 UTC (1492194439)
ata1.00: configured for UDMA/66
scsi 0:0:0:0: Direct-Access     ATA      ST38410A         3.03 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 16841664 512-byte logical blocks: (8.62 GB/8.03 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3 sda4 sda5
ata2.00: ATA-7: Maxtor 7Y250P0, YAR41BW0, max UDMA/133
ata2.00: 490234752 sectors, multi 0: LBA48
sd 0:0:0:0: [sda] Attached SCSI disk
ata2.00: configured for UDMA/66
scsi 1:0:0:0: Direct-Access     ATA      Maxtor 7Y250P0   1BW0 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 490234752 512-byte logical blocks: (251 GB/234 GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1
sd 1:0:0:0: [sdb] Attached SCSI disk
EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem
EXT4-fs (sda1): INFO: recovery required on readonly filesystem
EXT4-fs (sda1): write access will be enabled during recovery
EXT4-fs (sda1): recovery complete
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) readonly on device 8:1.
devtmpfs: mounted

INIT: version 2.88 booting


   OpenRC 0.23.2 is starting up Gentoo Linux (sparc64)

Press I to enter interactive boot mode

random: nonblocking pool is initialized
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...

--> [139 lines]

 * Mounting /proc ...
 * Mounting /proc ...
 * Mounting /proc ...
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy

--> [66 lines]

mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
proc is already mounted or /proc busy
mount: proc is already mounted or /proc busy
       proc is already mounted on /proc
       proc is already mounted on /proc
       proc is already mounted on /proc
       proc is already mounted on /proc

--> [61 lines]

       proc is already mounted on /proc
       proc is already mounted on /proc
       proc is already mounted on /proc
       proc is already mounted on /proc
 [ !! ]
 [ !! ]
 [ !! ]
 [ !! ]
 [ ok ]
 [ !! ]
 [ !! ]
 [ !! ]
 [ !! ]
 [ !! ]
mount invoked oom-killer: gfp_mask=0x24200ca, order=0, oom_score_adj=0
CPU: 0 PID: 3718 Comm: mount Not tainted 4.4.39-gentoo #1
Call Trace:
 [00000000004ba764] oom_kill_process+0x284/0x3a0
 [00000000004babf0] out_of_memory+0x2d0/0x320
 [00000000004bec20] __alloc_pages_nodemask+0x640/0x740
 [00000000004d4468] wp_page_copy.isra.91+0x48/0x3a0
 [00000000004d58cc] do_wp_page.isra.95+0x6c/0x580
 [00000000004d79b4] handle_mm_fault+0x614/0xc40
 [000000000044d8e8] do_sparc64_fault+0x268/0x720
 [0000000000407918] sparc64_realfault_common+0x10/0x20
Mem-Info:
active_anon:20159 inactive_anon:0 isolated_anon:0
 active_file:141 inactive_file:56 isolated_file:40
 unevictable:0 dirty:0 writeback:0 unstable:0
 slab_reclaimable:52 slab_unreclaimable:26727
 mapped:181 shmem:0 pagetables:9085 bounce:0
 free:346 free_pcp:33 free_cma:0
Normal free:2768kB min:2824kB low:3528kB high:4232kB active_anon:161272kB inactive_anon:0kB active_file:1128kB inactive_file:448kB unevictable:0kB isolated(anon):0kB isolated(file):320kB present:522456kB managed:499824kB mlocked:0kB dirty:0kB writeback:0kB mapped:1448kB shmem:0kB slab_reclaimable:416kB slab_unreclaimable:213816kB kernel_stack:45904kB pagetables:72680kB unstable:0kB bounce:0kB free_pcp:264kB local_pcp:264kB free_cma:0kB writeback_tmp:0kB pages_scanned:11944 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*8kB 47*16kB (UM) 31*32kB (M) 12*64kB (M) 2*128kB (M) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 2768kB
234 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
65307 pages RAM
0 pages HighMem/MovableOnly
2829 pages reserved
[ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
[  430]     0   430      251       32       3       1        0             0 init
[  431]     0   431      346       71       3       1        0             0 openrc
[  560]     0   560      480       82       3       1        0             0 init.sh
[  561]     0   561      480       82       3       1        0             0 init.sh
[  562]     0   562      480       82       3       1        0             0 init.sh
[  563]     0   563      480       82       3       1        0             0 init.sh
[  564]     0   564      480       82       3       1        0             0 init.sh

--> [121 lines]

[  686]     0   686      480       82       3       1        0             0 init.sh
[  687]     0   687      480       82       3       1        0             0 init.sh
[ 1226]     0  1226      380       69       3       1        0             0 mount
[ 1237]     0  1237      380       85       4       1        0             0 mount
[ 1240]     0  1240      380       69       4       1        0             0 mount
[ 1247]     0  1247      380       69       4       1        0             0 mount

--> [49 lines]

[ 1376]     0  1376      380       65       3       1        0             0 mount
[ 1380]     0  1380      380       68       4       1        0             0 mount
[ 1382]     0  1382      380       67       3       1        0             0 mount
[ 1383]     0  1383      324       44       4       1        0             0 fstabinfo
[ 1384]     0  1384      380       39       4       1        0             0 mount
[ 1385]     0  1385      380       24       4       1        0             0 mount
[ 1386]     0  1386      380       24       4       1        0             0 mount
[ 1387]     0  1387      380       69       3       1        0             0 mount
[ 1388]     0  1388      380       37       4       1        0             0 mount

[2227 lines, mostly with "mount" but a couple times "fstabinfo", a series of "openrc" and one "init.sh"]

[ 3946]     0  3946      380      170       3       1        0             0 mount
[ 3947]     0  3947      380           0 mount
[ 3948]     0  3948      380       59       3 0
Out mKcess )2 or sacrifillecess 3944 (mounB, anon-rss:184^@

Type  'go' to resume
ok power-off

lom>
LOM event: +0h50m1s host power off
lom>
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on Sparc All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum