| View previous topic :: View next topic |
| Author |
Message |
Colt45
Tux's lil' helper
Joined: 05 Sep 2007
Posts: 122
Location: Central Washington
|
 Posted: Sat Nov 08, 2014 3:41 pm Post subject: Kernel panicing when WAN down, IPv6 in v4 tunnel to blame? |
 |
|
So my internet went down a few days ago. First I tried to SSH into my router box, but couldnt connect. So I connected to my Moxa which is connected to the serial port.
I could watch it boot and then panic within a few seconds of coming up.
They always looked like this: http://pastebin.com/MsMfY95V
So I finally modified the kernel command line so it wouldnt start my 'services' softlevel, only booting the minimal set of services. Then I manually started one by one until I discovered that having the v6 tunnel up with named would cause a panic. I change the named config to disable v6 and left my IPv6 tunnel down. That seemed to fix it. At least, it was no longer panicing. It was after I did this I found out my cable modem had been disabled because my ISP somehow lost my card info then didnt tell me until they just shut it off for 2 months of backpayments due.
Looking at the stack, now Im no expert, but I see a lot of ipv6 stuff in it. I just tried it now, and I can reliably cause a panic by pinging a v6 address and rebooting my cable modem.
wtf is going on? |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Sat Nov 08, 2014 6:39 pm Post subject: |
|
|
What version of the kernel are you using? Using latest?
May just be a kernel bug...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Colt45
Tux's lil' helper
Joined: 05 Sep 2007
Posts: 122
Location: Central Washington
|
| Posted: Sat Nov 08, 2014 9:47 pm Post subject: |
|
|
| 3.15.10-hardened-r1 |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Sun Nov 09, 2014 12:41 am Post subject: |
|
|
Tried a later kernel? non-hardened kernel?
Not sure of any bugfixes but typically repeatable oops/aieees are kernel bugs, whether it real or someone sending you a bad packet.
I am currently running a dual stack IPV4/IPV6 but haven't seen any crashes, but I don't have the exact same setup as you....
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Colt45
Tux's lil' helper
Joined: 05 Sep 2007
Posts: 122
Location: Central Washington
|
| Posted: Sun Nov 09, 2014 2:35 am Post subject: |
|
|
Tried latest hardened 3.17.2
It still died, but slightly different output
http://pastebin.com/N1qEwE3D
Ill try a vanilla kernel next, after I decide to expend the energy. |
|
| Back to top |
|
|
Colt45
Tux's lil' helper
Joined: 05 Sep 2007
Posts: 122
Location: Central Washington
|
| Posted: Mon Nov 10, 2014 2:25 pm Post subject: |
|
|
I think its something to do with the hardened patch set. Vanilla 3.17.2 doesnt do it. I pinged a v6 address and rebooted my modem and I got "Destination unreachable: Address unreachable" while it was down.
It never got that far before. It died trying to give that message, if Im reading the stack correctly.
Actually, I figured out some more. I looked through the stack and googled the first one that looked a little off, "report_size_overflow"
Seems to be a PAX function.
From: https://rstforums.com/forum/62312-inside-size-overflow-plugin.rst
| Quote: | report_size_overflow
The plugin creates the report_size_overflow declaration in the start_unit_callback, but the definition is always in the current program. The plugin inserts only the report_size_overflow calls. This is a no-return function.
This function prints out the file name, the function name and the line number of the detected overflow. If the stmt's line number is not available in gcc then it prints out the caller's start line number. The last two strings are only debug information.
The report_size_overflow function's message looks like this (without PaX it uses SIZE_OVERFLOW instead of PAX):
| Code: | | PAX: size overflow detected in function main tests/main12.c:27 cicus.4_21 (max) |
|
Ive tried to 'dmesg -w' while causing the crash in order to see if I can pop this message, but it seems to be dying too quickly or something, as it never shows up. So the only info I have are the stack traces.
Ill probably disable this particular PAX feature until this is fixed. From what Im seeing, its a kernel bug that PAX is catching. Kind of a poor work around, but I don't know enough about programming to even know where to start looking for the problem. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21635
|
| Posted: Mon Nov 10, 2014 11:26 pm Post subject: |
|
|
| I interpret that stack dump to mean that a problem was detected, so PaX tried to force-exit the current task. Unfortunately, the current task was interrupt context, so the kernel panicked instead. You should check higher up to see if there are more informative messages preceding the kernel panic message. |
|
| Back to top |
|
|
Colt45
Tux's lil' helper
Joined: 05 Sep 2007
Posts: 122
Location: Central Washington
|
| Posted: Tue Nov 11, 2014 12:22 am Post subject: |
|
|
| Hu wrote: | | I interpret that stack dump to mean that a problem was detected, so PaX tried to force-exit the current task. Unfortunately, the current task was interrupt context, so the kernel panicked instead. You should check higher up to see if there are more informative messages preceding the kernel panic message. |
I agree.
Unfortunately, I dont get anything useful before the stack dumps. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21635
|
| Posted: Tue Nov 11, 2014 2:49 am Post subject: |
|
|
| In that case, file it as a bug with the PaX team. Crashing the kernel is a bug. Crashing it without even reporting adequate debugging information is worse. |
|
| Back to top |
|
|
creideiki
n00b
Joined: 19 Feb 2003
Posts: 61
Location: Linköping, Sweden
|
| Posted: Wed Nov 26, 2014 6:42 am Post subject: |
|
|
| Did you ever report this, and/or find a workaround? I think I'm seeing the same crash after upgrading my router and IPv6-in-IPv4 tunnel endpoint from 3.16.5-hardened to 3.17.3-hardened. Unfortunately, the backtrace doesn't fit on my screen, and I was more interested in getting my Internet back than in digging out a serial cable to catch the whole message, but the last 30-something lines look similar. |
|
| Back to top |
|
|
creideiki
n00b
Joined: 19 Feb 2003
Posts: 61
Location: Linköping, Sweden
|
| Posted: Wed Nov 26, 2014 4:38 pm Post subject: |
|
|
| Never mind, I found |
|
| Back to top |
|
|
|
Kernel & Hardware
