Difficult question - Keyboard not working on RE-boot only:

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

This one is an elusive question. When I power the computer up and boot on Gentoo (UEFI, through rEFInd), they keyboard, a Corsair USB K70 RGB, takes around 5 second on the udev devices line in the boot, then proceeds to work normally. Now comes the question, when I reboot the computer instead of powering it off, the udev devices line takes around 30 seconds and now the keyboard doesn't work anymore. I have to shut down the computer and turn on for it to work again.
What creepy issue could be causing this? Something that happens maybe on the BIOS that is not initializing it correctly?

Any ideas here?

Thank you,
Nelson

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

Some updates:

I've tried to change many options in MoBo EFI and nothing changed. I removed rEFInd as the bootloader and the same problem still occur. I've changed memory modules (because in some boot times, messages about checking for memory corruption every 60s appeared in the first line of the boot). Nothing seems to changed.
So, I download linux Mint liveCD and since it boots correctly every time, I'm not inclined to believe it is something wrong in either the kernel configuration or in the initram (which I used for FakeRaid).

Bottom line: What can be causing two booting process, consecutively done and without making any changes, detecting different hardware and displaying different messages? Isn't the boot time a deterministic process?

Thank you,
Nelson

NeddySeagoon · Posted: Fri Nov 25, 2016 8:31 pm Post subject:

nelsonwcf,

The two process differ in the way the hardware is reset.
When you power up the system, everything gets a reset pulse from the PSU. This hardware reset is missing when you do a shutdown -r now.

It gets worse. The boot process is not deterministic. Its mostly event driven.

I suspect the problem is USB related and changing the module load order will change things.
From a cold boot, post your kernel .config file - it needs to go onto a pastbin site.
The dmesg output - it needs to go onto a pastbin site too.
The output of lsusb -vvv - it needs to go onto a pastbin site too.
The output of lsmod

Now it gets harder ...
We need dmesg and lsmod after a restart too but your keyboard won't work.

Can you ssh in to get that?
Alternatively, a bit of a hack put the following into
/etc/local.d/savefiles.start and chmod +x /etc/local.d/savefiles.start

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

Hi NeddySeagoon,

Thanks for the help. Here are the pastebinned files:

http://pastebin.com/EwSFZLPP - .config
http://pastebin.com/HdNpCeem - dmesg
http://pastebin.com/BpQJvzPe - lsusb -vvv
http://pastebin.com/4ZDXCLcJ - lsmod

In this boot, the USB keyboard didn't work so I used an PS/2 keyboard (without disconnecting the USB keyboard) to be able to capture the problems.
If you need something else, just request it and I will post it here ASAP.

Thank you for the help.
Nelson

NeddySeagoon · Posted: Sat Nov 26, 2016 5:59 pm Post subject:

nelsonwcf,

I want to compare a working dmesg with a USB not working dmesg.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

NeddySeagoon · Posted: Sat Nov 26, 2016 6:31 pm Post subject:

nelsonwcf,

Your lsusb is interesting.

You have a total of four USB root hubs but only one appears to be used.

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

Hi,

I had some trouble getting the keyboard to work again on linux so I disconnect every single USB device from my USB ports with the exception of the mouse and keyboard.

Here are the pastebins for dmesg and lsmod with the keyboard working.

http://pastebin.com/PvidSA2J
http://pastebin.com/1RdF0UKj

The USB 2.0 load seems huge but there wasn't really anything plugged there, which makes me wonder what was happening. In addition, if the problem was overloading, why would the devices work correctly in Windows 10?

Any other insights on what could be happening here?

and again, thank you very much! very nice way of tracking the problem up!

NeddySeagoon · Posted: Sun Nov 27, 2016 11:52 pm Post subject:

nelsonwcf,

The lsmod outputs are identical between working and non working.
That tells that the module load order has not changed.
The first loaded module is at the bottom of the list.

Something that stands out like a sore thumb in dmesg is the difference in timestamp formats.
I've never seen what appears to be the incremental format in your reboot dmesg.

Your mouse and keyboard are now both on a USB3 bus.

Logicien · Posted: Mon Nov 28, 2016 4:25 am Post subject:

nelsonwcf, NeddySeagoon,

the problem can come from some Usb and/or keyboard supports who need to be in the kernel image or in the initramfs. With ArchLinux I had to put some modules in the initramfs for my Usb mouse working after boot with Gpm, the copy and paste mouse server at the console. Otherwise it was not working but I only had to restart the service. So, something was missing at boot time who was there after it.

The fact that Linux check for memory corruption every 60 seconds do not mean that your memory surely have something wrong. This mean that those 2 options are on in your kernel configuration:

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

Neddy,

I'm going through checking my BIOS and motherboard right now because after your comments, I found out that four ports that should be USB 3.0 are acting like USB 2.0 so this problem may be related. As soon I finish checking BIOS options and the if the connectors are in the right place I will give you more information.

Logicien,

There is a driver for this keyboard in corsair forum but the randomness of the keyboard being recognized or not is the problem. For example, the keyboard works in refind bootloader normally and also in the BIOS. Only when the kernel boots, the keyboard most of the time doesn't work but in some cases it is recognized. That is what the mystery is. I already installed the ckb driver but nothing happens if the keyboard doesn't start. Doesn't matter if I disconnect it and then reconnect. It is simply not detected.

I will get back to you soon.

Thanks!

1clue · Advocate Joined: 05 Feb 2006 Posts: 2569

I don't know how to decode the output files like Neddy's doing, but I have a question which may be pertinent.

Is your keyboard connected directly to a usb port on the motherboard? Meaning, no usb hub in the middle.

I had keyboard issues for awhile too, wound up calling paid support. They found out I had an external powered usb hub on my desk. When I moved it to direct connection the problem went away.

NeddySeagoon · Posted: Wed Nov 30, 2016 8:46 pm Post subject:

nelsonwcf,

There is a corsair driver in the kernel, which you are not using.

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

1clue: yes, right now it is but I've tried both. Same issue.

NeddySeagoon: yes, I saw this option but it is for old models only, not for this one.

As I told you guys before, I started tracking what was wrong with my USB ports so I plugged my pen drive in all of them and used a Windows tool (because everything seems to be working correctly on windows) and tracked all the ports. Something very weird is happening, sometimes the port is detected as USB 2 and sometimes as USB 3 in the USB Tree list. I went back to EFI BIOS and changed the legacy USB port from disable to auto to see if it fixed something.

The first time I booted on linux, it showed the following message:

http://picpaste.com/20161130_164426-EedvyzJG.jpg

Definitely something is wrong but I never saw such message but based on what is happening, it could be related to my keyboard not beings deterministically detected (similar messages happened before but I though it was something with me playing with the kernel settings).

NeddySeagoon · Posted: Thu Dec 01, 2016 12:35 pm Post subject:

nelsonwcf,

Do you have any USB3 devices?

A few USB3 hubs only run at USB2 if the first detected device is not USB3.
That's not supposed to happen, its a hardware bug.

If you have a USB3 device (I didn't see any) boot with nothing except the USB3 device connected.
This should force the USB hub to start in USB3 mode.
Now connect all the slower devices - order does not matter.

Something a bit more invasive. Remove USB2 support for the kernel, or unload the EHCI and XHCI modules, then reload only XHCI.

nelsonwcf · Tux's lil' helper Joined: 31 Oct 2012 Posts: 112

Okay. I did the things you suggested and I couldn't really see any difference:
- Tried to play with XHCI and EHCI and nothing changed;
- Tried to unplug and plug the keyboard again. When the keyboard doesn't work, normally the LED lights go off. When I unplug and plug it, the lights go on again but the keys still doesn't work;
- Tried to change some options in the BIOS but nothing changed.

Then I decided to try a different approach. I booted Linux mint and copied its .config to my current kernel and ran "make olddefconfig". I compiled the kernel (which took a lot of time) and then ran "genkernel initramfs --mdadm" because I need it for Fakeraid. That portion was interesting because it took almost 3 minutes for it to complete when previously it took less than 8s. After that, I Installed both new kernel and initramfs and rebooted. It took almost two times the boot time boot it booted and the keyboard worked correctly. Maybe it was just luck so I rebooted and it worked again correctly. I powered it off and turned it on and it worked correctly again. So far for now, it is working correctly. Now, my kernel is completely bloated with useless drivers and modules but it boot.
Interestingly, this new message appeared while still in the initramfs.img:

>> Activating mdev
>> Loading modules
:: Loading from pata then loading from sata, from scsi, from usb and there, the usbhid and hid-generic took around 5s to load then it proceed forward. This time, the initializing udev devices, the one who normally took 10s to proceed when the keyboard worked and around 20s when it didn't, this time takes 1s.

This solution may indicate that the problem was in the hardware configuration. Now, what could I have done differently that move loading the modules to the initramfs instead of the regular boot but which make the keyboard work correctly?

The problem with the rcu_shed didn't show up anymore so far.

This is eluding me because I want to clean up the kernel but I don't know what I can remove or not and;
why did genkernel this time took so long to finish and generated a more complete initramfs?

Logicien · Posted: Sat Dec 03, 2016 11:47 pm Post subject:

You know one thing, Linux can manage your keyboard on Gentoo. With your new kernel configuration taked from Mint you have to circle what was missing (maybe what was to much) in your previous kernel configuration. It may be a Usb and/or an Input support.

I suppose that your actual kernel is bigger and have more modules than before. It explain why your initramfs is bigger too. More stuff apply to your material. Generic kernels and initramfs are big.

When try to slim the kernel, you can remove without problem the support for cable and wireless Ethernet who do not apply. I hope you will find why Mint kernel work and not you own one so you will be able to use your previous one with corrections.
_________________
Paul