Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
IRQ seems to be using only CPU0.
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
BillWho
Veteran
Veteran


Joined: 03 Mar 2012
Posts: 1600
Location: US

PostPosted: Mon Dec 31, 2012 3:24 am    Post subject: Reply with quote

_______0,

Code:
laptop proc # dmesg|grep hpet
[    0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys ro
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys ro
[    0.000000] hpet clockevent registered
[    0.251867] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.252243] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    0.254325] Switching to clocksource hpet
[    0.546737] rtc0: alarms up to one month, y3k, 242 bytes nvram, hpet irqs

_______0 wrote:
ps: how did you put ahci on an MSI interrupt??

No idea, I didn't change anything.
_______0 wrote:
ps2: what about some un-obvious BIOS/UEFI settings?

I'll double check, but I doubt it.

What is the kernel parameters - is it hpet=force x2apic_phys clocksource=hpet :?:
_________________
Good luck :wink:

Since installing gentoo, my life has become one long emerge :)
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Mon Dec 31, 2012 3:54 am    Post subject: Reply with quote

BillWho wrote:

UPDATE: Almost forgot
Code:
bill@laptop /proc $ cat /sys/bus/clocksource/devices/clocksource0/current_clocksource
tsc


This:

Code:
hpet=force x2apic_phys clocksource=hpet


to change your clocksource to hpet
Back to top
View user's profile Send private message
BillWho
Veteran
Veteran


Joined: 03 Mar 2012
Posts: 1600
Location: US

PostPosted: Mon Dec 31, 2012 4:20 am    Post subject: Reply with quote

_______0,

Code:
laptop proc # cat /sys/bus/clocksource/devices/clocksource0/current_clocksource
hpet

laptop proc # cat cmdline
BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro

laptop proc # dmesg|grep x2apic
[    0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro

laptop proc # dmesg|grep hpet
[    0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[    0.000000] hpet clockevent registered
[    0.250202] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.250576] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    0.252657] Switching to clocksource hpet
[    0.544998] rtc0: alarms up to one month, y3k, 242 bytes nvram, hpet irqs

No change to interrupts :roll:

I'm starting to think this is processor related. I'm going to boot mint/debain and check it :?

UPDATE: I booted lmde and there was no difference with /proc/interrupts - all on cpu0 :?
_________________
Good luck :wink:

Since installing gentoo, my life has become one long emerge :)
Back to top
View user's profile Send private message
kerframil
l33t
l33t


Joined: 19 Apr 2002
Posts: 710
Location: London, UK

PostPosted: Thu Jan 03, 2013 7:44 am    Post subject: Reply with quote

If you have more than two cores, use sys-apps/irqbalance in conjunction with a >=3.3 kernel. I obtain exceptional results with it for spreading softirq load generated by a busy multiqueue NIC on a 12 core system. If your system is NUMA capable then be sure to set the corresponding USE flag, which is applicable to Opteron CPUs and some Intel CPUs as of the Nehalem microarchitecture. Some distros actually install and enable irqbalance by default, such as Ubuntu.

There is also an alternative going by the name of irqd. I have no experience of using this particular application.

EDIT: This is somewhat tangential but worth mentioning: RPS (receive packet steering) can be useful for increasing network scalability. I manage some Java application servers for a busy online service and, without IRQ balancing and RPS, each server would simply not be able to cope with its workload. However, I would not recommend this for general usage. For those that might benefit, here is a script to enable it. It accepts the name of an ethernet interface as its argument:

Code:
#!/bin/bash

# https://lwn.net/Articles/361440/
# http://thread.gmane.org/gmane.linux.network/179883/focus=179976

set -e

[[ $IFACE ]] || IFACE="$1"

regex='^eth([[:digit:]]+|\*)$'

[[ $IFACE =~ $regex ]] || { echo "Invalid interface: $IFACE"; exit 1; }

flow=16384
echo $flow > /proc/sys/net/core/rps_sock_flow_entries

queues=(/sys/class/net/$IFACE/queues/rx-*)

for rx in "${queues[@]}"; do
        echo $(sed -e 's/0/f/g' < $rx/rps_cpus) > $rx/rps_cpus
        echo $flow > $rx/rps_flow_cnt
done

logger -t "$(basename $0)" "Enabled RFS/RPS for ${#queues[@]} queues on $IFACE"
Back to top
View user's profile Send private message
roravun
Tux's lil' helper
Tux's lil' helper


Joined: 05 Sep 2012
Posts: 82

PostPosted: Thu Jan 03, 2013 9:01 am    Post subject: Reply with quote

Before I use the knowledge contained in this thread, I would like to ask if messages like
Quote:
Interrupt took horrible_amount_of_ns_order_of_K

may be caused by unballanced IRQs.

Of course as I discovered, my proc interrupts looks like this:
Code:
                     CPU0       CPU1       
  0:  107595256          3   IO-APIC-edge      timer
  1:     706460          0   IO-APIC-edge      i8042
  8:          1          0   IO-APIC-edge      rtc0
  9:      46059          0   IO-APIC-fasteoi   acpi
 12:   10466748          0   IO-APIC-edge      i8042
 14:      39423          0   IO-APIC-edge      ata_piix
 15:          0          0   IO-APIC-edge      ata_piix
 16:     589526          0   IO-APIC-fasteoi   uhci_hcd:usb1
 17:          0          0   IO-APIC-fasteoi   uhci_hcd:usb2
 18:      97973          0   IO-APIC-fasteoi   uhci_hcd:usb3
 19:          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
 40:          0          0   PCI-MSI-edge      PCIe PME
 41:          0          0   PCI-MSI-edge      PCIe PME
 42:          0          0   PCI-MSI-edge      PCIe PME
 43:          0          0   PCI-MSI-edge      PCIe PME
 44:          0          0   PCI-MSI-edge      PCIe PME
 45:    2703873          0   PCI-MSI-edge      ahci
 46:         96          0   PCI-MSI-edge      snd_hda_intel
 47:        806          0   PCI-MSI-edge      eth0
 48:   32929731          0   PCI-MSI-edge      iwl3945
 49:   36243940          0   PCI-MSI-edge      radeon
NMI:          0          0   Non-maskable interrupts
LOC:   77904422  134472172   Local timer interrupts
SPU:          0          0   Spurious interrupts
PMI:          0          0   Performance monitoring interrupts
IWI:          0          0   IRQ work interrupts
RTR:          0          0   APIC ICR read retries
RES:   90040347   90498321   Rescheduling interrupts
CAL:       3388      12283   Function call interrupts
TLB:    1964560    2268016   TLB shootdowns
TRM:          0          0   Thermal event interrupts
THR:          0          0   Threshold APIC interrupts
MCE:          0          0   Machine check exceptions
MCP:       1345       1330   Machine check polls
ERR:          0
MIS:          0




Also I propose this thread be made sticky. Who knows how many systems with unbalanced irqs are there ;-)
Back to top
View user's profile Send private message
kerframil
l33t
l33t


Joined: 19 Apr 2002
Posts: 710
Location: London, UK

PostPosted: Thu Jan 03, 2013 9:51 am    Post subject: Reply with quote

roravun wrote:
Before I use the knowledge contained in keframil post, I would like to ask if messages like
Quote:
Interrupt took horrible_amount_of_ns_order_of_K

may be caused by unballanced IRQs.


This is a different matter and related to the use of high-resolution timers. The circumstances under which that message is triggered are described here:

https://git.kernel.org/?p=linux/kernel/git/tip/tip.git;a=commitdiff;h=41d2e49

Regarding my post, I should add that spreading IRQ load with a perfectly even distribution isn't a realistic goal. Nor does interrupt spreading necessarily enhance performance (to the contrary in some cases). Power usage must also be taken into consideration; there's little sense in waking up cores that might otherwise be idle or parked unless there is a clear benefit in doing so. I wouldn't want people looking at /proc/interrupts and thinking that there is something intrinsically 'wrong'.

It's better to think of it as alleviating a potential bottleneck in situations where softirq load is high. When running as a daemon, irqbalance continually adjusts the affinity maps based on the amount of load, among other factors.
Back to top
View user's profile Send private message
roravun
Tux's lil' helper
Tux's lil' helper


Joined: 05 Sep 2012
Posts: 82

PostPosted: Thu Jan 03, 2013 10:14 am    Post subject: Reply with quote

Code:
This is a different matter and related to the use of high-resolution timers. The circumstances under which that message is triggered are described here:

I have just discovered I somehow disabled hpet, so it could be the cause.

As to the interrupts, I will try to move at least sound card to the other CPU. Sometimes when playing games (the CPU0 is 100% busy) sound crackles or lags a bit. When I used pulseaudio the problem was even more evident.
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Thu Jan 03, 2013 12:31 pm    Post subject: Reply with quote

roravun wrote:
sound crackles or lags a bit

I recommend raising the interrupt priority. Doesn't need a realtime kernel.
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Thu Jan 03, 2013 8:50 pm    Post subject: Reply with quote

kerframil wrote:
Power usage must also be taken into consideration; there's little sense in waking up cores that might otherwise be idle or parked unless there is a clear benefit in doing so. I wouldn't want people looking at /proc/interrupts and thinking that there is something intrinsically 'wrong'.


Some ppl have perfectly even irq balancing among cores. The case to save power would be fine with laptops even then, how do you determine which one is best? balanced or un-balanced?

But the mistery here is why and what makes irqs to be evenly distributed over cores and why some systems don't. I would like to know which piece in the kernel and option in makeconfig is taking this decision and a way to see /proc/interrupts either unbalanced and balanced in a controlled manner. There's gotta be an explanation.

On this system irqs are balanced out of the box, no need for irqbalance nor irqd.

And the fact that irqs are being spread in tandem means that something is doing a pretty damn well job in the kernel.

Sound card example:

Code:
16:       1985       1972       1990       1992


I still want to turn on NMI and PMI
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum