| View previous topic :: View next topic |
| Author |
Message |
BillWho
Veteran
Joined: 03 Mar 2012
Posts: 1600
Location: US
|
 Posted: Mon Dec 31, 2012 3:24 am Post subject: |
 |
|
_______0,
| Code: | laptop proc # dmesg|grep hpet
[ 0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys ro
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys ro
[ 0.000000] hpet clockevent registered
[ 0.251867] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[ 0.252243] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[ 0.254325] Switching to clocksource hpet
[ 0.546737] rtc0: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
|
| _______0 wrote: | | ps: how did you put ahci on an MSI interrupt?? |
No idea, I didn't change anything.
| _______0 wrote: | | ps2: what about some un-obvious BIOS/UEFI settings? |
I'll double check, but I doubt it.
What is the kernel parameters - is it hpet=force x2apic_phys clocksource=hpet 
_________________
Good luck 
Since installing gentoo, my life has become one long emerge  |
|
| Back to top |
|
 |
_______0
Guru
Joined: 15 Oct 2012
Posts: 521
|
| Posted: Mon Dec 31, 2012 3:54 am Post subject: |
|
|
| BillWho wrote: |
UPDATE: Almost forgot
| Code: | bill@laptop /proc $ cat /sys/bus/clocksource/devices/clocksource0/current_clocksource
tsc
|
|
This:
| Code: | | hpet=force x2apic_phys clocksource=hpet |
to change your clocksource to hpet |
|
| Back to top |
|
|
BillWho
Veteran
Joined: 03 Mar 2012
Posts: 1600
Location: US
|
| Posted: Mon Dec 31, 2012 4:20 am Post subject: |
|
|
_______0,
| Code: | laptop proc # cat /sys/bus/clocksource/devices/clocksource0/current_clocksource
hpet
laptop proc # cat cmdline
BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
laptop proc # dmesg|grep x2apic
[ 0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
laptop proc # dmesg|grep hpet
[ 0.000000] Command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/kernel root=/dev/sda8 rootfstype=ext3 hpet=force x2apic_phys clocksource=hpet ro
[ 0.000000] hpet clockevent registered
[ 0.250202] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[ 0.250576] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[ 0.252657] Switching to clocksource hpet
[ 0.544998] rtc0: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
|
No change to interrupts 
I'm starting to think this is processor related. I'm going to boot mint/debain and check it 
UPDATE: I booted lmde and there was no difference with /proc/interrupts - all on cpu0
_________________
Good luck
Since installing gentoo, my life has become one long emerge |
|
| Back to top |
|
|
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
| Posted: Thu Jan 03, 2013 7:44 am Post subject: |
|
|
If you have more than two cores, use sys-apps/irqbalance in conjunction with a >=3.3 kernel. I obtain exceptional results with it for spreading softirq load generated by a busy multiqueue NIC on a 12 core system. If your system is NUMA capable then be sure to set the corresponding USE flag, which is applicable to Opteron CPUs and some Intel CPUs as of the Nehalem microarchitecture. Some distros actually install and enable irqbalance by default, such as Ubuntu.
There is also an alternative going by the name of irqd. I have no experience of using this particular application.
EDIT: This is somewhat tangential but worth mentioning: RPS (receive packet steering) can be useful for increasing network scalability. I manage some Java application servers for a busy online service and, without IRQ balancing and RPS, each server would simply not be able to cope with its workload. However, I would not recommend this for general usage. For those that might benefit, here is a script to enable it. It accepts the name of an ethernet interface as its argument:
| Code: | #!/bin/bash
# https://lwn.net/Articles/361440/
# http://thread.gmane.org/gmane.linux.network/179883/focus=179976
set -e
[[ $IFACE ]] || IFACE="$1"
regex='^eth([[:digit:]]+|\*)$'
[[ $IFACE =~ $regex ]] || { echo "Invalid interface: $IFACE"; exit 1; }
flow=16384
echo $flow > /proc/sys/net/core/rps_sock_flow_entries
queues=(/sys/class/net/$IFACE/queues/rx-*)
for rx in "${queues[@]}"; do
echo $(sed -e 's/0/f/g' < $rx/rps_cpus) > $rx/rps_cpus
echo $flow > $rx/rps_flow_cnt
done
logger -t "$(basename $0)" "Enabled RFS/RPS for ${#queues[@]} queues on $IFACE" |
|
|
| Back to top |
|
|
roravun
Tux's lil' helper
Joined: 05 Sep 2012
Posts: 82
|
| Posted: Thu Jan 03, 2013 9:01 am Post subject: |
|
|
Before I use the knowledge contained in this thread, I would like to ask if messages like
| Quote: | | Interrupt took horrible_amount_of_ns_order_of_K |
may be caused by unballanced IRQs.
Of course as I discovered, my proc interrupts looks like this:
| Code: | CPU0 CPU1
0: 107595256 3 IO-APIC-edge timer
1: 706460 0 IO-APIC-edge i8042
8: 1 0 IO-APIC-edge rtc0
9: 46059 0 IO-APIC-fasteoi acpi
12: 10466748 0 IO-APIC-edge i8042
14: 39423 0 IO-APIC-edge ata_piix
15: 0 0 IO-APIC-edge ata_piix
16: 589526 0 IO-APIC-fasteoi uhci_hcd:usb1
17: 0 0 IO-APIC-fasteoi uhci_hcd:usb2
18: 97973 0 IO-APIC-fasteoi uhci_hcd:usb3
19: 0 0 IO-APIC-fasteoi uhci_hcd:usb4
40: 0 0 PCI-MSI-edge PCIe PME
41: 0 0 PCI-MSI-edge PCIe PME
42: 0 0 PCI-MSI-edge PCIe PME
43: 0 0 PCI-MSI-edge PCIe PME
44: 0 0 PCI-MSI-edge PCIe PME
45: 2703873 0 PCI-MSI-edge ahci
46: 96 0 PCI-MSI-edge snd_hda_intel
47: 806 0 PCI-MSI-edge eth0
48: 32929731 0 PCI-MSI-edge iwl3945
49: 36243940 0 PCI-MSI-edge radeon
NMI: 0 0 Non-maskable interrupts
LOC: 77904422 134472172 Local timer interrupts
SPU: 0 0 Spurious interrupts
PMI: 0 0 Performance monitoring interrupts
IWI: 0 0 IRQ work interrupts
RTR: 0 0 APIC ICR read retries
RES: 90040347 90498321 Rescheduling interrupts
CAL: 3388 12283 Function call interrupts
TLB: 1964560 2268016 TLB shootdowns
TRM: 0 0 Thermal event interrupts
THR: 0 0 Threshold APIC interrupts
MCE: 0 0 Machine check exceptions
MCP: 1345 1330 Machine check polls
ERR: 0
MIS: 0
|
Also I propose this thread be made sticky. Who knows how many systems with unbalanced irqs are there |
|
| Back to top |
|
|
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
| Posted: Thu Jan 03, 2013 9:51 am Post subject: |
|
|
| roravun wrote: | Before I use the knowledge contained in keframil post, I would like to ask if messages like
| Quote: | | Interrupt took horrible_amount_of_ns_order_of_K |
may be caused by unballanced IRQs. |
This is a different matter and related to the use of high-resolution timers. The circumstances under which that message is triggered are described here:
https://git.kernel.org/?p=linux/kernel/git/tip/tip.git;a=commitdiff;h=41d2e49
Regarding my post, I should add that spreading IRQ load with a perfectly even distribution isn't a realistic goal. Nor does interrupt spreading necessarily enhance performance (to the contrary in some cases). Power usage must also be taken into consideration; there's little sense in waking up cores that might otherwise be idle or parked unless there is a clear benefit in doing so. I wouldn't want people looking at /proc/interrupts and thinking that there is something intrinsically 'wrong'.
It's better to think of it as alleviating a potential bottleneck in situations where softirq load is high. When running as a daemon, irqbalance continually adjusts the affinity maps based on the amount of load, among other factors. |
|
| Back to top |
|
|
roravun
Tux's lil' helper
Joined: 05 Sep 2012
Posts: 82
|
| Posted: Thu Jan 03, 2013 10:14 am Post subject: |
|
|
| Code: | | This is a different matter and related to the use of high-resolution timers. The circumstances under which that message is triggered are described here: |
I have just discovered I somehow disabled hpet, so it could be the cause.
As to the interrupts, I will try to move at least sound card to the other CPU. Sometimes when playing games (the CPU0 is 100% busy) sound crackles or lags a bit. When I used pulseaudio the problem was even more evident. |
|
| Back to top |
|
|
PaulBredbury
Watchman
Joined: 14 Jul 2005
Posts: 7310
|
| Posted: Thu Jan 03, 2013 12:31 pm Post subject: |
|
|
| roravun wrote: | | sound crackles or lags a bit |
I recommend raising the interrupt priority. Doesn't need a realtime kernel. |
|
| Back to top |
|
|
_______0
Guru
Joined: 15 Oct 2012
Posts: 521
|
| Posted: Thu Jan 03, 2013 8:50 pm Post subject: |
|
|
| kerframil wrote: | | Power usage must also be taken into consideration; there's little sense in waking up cores that might otherwise be idle or parked unless there is a clear benefit in doing so. I wouldn't want people looking at /proc/interrupts and thinking that there is something intrinsically 'wrong'. |
Some ppl have perfectly even irq balancing among cores. The case to save power would be fine with laptops even then, how do you determine which one is best? balanced or un-balanced?
But the mistery here is why and what makes irqs to be evenly distributed over cores and why some systems don't. I would like to know which piece in the kernel and option in makeconfig is taking this decision and a way to see /proc/interrupts either unbalanced and balanced in a controlled manner. There's gotta be an explanation.
On this system irqs are balanced out of the box, no need for irqbalance nor irqd.
And the fact that irqs are being spread in tandem means that something is doing a pretty damn well job in the kernel.
Sound card example:
| Code: | | 16: 1985 1972 1990 1992 |
I still want to turn on NMI and PMI |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
