Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
High timer interrups (INT 0) with 3.4.9
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 5:50 am    Post subject: High timer interrups (INT 0) with 3.4.9 Reply with quote

I have three machines, two of which I recently upgraded to 3.4.9 and just noticed that they have extremly high timer interrupt count (INT 0) - one reached 10 millions
form Sept 15th, the other - 500 millions from August 27th reboot. At the same time, the only machine that stayed at 2.6.39 kernel, shows just 6000 timer interrupts
since its boot on August 1st.

What could be the reason ? I don't think I played with kernel config options that much (or differently). All machines are quite different - high interrupt count one are
Core 2 Dupo laptop and i7 desktop, Intel and nvidia video correspodingly, the one with low count is core2 duo desktop with nvidia video. So it seems it is a kernel version which matters
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Tue Sep 18, 2012 7:32 am    Post subject: Reply with quote

Did you have tickless kernel enabled? Is it incrementing like a hundred every second or whatever your HZ is?

This was fairly common and normal in old Linux kernels that don't support tickless...after enabling tickless it's much lower...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 2:36 pm    Post subject: Reply with quote

eccerr0r wrote:
Did you have tickless kernel enabled? Is it incrementing like a hundred every second or whatever your HZ is?

This was fairly common and normal in old Linux kernels that don't support tickless...after enabling tickless it's much lower...


I have tickless system, I double checked ( CONFIG_NO_HZ=y ) . Although 3.4.9 kernels have an additional option, CONFIG_RCU_FAST_NO_HZ and this one is not set. 2.6.39 does not have it.

I timed the interrupts - I get approximately 500 interrupts per second, and system timer is at 1000 HZ, so it increments at twice a slower rate.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Tue Sep 18, 2012 2:55 pm    Post subject: Reply with quote

Is it like this on fresh boot? Single user mode?

Do you have any special hardware? Can the drivers be removed one at a time to see if any of the drivers require frequent interrupts by the clock tick timer?

Do you have HPET enabled on the newer machines? APIC?

Hmm... thought I had a 3.4.9 machine, and doesn't seem to be exhibiting interrupt storm in hw INT0... But I get a lot of local interrupts...

Code:
mikuru $ uname -a
Linux mikuru 3.4.9-gentoo #2 SMP Sun Aug 26 09:37:03 MDT 2012 x86_64 Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz GenuineIntel GNU/Linux
mikuru $ cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       
  0:        127          0          0          0   IO-APIC-edge      timer
  1:        292       9536        146        311   IO-APIC-edge      i8042
  8:          2         11          0          0   IO-APIC-edge      rtc0
  9:        250        205          5         85   IO-APIC-fasteoi   acpi
 12:       7240     260815       3375       7179   IO-APIC-edge      i8042
 16:         84        132         30         38   IO-APIC-fasteoi   ehci_hcd:usb1
 23:          0         23          5          4   IO-APIC-fasteoi   ehci_hcd:usb2
 40:       5886       9660       3690       2082   PCI-MSI-edge      i915
 41:      19673      16300       4217       5245   PCI-MSI-edge      ahci
 42:      12821      30427       9406       4772   PCI-MSI-edge      xhci_hcd
 43:       6457      24409       3702       3569   PCI-MSI-edge      snd_hda_intel
 44:          0          0          0          0   PCI-MSI-edge      eth0
 45:      70034     267897      31547      40622   PCI-MSI-edge      iwlwifi
NMI:          0          0          0          0   Non-maskable interrupts
LOC:     688301     412667     713302     374754   Local timer interrupts
SPU:          0          0          0          0   Spurious interrupts
PMI:          0          0          0          0   Performance monitoring interrupts
IWI:          0          0          0          0   IRQ work interrupts
RTR:          3          0          0          0   APIC ICR read retries
RES:     389680     266397      32705      15704   Rescheduling interrupts
CAL:         67         82         73         77   Function call interrupts
TLB:      11227      13067       8305      12849   TLB shootdowns
TRM:          8          8          8          8   Thermal event interrupts
THR:          0          0          0          0   Threshold APIC interrupts
MCE:          0          0          0          0   Machine check exceptions
MCP:         19         19         19         19   Machine check polls
ERR:          0
MIS:          0

_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 2:58 pm    Post subject: Reply with quote

Good questions, I'll try to invesitage. It is a run-of-the-mill desktop, but has e-sata external drive attached. Last time it was rebooted on Sept 5th.

I'll play today with my laptop which shows the same, and for which I still have 2.6.39 kernel to boot in to compare.
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 4:17 pm    Post subject: Reply with quote

And, havn't yet done the investigation, my interrupt table is

Code:

# uname -a
Linux wheeler 3.4.9-gentoo #1 SMP Mon Aug 27 14:13:10 MDT 2012 x86_64 Intel(R) Core(TM) i7 CPU 930 @ 2.80GHz GenuineIntel GNU/Linux

#cat /proc/interrupts
  0:        116          0          0  541481683          0          0          0          0   IO-APIC-edge      timer
  1:          0          0          0     159774          0          0          0          0   IO-APIC-edge      i8042
  8:          0          0          0         64          0          0          0          0   IO-APIC-edge      rtc0
  9:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   acpi
 16:          0          0    9190560          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
 17:          0          0          0          0   10746994          0          0          0   IO-APIC-fasteoi   ahci, snd_hda_intel
 18:          0          0          0          0          0  102422837          0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb5, uhci_hcd:usb8, firewire_ohci
 19:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb7
 21:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
 23:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6
 44:   12579660          0          0          0          0          0          0          0   PCI-MSI-edge      ahci
 45:          0   37480974          0          0          0          0          0          0   PCI-MSI-edge      eth0
 47:          0          0     407490          0          0          0          0          0   PCI-MSI-edge      snd_hda_intel
 48:          0          0          0  118904701          0          0          0          0   PCI-MSI-edge      nvidia
NMI:          0          0          0          0          0          0          0          0   Non-maskable interrupts
LOC:  185727185  200928325  181580118   82962887   52201164   48264262   66709717   80949762   Local timer interrupts
SPU:          0          0          0          0          0          0          0          0   Spurious interrupts
PMI:          0          0          0          0          0          0          0          0   Performance monitoring interrupts
IWI:          0          0          0          0          0          0          0          0   IRQ work interrupts
RTR:          6          0          0          0          0          0          0          0   APIC ICR read retries
RES:  142684187   40531377    4161936    1538613     345423     289461     147759     164673   Rescheduling interrupts
CAL:    6568138    5452265    6344138    3496687    7217879    7221072    7226884    7222810   Function call interrupts
TLB:    1779471    1784771    1826604    1847394    1851097    1850794    1902552     970374   TLB shootdowns
TRM:          0          0          0          0          0          0          0          0   Thermal event interrupts
THR:          0          0          0          0          0          0          0          0   Threshold APIC interrupts
MCE:          0          0          0          0          0          0          0          0   Machine check exceptions
MCP:       6287       6287       6287       6287       6287       6287       6287       6287   Machine check polls
ERR:          0
MIS:          0



And your's is looking as I would expect it to look
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Tue Sep 18, 2012 4:22 pm    Post subject: Reply with quote

Being said that my system gets a far higher interrupt rate than yours (1.500/s), I can't see any significant difference between my 2.6.38 and 3.4.9
Which CONFIG_HZ_xyzt did you set ?
_________________
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 4:45 pm    Post subject: Reply with quote

aCOSwt wrote:
Being said that my system gets a far higher interrupt rate than yours (1.500/s), I can't see any significant difference between my 2.6.38 and 3.4.9
Which CONFIG_HZ_xyzt did you set ?


Mine is CONFIG_HZ_1000
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Tue Sep 18, 2012 4:48 pm    Post subject: Reply with quote

I have to mention that when I first posted it, it's been up only an hour or so. As of right now, it's up 3 hours 17 minutes and hasn't changed from the 127 count... Everything else has gone up though.

Weird.

INT0 should be the main interval timer...

CONFIG_NO_HZ=y
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_HPET=y
CONFIG_X86_LOCAL_APIC=y

update--
I just looked on my core2 quad machine and it's acquiring about 40 interrupts per core per second...Not quite the same issue but probably similar. My core2 duo machine is holding fairly steady on INT0 however.

The core2 quad machine has closed source software though (FGLRX).
It does have HZ=1000, not have CONFIG_HPET set, but it's running Linux-3.3.8-gentoo ...

update 2--
You guys made me boot my i7-2700k ... It's currently running 3.2.21-gentoo.
It does not appear to be getting INT0 bloat like the core2quad, but HPET=YES.

So for me it looks like HPET made the difference... Granted I should try 3.4.9 though...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 7:36 pm    Post subject: Reply with quote

eccerr0r wrote:

update--
I just looked on my core2 quad machine and it's acquiring about 40 interrupts per core per second...Not quite the same issue but probably similar. My core2 duo machine is holding fairly steady on INT0 however.

The core2 quad machine has closed source software though (FGLRX).
It does have HZ=1000, not have CONFIG_HPET set, but it's running Linux-3.3.8-gentoo ...

update 2--
You guys made me boot my i7-2700k ... It's currently running 3.2.21-gentoo.
It does not appear to be getting INT0 bloat like the core2quad, but HPET=YES.

So for me it looks like HPET made the difference... Granted I should try 3.4.9 though...


One both my offending machines I have HPET=YES, but dmesg says

Code:

ACPI: HPET 00000000afee6c00 00038 (v01 GBT    GBTUACPI 42302E31 GBTU 00000098)
ACPI: HPET id: 0x8086a201 base: 0xfed00000
HPET: 4 timers in total, 0 timers will be used for per-cpu timer


(other offending machine has 3 timers is total, but again 0 timers will be used for per-cpu timer)

Don't know what it means and I have to reboot to see what HPET is doing on the one machine that does not show high INT0

I remember very clearly how INT0 interrupts disappeared on all my machines when tickless kernels first appear few years ago.

I am also somewhat suprised about high counts of INT18 - on which some USB controllers sit. It is also at the level of 100 per second.
And I have just USB mouse and camera (almost always off), and SD controller (not used as well).
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Tue Sep 18, 2012 7:44 pm    Post subject: Reply with quote

Before understanding where the difference could come from, you can test the actual capabilities of your different systems in terms of interrupt rates.
(no need to single user mode...)
Code:

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
 
#define USECREQ 250
#define LOOPS   1000
 
void event_handler (int signum){
  static unsigned long cnt = 0;
  static struct timeval tsFirst;

if (cnt == 0){
  gettimeofday (&tsFirst, 0);}
cnt ++;
if (cnt >= LOOPS){
  struct timeval tsNow;
  struct timeval diff;

  setitimer (ITIMER_REAL, NULL, NULL);
  gettimeofday (&tsNow, 0);
  timersub(&tsNow, &tsFirst, &diff);
  unsigned long long udiff = (diff.tv_sec * 1000000) + diff.tv_usec;
  double delta = (double)(udiff/cnt)/1000000;
  int hz = (unsigned)(1.0/delta);
  printf ("kernel timer interrupt frequency is approx. %d Hz", hz);
  if (hz >= (int) (1.0/((double)(USECREQ)/1000000))){
   printf (" or higher");}       
  printf ("\n");
  exit (0);}}

int main (int argc, char **argv){

  struct sigaction sa;
  struct itimerval timer;
 
  memset (&sa, 0, sizeof (sa));
  sa.sa_handler = &event_handler;
  sigaction (SIGALRM, &sa, NULL);
  timer.it_value.tv_sec = 0;
  timer.it_value.tv_usec = USECREQ;
  timer.it_interval.tv_sec = 0;
  timer.it_interval.tv_usec = USECREQ;
  setitimer (ITIMER_REAL, &timer, NULL);
  while (1);}

(credits advenage)
Copy, paste, save, and gcc this.

On my current core 2 duo running ck-sources-3.4.9 with CONFIG_HZ=1000 / CONFIG_NOHZ=y / CONFGI_HPET* =y, I get :
Code:
acoswt@PrimaPratica ~ $ ./a.out
kernel timer interrupt frequency is approx. 4016 Hz or higher

_________________
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Tue Sep 18, 2012 7:52 pm    Post subject: Reply with quote

aCOSwt wrote:

On my current core 2 duo running ck-sources-3.4.9 with CONFIG_HZ=1000 / CONFIG_NOHZ=y / CONFGI_HPET* =y, I get :
Code:
acoswt@PrimaPratica ~ $ ./a.out
kernel timer interrupt frequency is approx. 4016 Hz or higher


On that one 'offending' i7 machine I am at right now (gentoo-sources-3.4.9 wth CONFIG_HZ=1000 / CONFIG_NOHZ=y / CONFGI_HPET* =y)
I get exactly the same

kernel timer interrupt frequency is approx. 4016 Hz or higher


Not sure about #define USECREQ 250, could it be we should change it to 1000 ? with this redifinition I get

kernel timer interrupt frequency is approx. 1002 Hz or higher which makes more sense
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Tue Sep 18, 2012 8:43 pm    Post subject: Reply with quote

This code appears to be measuring the precision of the timer, but because of tickless kernel, the timer may not be running at all times. There's something weird going on here... (Mine is also saying about 4KHz for that code, despite machines that are not triggering thousands of INT0's.)

I'm out of ideas now, could you post your .config's for correct and incorrect behavior? Maybe someone could scrutinize them for differences... http://users.vanade.com/~eccerr0r/mikuru.config has my .config for my i5 (now up 7 hours and still at 127 INT0s) that seems to keep counts low if interested...

(and there's a specific reason for interest... though performance might not really matter, my i5 is a laptop and reducing the number of interrupts equates to better battery life...)
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 2:06 am    Post subject: Reply with quote

eccerr0r wrote:


I'm out of ideas now, could you post your .config's for correct and incorrect behavior? Maybe someone could scrutinize them for differences... http://users.vanade.com/~eccerr0r/mikuru.config has my .config for my i5 (now up 7 hours and still at 127 INT0s) that seems to keep counts low if interested...

(and there's a specific reason for interest... though performance might not really matter, my i5 is a laptop and reducing the number of interrupts equates to better battery life...)


This is my main concern as well, since one of two machines with high INT0 is a laptop. I however just run powertop on it, and there is no trace that timer interrupts register in CPU wakeups. On a clean boot, with just kdm running, no network, no wireless, powertop shows 25 wakeups per second, with leading cause being swapper/0,
then kdm, then 'rescheduling interrupts'. But nothing like hundreds per second timer wakeups. CPU is nicely in C3 state >99% of the time.
I wonder, how to reconcile powertop and /proc/interrupts reading.

BTW, I booted into older 2.6.38-2.6.39 kernels on both machines, and they show as high timer count. Still a puzzle why my last machine shows none of those.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Wed Sep 19, 2012 4:24 am    Post subject: Reply with quote

I have a feeling some of those /proc/interrupts are double counted in that file, maybe powertop knows now to deal with the doublecounting.

And oddly enough I can simply not get powertop to get below 10 interrupts/second for some reason with Gentoo (it will print as green). Even with everything quiescent best I get was like 20 or so (brown). When in the 30s it will print in red.

For sure need to disable wireless, USB devices, ethernet, etc. to get it down.

But I still don't see how these affect INT0...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 4:56 am    Post subject: Reply with quote

eccerr0r wrote:
I have a feeling some of those /proc/interrupts are double counted in that file, maybe powertop knows now to deal with the doublecounting.

And oddly enough I can simply not get powertop to get below 10 interrupts/second for some reason with Gentoo (it will print as green). Even with everything quiescent best I get was like 20 or so (brown). When in the 30s it will print in red.

For sure need to disable wireless, USB devices, ethernet, etc. to get it down.

But I still don't see how these affect INT0...


based on the description I googled, powertop gets its info from /proc/timer_stats. There I do not see reference to INT0 timer as well. Say, my current is

Code:

cat /proc/timer_stats
Timer Stats Version: v0.2
Sample period: 0.507 s
    8,     0 swapper/1        tick_nohz_idle_exit (tick_sched_timer)
   1D,     8 kworker/1:0      do_dbs_timer (delayed_work_timer_fn)
    3,     0 swapper/0        tick_nohz_stop_sched_tick.clone.7 (tick_sched_timer)
   3D,  1046 kworker/0:2      do_dbs_timer (delayed_work_timer_fn)
   1D,  1046 kworker/0:2      process_one_work (delayed_work_timer_fn)
    3,     0 swapper/0        run_timer_softirq (cursor_timer_handler)
    1,  1832 ifplugd          schedule_hrtimeout_range_clock (hrtimer_wakeup)
20 total events


so it is at most 20 events per half a second
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 5:02 am    Post subject: Reply with quote

eccerr0r wrote:
I have a feeling some of those /proc/interrupts are double counted in that file, maybe powertop knows now to deal with the doublecounting.

And oddly enough I can simply not get powertop to get below 10 interrupts/second for some reason with Gentoo (it will print as green). Even with everything quiescent best I get was like 20 or so (brown). When in the 30s it will print in red.

For sure need to disable wireless, USB devices, ethernet, etc. to get it down.

But I still don't see how these affect INT0...


I don't think I was ever able to go below 10 as well. Currently , if I am in kde, but with no wireless/network, firefox not launched, and I don't touch keyboard/mouse, I am getting 10-12 wakeups/per sec, with some due to kwin and some due to LOC (Local timer interrupts). I remember when I played with
powertop a lot couple of years ago, it were these LOC's that was giving me minumum that I could not beat.

However and remarkably, if a switch from KDE to console (ALT-CNTRL-F1) I get twice more, 20 wakeups/s - frome somewhere swapper/0 appears that maintains this extra 10. Go figure why it is not present when I am inside KDE
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Wed Sep 19, 2012 9:55 am    Post subject: Reply with quote

dmpogo wrote:

Not sure about #define USECREQ 250, could it be we should change it to 1000 ? with this redifinition I get
kernel timer interrupt frequency is approx. 1002 Hz or higher which makes more sense

You put the value you want. I mean irrespective of the value of CONFIG_HZ you kernel was built with.
The purpose of the code listed above is to determine the capabilities of the timer, not to determine the frequency at which the kernel requests it to run.
So, with USECREQ (microseconds request) you define the value of the interval of time (timer.it_interval.tv_usec = USECREQ;) between two timer events.
In the example given : 250µs => corresponding to a 4000Khz frequency.
And the answer when you ran it was : Yes it can... and probably even higher.
If, on my system I want to check with USECREQ=1, the answer is yes again it can.

What I had suggested you was to run this on the system under which you see a very low number of interrupts, just in order to check if this system is actually capable of much more.

In a near to tickless system the value given as CONFIG_HZ is not the value your system will necessarily use. It is a sort of fallback.
In a totally tickless system, each time an event occurs, the kernel simply programs the timer to trigger an interrupt when the next earliest event is due irrespective of any potential cadenza.
The problem for going completely with this approach is that the kernel still needs to keep a certain track of relative time. So there still must be some period ticks somewhere.

On my system (two cores) with CONFIG_HZ=1000, I can sometimes see more than 3500 int0 per second. That is to say above the CONFIG_HZ value.
It averages around 1500, but I am running JACK. As soon as I kill JACK, the average looses the 1000hz rate jack was asking for.

In conclusion, provided you ensured that your low-int0-rate system is capable of a greater one, then the answer is that the difference must comes from what you are actually running on these systems. Is one most of the times iddle when the other one permanently running some audio server, multimedia player...?
Anyway, provided you ensured your machine is capable of more, then, contrarily to what you suggested in your first post, the discrepency is not kernel dependent.

EDIT : As a consequence of all this, selecting CONFIG_NO_HZ on your system heavily interrupted will not help you saving whatever power.
But it does save significant energy on the other one.
_________________
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 2:10 pm    Post subject: Reply with quote

aCOSwt wrote:
dmpogo wrote:

Not sure about #define USECREQ 250, could it be we should change it to 1000 ? with this redifinition I get
kernel timer interrupt frequency is approx. 1002 Hz or higher which makes more sense

You put the value you want. I mean irrespective of the value of CONFIG_HZ you kernel was built with.
The purpose of the code listed above is to determine the capabilities of the timer, not to determine the frequency at which the kernel requests it to run.
So, with USECREQ (microseconds request) you define the value of the interval of time (timer.it_interval.tv_usec = USECREQ;) between two timer events.
In the example given : 250µs => corresponding to a 4000Khz frequency.
And the answer when you ran it was : Yes it can... and probably even higher.
If, on my system I want to check with USECREQ=1, the answer is yes again it can.

What I had suggested you was to run this on the system under which you see a very low number of interrupts, just in order to check if this system is actually capable of much more.

In a near to tickless system the value given as CONFIG_HZ is not the value your system will necessarily use. It is a sort of fallback.
In a totally tickless system, each time an event occurs, the kernel simply programs the timer to trigger an interrupt when the next earliest event is due irrespective of any potential cadenza.
The problem for going completely with this approach is that the kernel still needs to keep a certain track of relative time. So there still must be some period ticks somewhere.

On my system (two cores) with CONFIG_HZ=1000, I can sometimes see more than 3500 int0 per second. That is to say above the CONFIG_HZ value.
It averages around 1500, but I am running JACK. As soon as I kill JACK, the average looses the 1000hz rate jack was asking for.

In conclusion, provided you ensured that your low-int0-rate system is capable of a greater one, then the answer is that the difference must comes from what you are actually running on these systems. Is one most of the times iddle when the other one permanently running some audio server, multimedia player...?
Anyway, provided you ensured your machine is capable of more, then, contrarily to what you suggested in your first post, the discrepency is not kernel dependent.

EDIT : As a consequence of all this, selecting CONFIG_NO_HZ on your system heavily interrupted will not help you saving whatever power.
But it does save significant energy on the other one.



Right, tried it on the machine that accumulated just 250 INT0 interrupts since it was rebooted 10 hours ago and the code gives same 4016 or higher.
It runs netatalk, backuppc (having performed backups within this 10 hours) and idle KDE session as we speak

As of what I run on machines with high (and persistent) INT0.
One is the laptop, fresh reboot, just logged in KDE, run wireless and firefox, uptime 28 minutes - 200000 interrupts, 120 interrupts/sec. It was the same in last try when I did not log into KDE or switched on wireless, just sat in console. It reached 320000 as I was typing this. These interrupts do not however register in powertop as wakeups.

The other is office desktop - run kde, some nfs disks, backuppc, numerical code from time to time, external sata drive for backup. It sits at 170 INT0 interrupts per second solid, since reboot yesterday 18 hours ago.

Also you are right that the difference is not kernel dependent, at least kernel version dependent. On both machines with high interrupts I booted in the old kernels, 2.6.39 and 2.6.38 and they showed the same high INT0. So this hypothesis can be put to rest. Kernel configs are, however different on all three machines, so there maybe a difference here, but I could not visualize the pattern. One thing common between high INT0 machines is that they use deadline scheduler and scheduler is built in, while low INT0 uses CFQ. Mere runtime switching to CFQ did not help my laptop, however maybe just compiling deadline into kernel causes some timer related code to run ?

Hardware also does not point to a pattern

Low INT0 - core 2 duo, nvidia, snd_hda_intel, no wifi, realtek ethernet

High INT0 - a) core 2 duo, intel graphics, wifi, snd_hda_intel, intel ethernet (off) laptop
b) 4 core i7, nvidia, snd_hda_intel, realtek ethernet, desktop


PS of course it can be coincidence, that there is some process, like wifi, which causes timer to run at 120 INT0/s on laptop, and something else on desktop.
I just tried to switch to CFQ with no deadline compiled in, and INT0 seems slowed to 35/s, but after longer run with wifi on it is back to 122/s
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Wed Sep 19, 2012 3:26 pm    Post subject: Reply with quote

dmpogo wrote:
One thing common between high INT0 machines is that they use deadline scheduler and scheduler is built in, while low INT0 uses CFQ.

I never used CFQ, moreover I use a ck-patched kernel which does its best in order to reduce the IO throughput, so I cannot really tell about this.
However, I used to have noop and deadline compiled into my kernel, switching from one to the other for testing purposes, then I realized deadline did not help so I no longer build the deadline scheduler.
I never watched any significant difference in terms of int0 rate between deadline and noop with deadline built, nor did I watch any significant difference since I no longer build deadline.
dmpogo wrote:
Hardware also does not point to a pattern
Low INT0 - core 2 duo, nvidia, snd_hda_intel, no wifi, realtek ethernet
High INT0 - a) core 2 duo, intel graphics, wifi, snd_hda_intel, intel ethernet (off) laptop
b) 4 core i7, nvidia, snd_hda_intel, realtek ethernet, desktop

I do not think the answer is to be searched around these.

Perhaps in the daemons running, almost certainly around X, kde & related qt apps.
Akgregator, Amarok... have had long histories of triggering high int0 rates when iddle.
You just cannot imagine the number of threads doing nothing else but polling as soon as these apps are launched.
And polling requires... timer interrupts.
And then depending on X, apps, qt, kde versions, polling rates can be increased, decreased...
_________________
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Wed Sep 19, 2012 4:52 pm    Post subject: Reply with quote

I run Gnome on pretty much all machines, network manager and all. Even blueman and the polling gweather, gnome-power-manager, etc.

The thing is, I thought that INT0 may only be used during boot or something. I reboot my i5 and it still came back up as 127 somehow.

My wifi does not contribute to INT0 counts (of course), just its corresponding MSI.

I wonder what the interrupt numbers would look like if you compiled the same kernel as mine?

The hardware on my laptop: Intel Graphics (HD4000), realtek gbit ethernet, intel wifi 2230N, Z77 chipset; it's a pretty basic laptop (HP Envy4t).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 5:11 pm    Post subject: Reply with quote

aCOSwt wrote:

Perhaps in the daemons running, almost certainly around X, kde & related qt apps.
Akgregator, Amarok... have had long histories of triggering high int0 rates when iddle.
You just cannot imagine the number of threads doing nothing else but polling as soon as these apps are launched.
And polling requires... timer interrupts.
And then depending on X, apps, qt, kde versions, polling rates can be increased, decreased...


What actually is practically identical between three machines is KDE setup. And it is -semantic-desktop on all three - so no nepomunks, amarok's
or anything like that even installed. It is as barebone KDE setup as I could get. Well it has weather applet - on all three machines, it has UPS and its applet on two - one with low INT0, one with high (desktops), X ? one high INT machine uses nvidia, the other - intel, low int machine is nvidia. External eSata drives ? both on low INT and high INT desktops, none on high INT laptop.

So go figure. One thing in common that both high INT machines have SSD drives (that's why I suspected deadline), but I'd suprised it has anything to do.

Finding what may poll is an interesting point - that's I believe why I have high interrupt counts on one USB device, it is probably usbdisks polling it all the time
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3264
Location: Canada

PostPosted: Wed Sep 19, 2012 5:13 pm    Post subject: Reply with quote

eccerr0r wrote:
I run Gnome on pretty much all machines, network manager and all. Even blueman and the polling gweather, gnome-power-manager, etc.

The thing is, I thought that INT0 may only be used during boot or something. I reboot my i5 and it still came back up as 127 somehow.

My wifi does not contribute to INT0 counts (of course), just its corresponding MSI.

I wonder what the interrupt numbers would look like if you compiled the same kernel as mine?

The hardware on my laptop: Intel Graphics (HD4000), realtek gbit ethernet, intel wifi 2230N, Z77 chipset; it's a pretty basic laptop (HP Envy4t).


What I'll try to do (need to find time) is copy a kernel config from my high INT desktop to a low INT one. They have similar hardware so it can be a close comparison.
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Wed Sep 19, 2012 6:59 pm    Post subject: Reply with quote

eccerr0r wrote:
So for me it looks like HPET made the difference

I thought HPET was set for both configs.
But I realize that even then, there might well be some difference because of the BIOS :
HPET Legacy Replacement Route option (hpet_lrr) wrote:

+
+HPET is capable of replacing the IRQ0 (connected INT0 PIN) routing for
+timer interrupt.
The capability register (at offset 0 of HPET
+base address) has a bit specifying if HPET chip is capbale of doing
+this. OS can read the bit either from HW or ACPI table. (HPET ACPI
+description table -> Event Timer block -> bit 15, page 30 of HPET
+spec). Ideally (I think so!) BIOS should set the ACPI table than letting
+the OS read H/W, which gives the BIOS a way to configure either legacy
+or Legacy replacement modes.

+
+Typically the motherboard has BIOS configured / hardwired IRQ0 to INT0
+(pin of APIC) connection. Linux assumes IRQ0 connected to INT0 unless it is
+supplied using an override parameter in the MPTable
. Some NVidia chipsets /
+BIOS initialization code had configured to override IRQ0 -> INT0 connection
+and later a parameter was introduce (acpi_skip_timer_override) to get IRQ0 ->
+INT0 connection right.
+
+But a number of bioses (both phoenix and AMI) are not working as
+expected. (I have an AMI BIOS which sets ACPI table bit 15 to 0 and then
+connect IRQ0 -> INT2 internally, Another bios I have sets the ACPI table bit
+15 to 0, but does not connect IRQ0 -> INT2.

Out of interest what can you read concerning hpet in your system log ?
On mine :
Code:

HPET: 4 timers in total, 0 timers will be used for per-cpu timer
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
Switching to clocksource hpet

_________________
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Wed Sep 19, 2012 7:48 pm    Post subject: Reply with quote

Hmm... It's probably not HPET: I just looked at one of my really ancient machines that does not have HPET and it's also got very few INT0
Code:
doujima:/var/log$ cat /proc/interrupts
           CPU0       
  0:         45   IO-APIC-edge      timer
  1:      23269   IO-APIC-edge      i8042
  4:    3759111   IO-APIC-edge   
  6:          5   IO-APIC-edge      floppy
  7:     147257   IO-APIC-edge      parport0
  9:          0   IO-APIC-fasteoi   acpi
 12:    1026863   IO-APIC-edge      i8042
 14:   21261716   IO-APIC-edge      pata_sis
 15:   20630591   IO-APIC-edge      pata_sis
 16:      23030   IO-APIC-fasteoi   uhci_hcd:usb4, radeon@pci:0000:01:00.0
 17:   41882707   IO-APIC-fasteoi   pata_pdc202xx_old, uhci_hcd:usb5
 18:  106239203   IO-APIC-fasteoi   pata_pdc2027x, ehci_hcd:usb1, SiS SI7012
 19:  226003861   IO-APIC-fasteoi   ohci_hcd:usb2, eth0
 22:          0   IO-APIC-fasteoi   eth1
 23:          0   IO-APIC-fasteoi   ohci_hcd:usb3
NMI:          0   Non-maskable interrupts
LOC: 2483032918   Local timer interrupts
SPU:          0   Spurious interrupts
PMI:          0   Performance monitoring interrupts
IWI:          0   IRQ work interrupts
TRM:          0   Thermal event interrupts
THR:          0   Threshold APIC interrupts
MCE:          0   Machine check exceptions
MCP:      22645   Machine check polls
ERR:         95
MIS:          0


This machine does not have GNOME installed, but as can be seen by the interrupt handlers, has a lot of hard drives attached (5 PATA.)
This is an old kernel though - 2.6.38. I need to update this soon...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum