| View previous topic :: View next topic |
| Author |
Message |
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
 Posted: Wed Mar 16, 2005 3:30 am Post subject: chugger - a program to benchmark interactivity |
 |
|
This thread gave me the motivation to write a program to benchmark interactivity. Interactivity is a difficult thing to benchmark because so many things will effect it. Basically, I want to do something similar to sysmark, but instead of using real apps, just use some muck I'll write in C (my first real C program).
Basically, here's how I think it will work: There will be 3 apps.
The first app is a control app that spawns the second 2, tabulates results, blady-blah.
The second will generate a system load. I have a very sweet prime finding algorhythm to do this. It will basically load the CPU to 100% and generate mild disk I/O and memory I/O as it writes a linked list of found primes back and forth from RAM to the HD and uses the list of primes to test for new ones. It's very effective at loading a system and incredibly fast at finding prime numbers. It's written in C++ right now (I'm a nob, I know), but I will port it to C tonight and make sure this is a sufficient load. I'd also like it variable and will include some way of limiting the CPU and RAM usage.
The third app is what actually tests the interactivity. Really, what I need to do is load a large number of libraries to RAM and utilize various functions. GTK, QT, etc. Then the app perform a disk I/O and then will die without doing anything visual. The time it takes to load these libraries and call the functions will vary linearly with interactivity. Large number = bad. Performance should vary with various build flags used on the libraries (CFLAGS on the app itself will be locked down with the configure script to make it respectibly repeatable), linker flags, kernel (I/O scheduler, driver performance), and filesystem type
Any thoughts/feedback? What sort of inputs should the control program take to ensure the system is being thoroughtly benchmarked beyond what I've touched on here?
Anyone want to partake? |
|
| Back to top |
|
 |
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
| Posted: Sat Mar 19, 2005 4:01 pm Post subject: |
|
|
Here is the first incarnation of primefinder. It has lost most of it's functionality, but serves very well as a benchmark. It appears to be quite sensitive to a variety of CFLAGS and the output of the first 32m prime numbers is identicle in every case (even with -ffast-math). The output has also been tested against an exhaustive searching algorhythm to make sure it's accurate. Compile it with | Code: | | gcc main.c -march=whatever -O2 -fomit-frame-pointers -funroll-loops -o primefinder -lm |
and run like: | Code: | | time echo 2^26 | bc | nice -n -19./primefinder &>/dev/null |
Adjust 2^26 to suite your amount of RAM. 2^26 is suggested for 512MB of RAM as it should be just barely swapping to be an effective test. The results are very repeatable, varying at most 20ms on my box. The time is so effected by CFLAGS the time can be reduced by over 180% and quite a few flags will make a difference. -march, -O(all levels except 3), and -fomit-frame-pointers, most notably. I'm testing right now to see if the results vary from kernel to kernel. My current average time over 4 runs is 3:20.888 (user) with 2.6.10-ck5 and a friend's 4292.60 bogomips 256k athlon XP with 333MHZ FSB. The results scale very nicely with FSB (even with roughly the same bogomips), leading me to believe I'm on the right track. I'll be back in a bit to let you know how fast 2.6.10-ck5-grsecurity is by comparison, and then vanilla 2.6.10.
| Code: |
//main.c
//This is primefinder. It will stress your system. Run with "time echo $stopnum | nice -n -19 ./primefinder" to use as a benchmark
//Licensed under the General Public License
#include <stdio.h>
#include <math.h>
extern double sqrt(); //Anjuta required this
//This is the linked list element type definition
struct numTable
{
long value;
struct numTable * next;
};
//What a stupid waste of RAM.
//Fills in a linked list of primes numbers from head->value to tail->value and slaps it btw these pointers
void numTableInit(struct numTable * head, struct numTable * tail)
{
struct numTable * current;
current=head;
long bigVal;
for ( bigVal=head->value+2; bigVal<=tail->value ; )
{
current->next = malloc(sizeof(struct numTable));
current=current->next;
current->value=bigVal;
bigVal+=2;
}
free(current->next);
current->next=tail;
}
//Quickely gives numTable's table the smack down
void freeTable(struct numTable * head)
{
struct numTable * pos = head;
struct numTable * tmp = pos->next;
while(tmp->next) {
free(pos);
pos=tmp;
tmp=tmp->next;
}
free(tmp);
}
//Test for primes using an exhaustive test of every odd number
//from 3 to sqrt(PrimeTest)
int primeTester(long Primetest)
{
if (Primetest == 3) return 1;
long checker;
for (checker=3;fmod(Primetest,checker) && checker<=sqrt(Primetest);checker+=2) ;
if (fmod(Primetest,checker)) return 1; else return 0;
}
//Test for primes using an exhaustive list of known primes at least as large as sqrt(PrimeTest)
int primeTesterQuick(long Primetest, struct numTable * ptHead)
{
if (Primetest == 3 || Primetest == 5) return 1;
struct numTable * pos;
for(pos=ptHead;pos!=NULL && fmod(Primetest,pos->value) && pos->value<=sqrt(Primetest);pos=pos->next) ;
if (fmod(Primetest,pos->value)) return 1; else return 0;
}
//builds a linked list of primes using candidate numbers from numlHead to numlTail.
//The list of primes spans from ptHead to ptTail
void primeTableInit(struct numTable * numlHead, struct numTable * numlTail, struct numTable * ptHead, struct numTable * ptTail)
{
long i=0; //prime number counter
ptHead->value=numlHead->value; //since the first numl value is prime
struct numTable * current; //floating pointer to list items
struct numTable * ptCurrent = ptHead;
for(current=numlHead;current!=numlTail;current=current->next) //increment up the list
if(primeTesterQuick(current->value,ptHead)) //if(prime)
{
ptCurrent->next=malloc(sizeof(struct numTable)); //make a new list item
ptCurrent = ptCurrent->next; //insert it
ptCurrent->next = ptTail; //tie up the laces
ptCurrent->value=current->value; //set the value
printf("%d\n",current->value); //again, for debugging, really.
//In future, this will be swapped to disk to measure I/O scheduler performance in with multiple threads
i++; //prime number counter ++
}
ptCurrent->next=ptTail; //should'a been set in the loop, but why not.
printf("Found %d primes\n",i);
ptTail->next=NULL; //will be moved, seems to work
}
int main(int argc, char *argv[])
{
long start=3;
long stop=5033164;
// printf("Please enter a start number: ");
// scanf("%u",&start);
// printf("Please enter the number to stop at: ");
// We are currently broken if starting at anything other than 3
//due to last of a list of initial primes to test against. Will be fixed.
scanf("%u",&stop);
//Interactive mode
/* while(start!=0) {
printf("Enter a number: ");
scanf("%u",&start);
if(!fmod(start,2)>0) start++;
if(primeTester(start)) printf("%d is Prime!\n",start);
else printf("%d is Not Prime!\n",start);
}
*/
struct numTable * ptHead = malloc(sizeof(struct numTable));//prime number table list head
struct numTable * ptTail = malloc(sizeof(struct numTable));//prime number table list tail
struct numTable * numlHead = malloc(sizeof(struct numTable));//head of a list of numbers to test against
struct numTable * numlTail = malloc(sizeof(struct numTable));//tail of a yadda yadda
//Here we're handling the input numbers because people will do silly things.
//Please don't pass values that overflow the type as this is untested (other
//than a few accidental segfaults)
//Make sure we're dealing with an odd numbers
if(!fmod(start,2)>0) start++;
if(!fmod(stop,2)>0) stop--;
//Advance to the next prime
long * x = malloc(sizeof(long));
for(*x = start;!primeTester(*x);*x+=2);
start=*x;
free(x);
//This is for debugging, really
printf("Starting search at %d\n",start);
printf("Stopping search at %d\n",stop);
//Setup the initial lists
numlHead->value=start;
numlTail->value=stop;
numlHead->next=numlTail;
numlTail->next=NULL;
ptHead->next=ptTail;
ptTail->next=NULL;
//This generates a list of numbers to be tested
//implimented this way for future use and so as to be memory intensive
numTableInit(numlHead, numlTail);//rough measure of ram performance here
//This builds a list of prime numbers from ptHead based on the test data within numlHead.
primeTableInit(numlHead, numlTail, ptHead, ptTail);
//gets CPU intensive when the numbers get big
//cleanup
freeTable(numlHead);
freeTable(ptHead);
return (0);
}
|
Last edited by thebigslide on Sat Mar 19, 2005 5:22 pm; edited 1 time in total |
|
| Back to top |
|
|
Need4Speed
Guru
Joined: 06 Jun 2004
Posts: 497
|
| Posted: Sat Mar 19, 2005 5:04 pm Post subject: |
|
|
nice program, just tried it with NO cflags and with my usual cflags. With no cflags, I just killed it after 10 minutes, but cflags made it MUCH faster. The results are probably a liitle exagerated compared to real world perforamace, but still pretty interesting.
| Code: | linux ~ # cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 10
model name : AMD Athlon(tm) XP
stepping : 0
cpu MHz : 2105.123
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse pni syscall mp mmxext 3dnowext 3dnow
bogomips : 4161.53
|
(note: fsb is currently around 440mhz)
| Code: | linux ~ # gcc main.c -march=athlon-xp -O2 -pipe -fomit-frame-pointer -ffast-math -fmerge-all-constants -ftracer -fweb -frename-registers -o primefinder -lm
linux ~ # time echo 2^26 | bc | nice -n -19 ./primefinder &>/dev/null
real 2m2.551s
user 0m52.933s
sys 0m2.796s
|
oh yeah, i think there's a typo in your time command, you have primefinderc, but you orginially called it primefinder |
|
| Back to top |
|
|
pilla
Bodhisattva
Joined: 07 Aug 2002
Posts: 7729
Location: Underworld
|
| Posted: Sat Mar 19, 2005 5:25 pm Post subject: |
|
|
Moved from P&P
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin |
|
| Back to top |
|
|
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
| Posted: Sat Mar 19, 2005 7:21 pm Post subject: |
|
|
I did a test run with the system running 'standard desktop software' using the vanilla and ck5 2.6.10 kernels:
The system was running a fluxbox desktop and an unused fluxbox VNC desktop. Xine was playing Season 3 of Trailer Park Boys Episode 2 on a continuous loop at stock resolution. Beep Media Player was playing Bob Marley's Burnin' and a Lootin' on a continuous loop. There was a top window updating every 1 second also open. I ran the program niced _below_ the desktop first to see how good each kernel was at isolating a niced load away from the user.
Here's what the system load looked like:
| Code: | Tasks: 60 total, 5 running, 55 sleeping, 0 stopped, 0 zombie
Cpu(s): 51.2% us, 1.0% sy, 40.5% ni, 0.0% id, 7.4% wa, 0.0% hi, 0.0% si
Mem: 515244k total, 511472k used, 3772k free, 127772k buffers
Swap: 2000052k total, 615244k used, 1384808k free, 27000k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11603 bigslide 21 1 555m 307m 188 R 40.6 61.2 2:07.21 primefinderc
10596 root 20 0 169m 11m 8656 R 39.6 2.3 17:40.00 X
11480 bigslide 20 0 212m 12m 8864 R 11.3 2.5 4:15.46 xine
11446 bigslide 20 0 44744 4664 2768 S 0.7 0.9 0:19.19 beep-media-play
10697 bigslide 20 0 8736 1368 784 S 0.3 0.3 0:04.04 fluxbox
11498 bigslide 20 0 2004 572 420 R 0.3 0.1 0:02.32 top
1 root 20 0 1380 244 228 S 0.0 0.0 0:00.22 init
2 root 39 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
3 root 15 -5 0 0 0 S 0.0 0.0 0:00.06 events/0
4 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
9 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
18 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
133 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
198 root 20 0 0 0 0 S 0.0 0.0 0:00.05 pdflush
199 root 20 0 0 0 0 S 0.0 0.0 0:00.09 pdflush
201 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0
200 root 20 0 0 0 0 S 0.0 0.0 0:05.08 kswapd0 |
Here are the times for the 2 kernels averaged over 4 trials:
| Code: | 2.6.10-r1
time echo 2^26 | bc | nice -n 1 ./primefinder > primes.lst
real 9m34.671s
user 3m34.205s
sys 0m3.513s
2.6.10-ck5
time echo 2^26 | bc | nice -n 1 ./primefinderc > primes.lst
real 9m37.914s +/- 2.700s!
user 3m34.746s +/- 0.100s
sys 0m3.416s +/- 0.050s |
When using the vanilla kernel, the desktop felt sluggish once the primefinder app was launched. I fired up those apps first and I didn't do anything once it was chuggin', just moved the mouse around, and it was 'skippy'. The DVD playback skipped a few times, sometimes seemingly at random, every other time beep changed tracks, the DVD would hiccup. When primefinder started another iteration (I ran the trials in a shell for loop) or when Xine changed tracks, the whole system would hiccup and lock up for a half a second or so, sometimes a couple of times in a row.
When using the ck kernel, the desktop felt smooth, even with everything running. When Xine changed tracks there was no stuttering. There was no choppiness or stuttering of the system at all except when primefinder started another iteration, the whole system would lockup for a half a second. Primefinder ran measureaby slower... barely |
|
| Back to top |
|
|
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
| Posted: Sat Mar 19, 2005 8:02 pm Post subject: |
|
|
I have tabulated some earlier results of running the program nice'd all the way up in an environment with few other processes running
| Code: | 2.6.10-ck5
time echo 2^26 | bc | nice -n -19./primefinderc &>primes2.lst
real 4m2.016s +/- 1.500s
user 3m20.888s +/- 0.050s
sys 0m2.086s +/- 0.200s
2.6.10-ck5-grsecurity
time echo 2^26 | bc | nice -n -19 ./primefinderc &>primes2.lst
real 4m0.113s +/- 1.500s
user 3m20.726s +/- 0.050s
sys 0m2.190s +/- 0.200s
2.6.10-r1
time echo 2^26 | bc | nice -n -19 ./primefinderc &>primes2.lst
real 3m56.809s +/- 1.500s
user 3m20.746s +/- 0.050s
sys 0m2.336s +/- 0.200s
|
From these, it seems that the kernels designed to be more 'interactive' will execute an individual task a little slower in favor of not allowing any one to hog the system. This is to be expected. I'm compiling a nitro and a love kernel and we'll see how these perform in an interactivity test similar to the first results I posted. I'm working on a way to quantify the 'choppiness' |
|
| Back to top |
|
|
cokey
Advocate
Joined: 23 Apr 2004
Posts: 3355
|
| Posted: Fri Apr 08, 2005 12:04 am Post subject: |
|
|
this will be interesting...
_________________
https://otw20.com/ OTW20 The new place for off the wall chat |
|
| Back to top |
|
|
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
| Posted: Fri Apr 08, 2005 6:15 pm Post subject: |
|
|
| Haven't had much time to work on this. I was sifting through the source code for firefox, k3b, some others the other day. Basically I'm going to rewrite main.c to make them 'go through the hoops' ala sysmark. |
|
| Back to top |
|
|
Tiger683
Veteran
Joined: 08 Jan 2005
Posts: 1347
Location: Heffner's House
|
| Posted: Fri Apr 08, 2005 7:14 pm Post subject: |
|
|
Oh man, this is so funny/strange
| Code: |
BuildHost:me # uname -r
2.6.12-rc2-jadeX1-1
BuildHost:me # gcc main.c -march=athlon-xp -O2 -fomit-frame-pointer -funroll-loops -o primefinder -lm
BuildHost:me # time echo 2^26 | bc | nice -n -19./primefinder &>/dev/null
real 0m0.006s
user 0m0.001s
sys 0m0.006s
|
_________________
Retired gentoo user |
|
| Back to top |
|
|
thebigslide
l33t
Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard
|
| Posted: Fri Apr 08, 2005 7:20 pm Post subject: |
|
|
| Perhaps it's segfaulting? Get rid of the &>/dev/null bit and see if there's output (should be a list of all the primes it finds) |
|
| Back to top |
|
|
darklegion
Guru
Joined: 14 Nov 2004
Posts: 468
|
| Posted: Sat Apr 09, 2005 1:53 am Post subject: |
|
|
| It must be run as root (caught me too) |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
