Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[HOWTO] Fun with io priorities aka ionice
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
pijalu
Guru
Guru


Joined: 04 Oct 2004
Posts: 365

PostPosted: Sun May 14, 2006 12:59 pm    Post subject: [HOWTO] Fun with io priorities aka ionice Reply with quote

Sometimes, when my sweet gentoo do some heavy background tasks on my laptop (eg: big fat emerge, update-db, prelink, etc...), my io usage becomes annoying (no cpu use... just hd is out for lunch)...and this, even if the tasks are niced
==> My desktop starts to lags... opening a console takes way too much time....

If you already experience this, here comes a tip to keep a pleasant desktop: messing with io priorities...
As usual, this is not garanteed, if you break something, get hacked, loose your job or even start to miss windows, don't complain to me ;).... This won't make your desktop go faster with IO... just reschedule things to keep a snappy desktop

So, let's start !
Check kernel settings:
* Be sure you have a PREEMT enabled kernel
Code:

$ zcat /proc/config.gz | grep PREEMT
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set

(normal or voluntary should do)
* be sure CFQ iosched is enabled
Code:

$ zcat /proc/config.gz | grep IOSCHED
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_IOSCHED="anticipatory"


* if CONFIG_IOSCHED_CFQ is set but CONFIG_DEFAULT_IOSCHED is not CFG (like here)
add to /etc/conf.d/local.start:
Code:

   for drive in  /sys/block/*/queue/scheduler
   do
      echo cfq > $drive
   done

(restart local)

* install ionice
a) get the source from kernel docs:
Code:

sed -n -e "/snip ionice.c/,/snip ionice.c/ p" /usr/src/linux/Documentation/block/ioprio.txt | sed "s/.\+snip ionice.\+//g" > ionice.c


If you want all users to have access to ionice, it will need to bet set uid root ==> this mean security hazard..... the following patch should avoid normal user setting higher ioprio than default and, last but not least, avoid ionice to start programs as root.... This is not bullet proof and can lead to security problems... jump to point e and skip f if you plan to use ionice as root only)
b) Patch it => create a file called ionice.patch and copy the following content
Code:

--- ionice.c.ori   2006-05-14 13:59:53.000000000 +0200
+++ ionice.c   2006-05-14 13:56:32.000000000 +0200
@@ -1,5 +1,3 @@
-
-
 #include <stdio.h>
 #include <stdlib.h>
 #include <errno.h>
@@ -99,14 +97,23 @@
          printf("%s: prio %d\n", to_prio[ioprio_class], ioprio);
       }
    } else {
+      if (getuid()!=0 && (ioprio_class < IOPRIO_CLASS_BE || ioprio < 4)  ) {
+         fprintf(stderr,"User cannot set higher ioprio\n");
+         return 1;
+      }
       if (ioprio_set(IOPRIO_WHO_PROCESS, pid, ioprio | ioprio_class << IOPRIO_CLASS_SHIFT) == -1) {
          perror("ioprio_set");
          return 1;
       }
 
-      if (argv[optind])
+      if (argv[optind]) {
+          if (seteuid(getuid())!=0 || setegid(getgid())!=0) {
+         perror("Error setting real user");
+         return 1;
+          }
          execvp(argv[optind], &argv[optind]);
    }
+   }
 
    return 0;
 }

c) check if patch is fine
Code:

patch --dry-run -l < ionice.patch


d) if fine, apply
Code:

patch -l < ionice.patch


e) compile
Code:

gcc -o ionice ionice.c


f) set rights (only if you patched ionice...):
Code:

# chown root:root ionice && chmod u+s,-rw ionice


g) copy to /usr/bin (/usr/sbin for a root only version)
Code:

# mv ionice /usr/bin


* Play with it:
I use ionice on every heavy io cron jobs.... Simply change the cron job scripts (/etc/cron.*) to prefix the calls to program with ionice...( do some backups first...;-) )

Some infos:
* Can be used with nice
* 3 prio:
RT (real time) ==> -c1 (take it no matter what)
BE (BEST EFFORT)==> -c2 (default)
IDLE ==> -c3 (disk will be used only if "free")

* Level: available for RT and BE, use via -nX
Levels allows you a finer tunning of the class (prio between same class) - this is not used for idle...
Default prio are BE level 4

Check /usr/src/linux/Documentation/block/ioprio.txt for more info ;-)

Examples
eg: Start an emerge sync as low priorities
Code:

# ionice -c3 emerge --sync


Not annoying cron slocate (update-db) - /etc/cron.daily/slocate :
Code:

#! /bin/sh

if [ -x /usr/bin/updatedb ]
then
   if [ -f /etc/updatedb.conf ]
   then
      /usr/bin/ionice -c3 nice /usr/bin/updatedb
   else
      /usr/bin/ionice -c3 nice /usr/bin/updatedb -f proc
   fi
   chown root:locate /var/lib/slocate/slocate.db
fi


Have fun....
Back to top
View user's profile Send private message
johntash
n00b
n00b


Joined: 30 Apr 2005
Posts: 20
Location: KS

PostPosted: Wed May 17, 2006 3:35 am    Post subject: Reply with quote

Nice.. I think I'll try this on my gentoo desktop later this week. Looks good :)
Back to top
View user's profile Send private message
JuNix
Apprentice
Apprentice


Joined: 05 Mar 2003
Posts: 204
Location: Sheffield

PostPosted: Wed May 17, 2006 8:58 am    Post subject: Reply with quote

Nice post!
Back to top
View user's profile Send private message
JuNix
Apprentice
Apprentice


Joined: 05 Mar 2003
Posts: 204
Location: Sheffield

PostPosted: Fri May 19, 2006 11:10 am    Post subject: Reply with quote

Just to follow up on this, this really works, very nicely indeed. So so useful

So I ran a benchmark

Completely idle disks
Code:
gatekeeper ~ # hdparm -T -t /dev/md2

/dev/md2:
 Timing cached reads:   2012 MB in  2.00 seconds = 1004.70 MB/sec
 Timing buffered disk reads:  282 MB in  3.02 seconds =  93.47 MB/sec


Lots of normally prioritised diskio using chkfrag.pl
Code:
gatekeeper ~ # hdparm -T -t /dev/md2

/dev/md2:
 Timing cached reads:   2188 MB in  2.00 seconds = 1092.19 MB/sec
 Timing buffered disk reads:   40 MB in  3.13 seconds =  12.77 MB/sec



So you can immediately see that while chkfrag.pl is running, I'm only getting 12.77 MB/s, so I have really degraded disk io bandwidth available, lots of expensive head seeks

and finally, chkfrag.pl is invoked with ionice -c3 ....

Code:
gatekeeper ~ # hdparm -T -t /dev/md2

/dev/md2:
 Timing cached reads:   2732 MB in  2.00 seconds = 1365.29 MB/sec
 Timing buffered disk reads:  310 MB in  3.02 seconds = 102.81 MB/sec



and for the curious

chkfrag.pl

Code:
#!/usr/bin/perl -w

#this script search for frag on a fs
use strict;

#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;

#search fs for all file
open (FILES, "find " . $ARGV[0] . " -xdev -type f -print0 |");

$/ = "\0";

while (defined (my $file = <FILES>)) {
        PrivoxyWindowOpen(FRAG, "-|", "filefrag", $file);
        my $res = <FRAG>;
        if ($res =~ m/.*:\s+(\d+) extents? found/) {
                my $fragment = $1;
                $fragments += $fragment;
                if ($fragment > 1) {
                        $fragfiles++;
                }
                $files++;
        } else {
                print ("$res : not understand for $file.\n");
        }
        close (FRAG);
}
close (FILES);

print ( $fragfiles / $files * 100 . "% non contiguous files, " . $fragments / $files . " average fragments.\n");
Back to top
View user's profile Send private message
hans0r
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jan 2005
Posts: 122
Location: Germany

PostPosted: Tue Jun 13, 2006 7:34 pm    Post subject: Reply with quote

great howto! works like a charm :)
Back to top
View user's profile Send private message
pussi
l33t
l33t


Joined: 08 May 2004
Posts: 727
Location: Finland

PostPosted: Mon Jul 10, 2006 2:17 pm    Post subject: Reply with quote

ionice is also avalibe from portage from sys-apps/util-linux-2.13_pre7, which is currently hard masked though.
Back to top
View user's profile Send private message
pijalu
Guru
Guru


Joined: 04 Oct 2004
Posts: 365

PostPosted: Mon Jul 10, 2006 3:24 pm    Post subject: Reply with quote

pussi wrote:
ionice is also avalibe from portage from sys-apps/util-linux-2.13_pre7, which is currently hard masked though.


Yep, but i didn't suceeded to compile this package on my system => the reason why I choose to take it directly from the kernel doc
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Thu Aug 10, 2006 12:42 pm    Post subject: Reply with quote

Isn't there any other scheduler supporting io priorities? The CFQ scheduler produces EXTREME lags on the desktop. For example, when editing within vi, the response can take up to a minute or more.... so that solution is far from optimal, but better than nothing :-). M;aybe that is due to ext3's incapability to maintain a high total throughput rate in high disk load situations (unlike XFS)?

If you are using fcrontab, you should consider running your jobs via:

Code:
%hourly *        rm -f /var/spool/cron/lastrun/cron.hourly
%daily * 5-23       rm -f /var/spool/cron/lastrun/cron.daily
%weekly * 5-23      rm -f /var/spool/cron/lastrun/cron.weekly
%monthly * * *   rm -f /var/spool/cron/lastrun/cron.monthly
@ 10 if /usr/bin/test -x /usr/bin/ionice; then /usr/bin/ionice -c3 /usr/sbin/run-crons; else /usr/sbin/run-crons; fi


BTW: the ionice utility is also available via my scripting tools package: https://stier.dynu.com/~myportage/sys-libs/mscript-tools/
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Thu Sep 14, 2006 2:01 pm    Post subject: Reply with quote

ionice -c3 seems a bit screwy.. I've seen it go ti 99% iowait and move EXTREAMLY slowly (keeping extreamly low latency desktop while doing it though), while -c2 -p7 works flawlessly.
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Thu Sep 14, 2006 2:08 pm    Post subject: Reply with quote

neuron wrote:
ionice -c3 seems a bit screwy.. I've seen it go ti 99% iowait and move EXTREAMLY slowly (keeping extreamly low latency desktop while doing it though), while -c2 -p7 works flawlessly.


Uhm, isn't that what -c3 says? To delay any io operations as long as there are higher-prio IOs? 99% iowait tells you that there is IO going on, so that seems perfect to me (as long as there actually IS some non -c3 IO).
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Thu Sep 14, 2006 2:19 pm    Post subject: Reply with quote

Cinquero wrote:
neuron wrote:
ionice -c3 seems a bit screwy.. I've seen it go ti 99% iowait and move EXTREAMLY slowly (keeping extreamly low latency desktop while doing it though), while -c2 -p7 works flawlessly.


Uhm, isn't that what -c3 says? To delay any io operations as long as there are higher-prio IOs? 99% iowait tells you that there is IO going on, so that seems perfect to me (as long as there actually IS some non -c3 IO).


yes, but it's a tad bit extreme.

Like in I can copy stuff in about 100kb/sec, while downloading something at 1mb/sec. It should not go anywhere near that slow. And ionice -c2 -p7 gives me 20-30mb/sec instead ;)
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Sat Sep 16, 2006 6:12 am    Post subject: Reply with quote

This looks great. Any idea when it will be unmasked in portage?
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Sun Sep 17, 2006 9:25 pm    Post subject: Reply with quote

Install my scripting tools package. It includes the ionice binary. Don't wait for Gentoo. You have no guarantee that they will ever support it.

https://stier.dynu.com/~myportage/sys-libs/mscript-tools/

Question:

does anyone else experience the following problem?

1.) ionice -c3 emerge ... some pkg

2.) rsync / target (in parallel to 1)

3.) some GUI (ie. tvtime) with regular io and cpu priorities hangs as long as rsync builds the filelist... that is really ANNOYING and definitely a bug.
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Mon Sep 18, 2006 12:11 am    Post subject: Reply with quote

Cinquero wrote:
Install my scripting tools package. It includes the ionice binary. Don't wait for Gentoo. You have no guarantee that they will ever support it.


it's in sys-apps/util-linux-2.13_pre7 ;)
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Mon Sep 18, 2006 12:15 am    Post subject: Reply with quote

neuron wrote:
it's in sys-apps/util-linux-2.13_pre7 ;)


Not unmasked.
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Mon Sep 18, 2006 12:23 am    Post subject: Reply with quote

Cinquero wrote:
neuron wrote:
it's in sys-apps/util-linux-2.13_pre7 ;)


Not unmasked.


no but "You have no guarantee that they will ever support it.", that's fairly unlikely ;)
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Mon Sep 18, 2006 12:29 am    Post subject: Reply with quote

neuron wrote:
no but "You have no guarantee that they will ever support it.", that's fairly unlikely ;)


Conversation considered useless.
Back to top
View user's profile Send private message
fgrosshans
n00b
n00b


Joined: 28 Apr 2004
Posts: 11
Location: Munich

PostPosted: Tue Sep 19, 2006 4:14 pm    Post subject: Thanks for the trick... Reply with quote

Thanks a lot. It makes emerge-by-working practical on my laptop !

It just lacks the way to (io)renice an already started process. For a started emerge, for exemple, you can find the relevant PID by
Code:
 # ps -A | grep emerge
22066 tty2     00:00:19 emerge

and then ...
Code:
 ionice -c3 -p2206

_________________
Frédéric Grosshans
Back to top
View user's profile Send private message
pijalu
Guru
Guru


Joined: 04 Oct 2004
Posts: 365

PostPosted: Tue Sep 19, 2006 6:44 pm    Post subject: Reply with quote

Something like this script may help
Code:

#!/bin/bash

cmd="$@"
if [ -z "$cmd" ]
 then
    echo "feed me something..."
    exit 0
fi

# get all options
params=`echo $@ | sed -e "s/ /\n/g" | sed -n "/^-/p"`
# get rid of extra \n
params=`echo $params`

# set a def val if needed
if [ -z "$params" ]
then
    params="-c3"
fi

# for each given prog name
for item in `echo $@ | sed "s/-.*//g" | sed "s/\ /\n/g"`
do
  # for each process hitting this
  for pid in `ps -A | grep $item | sed "s/[^ 0-9].*//g"`
  do
      echo "ionicing $item ($pid) with $params"
      ionice $params -p$pid
  done
done


usage: iorenice [partial prog name] [ionice options]
Note: This will take ALL hit to partial prog name (multiple should work) and apply the give class/prio (-c3 by def)
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Tue Sep 19, 2006 6:59 pm    Post subject: Reply with quote

Uhm, my mscript-tools script allows to nice a complete process hierarchy. Just give it the parent process id:

mscript-tools NiceAll `pidof emerge`

You just need to replace nice with ionice in it.
Back to top
View user's profile Send private message
MaratIK1
n00b
n00b


Joined: 21 Sep 2006
Posts: 3

PostPosted: Thu Sep 21, 2006 4:52 pm    Post subject: ebuild Reply with quote

Request for ebuild added:
https://bugs.gentoo.org/show_bug.cgi?id=148519
Back to top
View user's profile Send private message
Cinquero
Apprentice
Apprentice


Joined: 24 Jun 2004
Posts: 249

PostPosted: Sat Sep 30, 2006 6:50 pm    Post subject: Reply with quote

Summary of my experience with the CFQ scheduler: H-O-R-R-I-B-L-E. Just plain useless. When I do not use io prios, a simple "rm -rf /var/tmp/ccache" may lock the entire desktop until it has finished.... even NFS transfers have stalled completely. I'm switching back to AS.
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Mon Oct 02, 2006 9:58 am    Post subject: Re: ebuild Reply with quote

MaratIK1 wrote:
Request for ebuild added:
https://bugs.gentoo.org/show_bug.cgi?id=148519
According to this link, ionice is included in schedutils 1.5.0, which appears to be already in portage. Has anyone tried this out?

Last edited by bludger on Mon Oct 02, 2006 10:10 am; edited 1 time in total
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Mon Oct 02, 2006 10:09 am    Post subject: Reply with quote

Cinquero wrote:
Summary of my experience with the CFQ scheduler: H-O-R-R-I-B-L-E. Just plain useless. When I do not use io prios, a simple "rm -rf /var/tmp/ccache" may lock the entire desktop until it has finished.... even NFS transfers have stalled completely. I'm switching back to AS.

I haven't tried it out myself, so I can't really judge, but it seems contrary to the theory of CFQ as far as I understand it. According to this link, io should be distributed fairly between all processes. Can someone else confirm this phenomenon? I find it strange that Redhat have made this the default scheduler, if it really is so bad. Can you confirm that everything is running with IOPRIO_CLASS_BE (Best effort) and that nothing is running with IOPRIO_CLASS_RT (Real time)?

Maybe I'll have to try it out for myself. BTW if I reboot with the boot flag "elevator=cfq" will that select cfq as default? I would rather not recompile my kernel just for this test.
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Mon Oct 02, 2006 11:12 am    Post subject: Reply with quote

bludger wrote:
Maybe I'll have to try it out for myself. BTW if I reboot with the boot flag "elevator=cfq" will that select cfq as default? I would rather not recompile my kernel just for this test.

Just found this: /usr/src/linux/Documentation/block/switching-sched.txt. It is possible to change the scheduler on the fly with:

echo cfq > /sys/block/<device>/queue/scheduler
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum