Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Question about IO schedulers
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Etal
Veteran
Veteran


Joined: 15 Jul 2005
Posts: 1931

PostPosted: Wed Jul 05, 2017 12:14 am    Post subject: Question about IO schedulers Reply with quote

Linux 4.12 includes two new IO schedulers. Curiously, "make oldconfig" asked me if I wanted to enable them but didn't ask me what I wanted as the default, so I decided to double-check with menuconfig:
Code:
 .config - Linux/x86 4.12.0 Kernel Configuration
 → Enable the block layer → IO Schedulers ─────────────────────────────────────
  ┌───────────────────────────── IO Schedulers ─────────────────────────────┐
  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty │ 
  │  submenus ----).  Highlighted letters are hotkeys.  Pressing <Y>        │ 
  │  includes, <N> excludes, <M> modularizes features.  Press <Esc><Esc> to │ 
  │  exit, <?> for Help, </> for Search.  Legend: [*] built-in  [ ]         │ 
  │ ┌─────────────────────────────────────────────────────────────────────┐ │ 
  │ │    [*] Deadline I/O scheduler                                       │ │ 
  │ │    [*] CFQ I/O scheduler                                            │ │ 
  │ │        Default I/O scheduler (CFQ)  --->                            │ │ 
  │ │    [*] MQ deadline I/O scheduler                                    │ │ 
  │ │    [*] Kyber I/O scheduler                                          │ │ 
  │ │    [ ] BFQ I/O scheduler                                            │ │ 
  │ │                                                                     │ │ 
  │ │                                                                     │ │ 
  │ └─────────────────────────────────────────────────────────────────────┘ │ 
  ├─────────────────────────────────────────────────────────────────────────┤ 
  │        <Select>    < Exit >    < Help >    < Save >    < Load >         │ 
  └─────────────────────────────────────────────────────────────────────────┘

The default scheduler is CFQ, but oddly I don't have an option to change the default to any of the multi-queue ones:
Code:
 .config - Linux/x86 4.12.0 Kernel Configuration
 → Enable the block layer → IO Schedulers ─────────────────────────────────────

       ┌──────────────────── Default I/O scheduler ────────────────────┐
       │  Use the arrow keys to navigate this window or press the      │ 
       │  hotkey of the item you wish to select followed by the <SPACE │ 
       │  BAR>. Press <?> for additional information about this        │ 
       │ ┌───────────────────────────────────────────────────────────┐ │ 
       │ │                       ( ) Deadline                        │ │ 
       │ │                       (X) CFQ                             │ │ 
       │ │                       ( ) No-op                           │ │ 
       │ │                                                           │ │ 
       │ │                                                           │ │ 
       │ │                                                           │ │ 
       │ └───────────────────────────────────────────────────────────┘ │ 
       ├───────────────────────────────────────────────────────────────┤ 
       │                    <Select>      < Help >                     │ 
       └───────────────────────────────────────────────────────────────┘

Well then, CFQ it is … right? I boot up:
Code:
$ dmesg | grep scheduler
[    0.546146] io scheduler noop registered
[    0.546147] io scheduler deadline registered
[    0.546169] io scheduler cfq registered (default)
[    0.546171] io scheduler mq-deadline registered
[    0.546172] io scheduler kyber registered

And check what my schedulers are…
Code:
$ grep -H . /sys/block/*/queue/scheduler
/sys/block/bcache0/queue/scheduler:none
/sys/block/loop0/queue/scheduler:[mq-deadline] kyber none
/sys/block/sda/queue/scheduler:[mq-deadline] kyber none
/sys/block/sdb/queue/scheduler:[mq-deadline] kyber none

That's not what I expected!

How did my scheduler end up being MQ Deadline and why can't I even choose CFQ?
_________________
“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6920

PostPosted: Wed Jul 05, 2017 6:55 am    Post subject: Reply with quote

The menu choice for default is only for legacy single-queue schedulers (the ones above it), the ones at the bottom are multiqueue ones. You can only be using one of the two lists at any time depending on if you have CONFIG_SCSI_MQ_DEFAULT or the corresponding bootparam set. The unused ones are dead code and you may as well remove them from your kernel.

There isn't a menu option for default mq scheduler, it picks a per-device default in block/elevator.c based on whether the hardware reports native multiqueue support or not. If you want to override it, the only way right now is to remove mq-deadline or use an /etc/local.d/ script.
Back to top
View user's profile Send private message
mbar
Veteran
Veteran


Joined: 19 Jan 2005
Posts: 1990
Location: Poland

PostPosted: Wed Jul 05, 2017 7:04 am    Post subject: Reply with quote

Also https://www.phoronix.com/scan.php?page=article&item=linux412-hddssd-io&num=1
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana

PostPosted: Wed Jul 05, 2017 12:35 pm    Post subject: Reply with quote

Phoronix test does not mention what filesystem was used. XFS wiki.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
CaptainBlood
Advocate
Advocate


Joined: 24 Jan 2010
Posts: 3596

PostPosted: Wed Jul 05, 2017 10:36 pm    Post subject: Reply with quote

Jaglover wrote:
Phoronix test does not mention what filesystem was used. XFS wiki.
Are SSD results relevant for /var/tmpfs [RAM] emerging?
Thks 4 ur attention.
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana

PostPosted: Wed Jul 05, 2017 10:53 pm    Post subject: Reply with quote

CaptainBlood wrote:
Thks 4 ur attention.

What you intend to achieve by insulting people?
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
BlueFusion
Guru
Guru


Joined: 08 Mar 2006
Posts: 371

PostPosted: Wed Jul 05, 2017 11:17 pm    Post subject: Reply with quote

I just upgraded to kernel 4.12 today and wanted to take advantage of in-kernel BFQ IO scheduler. Perhaps, I misunderstand it's usefulness, however.

I disabled all IO schedulers except BFQ, which is compiled in-kernel. I added the following to my kernel command line:
Code:
scsi_mod.use_blk_mq=y dm_mod.use_blk_mq=y elevator=bfq


All 3 of my systems I rolled this change out on come up with the "none" scheduler for all drives:
Code:
proton ~ # cat /sys/block/sd*/queue/scheduler
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq

NOTE: This server is entirely HDD. My desktop has the same issue with a single SSD installed.

I can manually activate bfq, or write a script to do so, but why is the elevator kernel option not working?

Code:
proton ~ # echo bfq > /sys/block/sda/queue/scheduler
proton ~ # cat /sys/block/sd*/queue/scheduler
[bfq] none
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq
[none] bfq

_________________
i7-940 2.93Ghz | ASUS P6T Deluxe (v.1) | 24GB Triple Channel RAM | nVidia GTX660
4x 4TB Seagate NAS HDD (Btrfs raid5) | 2x 120GB Samsung 850 EVO SSD (Btrfs raid1)
Back to top
View user's profile Send private message
thumper
Guru
Guru


Joined: 06 Dec 2002
Posts: 552
Location: Venice FL

PostPosted: Wed Jul 05, 2017 11:41 pm    Post subject: Reply with quote

Blk-mq-based device drivers bypass the previous Linux I/O scheduler.

This should get you started in the proper direction.
https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq)

George

[Moderator edit: fixed [url] tag; auto-linking does not work with embedded parentheses. -Hu]
Back to top
View user's profile Send private message
BlueFusion
Guru
Guru


Joined: 08 Mar 2006
Posts: 371

PostPosted: Thu Jul 06, 2017 1:44 am    Post subject: Reply with quote

I was under the impression that there are 3 distinct blk-mq schedulers - Kyber, Deadline-mq, and BFQ (maybe a noop-style one, too?).

If I remove the scsi_mod.use_blk_mq=y dm_mod.use_blk_mq=y from the kernel command line, and just specify elevator=bfq, the system does not boot - it stops just after loading modules. The fact that I can also manually activate the bfq scheduler via the command referenced in my previous post also gives the impression that you can specify specific blk-mq schedulers but it is not possible to activate non-blk-mq schedulers (i.e. CFQ). BFQ is what used to be called bfq-mq, no?

Thanks for the link, George. I stumbled upon that earlier in my Google searching. It seems to be somewhat dated, however. I've read elsewhere that the reason it bypassed all other IO schedulers is simply because they needed to be rewritten to work together or something of the sort. Perhaps that is why we can now switch between them?
_________________
i7-940 2.93Ghz | ASUS P6T Deluxe (v.1) | 24GB Triple Channel RAM | nVidia GTX660
4x 4TB Seagate NAS HDD (Btrfs raid5) | 2x 120GB Samsung 850 EVO SSD (Btrfs raid1)
Back to top
View user's profile Send private message
mbar
Veteran
Veteran


Joined: 19 Jan 2005
Posts: 1990
Location: Poland

PostPosted: Thu Jul 06, 2017 6:03 am    Post subject: Reply with quote

Jaglover wrote:
Phoronix test does not mention what filesystem was used. XFS wiki.


EXT4, it's mentioned in the table on the first page. And in the text of the article:

Quote:
The I/O schedulers were tested on a Linux 4.12 Git kernel on Ubuntu 17.04 x86_64. An EXT4 file-system was used on each drive with the stock mount options. The only changes made during the benchmarking process was changing the I/O scheduler in use.
Back to top
View user's profile Send private message
Etal
Veteran
Veteran


Joined: 15 Jul 2005
Posts: 1931

PostPosted: Sat Jul 08, 2017 6:12 pm    Post subject: Reply with quote

Thanks for the pointers, everyone!

From the source file Ant P. pointed out, it looks like that it completely ignores any parameters with MQ schedulers and just selects "mq-deadline" if available and the the device has 1 hardware queue, or "none" otherwise. So the only way to change it is through an init script.

By the way, is there a way to check how many hardware queues a device has?
_________________
“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010
Back to top
View user's profile Send private message
Juippisi
Developer
Developer


Joined: 30 Sep 2005
Posts: 723
Location: /home

PostPosted: Sun Jul 09, 2017 6:39 am    Post subject: Reply with quote

The mainline kernel didnt update their Kconfig when they added new schedulers, theres a patch that allows you to select BFQ as default in menuconfig.
https://patchwork.kernel.org/patch/9825423/

I think you also have enable few things in kernel config in order for the system to boot with BFQ.
Code:

CONFIG_SCSI_MQ_DEFAULT=y
CONFIG_DM_MQ_DEFAULT=y

-- https://pf.natalenko.name/news/?p=281
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum