View previous topic :: View next topic |
Author |
Message |
yoshi314 l33t
Joined: 30 Dec 2004 Posts: 850 Location: PL
|
|
Back to top |
|
|
Jorgo n00b
Joined: 18 Jun 2004 Posts: 62 Location: Bochum, Germany
|
Posted: Wed Aug 04, 2010 3:52 pm Post subject: |
|
|
i tried with gentoo-sources-2.6.35 but the second patch part fails ...
so restored the original file. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Wed Aug 04, 2010 8:06 pm Post subject: |
|
|
Jorgo wrote: | i tried with gentoo-sources-2.6.35 but the second patch part fails ...
so restored the original file. |
yeah, it's failing because naiming of items / stuff changed:
where @line 1337 (interesting line number eh ? )
it was looking for:
Quote: | --- mm/vmscan.c 2010-07-20 11:21:08.000000000 +0800
+++ mm/vmscan.c 2010-08-01 16:47:52.000000000 +0800
@@ -1337,14 +1378,8 @@
nr_reclaimed = shrink_page_list(&page_list, sc, PAGEOUT_IO_ASYNC);
- /*
- * If we are direct reclaiming for contiguous pages and we do
- * not reclaim everything in the list, try again and wait
- * for IO to complete. This will stall high-order allocations
- * but that should be acceptable to the caller
- */
- if (nr_reclaimed < nr_taken && !current_is_kswapd() &&
- sc->lumpy_reclaim_mode) {
+ /* Check if we should syncronously wait for writeback */
+ if (should_reclaim_stall(nr_taken, nr_reclaimed, priority, sc)) {
congestion_wait(BLK_RW_ASYNC, HZ/10);
/* |
it's should be something like (I changed this manually so it would be better if you changed it manually - a real diff will follow later if it was successful):
Quote: | --- mm/vmscan.c 2010-07-20 11:21:08.000000000 +0800
+++ mm/vmscan.c 2010-08-01 16:47:52.000000000 +0800
@@ -1244,14 +1356,8 @@
nr_freed = shrink_page_list(&page_list, sc, PAGEOUT_IO_ASYNC);
- /*
- * If we are direct reclaiming for contiguous pages and we do
- * not reclaim everything in the list, try again and wait
- * for IO to complete. This will stall high-order allocations
- * but that should be acceptable to the caller
- */
- if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->lumpy_reclaim_mode) {
+ /* Check if we should syncronously wait for writeback */
+ if (should_reclaim_stall(nr_taken, nr_freed, priority, sc)) {
congestion_wait(BLK_RW_ASYNC, HZ/10);
/* |
confusing:
nr_reclaimed changed into nr_freed
should_reclaim_stall <-- didn't change
edit:
don't forget the 2nd patch ! _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Wed Aug 04, 2010 8:19 pm Post subject: |
|
|
here you go:
desktop-responsiveness_2.6.35_fix.patch
Code: | --- /usr/src/sources/kernel/zen-upstream/mm/vmscan.c 2010-07-21 17:01:20.911512995 +0200
+++ mm/vmscan.c 2010-08-04 22:11:43.663379966 +0200
@@ -1113,6 +1113,47 @@
}
/*
+ * Returns true if the caller should wait to clean dirty/writeback pages.
+ *
+ * If we are direct reclaiming for contiguous pages and we do not reclaim
+ * everything in the list, try again and wait for writeback IO to complete.
+ * This will stall high-order allocations noticeably. Only do that when really
+ * need to free the pages under high memory pressure.
+ */
+static inline bool should_reclaim_stall(unsigned long nr_taken,
+ unsigned long nr_freed,
+ int priority,
+ struct scan_control *sc)
+{
+ int lumpy_stall_priority;
+
+ /* kswapd should not stall on sync IO */
+ if (current_is_kswapd())
+ return false;
+
+ /* Only stall on lumpy reclaim */
+ if (!sc->lumpy_reclaim_mode)
+ return false;
+
+ /* If we have relaimed everything on the isolated list, no stall */
+ if (nr_freed == nr_taken)
+ return false;
+
+ /*
+ * For high-order allocations, there are two stall thresholds.
+ * High-cost allocations stall immediately where as lower
+ * order allocations such as stacks require the scanning
+ * priority to be much higher before stalling.
+ */
+ if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ lumpy_stall_priority = DEF_PRIORITY;
+ else
+ lumpy_stall_priority = DEF_PRIORITY / 3;
+
+ return priority <= lumpy_stall_priority;
+}
+
+/*
* shrink_inactive_list() is a helper for shrink_zone(). It returns the number
* of reclaimed pages
*/
@@ -1202,15 +1243,8 @@
nr_scanned += nr_scan;
nr_freed = shrink_page_list(&page_list, sc, PAGEOUT_IO_ASYNC);
- /*
- * If we are direct reclaiming for contiguous pages and we do
- * not reclaim everything in the list, try again and wait
- * for IO to complete. This will stall high-order allocations
- * but that should be acceptable to the caller
- */
- if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->lumpy_reclaim_mode) {
- congestion_wait(BLK_RW_ASYNC, HZ/10);
+ /* Check if we should syncronously wait for writeback */
+ if (should_reclaim_stall(nr_taken, nr_freed, priority, sc)) {
/*
* The attempt at page out may have made some |
kudos to Wu Fengguang and KOSAKI Motohiro _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
Jorgo n00b
Joined: 18 Jun 2004 Posts: 62 Location: Bochum, Germany
|
Posted: Thu Aug 05, 2010 6:57 am Post subject: |
|
|
Thanks a lot for your work, but it is not working for me.
I'm using gentoo-sources 2.6.35. You seems to use zen-sources ...
I tried to adopt but i get a reject error.
Code: | mm/vmscan.c | 51 ++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 43 insertions(+), 8 deletions(-)
--- mmotm.orig/mm/vmscan.c 2010-07-20 11:21:08.000000000 +0800
+++ mmotm/mm/vmscan.c 2010-08-01 16:47:52.000000000 +0800
@@ -1113,6 +1113,47 @@ static noinline_for_stack void update_is
}
/*
+ * Returns true if the caller should wait to clean dirty/writeback pages.
+ *
+ * If we are direct reclaiming for contiguous pages and we do not reclaim
+ * everything in the list, try again and wait for writeback IO to complete.
+ * This will stall high-order allocations noticeably. Only do that when really
+ * need to free the pages under high memory pressure.
+ */
+static inline bool should_reclaim_stall(unsigned long nr_taken,
+ unsigned long nr_freed,
+ int priority,
+ struct scan_control *sc)
+{
+ int lumpy_stall_priority;
+
+ /* kswapd should not stall on sync IO */
+ if (current_is_kswapd())
+ return false;
+
+ /* Only stall on lumpy reclaim */
+ if (!sc->lumpy_reclaim_mode)
+ return false;
+
+ /* If we have relaimed everything on the isolated list, no stall */
+ if (nr_freed == nr_taken)
+ return false;
+
+ /*
+ * For high-order allocations, there are two stall thresholds.
+ * High-cost allocations stall immediately where as lower
+ * order allocations such as stacks require the scanning
+ * priority to be much higher before stalling.
+ */
+ if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ lumpy_stall_priority = DEF_PRIORITY;
+ else
+ lumpy_stall_priority = DEF_PRIORITY / 3;
+
+ return priority <= lumpy_stall_priority;
+}
+
+/*
* shrink_inactive_list() is a helper for shrink_zone(). It returns the number
* of reclaimed pages
*/
@@ -1202,15 +1243,8 @@
nr_scanned += nr_scan;
nr_freed = shrink_page_list(&page_list, sc, PAGEOUT_IO_ASYNC);
- /*
- * If we are direct reclaiming for contiguous pages and we do
- * not reclaim everything in the list, try again and wait
- * for IO to complete. This will stall high-order allocations
- * but that should be acceptable to the caller
- */
- if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->lumpy_reclaim_mode) {
- congestion_wait(BLK_RW_ASYNC, HZ/10);
+ /* Check if we should syncronously wait for writeback */
+ if (should_reclaim_stall(nr_taken, nr_freed, priority, sc)) {
/*
* The attempt at page out may have made some |
Code: | patching file mm/vmscan.c
Hunk #1 succeeded at 1194 (offset 81 lines).
patch unexpectedly ends in middle of line
Hunk #2 FAILED at 1243.
1 out of 2 hunks FAILED -- saving rejects to file mm/vmscan.c.rej
|
vmscan.c.rej:
Code: | --- mm/vmscan.c 2010-07-20 11:21:08.000000000 +0800
+++ mm/vmscan.c 2010-08-01 16:47:52.000000000 +0800
@@ -1243,15 +1284,8 @@
nr_scanned += nr_scan;
nr_freed = shrink_page_list(&page_list, sc, PAGEOUT_IO_ASYNC);
- /*
- * If we are direct reclaiming for contiguous pages and we do
- * not reclaim everything in the list, try again and wait
- * for IO to complete. This will stall high-order allocations
- * but that should be acceptable to the caller
- */
- if (nr_freed < nr_taken && !current_is_kswapd() &&
- sc->lumpy_reclaim_mode) {
- congestion_wait(BLK_RW_ASYNC, HZ/10);
+ /* Check if we should syncronously wait for writeback */
+ if (should_reclaim_stall(nr_taken, nr_freed, priority, sc)) {
/*
mm/vmscan.c.rej lines 1-20/20 (END)
|
|
|
Back to top |
|
|
SlashBeast Retired Dev
Joined: 23 May 2006 Posts: 2922
|
|
Back to top |
|
|
Shining Arcanine Veteran
Joined: 24 Sep 2009 Posts: 1110
|
Posted: Thu Aug 05, 2010 1:09 pm Post subject: |
|
|
For any ignorant people like me that are not well versed in the use of patch, you can do the following:
Code: | wget -O desktop-responsiveness_2.6.35_fix.patch http://paste.pocoo.org/raw/r59CezZil6xEbcKH5laK/
cd /usr/src/linux
patch -p6 < $OLDPWD/desktop-responsiveness_2.6.35_fix.patch |
This assumes that you eselected the appropriate kernel.
By the way, I am using sys-kernel/vanilla-sources-2.6.35 and patch is emitting a minor warning:
Quote: | # patch -p6 < /root/desktop-responsiveness_2.6.35_fix.patch
patching file mm/vmscan.c
Hunk #1 succeeded at 1112 (offset -1 lines).
patch unexpectedly ends in middle of line
Hunk #2 succeeded at 1242 with fuzz 1 (offset -1 lines). |
Anyway, I am recompiling my kernel now. I did not know that there was a problem, but I am happy to have it fixed.
Edit: I have an Intel X25-M SSD. Someone at Phoronix claimed dd if=/dev/zero of=test bs=1M count=5024 && rm test -f would bring unpatched systems to a crawl, but I have run it on my system in both a patched and unpatched state and I am having difficulty seeing an improvement. Minor lag is still there when opening programs with that command running. The system can also be unresponsive when switching between open programs with kwin while the command is running. Some lag also occurs in closing programs. The lag does not last more than few seconds, but nevertheless, it is there, regardless of whether or not my system is patched. On the other hand, Google Chromium feels more responsive. I am not sure if it is because my system has not been running long enough for there to be a slow-down (occasionally, I can notice tearing due to repaints), but so far, those issues have not appeared. |
|
Back to top |
|
|
SlashBeast Retired Dev
Joined: 23 May 2006 Posts: 2922
|
Posted: Thu Aug 05, 2010 3:06 pm Post subject: |
|
|
Shining Arcanine wrote: | Someone at Phoronix claimed dd if=/dev/zero of=test bs=1M count=5024 && rm test -f would bring unpatched systems to a crawl, but I have run it on my system in both a patched and unpatched state and I am having difficulty seeing an improvement. |
Orly? 'piotr' at phoronix is me. And I only said, that with this patch I have lags when doing much disk activity. I haven't said that without it is better or worst, I see no improvement. |
|
Back to top |
|
|
NaterGator n00b
Joined: 14 Jan 2008 Posts: 69 Location: Gainesville, Fl
|
Posted: Thu Aug 05, 2010 3:13 pm Post subject: |
|
|
I patched a few minute ago, and copying large (1.3GB) video files across two disks definitely resulted in a GUI slowdown/near halt before the patch. With the patch things are markedly improved, but not perfect.
Looking at IO wait in top still shows some significant stalling, but apparently the patch did make a difference. FWIW I noticed the issue the most when performing IO operations with particularly slow devices, like old USB sticks. |
|
Back to top |
|
|
Shining Arcanine Veteran
Joined: 24 Sep 2009 Posts: 1110
|
Posted: Thu Aug 05, 2010 4:06 pm Post subject: |
|
|
SlashBeast wrote: | Shining Arcanine wrote: | Someone at Phoronix claimed dd if=/dev/zero of=test bs=1M count=5024 && rm test -f would bring unpatched systems to a crawl, but I have run it on my system in both a patched and unpatched state and I am having difficulty seeing an improvement. |
Orly? 'piotr' at phoronix is me. And I only said, that with this patch I have lags when doing much disk activity. I haven't said that without it is better or worst, I see no improvement. |
Sorry, I misread your post at Phoronix. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Thu Aug 05, 2010 6:33 pm Post subject: |
|
|
NaterGator wrote: | I patched a few minute ago, and copying large (1.3GB) video files across two disks definitely resulted in a GUI slowdown/near halt before the patch. With the patch things are markedly improved, but not perfect.
Looking at IO wait in top still shows some significant stalling, but apparently the patch did make a difference. FWIW I noticed the issue the most when performing IO operations with particularly slow devices, like old USB sticks. |
this
I've ZFS on top of dm / luks on an old USB harddrive and there are no interruptions in the GUI noticable anymore !
besides those 2 there are 5 additional patches but I don't have the time to port those since it's somewhat more tricky to do those - hopefully the zen devs will be able to port them
on non-slow volumes it's sometimes better, sometimes worse - overall it's better than before _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
Jorgo n00b
Joined: 18 Jun 2004 Posts: 62 Location: Bochum, Germany
|
Posted: Fri Aug 06, 2010 7:56 am Post subject: |
|
|
This patch for 2.6.35 applies with without error but when i compile:
Code: | CC mm/vmscan.o
CC kernel/sysctl.o
mm/vmscan.c:1163: Fehler: Redefinition von »should_reclaim_stall«
mm/vmscan.c:1122: Anmerkung: Vorherige Definition von »should_reclaim_stall« war hier
mm/vmscan.c:1204: Fehler: Redefinition von »should_reclaim_stall«
mm/vmscan.c:1163: Anmerkung: Vorherige Definition von »should_reclaim_stall« war hier
make[1]: *** [mm/vmscan.o] Fehler 1
make: *** [mm] Fehler 2
|
https://bugzilla.kernel.org/attachment.cgi?id=27314
EDIT: File seems to be patched before by gentoo-patchset.
Working with file from vanilla sources.
Last edited by Jorgo on Fri Aug 06, 2010 8:09 am; edited 1 time in total |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
Jorgo n00b
Joined: 18 Jun 2004 Posts: 62 Location: Bochum, Germany
|
Posted: Fri Aug 06, 2010 9:03 am Post subject: |
|
|
Playing around for 1 hour now with patched system.
Still some hick-ups but not so bad as before.
So if there are some other patches i think they found the first part of the issue but not all ... |
|
Back to top |
|
|
lagalopex Guru
Joined: 16 Oct 2004 Posts: 564
|
Posted: Fri Aug 06, 2010 12:20 pm Post subject: |
|
|
While the original patch on lkml did not remove the "congestion_wait" call, it is removed in all the posted patches in here.
And the patch was not for the mainline 2.6, its for the "-mm tree of the moment"-kernel. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
DaggyStyle Watchman
Joined: 22 Mar 2006 Posts: 5910
|
Posted: Mon Aug 09, 2010 9:33 am Post subject: |
|
|
any known source has this patch? _________________ Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 2998 Location: Bay Area, CA
|
Posted: Mon Aug 09, 2010 8:16 pm Post subject: |
|
|
Couple of question folks:
1. Which patch has all the fixes?
2. Has anybody seen any side effects like reduced throughput or copy speed? |
|
Back to top |
|
|
yoshi314 l33t
Joined: 30 Dec 2004 Posts: 850 Location: PL
|
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
darklegion Guru
Joined: 14 Nov 2004 Posts: 468
|
Posted: Mon Sep 13, 2010 11:42 am Post subject: |
|
|
It seems that using the deadline i/o scheduler is still useful with some systems. I'm using 2.6.35-zen2 (which includes the vmscan patches) and certain large file copies results in the dreaded multiple second pauses with cfq and bfq. Using ionice helps with bfq and cfq, but at the cost of greatly reduced throughput. With deadline the pauses are gone, and throughput seems to be fine. I realise that this may not help with all systems, but it certainly helped in my case.
EDIT: Never mind, the pauses are still there. Reduced somewhat, but still there. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Sep 13, 2010 3:31 pm Post subject: |
|
|
darklegion wrote: | It seems that using the deadline i/o scheduler is still useful with some systems. I'm using 2.6.35-zen2 (which includes the vmscan patches) and certain large file copies results in the dreaded multiple second pauses with cfq and bfq. Using ionice helps with bfq and cfq, but at the cost of greatly reduced throughput. With deadline the pauses are gone, and throughput seems to be fine. I realise that this may not help with all systems, but it certainly helped in my case.
EDIT: Never mind, the pauses are still there. Reduced somewhat, but still there. |
2.6.36 in that regard is much better _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 2998 Location: Bay Area, CA
|
Posted: Mon Sep 13, 2010 6:00 pm Post subject: |
|
|
kernelOfTruth wrote: | darklegion wrote: | It seems that using the deadline i/o scheduler is still useful with some systems. I'm using 2.6.35-zen2 (which includes the vmscan patches) and certain large file copies results in the dreaded multiple second pauses with cfq and bfq. Using ionice helps with bfq and cfq, but at the cost of greatly reduced throughput. With deadline the pauses are gone, and throughput seems to be fine. I realise that this may not help with all systems, but it certainly helped in my case.
EDIT: Never mind, the pauses are still there. Reduced somewhat, but still there. |
2.6.36 in that regard is much better | Does 2.6.36 include the fix from Wu Fengguang? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Sep 13, 2010 6:06 pm Post subject: |
|
|
devsk wrote: | kernelOfTruth wrote: | darklegion wrote: | It seems that using the deadline i/o scheduler is still useful with some systems. I'm using 2.6.35-zen2 (which includes the vmscan patches) and certain large file copies results in the dreaded multiple second pauses with cfq and bfq. Using ionice helps with bfq and cfq, but at the cost of greatly reduced throughput. With deadline the pauses are gone, and throughput seems to be fine. I realise that this may not help with all systems, but it certainly helped in my case.
EDIT: Never mind, the pauses are still there. Reduced somewhat, but still there. |
2.6.36 in that regard is much better | Does 2.6.36 include the fix from Wu Fengguang? |
you mean the posted 2 patches which also were written about on phoronix.com ?
sure, they went in pretty early from the start _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|