View previous topic :: View next topic |
Author |
Message |
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Tue Feb 15, 2011 8:59 pm Post subject: |
|
|
Update8:
(re-uploaded & added missing patches to changelog and patch(set))
Everyone's advised to update to v16 !
(as always - this is a cumulative update of all patches from the beginning)
Quote: | changes (for v15 -> v16):
"25 kztmem" --> "25 zcache"
2 [PATCH] staging: zcache: fix memory leak
3 [PATCH] zcache: Fix build error when sysfs is not defined
27 even more changes ->
13 fs: remove 8 bytes of padding from block_device on 64bit builds
14 (V2) Fix prlimit64 for suid_sgid processes
23 post-2.6.37-fixes -> 4 ext4 -> 12 new_v16
1 [patch] ext4: off by one check in ext4_groupinfo_create_slab()
3 [PATCH] jbd2: call __jbd2_log_start_commit with j_state_lock write locked
4 [PATCH] ext4: allow inode_readahead_blks=0 (linux-2.6.37)
20 2.6.38_patches -> 7 CFS -> 11 v16_sd_idle and first_idle_cpu
1 sched-Resolve-sd_idle-and-first_idle_cpu-Catch-22---v1
2 [PATCH] sched: Wholesale removal of sd_idle logic
20 2.6.38_patches -> 13_timer ->
5 (feb6) [GIT\ PULL\]\ timer\ fixes.txt
20 2.6.38_patches -> 15 x86\ platform ->
8 [PATCH\]\ x86\:\ Fix\ io_bitmap_ptr\ memory\ leak\ on\ copy_process\(\)
32 linux-phc_phc-intel
phc-intel-0.3.2.12.1-2.6.37.patch
"31 btrfs"
-> included latest btrfs changes from 2.6.37 to 2.6.38-rc4 (as of February 15th) (commit: c26a920373a983b52223eed5a13b97404d8b4158 (as of February 15th) in http://git.eu.kernel.org/?p=linux/kernel/git/mason/btrfs-unstable.git;a=summary)
1 btrfs-2.6.38-rc4_feb-15th.patch
2 [PATCH] fix (latent?) memory corruption in btrfs_encode_fh() |
you can try to use btrfs with zcache but most likely a (yet to be fixed) BUG will be triggered: https://forums.gentoo.org/viewtopic-p-6571799.html#6571799
upstream has been notified - no solution or patch to fix it has been posted yet
- more efficiency in CFS (cpu scheduler): removed cruft from HT scheduler
- more memory efficiency: fixed memory leaks in zcache, filesystem, and "fs: remove 8 bytes of padding from block_device on 64bit builds"
- latest btrfs from February 15th
- fixes for ext4 & ability to disable inode_readahead_blks (inode_readahead_blks=0)
- linux-phc (undervolting of the CPU to prolong battery runtime on note- and netbooks)
kudos to all devs making this possible ! _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004
Last edited by kernelOfTruth on Fri Feb 18, 2011 1:13 am; edited 1 time in total |
|
Back to top |
|
|
raj7095 n00b
Joined: 08 Feb 2011 Posts: 59
|
Posted: Thu Feb 17, 2011 11:49 pm Post subject: |
|
|
whoa whoa. did i just see over- and underclocking? holy ****. that is amazing. does it overclock higher than the intended speed or just adjusting the amount of voltage within a clock speed limit? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
raj7095 n00b
Joined: 08 Feb 2011 Posts: 59
|
Posted: Fri Feb 18, 2011 4:08 am Post subject: |
|
|
not a bad feature still. but, how do i change the voltage? |
|
Back to top |
|
|
raj7095 n00b
Joined: 08 Feb 2011 Posts: 59
|
Posted: Fri Feb 18, 2011 4:29 am Post subject: |
|
|
btw, very good job on the latest update. my performance improved just the amount that the previous kernel was lacking, that small imperfection that bothered me is gone. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
energyman76b Advocate
Joined: 26 Mar 2003 Posts: 2048 Location: Germany
|
Posted: Fri Feb 18, 2011 8:24 pm Post subject: |
|
|
reed1 wrote: | why ?? I'm moving my partition to reiser4 as I type this . Seeing u'r last statement , is reiser4 not recommended for everyday use ? I did not see u/anyone else mention that in other reiser4 thread |
I have several raid1 and raid5 partitons + a ssd running reiser4 with lzo and gzip compression. And I am fine. Just make sure to monitor reiserfs-devel. 2.6.36+latest r4 patch is a very good solution. _________________ Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
I identify as a dirty penismensch. |
|
Back to top |
|
|
raj7095 n00b
Joined: 08 Feb 2011 Posts: 59
|
Posted: Sat Feb 19, 2011 12:20 am Post subject: |
|
|
kerneloftruth, how do you install catalyst on a new kernel. my friend installed this kernel and his display doesnt work. he installed catalyst from ati's website. any way to fix it? |
|
Back to top |
|
|
energyman76b Advocate
Joined: 26 Mar 2003 Posts: 2048 Location: Germany
|
Posted: Sat Feb 19, 2011 12:26 am Post subject: |
|
|
raj7095 wrote: | kerneloftruth, how do you install catalyst on a new kernel. my friend installed this kernel and his display doesnt work. he installed catalyst from ati's website. any way to fix it? |
a) use the ebuilds
b) post Xorg.0.log
c) seriously, if you plan to use binary modules, use as few patches as possible. _________________ Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
I identify as a dirty penismensch. |
|
Back to top |
|
|
aTan Tux's lil' helper
Joined: 06 Jan 2007 Posts: 134 Location: Czech Republic (Ukraine)
|
Posted: Sat Feb 19, 2011 10:16 pm Post subject: |
|
|
Does it have Fbcondecor? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Sat Feb 19, 2011 10:41 pm Post subject: |
|
|
@energyman76b:
thanks !
aTan wrote: | Does it have Fbcondecor? |
not yet - I'll include it in the next update:
spock's devsite <--
just apply: fbcondecor-0.9.6-2.6.37.patch
that should apply cleanly
I haven't tested it since I'm having some issues (with my card) running KMS (having to load the radeon module manually otherwise it would hardlocks - also on vanilla)
but it should work fine if you're using KMS in general: Intel KMS + fbsplash
Quote: | Can fbsplash be used with KMS (Kernel Mode Setting)?
Absolutely! KMS provides a standard framebuffer device which allows fbsplash to operate in the usual fashion. In the case of Intel hardware, the framebuffer driver is called inteldrmfb, and is provided by the Intel DRM module, not by intelfb. In fact, you don't even need to have intelfb enabled if you're using KMS. This also means that in case you want to take advantage of the early silent splash feature (this requires the fbcondecor kernel patch), you need to have the Intel DRM module built into the kernel. |
thanks for testing ! _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
aTan Tux's lil' helper
Joined: 06 Jan 2007 Posts: 134 Location: Czech Republic (Ukraine)
|
Posted: Sun Feb 20, 2011 12:02 am Post subject: |
|
|
So I've tried it (by the way fbcon patch works well) but it is the same like with zen-2.6.37. system hangs when X is being started. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
aTan Tux's lil' helper
Joined: 06 Jan 2007 Posts: 134 Location: Czech Republic (Ukraine)
|
Posted: Sun Feb 20, 2011 12:46 am Post subject: |
|
|
vesafb and nvidia-drivers-260.19.36. I will try it without starting X. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
bingyuan n00b
Joined: 20 Feb 2011 Posts: 13
|
Posted: Sun Feb 20, 2011 1:35 pm Post subject: |
|
|
What's the advantage of your zcache patch? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Sun Feb 20, 2011 2:18 pm Post subject: |
|
|
bingyuan wrote: | What's the advantage of your zcache patch? |
Hi bingyuan !
Welcome to Gentoo & thanks for your interest
do you mean in general ?
it's sort of an advancement of Nitin Gupta's zcache & ramzswap
and the result of a collaboration of Nitin Gupta and Dan Magenheimer
in that patch (cumulative patch) is included frontswap, cleancache and zcache
*) cleancache is kind of the "manager" of those three, providing kind of a plugin-structure ("hooks" in VFS) to add support for several filesystems to work with cleancache - currently there are hooks for ext3, ext4, ocfs2 and btrfs: Cleancache [PATCH 0/7] (was Transcendent Memory): overview , [PATCH V5 0/8] Cleancache: overview
*) frontswap is the equivalent of ramzswap without its disadvantage to having to pre-allocate memory and going through the block device layer (afaik), it's providing a swap-device in your memory [PATCH V3 0/4] Frontswap: overview
*) zcache helps to keep more pages of the (filesystem) page cache longer in the memory by providing an additional cache using lzo compression & thus improving i/o operations [PATCH V2 0/3] drivers/staging: zcache: dynamic page cache/swap
benchmark results:
*) zcacheIOzone (that is from the "old" zcache - the "new" one is kztmem with improvements and renamed to zcache), I haven't done any "benchmarks" with bonnie, iozone, etc. but according to Dan Magenheimer the results with (the new) zcache should be at least comparable or even better than that
*) in my own experience it helps to cut down large i/o operation significantly, e.g. when syncing several hundreds of GiB (of the same backup job to different backup drives one after another) it'll help you to cut & save several minutes
*) the latencies under memory pressure is lower - with this kernel (and zen-kernel) it's significantly improved in addition to that due to the other patchsets - so you can do several things all at the same time with no stalls (of course also depending on your hardware)
technical information:
Dan Magenheimer wrote: | Cleancache can be thought of as a page-granularity victim cache for clean
pages that the kernel's pageframe replacement algorithm (PFRA) would like
to keep around, but can't since there isn't enough memory. So when the
PFRA "evicts" a page, it first attempts to put it into a synchronous
concurrency-safe page-oriented pseudo-RAM device (such as Xen's Transcendent
Memory, aka "tmem", or in-kernel compressed memory, aka "zmem", or other
RAM-like devices) which is not directly accessible or addressable by the
kernel and is of unknown and possibly time-varying size. And when a
cleancache-enabled filesystem wishes to access a page in a file on disk,
it first checks cleancache to see if it already contains it; if it does,
the page is copied into the kernel and a disk access is avoided.
This pseudo-RAM device links itself to cleancache by setting the
cleancache_ops pointer appropriately and the functions it provides must
conform to certain semantics as follows:
Most important, cleancache is "ephemeral". Pages which are copied into
cleancache have an indefinite lifetime which is completely unknowable
by the kernel and so may or may not still be in cleancache at any later time.
Thus, as its name implies, cleancache is not suitable for dirty pages. The
pseudo-RAM has complete discretion over what pages to preserve and what
pages to discard and when. |
Dan Magenheimer wrote: | [PATCH V2 0/3] drivers/staging: zcache: dynamic page cache/swap compression
(Historical note: This "new" zcache patchset supercedes both the
kztmem patchset and the "old" zcache patchset as described in:
http://lkml.org/lkml/2011/2/5/148)
HIGH LEVEL OVERVIEW
Zcache doubles RAM efficiency while providing a significant
performance boost on many workloads.
Summary for kernel DEVELOPERS: Zcache uses lzo1x compression to increase
RAM efficiency for both page cache and swap resulting in a significant
performance increase (3-4% or more) on memory-pressured workloads
due to a large reduction in disk I/O. To do this, zcache uses an
in-kernel (no virtualization required) implementation of transcendent
memory ("tmem"), which has other proven uses and intriguing future uses
as well.
Summary for kernel MAINTAINERS: Zcache is a fully-functional,
in-kernel (non-virtualization) implementation of transcendent memory
("tmem"), providing an in-kernel user for cleancache and frontswap.
The patch is based on 2.6.37 and requires either the cleancache patch
or the frontswap patch or both. The patch is proposed as a staging
driver to obtain broader exposure for further evolution and,
GregKH-willing, is merge-able at the next opportunity. Zcache will
hopefully also, Linus-and-akpm-willing, remove the barrier to merge-ability
for cleancache and frontswap. Please note that there is a dependency
on xvmalloc.[ch], currently in drivers/staging/zram. |
Dan Magenheimer wrote: | Frontswap is so named because it can be thought of as the opposite of
a "backing" store for a swap device. The storage is assumed to be
a synchronous concurrency-safe page-oriented pseudo-RAM device (such as
Xen's Transcendent Memory, aka "tmem", or in-kernel compressed memory,
aka "zmem", or other RAM-like devices) which is not directly accessible
or addressable by the kernel and is of unknown and possibly time-varying
size. This pseudo-RAM device links itself to frontswap by setting the
frontswap_ops pointer appropriately and the functions it provides must
conform to certain policies as follows:
An "init" prepares the pseudo-RAM to receive frontswap pages and returns
a non-negative pool id, used for all swap device numbers (aka "type").
A "put_page" will copy the page to pseudo-RAM and associate it with
the type and offset associated with the page. A "get_page" will copy the
page, if found, from pseudo-RAM into kernel memory, but will NOT remove
the page from pseudo-RAM. A "flush_page" will remove the page from
pseudo-RAM and a "flush_area" will remove ALL pages associated with the
swap type (e.g., like swapoff) and notify the pseudo-RAM device to refuse
further puts with that swap type.
Once a page is successfully put, a matching get on the page will always
succeed. So when the kernel finds itself in a situation where it needs
to swap out a page, it first attempts to use frontswap. If the put returns
non-zero, the data has been successfully saved to pseudo-RAM and
a disk write and, if the data is later read back, a disk read are avoided.
If a put returns zero, pseudo-RAM has rejected the data, and the page can
be written to swap as usual.
Note that if a page is put and the page already exists in pseudo-RAM
(a "duplicate" put), either the put succeeds and the data is overwritten,
or the put fails AND the page is flushed. This ensures stale data may
never be obtained from pseudo-RAM. |
_________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Sun Feb 20, 2011 2:34 pm Post subject: |
|
|
Hi folks,
I forgot:
when you're using rsync a lot and transferring lots of data
the following might improve interactivity (due to less hdd trashing) a lot:
create a directory:
/etc/portage/patches/net-misc/rsync/
go to http://tobi.oetiker.ch/patches/ and download the "rsync-3.0.7-fadvise.patch" named patch
make sure that the permissions are correct
after that on the next emerge that patch should be applied automagically to (e.g.) rsync-3.0.7-r2
since the in-tree ebuild already includes: epatch_user
description:
[PATCH v5 0/4] fadvise(DONTNEED) support
Kudos & thanks to Minchan Kim !
Minchan Kim wrote: | Recently, there was a reported problem about thrashing.
(http://marc.info/?l=rsync&m=128885034930933&w=2)
It happens by backup workloads(ex, nightly rsync).
That's because the workload makes just use-once pages
and touches pages twice. It promotes the page into
active list so that it results in working set page eviction.
So app developer want to support POSIX_FADV_NOREUSE but other OSes include linux
don't support it. (http://marc.info/?l=linux-mm&m=128928979512086&w=2)
By other approach, app developers use POSIX_FADV_DONTNEED.
But it has a problem. If kernel meets page is going on writing
during invalidate_mapping_pages, it can't work.
It makes application programmer to use it hard since they always
consider sync data before calling fadivse(..POSIX_FADV_DONTNEED) to
make sure the pages couldn't be discardable. At last, they can't use
deferred write of kernel so see performance loss.
(http://insights.oetiker.ch/linux/fadvise.html)
In fact, invalidation is very big hint to reclaimer.
It means we don't use the page any more. So the idea in this series is that
let's move invalidated pages but not-freed page until into inactive list.
It can help relcaim efficiency very much so that it can prevent
eviction working set. |
so in a nutshell: in addition to zcache this also helps to keep (relevant) pages in the filesystem cache longer so that it doesn't have to swap and re-read those again and again due to memory pressure (not enough memory) - resulting in a more responsive system
for those zen-kernel users out there: this is also included in the zen-kernel (V5 afaik, right now - the latest; my kernel patch(set) still has V4 but should also work reasonably well) _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
aTan Tux's lil' helper
Joined: 06 Jan 2007 Posts: 134 Location: Czech Republic (Ukraine)
|
Posted: Sun Feb 20, 2011 6:28 pm Post subject: |
|
|
aTan wrote: | So I've tried it (by the way fbcon patch works well) but it is the same like with zen-2.6.37. system hangs when X is being started. |
Removing cdemud resolved all problems. |
|
Back to top |
|
|
bingyuan n00b
Joined: 20 Feb 2011 Posts: 13
|
Posted: Mon Feb 21, 2011 1:14 pm Post subject: |
|
|
Will zcache be merged to zen patchset? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
SlashBeast Retired Dev
Joined: 23 May 2006 Posts: 2922
|
Posted: Mon Feb 21, 2011 5:36 pm Post subject: |
|
|
Nice work.
I would like follow your changes however there is no broken-out tarball, I would like see separated patches so I could apply them as needed. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Feb 21, 2011 5:53 pm Post subject: |
|
|
SlashBeast wrote: | Nice work.
I would like follow your changes however there is no broken-out tarball, I would like see separated patches so I could apply them as needed. |
thanks !
the list of the included patches actually was made specifically for this: to revert a patch(set) that you don't like
how's this different to having them on your harddrive (besides having to download them on your own) ?
the "problem" is that I haven't really got used to git yet
doing this manually can be quite cumbersome
if someone could point me to a nice write-up
I'd be much easier to maintain them - in that case I could also provide correctly applying patches (without rejects) _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
SlashBeast Retired Dev
Joined: 23 May 2006 Posts: 2922
|
Posted: Mon Feb 21, 2011 10:59 pm Post subject: |
|
|
kernelOfTruth wrote: | the list of the included patches actually was made specifically for this: to revert a patch(set) that you don't like |
There is a list? First post is a mess, I downloaded one patch but it is monolitic. Also list is not enough, to revert I need full patches.
kernelOfTruth wrote: | how's this different to having them on your harddrive (besides having to download them on your own) ? |
I need more than 'ext4 fixes' for example, With separated patches I can review changes, with one big patch I can't do it.
You can easly export 'commits' to patches from git. |
|
Back to top |
|
|
bingyuan n00b
Joined: 20 Feb 2011 Posts: 13
|
Posted: Tue Feb 22, 2011 11:13 am Post subject: |
|
|
Would you please make ck's BFS alternative? |
|
Back to top |
|
|
|