Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] how to enable CLEANCACHE and ZCACHE ?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Mon Jul 25, 2011 5:02 pm    Post subject: [SOLVED] how to enable CLEANCACHE and ZCACHE ? Reply with quote

is there any specific way to enable cleancache and zcache features in gentoo-sources-3.0 ?

I've configured CONFIG_CLEANCACHE and CONFIG_ZCACHE in kernel config,
put zcache kernel boot option in grub.conf,
but there's nothing useful in dmesg:

Code:
tnt cleancache # grep CLEANCACHE /usr/src/linux/.config
CONFIG_CLEANCACHE=y
tnt cleancache # cat /var/log/dmesg |grep cleancache
tnt cleancache # grep ZCACHE /usr/src/linux/.config
CONFIG_ZCACHE=y
tnt cleancache # cat /var/log/dmesg |grep zcache
Command line: root=/dev/sda1 zcache vga=0x31B
Kernel command line: root=/dev/sda1 zcache vga=0x31B
tnt cleancache #

_________________
gentoo user


Last edited by tnt on Sat Jul 30, 2011 8:26 pm; edited 1 time in total
Back to top
View user's profile Send private message
pmootrey
n00b
n00b


Joined: 23 Jul 2011
Posts: 15

PostPosted: Mon Jul 25, 2011 11:23 pm    Post subject: Reply with quote

I was also wondering this. I also appended zcache to the kernel line, but I don't see anything in dmesg.
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Wed Jul 27, 2011 12:12 pm    Post subject: Reply with quote

is there anyone successfully using cleancache and zcache ?
_________________
gentoo user
Back to top
View user's profile Send private message
skunk
Guru
Guru


Joined: 28 May 2003
Posts: 581
Location: granada, spain

PostPosted: Wed Jul 27, 2011 3:02 pm    Post subject: Reply with quote

http://lwn.net/Articles/397894/
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Wed Jul 27, 2011 3:58 pm    Post subject: Reply with quote

skunk wrote:
http://lwn.net/Articles/397894/


thx for the link, but...

I've tried compcache/ramzswap/zram before. back then it was ok, but it did not release used memory once drive/swap shrinked.

anyway, right now I'm looking for:

Code:
CONFIG_ZCACHE:                                                                                                                                                                                 Zcache doubles RAM efficiency while providing a significant
performance boosts on many workloads.  Zcache uses lzo1x
compression and an in-kernel implementation of transcendent
memory to store clean page cache pages and swap in RAM,
providing a noticeable reduction in disk I/O.

_________________
gentoo user
Back to top
View user's profile Send private message
tclover
Guru
Guru


Joined: 10 Apr 2011
Posts: 300

PostPosted: Thu Jul 28, 2011 1:40 pm    Post subject: Reply with quote

I'd advice to build zram as a module and not build in in the kernel as it's still... a staging driver with no warranty for whatever reason.
And then try to load the module with argument[s] like `num_devices=4' if ever you wanted four devices. Last time I tried to make use of it quickly discouraged me altogether. I'd see later what I can do with it--I'm primary interested to use for /var/{log,run} and my firefox home profile folder, well var/run is not that important because I can put it in a tmpfs based disk.
Of course you can go read the very outdated documentation on their code.google page.
Note that if your remove the module some `/dev/block/253:*' will remain in the directory--253 is the major number of /dev/zram* devices, you'll have to rm them manually. Then you can try to play with a few features in /sys/block/zram* to setup the size and such. I tried to make a reiserfs or reiser4 based system on a device only to fail. Again, it was a quick test, but the first attempt failed badly!

EDIT: I'm just relizing that I can do all that--set a disksize,mkfs.reiser*--being root! sudo doesn't seem work. Let's see what I can do with it.
Back to top
View user's profile Send private message
pmootrey
n00b
n00b


Joined: 23 Jul 2011
Posts: 15

PostPosted: Fri Jul 29, 2011 11:14 pm    Post subject: Reply with quote

I just found this patch for linux-3.0 that supposedly fixes zcache not loading. Testing now will report back shortly.

https://patchwork.kernel.org/patch/903912/
Back to top
View user's profile Send private message
pmootrey
n00b
n00b


Joined: 23 Jul 2011
Posts: 15

PostPosted: Fri Jul 29, 2011 11:17 pm    Post subject: Reply with quote

Patch fixes the issue.

dmesg now reports:

zcache: cleancache enabled using kernel transcendent memory and compression buddies
zcache: created ephemeral tmem pool, id=0
zcache: created ephemeral tmem pool, id=1

[Update]

Noticeable performance increase.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Jul 30, 2011 1:48 pm    Post subject: Reply with quote

pmootrey wrote:
I just found this patch for linux-3.0 that supposedly fixes zcache not loading. Testing now will report back shortly.

https://patchwork.kernel.org/patch/903912/


awesome thanks !

I had stumbled over that patch back in June but forgot about it

and now wondered why zcache wouldn't work despite zram, zcache and cleancache being enabled in 3.0 :roll:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Sat Jul 30, 2011 4:01 pm    Post subject: Reply with quote

after applying patch:

Code:
zcache: cleancache enabled using kernel transcendent memory and compression buddies
zcache: created ephemeral tmem pool, id=0


so, that's it? now it works?

no need to put some extra settings somewhere?
_________________
gentoo user
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Jul 30, 2011 8:19 pm    Post subject: Reply with quote

tnt wrote:
after applying patch:

Code:
zcache: cleancache enabled using kernel transcendent memory and compression buddies
zcache: created ephemeral tmem pool, id=0


so, that's it? now it works?

no need to put some extra settings somewhere?


yes, that should be all :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Sat Jul 30, 2011 8:26 pm    Post subject: Reply with quote

thx a lot!

I'll mark thread as solved.
_________________
gentoo user
Back to top
View user's profile Send private message
psomas
Developer
Developer


Joined: 23 Sep 2006
Posts: 212
Location: Greece

PostPosted: Sat Jul 30, 2011 8:50 pm    Post subject: Reply with quote

This is a known bug in linux-3.0. Hopefully, it'll be fixed in 3.1, and probably 3.0.1.
_________________
myblog
FOSS NTUA Community
Back to top
View user's profile Send private message
tclover
Guru
Guru


Joined: 10 Apr 2011
Posts: 300

PostPosted: Sun Jul 31, 2011 4:12 am    Post subject: Reply with quote

And to finish puting my 2 cent post, with kernel .38.8{zen,pf}, I have nothing in dmesg as well, however `/sys/kernel/mm/zcache/' are populated with 3 pool folders.

Now, If somebody managed to get something out of zram, do not hesitate to post something.

EDIT: This is not a joke... with my setup with {,s}bin:lib{32,64}:usr:var/portage:var/lib/layman squashed (squashfs+aufs) and running e17, my system is just using a bit over 512MB of RAM with firefox using almost 200MB without taking into account my $HOME/.mozilla/firefox/h*.default which is mounted on 128MB tmpfs! Now, I'm going to see if I can effectively put my profile on a zram based device.
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Tue Aug 02, 2011 2:03 am    Post subject: Reply with quote

on one of my boxes, I've got:

Code:
Aug  2 02:22:26 titan kernel: Command line: root=/dev/md1 rootflags=data=writeback zcache acpi_enforce_resources=lax
Aug  2 02:22:26 titan kernel: Kernel command line: root=/dev/md1 rootflags=data=writeback zcache acpi_enforce_resources=lax
Aug  2 02:22:26 titan kernel: zcache: cleancache enabled using kernel transcendent memory and compression buddies
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=0
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=1
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=2
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=3
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=4
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=5
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=6
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=7
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=8
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=9
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=10
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=11
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=12
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=13
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=14
Aug  2 02:22:26 titan kernel: zcache: created ephemeral tmem pool, id=15
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded
Aug  2 02:22:26 titan kernel: zcache: pool creation failed: max exceeded


I needed more then 16 zcache pools (server with 33 ext4 filesystems), so I've made change in one file to enable more then 16 pools.

here it is, just in case someone's interested:

Code:
--- a/drivers/staging/zcache/zcache_drv.c       2011-08-02 01:30:08.991848159 +0200
+++ b/drivers/staging/zcache/zcache_drv.c       2011-08-02 02:34:25.898160421 +0200
@@ -677,7 +677,7 @@
 static unsigned long zcache_failed_eph_puts;
 static unsigned long zcache_failed_pers_puts;
 
-#define MAX_POOLS_PER_CLIENT 16
+#define MAX_POOLS_PER_CLIENT 36
 
 static struct {
        struct tmem_pool *tmem_pools[MAX_POOLS_PER_CLIENT];

_________________
gentoo user
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Sat Aug 06, 2011 11:57 am    Post subject: Reply with quote

ok, just an info:

I gave up on cleancache on that specific server.

my C2D 4MB L2 cache 2.33GHz CPU cannot cope with the amount of data it needs to compress during heavy sequential IO.

I get 60-70MB/s from my RAID array and both CPU cores are saturated with system (kernel) time.
without cleancache, RAID gives me more then 300MB/s.

so, using cleancache in scenarios with a lot of non-random data throughput can lead to very ugly performance degradation.

guess cleancache/zcache should implement some kind of compression throttling vavle:
if there's available CPU time, then compress;
if there's no avalable CPU time, just store (or throw away);
_________________
gentoo user
Back to top
View user's profile Send private message
tclover
Guru
Guru


Joined: 10 Apr 2011
Posts: 300

PostPosted: Sun Aug 07, 2011 3:38 pm    Post subject: Reply with quote

Hi tnt, you may, and I'd rather say "should" if I can say, post or email your findings to Dan and to the other devs who are working on it. You can get the emails adresses easily on lkml related patch post.

I had a little time to play with zram but I did not manage to use it for my firefox $HOME/.mozilla/firefox/h*.default profile. I can eventually--as root only and it seems sudo command is not safe with zram--set the size of the device; make a fs--I tried reiser4 and reiserfs, I gave up on reiser4 because it was more troublesome to set up;--mount the fs on $HOME/?mozilla/firefox/h*.default profile; remount again with the right `-o gid=$(id -g),uid=$(id -u)' because I had some trouble in the first mount with combining them with `nodev' mount options. And everything should be set to use it as my firefox profile underlying device backend. However, when extracting a tarball archive containing my firefox profile data to it, tar unexpectedly hang on the task for some reason. So I tried to extract the tarball as root with passing the `p' option to see if I could get something and again... tar hang on the task. I could not finish my test with reiserfs because of it.
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Sun Aug 07, 2011 4:32 pm    Post subject: Reply with quote

I've already made a gentoo bug report. I'll report that on lkml, too.
https://bugs.gentoo.org/show_bug.cgi?id=377975

I guess that using any "complicated" fs on top of zram is useless: even linear search through zram should be fast enough and we should not need any B-tree optimisations.

have you tried ext2 or maybe ext4 without a jurnal?
_________________
gentoo user
Back to top
View user's profile Send private message
Ormaaj
Guru
Guru


Joined: 28 Jan 2008
Posts: 316

PostPosted: Mon Aug 15, 2011 5:32 am    Post subject: Reply with quote

Watching this since patching and booting to ext4 - Little formatting:
Code:
while :; do for i in /sys/kernel/mm/cleancache/*; do printf '%-15s %d\n' "${i##*/}:" "$(< $i)"; done; sleep .5; clear; done

Not looking good.
Code:
failed_gets:    244717
flushes:        15416
puts:           0
succ_gets:      0
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Tue Aug 16, 2011 7:49 pm    Post subject: Reply with quote

you have to wait until your memory is filled up with caches/buffers.

try copying some big amount of files (size of files should exceed your RAM).
_________________
gentoo user
Back to top
View user's profile Send private message
Ormaaj
Guru
Guru


Joined: 28 Jan 2008
Posts: 316

PostPosted: Wed Aug 17, 2011 12:26 pm    Post subject: Reply with quote

Sure enough. My page cache usually fills faster.
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1129

PostPosted: Tue Aug 30, 2011 4:06 am    Post subject: Reply with quote

@tclover

just for reference, here's the report of the problem on lkml:
https://lkml.org/lkml/2011/8/17/351

;)
_________________
gentoo user
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2584

PostPosted: Thu Jan 05, 2012 2:04 pm    Post subject: Reply with quote

From the referenced lkml thread and other places, I'm getting the distinct impression that I should just turn off cleancache and zcache until frontswap makes it in. Unfortunately frontswap doesn't appear to have made it into the newly released 3.2, or at least I see nothing for it in the kernel configuration options.

Comment?
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
papu
Guru
Guru


Joined: 25 Jan 2008
Posts: 390
Location: Sota algun pi de l'empordà

PostPosted: Mon May 13, 2013 1:36 pm    Post subject: Reply with quote

hi, i discovered zcache yesterday , i am trying to understand it: seems it's like a compresed buffer on system ram to avoid swap and i/o hd reads.

i have a dude if with a large ram(16GB) system, like my pc, it's necessary to activate zcache or it does not get any benefit.

Code:
dmesg | grep -i zcache
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.9.2-gentoo root=UUID=d9d69626-589a-49a8-b37a-fdabdbfea62f ro radeon.audio=1 zcache
[    0.451075] zcache: using lzo compressor
[    0.451181] zcache: cleancache enabled using kernel transcendent memory and compression buddies
[    0.451215] zcache: cleancache: ignorenonactive = 1
[    0.451242] zcache: frontswap enabled using kernel transcendent memory and compression buddies
[    0.451275] zcache: frontswap: excl gets = 1 active only = 1
[    1.892208] zcache: created ephemeral local tmem pool, id=0
[    1.943475] zcache: destroyed local pool id=0
[    1.995064] zcache: created ephemeral local tmem pool, id=0
[   19.041864] zcache: created ephemeral local tmem pool, id=1
[   19.233923] zcache: created persistent local tmem pool, id=2



thanks so much :)
_________________
--Intel i5 3570k --Asrock z77 ext. 4 --Skill 32GB dd3 --Radeon r9 270 --S.O Gentoo & Win 64bits
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun May 26, 2013 12:16 pm    Post subject: Reply with quote

the following might be useful:

https://www.ibm.com/developerworks/community/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/entry/new_linux_zswap_compression_functionality7?lang=en

http://queen.run.montefiore.ulg.ac.be/~martin/wordpress/wp-content/uploads/2013/02/os-zswap.pdf

https://oss.oracle.com/projects/tmem/dist/documentation/presentations/LSFMM12-zcache-final.pdf
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum