Questions about ZRAM, ZSwap

[kgdrenefort]

Hello,

Today I tried to setup some ZRAM into my main system.

I followed this page: https://wiki.gentoo.org/wiki/Zram

And did actually this step: https://wiki.gentoo.org/wiki/Zram#Using_systemd_zramctl

Which on my system, this was made:

Code: Select all

Mephistopheles ~ # modprobe zram
Mephistopheles ~ # zramctl /dev/zram0 --size 4G --algorithm lz4
Mephistopheles ~ # swapoff -a
Mephistopheles ~ # mkswap /dev/zram0
Configure l'espace d'échange (swap) en version 1, taille = 4 GiB (4294963200 octets)
pas d'étiquette, UUID=8b1bddbb-8c9a-4adb-80eb-0d3e2528cc88
Mephistopheles ~ # swapon /dev/zram0

Previously, I was using a swapfile (/swapfile) of 16GiB, which was as seen disabled (for now) for this process.

Now, I have effectively a 4GiB swap space:

Code: Select all

meself@Mephistopheles ~ $ LC_ALL=C free -m
               total        used        free      shared  buff/cache   available
Mem:           15905       12687        2612        3976        4910        3217
Swap:           4095        4095           0

First question, is the value showed below normal ?

I tried to check if it was effectively used as intented, as showed on the Zram wiki page:

Code: Select all

$ grep zram /proc/swaps
/dev/zram0                              partition	4194300		3391232		-2

First, I wonder why the last value is -2, while on the wiki it shows 16383, which seems to represent in the example 16GiB of swap. Here, I should not have something like 4096 ?

I'm a bit perplexe, since I see a swap space in /dev/zram0 as set earlier (see my example above).

I guess something is not understood by me.

---

Second question, isn't this zram actually zswap ?

As explained on the ZSwap page on the wiki: https://wiki.gentoo.org/wiki/Zswap

Next to zswap there is something related called zram. Zram can also be used to create a swap device for compressed pages in memory.

I'm in big doubt, because I followed the ZRam page and ended up with new swap space, as it seems compressed into my RAM.

Could you help me understand this, please ? 

Regards,
GASPARD DE RENEFORT Kévin

[Zucca]

Zswap tries to compress memory pages to aviod them being swapped. Zswap requires swap to function.
Zram is, like you have observed, a method of creating a virtual (compressed) storage device, that always* resides in RAM.
That's their difference.

* you can specify a backing device for zram device, so that its contents can (automatically, dynamically) be moved into non-virtual device.

As for the priority: You can pass --priority/-p for swapon command to adjust the priority of said swap device.

[pingtoo]

zswap,

Zswap is a lightweight compressed cache for swap pages. It takes pages that are in the process of being swapped out and attempts to compress them into a dynamically allocated RAM-based memory pool. zswap basically trades CPU cycles for potentially reduced swap I/O. This trade-off can also result in a significant performance improvement if reads from the compressed cache are faster than reads from a swap device

zram,

The zram module creates RAM-based block devices named /dev/zram<id> (<id> = 0, 1, ...). Pages written to these disks are compressed and stored in memory itself. These disks allow very fast I/O and compression provides good amounts of memory savings. Some of the use cases include /tmp storage, use as swap disks, various caches under /var and maybe many more

So as you can see, zswap is cache for swap. Pages that need to store in swap could store in zswap first and would be retrieved faster when needed.

zram is just a disk use your computer ram, you can make a swap on top of that disk (ram disk) or you can make a filesystem on the disk and mount for /var/tmp for example.

I use both. I use zram from multiple nodes over NBD then pool them together with LVM and finially make a filesystem on the entire pool and mount it on /var/tmp for my stage4 build process.

I setup zswap for each nodes that is distcc helper in participle in above setup so I can confident that compiler jobs will not run out of memory.

[kgdrenefort]

Hello,

Thanks for the replies.

So, if I get it right:

ZRAM will actually takes some RAM free space and stores data into it, instead of storing it to a disk, which is slower by the way. It could be seen as a virtual disk (/dev/zram0 for example) that is only on the RAM, and will replace the usual process storing data into a real disk, for this virtual, which is in RAM, so it's faster.

ZSwap in the other hand, looking a lot like ZRAM, is a cache space took from free RAM space and would be used once the data are compressed, instead of the usual swap space which is very slow, since it's on disk.

I was wondering why having these two ways, but I think:

ZRAM will allow faster processing of data (example, a big update) because instead of storing temporary data into a disk, it'll do it in a virtual disk (/dev/zramX) which is in RAM.

ZSwap will allow faster processing of data if they were about to be swapped, which is slow. Instead, another space is taken from RAM (but not like with ZRAM, creating a virtual disk device in RAM) and use it as a cache from SWAP. Before anything is actually written to Swap, the system will try to compress and store what it can into this, making things possibly faster.

Is this correct ? I'm annoying, sorry, but I need to understand the difference of both before actually sets it widely on my systems.

Regards,
GASPARD DE RENEFORT Kévin

[pingtoo]

Hello,

Thanks for the replies.

So, if I get it right:

ZRAM will actually takes some RAM free space and stores data into it, instead of storing it to a disk, which is slower by the way. It could be seen as a virtual disk (/dev/zram0 for example) that is only on the RAM, and will replace the usual process storing data into a real disk, for this virtual, which is in RAM, so it's faster.

ZSwap in the other hand, looking a lot like ZRAM, is a cache space took from free RAM space and would be used once the data are compressed, instead of the usual swap space which is very slow, since it's on disk.

I was wondering why having these two ways, but I think:

ZRAM will allow faster processing of data (example, a big update) because instead of storing temporary data into a disk, it'll do it in a virtual disk (/dev/zramX) which is in RAM.

ZSwap will allow faster processing of data if they were about to be swapped, which is slow. Instead, another space is taken from RAM (but not like with ZRAM, creating a virtual disk device in RAM) and use it as a cache from SWAP. Before anything is actually written to Swap, the system will try to compress and store what it can into this, making things possibly faster.

Is this correct ? I'm annoying, sorry, but I need to understand the difference of both before actually sets it widely on my systems.

Regards,
GASPARD DE RENEFORT Kévin

You are correct.

zram/zswap kind of share code inside kernel. so they kind of same thing except one is dedicate for kernel swap function and zram is open to any process that use storage I/O functions.

[sublogic]

[ . . . ] ZRAM will allow faster processing of data (example, a big update) because instead of storing temporary data into a disk, it'll do it in a virtual disk (/dev/zramX) which is in RAM.

If you mean running "mkfs /dev/zramX" and mounting the resulting file system: yes, you can do that, but you would be better off with a tmpfs .

[Zucca]

[ . . . ] ZRAM will allow faster processing of data (example, a big update) because instead of storing temporary data into a disk, it'll do it in a virtual disk (/dev/zramX) which is in RAM.

If you mean running "mkfs /dev/zramX" and mounting the resulting file system: yes, you can do that, but you would be better off with a tmpfs .

Yes. Zram has the overhead of a actual filesystem too. Basically:

• Zram: RAM --> compressor --> filesystem --> files
• tmpfs: RAM --> files

... to put very simply. By default zram contents reside only in RAM, while tmpfs contents can* be swapped.

*)

Code: Select all

       noswap(since Linux 6.4)
              Disables  swap.  Remounts must respect the original
              settings.  By default swap is enabled.

... (this was actually new to me)

[kgdrenefort]

After some testing and a few thinking, I decided to try this for now:

- Decreasing my swapfile from 16GiB to 12GiB
- Setting as zswap 25% of my RAM, so it's 4GiB on 16GiB

Once it was fixed in my system by adding this:

Code: Select all

GRUB_CMDLINE_LINUX="zswap.enabled=1 zswap.compressor=lz4"

And regenerating my grub config, I rebooted.

Now I see a swap space of 12GiB.

And I do not see how to see or test if the zswap is currently working on.

In the next hours or days I could guess that: My system is always using 4 to 8GiB of swap space, if it does not go above 4GiB I could safely assume it's working, specially if I see a bigger usage of RAM than I can usually see.

Looking at free -m output:

Code: Select all

$ LC_ALL=C free -m
               total        used        free      shared  buff/cache   available
Mem:           15905        3216       11468          72        1574       12688
Swap:          12287           0       12287

My system just booted and see there is 1574MiB cached, but I can't say if it's the normal behavior I usually gets.

Any tips regarding this ?

[pingtoo]

You can see zswap setting

Code: Select all

grep "" /sys/module/zswap/parameters/*

and for statistic you can get it from

Code: Select all

grep "" /sys/kernel/debug/zswap/*

[kgdrenefort]

Hello and thanks for your answer.

The output:

Code: Select all

# grep "" /sys/module/zswap/parameters/*
/sys/module/zswap/parameters/accept_threshold_percent:90
/sys/module/zswap/parameters/compressor:lz4
/sys/module/zswap/parameters/enabled:Y
/sys/module/zswap/parameters/exclusive_loads:Y
/sys/module/zswap/parameters/max_pool_percent:20
/sys/module/zswap/parameters/non_same_filled_pages_enabled:Y
/sys/module/zswap/parameters/same_filled_pages_enabled:Y
/sys/module/zswap/parameters/zpool:zbud

Code: Select all

# grep "" /sys/kernel/debug/zswap/*
/sys/kernel/debug/zswap/duplicate_entry:0
/sys/kernel/debug/zswap/pool_limit_hit:0
/sys/kernel/debug/zswap/pool_total_size:910716928
/sys/kernel/debug/zswap/reject_alloc_fail:0
/sys/kernel/debug/zswap/reject_compress_poor:7355
/sys/kernel/debug/zswap/reject_kmemcache_fail:0
/sys/kernel/debug/zswap/reject_reclaim_fail:0
/sys/kernel/debug/zswap/same_filled_pages:58519
/sys/kernel/debug/zswap/stored_pages:467007
/sys/kernel/debug/zswap/written_back_pages:0

First is showing indeed the configuration file, remembering me to show as well this line from /etc/default/grub:

Code: Select all

GRUB_CMDLINE_LINUX="zswap.enabled=1 zswap.compressor=lz4"

For the second output, if I understand:

- pool_total_size:910716928 : Shows in byte the size of my zram pool, if I understood, making ~910MiB. Which after a quick Google search gives me this information from this page:

Total size of the compressed cache pool in bytes.

So, to be precise, it's the actual pool size of my zswap, cached and compressed. This value could I guess grow up for the needs of my system.

If there is a pool size with ~910MiB compressed, it should means actually my system has cached 910 compressed MiB, which is between 1.5 and 3 times the original size of the data, uncompressed ?

Sorry to be such curious, but it's very new tech to me :).

Regards,
GASPARD DE RENEFORT Kévin

[pingtoo]

Did you tried with the shell script in the link you posted?

I don't know much of statistic thing so I don't have any comment of how that report mean.

I think the real question is did it work as you set out for? dose it reduce I/O? does it make thing run faster? (or slower) is it stable?

To me it is good because I mainly use SBC with small memory capacity (4G~8G) and SD storage that is why I wan to use it.

[kgdrenefort]

Hard to say for now. To test how it reacts I runned a compilation against qtwebengine at -j4 instead of -j3, while using heavily my system aside.

For more than a few hours, now, I noticed two some sort big latency, while the load average isn't above 5.XX, the memory is fine and for the swap I have 772MiB reported by htop, while the plasma widget tells me 4.2GiB.

Did not noticed from plasma widget for now any bad things regarding I/O on my severals disks.

htop shows that most of the usage of swap as gray, leading me to think it's cached.

The system is pretty much ok, beside these two latency for each of them about a minute.

I'll do test it more these next days, but it seems pretty much ok here.

Thanks :).

[pingtoo]

try atop instead htop.

I have several version installed, one of them is capable show zswap stat. This one is from alpinelinux.