View previous topic :: View next topic |
Author |
Message |
bmike1 n00b
Joined: 05 Sep 2021 Posts: 22
|
Posted: Mon Sep 06, 2021 4:50 pm Post subject: f2fs or ext4 |
|
|
I was wondering what the recommended file system was for for a ssd seeing as they are basically a flash drive. f2fs or ext4? |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Mon Sep 06, 2021 4:52 pm Post subject: |
|
|
I suggest you try searching the forums using Google
Code: | site:forums.gentoo.org f2fs ext4 |
as this has been discussed at length several times in the past few years _________________ Greybeard |
|
Back to top |
|
|
mike155 Advocate
Joined: 17 Sep 2010 Posts: 4438 Location: Frankfurt, Germany
|
|
Back to top |
|
|
The_Great_Sephiroth Veteran
Joined: 03 Oct 2014 Posts: 1602 Location: Fayetteville, NC, USA
|
Posted: Tue Sep 07, 2021 6:14 pm Post subject: |
|
|
That post is not accurate. Been using F2FS on solid-state media (SATA SSD and M.2 NVME) for ages and it beats the ever-loving pants off of ext4 and even BTRFS. The problem is that people seem to not want to change. ext4 has been around since before solid-state media and cling to it despite it being outdated feature-wise. CoW? No. Snapshots? No. Subvolumes? No. Designed for mechanical or solid-state media? mechanical. It is aging well and I am not against it, but most people simple accept the default (ext4) and assume it is godly without ever trying anything else. I personally left ext4 behind nearly ten years ago. I run BTRFS on mechanical disks and use it for RAID even with SSDs. I use F2FS in single-SSD setups and even on a few USB sticks which are used exclusively by Linux. I even run BTRFS on my Windows rigs to protect my data.
Ext4 is good, but it is dated. Slower than F2FS on solid-state media, far less feature-rich than BTRFS. There are many others which work better also. _________________ Ever picture systemd as what runs "The Borg"? |
|
Back to top |
|
|
bmike1 n00b
Joined: 05 Sep 2021 Posts: 22
|
Posted: Tue Sep 07, 2021 6:23 pm Post subject: |
|
|
very interesting. well, I was going to go against advice and install it on my thumb drive but was not able to. I'll investigate the whys of this when I get around to putting it on the ssd of the computer I am on as we tip-tap! |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21624
|
Posted: Tue Sep 07, 2021 7:14 pm Post subject: |
|
|
Filesystems are difficult to swap out since there is no in-place conversion for most source->destination pairs. Filesystem bugs that cause data loss are seen as very dire for most use cases, so people are generally averse to running a filesystem without a long history of good service. Risk of data loss bugs rises with the complexity of the filesystem, so the more features it offers, the higher the risk that there is a problem somewhere. From the feature angle, people usually acclimate to the currently available features and design their workflows accordingly. Then, when presented with a new feature, such as subvolumes, the first response is to observe that the workflow has no use for the feature. If the workflow has no use for it, then there is no value to switching to a filesystem that offers it. With no added value, but with some theoretical added risk, most people will decide to stay away. To drive adoption of the new generation of filesystems, people need compelling reasons to use those new features. |
|
Back to top |
|
|
mike155 Advocate
Joined: 17 Sep 2010 Posts: 4438 Location: Frankfurt, Germany
|
Posted: Tue Sep 07, 2021 7:44 pm Post subject: |
|
|
The_Great_Sephiroth wrote: | That post is not accurate. Been using F2FS on solid-state media (SATA SSD and M.2 NVME) for ages and it beats the ever-loving pants off of ext4 and even BTRFS. The problem is that people seem to not want to change. ext4 has been around since before solid-state media and cling to it despite it being outdated feature-wise. CoW? No. Snapshots? No. Subvolumes? . |
The most important feature of a filesystem is that it doesn't eat my data. I switched my machines to ext4 12 years ago - and I've never had a broken filesystem or lost data due to a filesystem bug since then. ext4 is perfect!
BTRFS? So many people have lost their data in last 10 years! Even our handbooks warns of this filesystem:
Quote: | Kernels prior to 5.4.y are not guaranteed to be safe to use with btrfs in production because fixes for serious issues are only present in the more recent releases of the LTS kernel branches. Filesystem corruption issues are common on older kernel branches, with anything older than 4.4.y being especially unsafe and prone to corruption. Corruption is more likely on older kernels (than 5.4.y) when compression is enabled. RAID 5/6 and quota groups unsafe on all versions of btrfs. Furthermore, btrfs can counter-intuitively fail filesystem operations with ENOSPC when df reports free space due to internal fragmentation (free space pinned by DATA + SYSTEM chunks, but needed in METADATA chunks). Additionally, a single 4K reference to a 128M extent inside btrfs can cause free space to be present, but unavailable for allocations. This can also cause btrfs to return ENOSPC when free space is reported by df. Installing sys-fs/btrfsmaintenance and configuring the scripts to run periodically can help to reduce the possibility of ENOSPC issues by rebalancing btrfs, but it will not eliminate the risk of ENOSPC when free space is present. Some workloads will never hit ENOSPC while others will. If the risk of ENOSPC in production is unacceptable, you should use something else. If using btrfs, be certain to avoid configurations known to have issues. |
|
|
Back to top |
|
|
pietinger Moderator
Joined: 17 Oct 2006 Posts: 4148 Location: Bavaria
|
|
Back to top |
|
|
The_Great_Sephiroth Veteran
Joined: 03 Oct 2014 Posts: 1602 Location: Fayetteville, NC, USA
|
Posted: Wed Sep 08, 2021 3:48 am Post subject: |
|
|
BTRFS data-loss bugs revolve almost entirely around the yet-to-be-completed RAID 5 and 6. I do not use 5 or 6 anywhere, so I don't care. I normally do 10 or 1. As for the patches, it shows the insane age and lack of maintenance. No patches do not mean perfect. It means lack of development. No standard features of modern file-systems. Heck, by your logic, Fat12 is freaking godly. Seen any patches for it lately? Me either, so it must be good. BTRFS is constantly be added to. New features, better performance, etc etc. I HAVE lost data on ext4 due to bit-rot. I would not have lost the data on BTRFS and I'd have known of corruption on F2FS.
Again, when speaking of losing data on BTRFS it is almost always due to BTRFS RAID 5/6. Filter those out and then show me some data loss. I couldn't find anything about data loss in DUP or SINGLE mode except when a disk died.
I do agree with one thing though. People don't switch if it isn't easy. It doesn't matter how much faster, more secure, or stable something is. If most people can't hit the "easy button" they won't do it. Personally if something came out tomorrow which was as good and fast for an SSD as F2FS is and had the features of BTRFS, I'd wipe and reload. No problem. I'd be down a few days but then I get the best of the two file-systems I use the most. _________________ Ever picture systemd as what runs "The Borg"? |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Wed Sep 08, 2021 8:13 am Post subject: |
|
|
Hu wrote: | Filesystems are difficult to swap out since there is no in-place conversion for most source->destination pairs.... |
The lack of in-place conversion doesn't make it difficult, but agreed it is tedious. It's multi-step process:
- find a spare device with enough space;
- for the paranoid, arrange the filesystem to be read-only - maybe boot in single-user mode and "remount -o ro", maybe boot a thumb drive with a recovery system
- create a tarball with all the right options
- mkfs.foo
- restore tarball
but all but the last two steps are an example of what's known as creating a backup. (I included the link, as this is a sophisticated technique known to few outside a select clique .) The last two steps are what's known as "restoring a backup", something tried even less often .
I do this every time I get a new motherboard; it's useful practice. _________________ Greybeard |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Wed Sep 08, 2021 8:19 am Post subject: |
|
|
In answer to the OP, I'm a fan of f2fs; been using it for years on Raspberry Pi's with SD cards, the laptop's SSD drive on which this post is being composed, and a big floorstander's NVMe root filesystem. Apart from one dead SD card in very early days, never had a problem. It seems very good at handling unscheduled power-offs, but you'd expect that as it's a journalling file system
(Oh, and maybe using it on my Android phone; IIUC Google use f2fs for phones, but I've not checked what mine actually uses.) _________________ Greybeard |
|
Back to top |
|
|
pa4wdh l33t
Joined: 16 Dec 2005 Posts: 812
|
Posted: Wed Sep 08, 2021 9:24 am Post subject: |
|
|
I've been using f2fs on my RPi for a few years now, it works nicely at the sdcard.
All my other disks (actual disks, usb drives and ssd's) are all ext*, not all necessarily ext4 since some are quite old . I agree ext* doesn't have all the fancy features, but in my case i don't want or need them. _________________ The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3342 Location: Rasi, Finland
|
Posted: Wed Sep 08, 2021 10:26 am Post subject: |
|
|
f2fs doesn't like sudden power outages, so it fits for laptops pretty well, but for others have UPS and backups.
EDIT: Always have backups, unless you really can afford to lose you data. _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Wed Sep 08, 2021 10:48 am Post subject: |
|
|
I happened across the Gentoo wiki article on NVMe, which refers to the one for SSDs. A lot to read about mkfs.ext format options, discard and trim support etc. An advantage of f2fs is that you don't need to worry about these, as it's SSD-aware, and (OK, it's a presumption) should do the right thing by these. The wiki stuff about reducing writes remains valid on f2fs, though IIUC modern NVMes and SSDs have very long lifetimes under normal use patterns.
My 500GB Samsung EVO 970 plus is warranted for writing 300 TB; I've written 1.3 GB in 202 hours power-on time according to smartctl, over the 20 months I've had it installed as rootfs (but not /home, that's on HDD), which gives a lifetime of 300/1.3*20/12=385 years. (I don't trust that 202 hours power-on time; at a guess it should be about 10 times larger). _________________ Greybeard |
|
Back to top |
|
|
pa4wdh l33t
Joined: 16 Dec 2005 Posts: 812
|
Posted: Wed Sep 08, 2021 1:57 pm Post subject: |
|
|
I wouldn't call F2FS SSD aware. It's a filesystem intended for raw flash devices, the ones that usually show op as mmcblk devices. These devices usually don't do the clever stuff modern SSD's do, like wear leveling, sector remapping, etc.. Instead F2FS tries to do those things for you. So having two layers (one in the SSD and one in F2FS) trying to do similar things without knowing what the other does might have a different effect than you think (but you don't know what because the firmware in SSD's is usually not readable for us). _________________ The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Wed Sep 08, 2021 4:27 pm Post subject: |
|
|
My (old) android phone uses ext4fs for it's 64G emmc.
Me I keep things simple, modern SSDs were designed to be replacements for regular hard drives and have excellent wear leveling, so run any filesystem you want on it. ext4fs works just fine.
However when it comes to compromised flash storage like SD-cards and raw MTD devices, F2FS is helpful to necessary. I put SD cards in this category only because some of them do not do wear leveling properly.
IMHO flash memory hardware that doesn't or poorly internally wear levels is extremely risky. Unless the erase metadata is kept on an out of band device, if someone cat /dev/zero >/dev/your_f2fs_mtd ... you lose all your wear leveling information and technically your memory device is tough to trust as you'll have no idea how many write cycles are on each page anymore.
All other things being equal it shouldn't hurt running f2fs on a fully wear leveled device. Just that it does have a bit of write amplification as it would be writing unnecessary metadata. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3342 Location: Rasi, Finland
|
Posted: Wed Sep 08, 2021 6:26 pm Post subject: |
|
|
pa4wdh wrote: | It's a filesystem intended for raw flash devices, the ones that usually show op as mmcblk devices. | My "sources" may be wrong, but from what I've gathered around the web f2fs is made for flash devices with controller. Meaning the normal SSDs.
Then for raw flash there are JFFS2, UBIFS and YAFFS ... for example (is there more..?).
Arch Wiki - F2FS wrote: | F2FS (Flash-Friendly File System) is a file system intended for NAND-based flash memory equipped with Flash Translation Layer. Unlike JFFS or UBIFS it relies on FTL to handle write distribution. It is supported from kernel 3.8 onwards.
An FTL is found in all flash memory with a SCSI/SATA/PCIe/NVMe interface, opposed to bare NAND Flash and SmartMediaCards. | Also Gentoo wiki article states quite the same.
I've seen this "F2FS is for raw flash" statement quite often. There must be a reason for it. Was F2FS originally only for raw flash? Or does it support raw flash too? _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Thu Sep 09, 2021 1:25 am Post subject: |
|
|
hmm. then it almost seems it's a solution looking for a problem, unless the main problem is corruption due to power outages which shouldn't happen often and standard SSDs handle with seconds of extra power to ensure things get written in a consistent state. The wear leveling is the main problem that needs to be addressed when using flash, everything else is just gravy.
Perhaps ensuring things that are likely to be deleted together should best be stuck on the same erase page helps, but eventually when it comes to running out of space, you'll still need to do garbage collection because it's difficult to predict what will be deleted together. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
pa4wdh l33t
Joined: 16 Dec 2005 Posts: 812
|
Posted: Sat Sep 11, 2021 10:29 am Post subject: |
|
|
@Zucca: It seems you are right, maybe f2fs has changed its goals. Every source i can find now also states it's for SSD's.
I'm still struggling why the FS needs to be aware of an SSD while the SSD itself goes through the effort of presenting a "simple" block device and doing wear leveling etc. The only thing left is trim, and opinions on that differ. _________________ The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
Back to top |
|
|
Tony0945 Watchman
Joined: 25 Jul 2006 Posts: 5127 Location: Illinois, USA
|
Posted: Sat Sep 11, 2021 2:34 pm Post subject: |
|
|
pa4wdh wrote: | I'm still struggling why the FS needs to be aware of an SSD while the SSD itself goes through the effort of presenting a "simple" block device and doing wear leveling etc. |
Aping Windows where everything is connected to everything. The UNIX philosophy of minimal sized single purpose programs connected by well tested generic mechanisms is dead to today's software designers who aspire to Gates' greatness. |
|
Back to top |
|
|
Zucca Moderator
Joined: 14 Jun 2007 Posts: 3342 Location: Rasi, Finland
|
Posted: Sat Sep 11, 2021 2:38 pm Post subject: |
|
|
pa4wdh wrote: | I'm still struggling why the FS needs to be aware of an SSD while the SSD itself goes through the effort of presenting a "simple" block device and doing wear leveling etc.. | Me too. pa4wdh wrote: | The only thing left is trim, and opinions on that differ. | Automatic trimming seems to be the feature of f2fs. I just read it from... somewhere. Can't really remember where. I need to recheck that.
Other features of f2fs are compression and checksumming of some sort. And it seems some level on encryption. It also supports multi-device fs at some level (max 8 device iirc).
Maybe the wiki page should list the benefits of f2fs so that users choosing fs for their SSD could make comparisons to other filesystems. _________________ ..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Sat Sep 11, 2021 4:24 pm Post subject: |
|
|
As a thought question, how much of the hardware should be exposed to the software?
Sounds like people don't want wear leveling details exposed to the software? _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Sat Sep 11, 2021 5:11 pm Post subject: |
|
|
Zucca wrote: | pa4wdh wrote: | I'm still struggling why the FS needs to be aware of an SSD while the SSD itself goes through the effort of presenting a "simple" block device and doing wear leveling etc.. | Me too. pa4wdh wrote: | The only thing left is trim, and opinions on that differ. | Automatic trimming seems to be the feature of f2fs. I just read it from... somewhere. Can't really remember where. I need to recheck that.
Other features of f2fs are compression and checksumming of some sort. And it seems some level on encryption. It also supports multi-device fs at some level (max 8 device iirc).
Maybe the wiki page should list the benefits of f2fs so that users choosing fs for their SSD could make comparisons to other filesystems. |
AFAIK, the main benefits are (1) it's a journaling fs, so you should never lose writes (2) it's tuned for performance - it usually leads Phoronix benchmarks (3) internally it manages several queues for garbage collection; IIUC this is smarter than either trim or discard (4) it was originally written by Samsung, so presumably it understands (at least their, probably other) flash storage products well; it has input from Google, as they use it in more recent versions of Android.
The checksum stuff is optional and for metadata only - it was added in response to a paper comparing filesystems which complained that it didn't have checksums... The compression, and also encryption, are Linux kernel features that any fs could support, though at the moment it's only ext4, f2fs and possibly another I'be forgotten. _________________ Greybeard |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21624
|
Posted: Sat Sep 11, 2021 5:17 pm Post subject: |
|
|
eccerr0r wrote: | As a thought question, how much of the hardware should be exposed to the software?
Sounds like people don't want wear leveling details exposed to the software? | To counter the question, is this about details that are exposed so that software has the option of being clever, or detailed that are exposed so that the hardware can be stupid and expect the software to be clever enough to compensate? A flash device with no FTL that merely exposes wear leveling to the OS and expects the OS to implement wear leveling in software is stupid hardware. It can still be good hardware if the software does a good enough job. An SSD that manages FTL internally, and lets you inspect it so you can judge in detail how much wear you have accumulated, is clever hardware that tries to let the software be clever too. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9679 Location: almost Mile High in the USA
|
Posted: Sat Sep 11, 2021 5:38 pm Post subject: |
|
|
In this case, the physical hardware has no idea of how many times it gets erased. There must be some microcontroller or something to keep track of erase cycles, so it's a software problem regardless of how it's being looked at - so whether this software should be brought up so it's visible to the OS or not.
Given good underlying firmware that can tell the OS in excruciating detail what's underneath is also a possibility but ignore this for now... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
|