Very fast writing to NTFS with kernel 5.15

Message

Irre · Post by **Irre** » Mon Oct 18, 2021 7:20 pm

I have built 5.15.0-rc6 with ntfs3 support. We have no mount command for ntfs3 but /etc/fstab "works":

Linux localhost 5.15.0-rc6-x86_64 #2 SMP Mon Oct 18 14:21:27 CEST 2021 x86_64 Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz GenuineIntel GNU/Linux

cat /etc/fstab | grep ntfs3
/dev/sda2               /tmp2           ntfs3           noauto,prealloc,noatime,rw       0 0

mount /tmp2

mount -v | grep ntfs3
/dev/sda2 on /tmp2 type ntfs3 (rw,noatime,uid=0,gid=0,iocharset=utf8,prealloc)

Test with new kernel:

Code: Select all

sync; echo 3 >/proc/sys/vm/drop_caches; time ( dd if=/dev/zero of=/tmp2/tempfile bs=1M count=2048; sync )
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 0.846204 s, 2.5 GB/s

real    0m4.967s
user    0m0.005s
sys     0m0.847s

Test with old ntfs-3g driver:

Code: Select all

sync; echo 3 >/proc/sys/vm/drop_caches; time ( dd if=/dev/zero of=/tmp2/tempfile bs=1M count=2048; sync )
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 16.3061 s, 132 MB/s

real    0m17.862s  
user    0m0.001s
sys     0m2.596s

Really fast with new kernel

mike155 · Post by **mike155** » Mon Oct 18, 2021 8:52 pm

Hmm. 2.5 GB/s to a SATA device? Please explain!

Irre · Post by **Irre** » Mon Oct 18, 2021 10:28 pm

mike155 wrote:Hmm. 2.5 GB/s to a SATA device? Please explain!

No, 2 GB in 5 s!

mike155 · Post by **mike155** » Mon Oct 18, 2021 10:36 pm

Ah! You compare "real 0m17.862s" with "real 0m4.967s". That makes more sense!

But it's still a huge difference! Is a filesystem driver in userspace so much slower? Or is it just ntfs3g, which is slower than it could be?

Anyway, thanks for posting your measurement results. Very appreciated!

Irre · Post by **Irre** » Mon Oct 18, 2021 10:47 pm

mike155 wrote:Ah! You compare "real 0m17.862s" with "real 0m4.967s". That makes more sense!

But it's still a huge difference! Is a filesystem driver in userspace so much slower?

Yes ntfs3 seems to be almost 4 times faster than ntfs-3g!

Tony0945 · Post by **Tony0945** » Tue Oct 19, 2021 1:35 am

Irre wrote:
mike155 wrote:Hmm. 2.5 GB/s to a SATA device? Please explain!
No, 2 GB in 5 s!

Sounds reasonable. I get around 450MB/sec on my SSD's.
NVME is over 1100MB/sec

Atha · Post by **Atha** » Thu Oct 21, 2021 11:14 am

Well, comparing apples and oranges, because ntfs-3g runs in user mode (FUSE, file system in user mode) and ntfs3 runs in kernel mode. Naturally a file system driver is faster in-kernel...

But is ntfs3 as compatible as ntfs-3g is in a real life scenario?

Irre · Post by **Irre** » Thu Oct 21, 2021 1:26 pm

Atha wrote:Well, comparing apples and oranges, because ntfs-3g runs in user mode (FUSE, file system in user mode) and ntfs3 runs in kernel mode. Naturally a file system driver is faster in-kernel...

But is ntfs3 as compatible as ntfs-3g is in a real life scenario?

Ntfs3 is much better! I tested to write 800 gb data to an external usb disk. Fast and unfragmented result.

dmpogo · Post by **dmpogo** » Thu Oct 21, 2021 8:13 pm

Atha wrote:Well, comparing apples and oranges, because ntfs-3g runs in user mode (FUSE, file system in user mode) and ntfs3 runs in kernel mode. Naturally a file system driver is faster in-kernel...

But is ntfs3 as compatible as ntfs-3g is in a real life scenario?

As far as it goes, it is not apple and oranges. It is two alternative tools expected to do the same job. If one is as feature full ( and here is a question mark ) and faster, it is undeniably better.

Atha · Post by **Atha** » Sun Oct 24, 2021 5:01 pm

Okay, from what I've read now about it, it seems to be true. NTFS3 is a complete rewrite of an NTFS3.1 driver for the Linux kernel by Paragon. (The old driver "ntfs" was never really capable of writing to an NTFS partition, and –the worst part– development had stopped a long time ago... Which was the reason NTFS-3G got developed BTW.)

Paragon in the past had its own proprietary NTFS driver for various operating systems, like Mac OS X. My perception has been that some people had problems with this driver, and I trusted NTFS-3G more than the Paragon NTFS driver.

NTFS3 on the other hand is a complete rewrite... and it will be 100% open source and the Paragon developers already incorporated changes requested by the kernel developers, hence it certainly meets the quality standards for Linux kernel code.

Me, personally, I will stick to the well tested NTFS-3G FUSE implementation for now. Let's see how the new driver performs when it gets tested more thoroughly after the release of 5.15. Also, let's see if the in-kernel driver gets the same useful functions that NTFS-3G has, like hide_dot_files (which I personally just love) and the user mapping.