View previous topic :: View next topic |
Author |
Message |
dman777 Veteran
Joined: 10 Jan 2007 Posts: 1004
|
Posted: Sun Jan 24, 2010 12:37 am Post subject: Where is trim in ext4? How do I know if it's enabled? |
|
|
I have a ssd and I am using ext4. Where is the trim feature? Is it implemented in ext4? If trim is a feature of ext4 and not the kernel, how do I know if it's enabled in ext4? |
|
Back to top |
|
|
cach0rr0 Bodhisattva
Joined: 13 Nov 2008 Posts: 4123 Location: Houston, Republic of Texas
|
|
Back to top |
|
|
poOoch Tux's lil' helper
Joined: 05 Nov 2003 Posts: 77 Location: Ulm / Germany
|
Posted: Sat Feb 27, 2010 11:06 am Post subject: |
|
|
From Documentation/filesystems/ext4.txt in 2.6.33
Code: |
discard Controls whether ext4 should issue discard/TRIM
nodiscard(*) commands to the underlying block device when
blocks are freed. This is useful for SSD devices
and sparse/thinly-provisioned LUNs, but it is off
by default until sufficient testing has been done.
| So to enable it now, use discard mount option for ext4. |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
Posted: Fri Apr 02, 2010 4:51 pm Post subject: |
|
|
I've found a test for TRIM on OCZ forum:
Quote: | if it really work you can test it:
get the used sectors for a file
hdparm --fibmap filename
read a sector from the file eg. with
sudo hdparm --read-sector 66385920 /dev/sda
delete the file and sync
rm filename;sync
and read the sector a second time |
and that should look something like this:
Code: | # dd if=/dev/urandom of=tempfile count=100 bs=512k oflag=direct
100+0 records in
100+0 records out
52428800 bytes (52 MB) copied, 6.47137 s, 8.1 MB/s
# hdparm --fibmap tempfile
tempfile:
filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
byte_offset begin_LBA end_LBA sectors
0 37094400 37196799 102400
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3
0ef9 fa10 b172 89f8 186f 0194 4cb1 e190
d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f
ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f
af71 dea7 ac23 a55d e31e ff83 164e bb76
8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8
90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2
c506 c3ee c120 3732 17eb 6e68 11aa 721c...
# rm tempfile
# sync
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3
0ef9 fa10 b172 89f8 186f 0194 4cb1 e190
d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f
ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f
af71 dea7 ac23 a55d e31e ff83 164e bb76
8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8
90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2...
(wait a few minutes)
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 |
unfortunately, this test shows that there is no trim enabled for me, although I have:
Code: | uname -a
Linux tnt 2.6.33-gentoo #3 SMP Fri Mar 26 12:35:12 CET 2010 x86_64 Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz GenuineIntel GNU/Linux |
Code: | rootfs / rootfs rw 0 0
/dev/root / ext4 rw,noatime,commit=100,barrier=0,nobh,stripe=128,data=writeback,discard 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
rc-svcdir /lib64/rc/init.d tmpfs rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
none /tmp tmpfs rw,relatime,size=2097152k 0 0
none /var/tmp tmpfs rw,relatime,size=2097152k,nr_inodes=1048576 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0 |
manual TRIM of that sector using hdparm works (I get zeroes) and that means SSD is TRIM enabled, but waiting after rm and sync gives no results.
where could I go wrong?
some kernel configuration option missing? _________________ gentoo user |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
Posted: Sat Apr 03, 2010 8:13 am Post subject: |
|
|
removing data=writeback from mount and rootflag grub options resulted in working ext4 TRIM support.
is data=writeback and TRIM hate bug or a feature? _________________ gentoo user |
|
Back to top |
|
|
mbar Veteran
Joined: 19 Jan 2005 Posts: 1990 Location: Poland
|
Posted: Wed Apr 28, 2010 11:45 am Post subject: |
|
|
tnt wrote: | is data=writeback and TRIM hate bug or a feature? |
Is this confirmed? |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
|
Back to top |
|
|
Smeagel n00b
Joined: 09 Apr 2007 Posts: 13
|
Posted: Wed May 12, 2010 12:01 am Post subject: Thx |
|
|
Just wanted to write and say thanks tnt, this post proved helpful for me when setting my ssd up. _________________ Redhat? Really? Sigh. |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
Posted: Wed May 12, 2010 1:19 am Post subject: |
|
|
I'm very happy if I could help anyone adopting this great old technology called Linux and this great new technology called SSD, and mixing them together. _________________ gentoo user |
|
Back to top |
|
|
dmitryilyin n00b
Joined: 08 Apr 2008 Posts: 27 Location: Netherlands
|
Posted: Tue Jul 26, 2011 3:15 pm Post subject: |
|
|
It doesn't work(
I don't get zeros when I hdparm --read-sector after file delete.
Even hdparm --trim-sector-ranges doesn't help, but returns success.
Does trim realy zeros sector or just tells drive to use this sector for wear leveling when needed?
Why some drives have only
"Data Set Management TRIM supported"
But other also have
"Deterministic read ZEROs after TRIM"
Does second mean that drive zeroes trimed secotrs instead of only marking them as unused?
I tried Intel, Kingston and OCZ ssds with same result.
Mounted ext4 with discard,noatime,data=ordered
Kernel 2.6.35-23-generic
Maybe I need AHCI enabled controller instead of my Promise and NForce like in FreeBSD?
By the way to enable trim in FreeBSD 8.2+
newfs -tU /dev/disk or tunefs -t enable /dev/disk
-t Turn on the TRIM enable flag. If enabled, and if the underlying
device supports the BIO_DELETE command, the file system will send
a delete request to the underlying device for each freed block.
The trim enable flag is typically set when the underlying device
uses flash-memory as the device can use the delete command to
pre-zero or at least avoid copying blocks that have been deleted.
It only works with new ada disks using mvs module for marvel controller or modern Intel controller.
If you try mount ad disk kernel will complain that trim is enabled but not supported.
Looks like I need to try newer kernel and better hardware.
Also I cannot get more then 100Mb/s from ssd on Linux box but I can get 220 and more on FreeBSD with modern controller. |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
Posted: Tue Jul 26, 2011 3:23 pm Post subject: |
|
|
you need an AHCI enabled controller and driver to get TRIM command passed to the SSD. _________________ gentoo user |
|
Back to top |
|
|
dmitryilyin n00b
Joined: 08 Apr 2008 Posts: 27 Location: Netherlands
|
Posted: Tue Jul 26, 2011 4:00 pm Post subject: |
|
|
Tried with 39 kernel and Marvel controller (no shure is it AHCI) with same result.
I also noticed that hdparm returns "Deterministic read data after TRIM" for my OCZ and only trim supported for Kingston and Intel.
But some drives return "Deterministic read ZEROs after TRIM".
Maybe first means that drive doesnt zero trimed sectors and second means that it does zero and only supported means that behavior is unknown/unreported? |
|
Back to top |
|
|
tnt Veteran
Joined: 27 Feb 2004 Posts: 1221
|
Posted: Tue Jul 26, 2011 4:15 pm Post subject: |
|
|
nowdays, all OCZ and Intel drives should support TRIM. Kingston is different story (they use many different controllers, including JMicrons... ).
when drive receive TRIM command, it does not have to fill up zeros instantly.
go to BIOS (of the motherboard or controller itself) and try to put Marvell into the AHCI mode.
after that, try booting with "ahci.marvell_enable=1" kernel option.
then, try looking for something similar to this:
Code: | grep AHCI /var/log/dmesg
ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x39 impl SATA mode |
_________________ gentoo user |
|
Back to top |
|
|
dmitryilyin n00b
Joined: 08 Apr 2008 Posts: 27 Location: Netherlands
|
Posted: Tue Jul 26, 2011 4:40 pm Post subject: |
|
|
How do I find out is trim working or not if trim doesnt determenistic zero trimed sectors?
Perhaps test that returned zeros you quoted was with drive that "Deterministic return zeros"?
Or should it zero after "wait few minutes" or after some random writes?
At least FreeBSD writes that "trim is enabled, but not supported" if i use AHCI-less controller/driver, but Linux does say nothing, just mounted ext4 fs with discard option.
I also found that hdparm --security-erase should zero and set as unused entire drive and it does it almost instantly! |
|
Back to top |
|
|
Moriah Advocate
Joined: 27 Mar 2004 Posts: 2351 Location: Kentucky
|
Posted: Fri Sep 02, 2011 7:01 pm Post subject: |
|
|
I have been running an OCZ Vertex 256 GB SSD on my laptop for almost 2 years now. It died night before last. OCZ said to reflash the firmware to fix it, which I have not had a chance to try yet, but when I do, I will have firmware version 1.6 with trim and garbage collection.
I use my laptop for my business, and it contains confidential and proprietary information for both my company and my clients.
I am running luks to encrypt the entire disk. I boot from a USB stick and enter a passphrase to decrypt the drive. I have lvm running on the decrypted device manager device, then xfs on a logical volume under lvm. My SSD firmware has only had garbage collection up till now, as when I bought the drive, trim was still very much bleeding edge, and there was only flakey support under windows7, and none under linux.
Given that garbage collection needs some way to tell whether a sector is being used or not, it must have some knowledge of the filesystem being used on the drive. I would be very surprised if it could make any sense of a fully encrypted disk, since the ssd controller can only see the raw bits on the drive, and has no knowledge of the encryption.
I would even be suprised if the garbage collection firmware could even make sense of an xfs file system anyway. Add to that the fact that there are no partition tables on the drive, since it is running lvm on the decrypted device manager device, and I do not see how garbage collection could possibly work.
It would seem to me the only way the drive could know that a sector was being freed was that a write command had just been issued to that sector. Given that the ssd controller maintains a pool of free chunks of flash above and beyond what is visible to the sata controller on the motherboard of the computer, this allows the drive to still perform reasonably well, but nowhere near what it could do if trim was working properly.
So I see that the ext4 filesystem can now support trim. I also read that the kernel 2.6.37 permits lvm to pass trim commands thru to the ssd, but not with snapshot or crypt targets. That's too bad, because the main reason I use lvm is for snapshots, as they are needed to do a clean backup, which my backup server does every night automatically.
Since my nightly backup uses a snapshot, but does not write to the snapshot logical device, does this mean I am ok to use trim with my read-only snapshots?
How about luks for whole drive encryption? Will that also pass trim commands?
Until this is all resolved, I guess I am stuck with xfs and garbage collection. I really do want trim. _________________ The MyWord KJV Bible tool is at http://www.elilabs.com/~myword
Foghorn Leghorn is a Warner Bros. cartoon character. |
|
Back to top |
|
|
cach0rr0 Bodhisattva
Joined: 13 Nov 2008 Posts: 4123 Location: Houston, Republic of Texas
|
Posted: Fri Sep 02, 2011 9:32 pm Post subject: |
|
|
Moriah wrote: |
How about luks for whole drive encryption? Will that also pass trim commands?
|
ive been reading about this recently, as a nice Thinkpad with SSD just hit craigslist, and I too insist on full disk crypto
googling around, looks like dmcrypt does not yet support TRIM, and while they're planning on adding it (there's a patch already i think?) there are some concerns in doing so as it leaves visible information on which blocks are used/unused.
something like that. I'm too lazy to google again and find the long discussion (heaps of discussions, only one i found that discussed it in detail), but was enough to convince me to keep my rotational disk for the time being. _________________ Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash |
|
Back to top |
|
|
Moriah Advocate
Joined: 27 Mar 2004 Posts: 2351 Location: Kentucky
|
Posted: Fri Sep 02, 2011 9:53 pm Post subject: |
|
|
Well I have been running almost 2 years with an OCZ Vertex 256 GB drive using luks/dmcrypt and lvm with an xfs filesystem, and I've been loving it -- until 2 days ago when the SSD died. It actually started getting flakey 2 days before that, but I just thought it was a filesystem problem because I had to force a power down on it a few days earlier.
I ran xfs_check and it showed problems, so I ran xfs_repair. I then booted and everything seemed happy again, but about 2 days later, I got a segfault during a routine emerge, so I immeadiatly shut down in an orderly fashion and ran xfs_check again. It was sick again. I ran xfs_repair, then xfs_check right afterwards. It still showed one problem area -- I don't recall the details. This was enough for me to say it was time to run mkfs.xfs and reload the filesystem from backup. After I did that, it would not boot. It was my fault. I messed up the perms during the restore.
Being lazy, I then did the hdparm trick to do a data security erase on the drive to reset its free pool. I then wanted to restore a known good backup image -- yes, I ran xfs_check on the image first to be sure -- and when I booted the system to a live usb, I got bios errors saying the sata controller had problems. So I shut down and tried the drive in the other sata slot on my laptop. Same thing. OK, I tried pulling the drive and hot-plugging it after a clean boot without the drive installed. When I hot plugged the drive, I got ugly ATA errors about the drive not being good from gentoo.
I contacted OCZ and they said to reflash the firmware, which I am about to do now. Supposedly, they have seen this before, and a re-flash fixes it. If not, then I will have to RMA the drive. Meanwhile, I am running my laptop on a 7200 RPM HDD, and it sure seems slow after the SSD.
I will post back here after the re-flash and restore of the disk image to say how it went. _________________ The MyWord KJV Bible tool is at http://www.elilabs.com/~myword
Foghorn Leghorn is a Warner Bros. cartoon character. |
|
Back to top |
|
|
Moriah Advocate
Joined: 27 Mar 2004 Posts: 2351 Location: Kentucky
|
Posted: Sat Sep 03, 2011 2:49 am Post subject: |
|
|
It went lousy. In fact, it didn't go at all. The re-flash program, which runs under windoze, detected a data corruption error, and left a pair of files, named good.bin and bad10.bin, in the current directory. I hexdumped them and diffed them with kdiff3, and the errors all look like they occur on 512 byte boundries, like an address line to the memory was stuck or something.
So I emailed OCZ with the screen output from the flashing program and the 2 .bin files. I am waiting to hear from them about it, but with it being Labor Day Weekend, I don'r expect to hear anything before Tuesday. _________________ The MyWord KJV Bible tool is at http://www.elilabs.com/~myword
Foghorn Leghorn is a Warner Bros. cartoon character. |
|
Back to top |
|
|
Stufo76 n00b
Joined: 30 Oct 2011 Posts: 2
|
Posted: Tue Dec 13, 2011 4:59 pm Post subject: |
|
|
tnt wrote: | I've found a test for TRIM on OCZ forum:
Quote: | if it really work you can test it:
get the used sectors for a file
hdparm --fibmap filename
read a sector from the file eg. with
sudo hdparm --read-sector 66385920 /dev/sda
delete the file and sync
rm filename;sync
and read the sector a second time |
and that should look something like this:
Code: | # dd if=/dev/urandom of=tempfile count=100 bs=512k oflag=direct
100+0 records in
100+0 records out
52428800 bytes (52 MB) copied, 6.47137 s, 8.1 MB/s
# hdparm --fibmap tempfile
tempfile:
filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
byte_offset begin_LBA end_LBA sectors
0 37094400 37196799 102400
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3
0ef9 fa10 b172 89f8 186f 0194 4cb1 e190
d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f
ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f
af71 dea7 ac23 a55d e31e ff83 164e bb76
8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8
90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2
c506 c3ee c120 3732 17eb 6e68 11aa 721c...
# rm tempfile
# sync
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3
0ef9 fa10 b172 89f8 186f 0194 4cb1 e190
d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f
ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f
af71 dea7 ac23 a55d e31e ff83 164e bb76
8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8
90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2...
(wait a few minutes)
# hdparm --read-sector 37094400 /dev/sdb
/dev/sdb:
reading sector 37094400: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 |
unfortunately, this test shows that there is no trim enabled for me, although I have:
Code: | uname -a
Linux tnt 2.6.33-gentoo #3 SMP Fri Mar 26 12:35:12 CET 2010 x86_64 Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz GenuineIntel GNU/Linux |
Code: | rootfs / rootfs rw 0 0
/dev/root / ext4 rw,noatime,commit=100,barrier=0,nobh,stripe=128,data=writeback,discard 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
rc-svcdir /lib64/rc/init.d tmpfs rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
none /tmp tmpfs rw,relatime,size=2097152k 0 0
none /var/tmp tmpfs rw,relatime,size=2097152k,nr_inodes=1048576 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0 |
manual TRIM of that sector using hdparm works (I get zeroes) and that means SSD is TRIM enabled, but waiting after rm and sync gives no results.
where could I go wrong?
some kernel configuration option missing? |
Hi, I've done this test, but:
- immediately before sync I have random
- immediately after sync I have all ffff istead of zeroes, and this remains after some minutes
What's up?
Last edited by Stufo76 on Wed Dec 14, 2011 8:35 am; edited 1 time in total |
|
Back to top |
|
|
Moriah Advocate
Joined: 27 Mar 2004 Posts: 2351 Location: Kentucky
|
Posted: Tue Dec 13, 2011 5:57 pm Post subject: |
|
|
Because I am runnung both LVM and dmcrypt on my OCZ Vertex-I 256 GB SSD, I have decided not to use TRIM after all. Because I need to run some additional Vmware Workstation vm's, I purchased an OCZ Vertex-II 480 GB SSD. Turns out the 256 GB drive was really dead, and OCZ replaced it, but I needed more space, so I got the 480 GB drive, which I just installed last Saturday evening. My first use of the additional space was to install w7-32bit for compatibility with my current client.
I wanted to put all the vmdk vmware virtual disks on a LVM logical device with dedup, but I could not find any dedup filesystems that I felt comfortable with. For vmware vmdk virtual disks, a 4 KB chunk size is recommended for a block level dedup scheme. ZFS is massive overkill, trying to be all things to all people. Lessfs and btrfs still do not appear ready for me to trust actual income generating work to. So, I just made another logical device under LVM and formatted it for XFS, and the vmdk's are going there. _________________ The MyWord KJV Bible tool is at http://www.elilabs.com/~myword
Foghorn Leghorn is a Warner Bros. cartoon character. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|