ssd wear level counts...

[eccerr0r]

Well I've been using Gentoo for years. And SSDs for years too.

I still use SATA SSDs as primary root disks, and short of a few of the disks, I can't seem to get the wear level count to increase with "normal" use.

One of my SSDs (180GB, Sandforce controller)

Code: Select all

233 Media_Wearout_Indicator 0x0032   100   100   000    Old_age   Always       -       0
241 Host_Writes_32MiB       0x0032   100   100   000    Old_age   Always       -       484905

If this is right, I wrote 14.8 TB to this 180GB SSD and IF wear leveling was perfect, I should have erased every block in the SSD at least 84 times. This is a MLC (2bpc) unit so the expected endurance is around 3000-5000 cycles. Assuming 5000, I should have at least hit 1.7% wear cycles used and that should clearly show up in field 233.

I'm just worried that it's lying and I really have no indication when the disk will run out of erase cycles.

This disk was used as a root disk but subsequently repurposed as a dm-cache for a 4TB RAID with rootfs. Maybe I'm not emerge -e @world often enough? Stop using tmpfs and swap directly to the SSD? Or is something more sinister going on? Or if this is a 10000 cycle MLC (which is kind of stretching it) ... ?

[Anon-E-moose]

Media_Wearout_Indicator should start at 100 and work it's way to 0

Easy, use the Media_Wearout_Indicator instead. Media wearout you ask? Sounds great! It is great. This value starts at 100 and is a percentage of the drive's estimated lifetime with regards to writes. If it's at 100, the drive says you have 100% of your writes left. If it's at 80, you have 80% of life left. Etc etc. Intel uses this value to determine warranty. If it hits 0, your warranty is up.

With my samsung's the counter goes the other way, and it's called wear_level_count.
Edit to add: I was wrong, the samsungs start at 100 counting down also.

[NeddySeagoon]

eccerr0r,

241 Host_Writes_32MiB ... is the amount of data the host commanded the SSD to write.
Than not the same thing as what actually gets written and counts towards wear levelling.
e.g. Null blocks are just thrown away.

Writing zeros is not the same as erased, the drive knows this, so it leaves the block as erased and saves wear.

[pjp]

...

Yikes. I presume the RAW_VALUE is the relevant indicator, not the WORST value. I've tried to minimize writes to the drive, so if using 95% of the drive's life in 5 years with minimized writing is normal, I can't say I'm eager to buy another one.

Code: Select all

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       32581
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       102
177 Wear_Leveling_Count     0x0013   099   099   000    Pre-fail  Always       -       5
235 POR_Recovery_Count      0x0012   099   099   000    Old_age   Always       -       45
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       8615405265

[NeddySeagoon]

pjp,

Ignore the raw values.
The are often a 32 bit quantity of packed bit fields. Unless you know how to decode them. Don't fret :)

The VALUE WORST THRESH fields are all normalised.
The pass criteria is that VALUE and WORST must be greater than THRESH.

As 8615405265 is an odd number it way well be right.
That's 4206740 MB or 4.2TB.

We don't know the FLASH type or the size of the drive but as

Code: Select all

177 Wear_Leveling_Count     0x0013   099   099   000    Pre-fail  Always       -       5 

hasn't changed.
It should be good for 100x the writes you have done. (500 years).
You won't care if it fails then. :)
It will be too small to be useful.

Tell us the drive part number and the size and we can do better.

[eccerr0r]

I don't know about you guys but I don't write that many blocks of all zeroes... so it should be a very small portion of the 14+TBs written to the disk...

[Anon-E-moose]

Value starts at 100 heading towards 1.

Raw value is the number of cycles the NAND media has undergone.

https://www.thomas-krenn.com/en/wiki/SM ... Intel_SSDs

also https://serverfault.com/questions/89095 ... mal-bounds which has a link to above.

[Anon-E-moose]

...

Yikes. I presume the RAW_VALUE is the relevant indicator, not the WORST value. I've tried to minimize writes to the drive, so if using 95% of the drive's life in 5 years with minimized writing is normal, I can't say I'm eager to buy another one.

Code: Select all

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       32581
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       102
177 Wear_Leveling_Count     0x0013   099   099   000    Pre-fail  Always       -       5
235 POR_Recovery_Count      0x0012   099   099   000    Old_age   Always       -       45
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       8615405265

Been using it since the end of 2014

Code: Select all

177 Wear_Leveling_Count     0x0013   098   098   000    Pre-fail  Always       -       17
...
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       12851451228

The Wear Leveling Count (WLC) SMART value gives us all the data we need. The current value stands for the remaining endurance of the drive in percentage, meaning that it starts from 100 and decreases linearly as the drive is written to. The raw WLC value counts the consumed P/E cycles, so if these two values are monitored while writing to the drive, sooner than later we will find the spot where the normalized value drops by one.

[pjp]

...

Thanks. I presumed I was wrong, but for some values, the raw value is the only one that makes sense. 500 years or not, 4.2TB sounds like a lot, but I can't say how much binary updates would have achieved... I guess somewhere less than 4.2TB :)

In 500 years, the bigger concern is most likely going to be whether or not there is an interface that could use it.

The drive is a Samsung SSD 860 EVO 500GB.

[Anon-E-moose]

Mine is a 250g 840. And for a significant portion of it's use I had /usr/portage on it (written daily), so a fair amount of use, still plenty of life left.

Eventually sata connectors will go the way of ide connectors, just not right now. Now when nvme gets a lot more common ....

[eccerr0r]

I can see pjp's and Anon-E-moose's examples are actually "working" - in the case that a known amount of wear has been put on the device and the expected lifetime has gone down a tick. I just am a bit concerned - I wrote over 3x as much data as pjp and over 2x as much as Anon-E-moose, and still haven't seen the tick down on my SMART values, either implying that the endurance of the drive is much higher than expected, or the tracking is not working as expected - and the latter is a bit concerning.

Note that host writes 32MB is because the tick counter in the Sandforce SSD ticks every 65536 (2^16) LBA's instead of every LBA...

I have some Micron-controller SSDs that have their endurance tick counters go down so I'm more confident I know these are actually counting. Just these two Sandforce-controller SSDs I don't know what's going on. Perhaps if I see it tick down one after all these years I'd be more confident.

The above SSD is 10 years old now I believe...

[pjp]

Eventually sata connectors will go the way of ide connectors, just not right now. Now when nvme gets a lot more common ....

Are there nvme devices that don't require direct interface with the motherboard? I prefer the convenience of connectors that add a comfortable working distance from the motherboard :).


@eccerr0r, sorry for the tangent... let me know if it is too much of a detraction or distraction.

[eccerr0r]

It's okay, it's good to know someones disks are working.

Anyway, this is one of my Micron SSDs:

Code: Select all

173 Avg_Erase_Count         0x0033   092   092   010    Pre-fail  Always       -       250

This disk appears to have 13.4TB written (according to a different field in smartctl -x, it does not show up in smartctl -a, unfortunately now who knows what each are based off of...) and this 250GB disk would technically have 53 writes with perfect leveling and zero write amplification. At 250 erases that seems really high compared to theoretical... At 8% used I don't think it's way too bad however, implying that this disk is a 3000-erase cycle disk.
(BTW this Micron SSD I bought second hand... so I don't know what the previous owner had been doing to cause all those P/E cycles.)

[Anon-E-moose]

It's okay, it's good to know someones disks are working.

Anyway, this is one of my Micron SSDs:

Code: Select all

173 Avg_Erase_Count         0x0033   092   092   010    Pre-fail  Always       -       250

This disk appears to have 13.4TB written (according to a different field in smartctl -x, it does not show up in smartctl -a, unfortunately now who knows what each are based off of...) and this 250GB disk would technically have 53 writes with perfect leveling and zero write amplification. At 250 erases that seems really high compared to theoretical... At 8% used I don't think it's way too bad however, implying that this disk is a 3000-erase cycle disk.
(BTW this Micron SSD I bought second hand... so I don't know what the previous owner had been doing to cause all those P/E cycles.)

Not sure which micron/controller, but those that show 173 for avg erase count, also have
202 perc rated life use, might give a clue.

[eccerr0r]

This SSD doesn't have a field 202 unfortunately, but I think enough information is available in 173 to make a judgment. Not so with the Sandforce field 233, at least as of yet.

[juniper]

I hope the OP doesn't mind me posting mine...

Code: Select all

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x0032   095   095   050    Old_age   Always       -       0/52230769
  5 Retired_Block_Count     0x0033   100   100   003    Pre-fail  Always       -       0
  9 Power_On_Hours_and_Msec 0x0032   085   085   000    Old_age   Always       -       13268h+20m+30.390s
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       436
171 Program_Fail_Count      0x000a   100   100   000    Old_age   Always       -       0
172 Erase_Fail_Count        0x0032   100   100   000    Old_age   Always       -       0
174 Unexpect_Power_Loss_Ct  0x0030   000   000   000    Old_age   Offline      -       19
177 Wear_Range_Delta        0x0000   000   000   000    Old_age   Offline      -       2
181 Program_Fail_Count      0x000a   100   100   000    Old_age   Always       -       0
182 Erase_Fail_Count        0x0032   100   100   000    Old_age   Always       -       0
187 Reported_Uncorrect      0x0012   100   100   000    Old_age   Always       -       0
189 Airflow_Temperature_Cel 0x0000   028   048   000    Old_age   Offline      -       28 (Min/Max 13/48)
194 Temperature_Celsius     0x0022   028   048   000    Old_age   Always       -       28 (Min/Max 13/48)
195 ECC_Uncorr_Error_Count  0x001c   120   120   000    Old_age   Offline      -       0/52230769
196 Reallocated_Event_Count 0x0033   100   100   003    Pre-fail  Always       -       0
201 Unc_Soft_Read_Err_Rate  0x001c   120   120   000    Old_age   Offline      -       0/52230769
204 Soft_ECC_Correct_Rate   0x001c   120   120   000    Old_age   Offline      -       0/52230769
230 Life_Curve_Status       0x0013   100   100   000    Pre-fail  Always       -       100
231 SSD_Life_Left           0x0000   095   095   011    Old_age   Offline      -       4294967297
233 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       9067
234 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       8854
241 Lifetime_Writes_GiB     0x0032   000   000   000    Old_age   Always       -       8854
242 Lifetime_Reads_GiB      0x0032   000   000   000    Old_age   Always       -       8685
244 Unknown_Attribute       0x0000   098   098   010    Old_age   Offline      -       7798860

I understand that THRESH < WORST. but I don't get the bottom 4 values. all are just 000 so THRESH, VALUE and WORST are all 000.

[eccerr0r]

I have seen value=worst=threshold=0 before, forgot where, but usually if threshold=0 it means this field isn't tracked for drive health -- because it's impossible to go below 0 in those fields. However I do suspect some drives take equal = fail, so dependent on drive firmware.

So it looks like sandforce have different revs too, hmm. But at least at least yours does show some sort of wear. So it looks like you've written 8.8TiB and have 95% life left.

But depending on the size of this drive, could this be within the margin of error why my 180GB drive, with 14.8TiB written (and this does not include write amplification!), still shows 100% life left?

Unfortunately I no longer have orthogonal write tracking, before the repurposing, I had one large ext4fs which does track writes. But now that it's fully allocated as logical cache volumes in an LVM system, not sure how many lifetime writes it wrote, nevermind I don't have the ext4fs data before I deleted it.

[juniper]

the drive is 256 (though partition manager registers 223).

[NeddySeagoon]

pjp,

The 2018 data sheet says that your drive is TLC and rated 300TB written.
That's the warranty value, so it should be good for at least that.

That's 600 erase cycles.

On the down side, I have two 1TB Samsung SSDs drives that broke at about 400GB written.
Having dealt with Samsung support and got warranty replacements, it's not an experience I want to repeat.

[eccerr0r]

the drive is 256 (though partition manager registers 223).

Yeah you also have a TLC drive here, 600-700 erase cycle lifetime, and sort of jives with my smaller drive but with at least 3000 (5x) erase cycle lifetime. If you used 5% of your lifetime, then mine would be 1% and this might be in the margin of error and my drive should(...better that it does!!!) hit 1% soon... We'll see...

On the down side, I have two 1TB Samsung SSDs drives that broke at about 400GB written.
Having dealt with Samsung support and got warranty replacements, it's not an experience I want to repeat.

Damn...what were the symptoms, did they outright become unusable or bad block galore?

I worry about 1TB SSDs. I have no way of backing them up. My SSDs are all small compared to mechanical disks/RAID and back them up to mechanical disks at the moment...

[pjp]

pjp,

The 2018 data sheet says that your drive is TLC and rated 300TB written.
That's the warranty value, so it should be good for at least that.

That's 600 erase cycles.

On the down side, I have two 1TB Samsung SSDs drives that broke at about 400GB written.
Having dealt with Samsung support and got warranty replacements, it's not an experience I want to repeat.

It seems common practice to try dissuading customers from thinking they can use warranties. Hopefully it will last long enough to not be a problem, and with some luck, serve as a cache device after that.

[eccerr0r]

using questionable SSDs as cache devices is kind of dangerous for me, mainly because I'd want to writeback cache...

[pjp]

I mainly meant anything that wasn't critical. My initial thought was for large packages too big for a tmpfs. Maybe even swap.

[juniper]

Yeah you also have a TLC drive here, 600-700 erase cycle lifetime, and sort of jives with my smaller drive but with at least 3000 (5x) erase cycle lifetime. If you used 5% of your lifetime, then mine would be 1% and this might be in the margin of error and my drive should(...better that it does!!!) hit 1% soon... We'll see...

I also did a long test. here are the results.

Code: Select all

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     13270         -
# 2  Short offline       Completed without error       00%     13268         -

I guess it's fine.

[eccerr0r]

Yeah it should be fine.

I'm just waiting for the first tickdown of life percentage on this 180GB SSD since forever... it's been long in the tooth, not to mention that it's POH field is bugged and the drive has existed since the early 1900's...

I have a hypothesis that I am hesitant to try: Perhaps the reason why it sustained so little "wear" because it is Sandforce versus some other controller. Neddy's suggestion earlier is still possible: while I'm not writing all zeros or ones, perhaps the data I'm writing is highly compressible - text files, web pages, even some binary executables, though few uncompressible files like zip files, MP3s, MPEGs, JPEGs, or encrypted files... and that staved off a lot of wear. Despite this I know I did have *some* large quantity of uncompressible files (game texture map files) but perhaps the ratio still made a huge difference.