[solved] Massive slowdowns when handling many small files

[NeddySeagoon]

Jimini,

Did you do this on SSD or magnetic HDD?

Writes to SSD are slow if you need to wait for erase cycles, as FLASH memory cells cannot be overwritten.
I don't think forcing an erase is possible. It depends how the drive implements trim.

Its supposed to make notes, then do the erase before it slows down writes.
In some drives, it is an immediate command, which is a bad thing for write amplification.
Use fstrim now and again. No the discard mount option.

Magnetic HDD just overwrite previously written space. No prior slow erase required.

[dmpogo]

Jimini,

Did you do this on SSD or magnetic HDD?

Writes to SSD are slow if you need to wait for erase cycles, as FLASH memory cells cannot be overwritten.
I don't think forcing an erase is possible. It depends how the drive implements trim.

Its supposed to make notes, then do the erase before it slows down writes.
In some drives, it is an immediate command, which is a bad thing for write amplification.
Use fstrim now and again. No the discard mount option.

Magnetic HDD just overwrite previously written space. No prior slow erase required.

But that would not explain the difference between the kernel versions ?

[Jimini]

Did you do this on SSD or magnetic HDD?

Only a RAID6 consisting of 7 HDD (1x Toshiba MG08ACA16TE, 1x Seagate Exos X18, 5x Seagate Exos X16) is affected here.

[NeddySeagoon]

dmpogo,

Correct ... unless the SSDs decided to do a block of erase cycles when the suspect kernel was running. Then it would be coincidence. :)
Anyway, its all rotating rust, so we can rule out random SSD oddities.

Jimini,

I suspect your drives all all OK but can you put

Code: Select all

smartctl -x /dev/...

for each drive onto a pastebin please?
If one drive was being slow, or needing the kernel error handler, it would probably be kicked out of the set.
The error handler makes lots of noise in dmesg.

[Jimini]

Neddy, of course, here you are:
https://paste.gentoo.zip/EhoatfiT
https://paste.gentoo.zip/JMwdci4E
https://paste.gentoo.zip/SdndfX1M
https://paste.gentoo.zip/maMgWzab
https://paste.gentoo.zip/cdbfCRTg
https://paste.gentoo.zip/evchNp90
https://paste.gentoo.zip/9JhwJwe1

I am currently in a hurry and cannot check the results by myself right now, will do this later and, if necessary, edit my post.

Kind regards
Jimini

[NeddySeagoon]

Jimini,

That are all good.

Code: Select all

Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0

and nothing in the error logs either.

That's another thing that it isn't :)

[Jimini]

Dear all,

it's been one and a half months since my last update here. All of my systems are running a 6.12.* kernel, and the initial problem did not occure one single time.
Thus, I have marked this thread as "solved". If someone should be interested in further analysis or troubleshooting, please let me know :)

Best regards
Jimini