| View previous topic :: View next topic |
| Author |
Message |
SarahS93
l33t
Joined: 21 Nov 2013
Posts: 693
|
 Posted: Sun Dec 10, 2017 1:22 am Post subject: strange things with md5sum |
 |
|
in my system i have a disk only for downloads (sdc), in the past i download a lot of stuff to this drive and extract it (with unrar) to my data hdd (sdg).
i extract from sdc with unrar to my backup drive sdh too, and then i run md5sum on both files and become not the same checksum, how can that be possible?
with smartctl i check sdc, sdg and sdh
sdg and sdh are new drives, only two weeks old, all smartctl infos looks good.
but my download/temp drive sdc looks very ungly
| Code: | smartctl --all /dev/sdc
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.12.12-gentoo] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate NAS HDD
Device Model: ST4000VN000-1H4168
Serial Number: XXXXXXXX
LU WWN Device Id: 5 000c50 067120e15
Firmware Version: SC44
User Capacity: 4.000.787.030.016 bytes [4,00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5900 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Sun Dec 10 02:15:54 2017 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 117) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 521) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x10bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 108 099 006 Pre-fail Always - 20486048
3 Spin_Up_Time 0x0003 094 091 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 85
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 074 060 030 Pre-fail Always - 27766222
9 Power_On_Hours 0x0032 066 066 000 Old_age Always - 30079
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 73
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 077 077 000 Old_age Always - 23
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 067 051 045 Old_age Always - 33 (Min/Max 21/33)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 30
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 85
194 Temperature_Celsius 0x0022 033 049 000 Old_age Always - 33 (0 18 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 8
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
SMART Error Log Version: 1
ATA Error Count: 23 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 23 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 43d+05:50:41.678 READ FPDMA QUEUED
e5 00 00 00 00 00 00 00 43d+05:50:41.678 CHECK POWER MODE
ef 10 02 00 00 00 a0 00 43d+05:50:41.678 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 43d+05:50:41.678 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 43d+05:50:41.677 IDENTIFY DEVICE
Error 22 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 43d+05:50:37.606 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 43d+05:50:37.606 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 43d+05:50:37.606 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 43d+05:50:37.605 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 43d+05:50:37.605 SET FEATURES [Set transfer mode]
Error 21 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 43d+05:50:33.608 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 43d+05:50:33.608 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 43d+05:50:33.607 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 43d+05:50:33.607 READ FPDMA QUEUED
60 00 08 ff ff ff 4f 00 43d+05:50:33.607 READ FPDMA QUEUED
Error 20 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 00 ff ff ff 4f 00 43d+05:50:29.447 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 43d+05:50:29.446 READ FPDMA QUEUED
60 00 00 ff ff ff 4f 00 43d+05:50:29.271 READ FPDMA QUEUED
60 00 80 ff ff ff 4f 00 43d+05:50:29.270 READ FPDMA QUEUED
60 00 00 ff ff ff 4f 00 43d+05:50:28.319 READ FPDMA QUEUED
Error 19 occurred at disk power-on lifetime: 25208 hours (1050 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 ff ff ff 4f 00 43d+04:30:46.995 READ FPDMA QUEUED
ef 10 02 00 00 00 a0 00 43d+04:30:46.994 SET FEATURES [Enable SATA feature]
27 00 00 00 00 00 e0 00 43d+04:30:46.994 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
ec 00 00 00 00 00 a0 00 43d+04:30:46.994 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 43d+04:30:46.994 SET FEATURES [Set transfer mode]
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay. |
does unrar check the files for corruption?
[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu] |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Sun Dec 10, 2017 1:50 am Post subject: |
|
|
Yes, rar does have checksums embedded during decompression. Most of the time, if there was a problem reading the file, unrar should flag this.
Usually if there is corruption read from the hard drive, a bad sector warning will pop up during read as the hard drive also has error detection, but the OS will be notified before passing the corrupt data to your application. It does look like your hard drive had problems reading a few sectors so you're rightly inclined to blame it, though it should have reported an i/o error to the application (unrar).
If it didn't flag any errors during decompression possibly it was corrupted in ram buffers, make sure your RAM doesn't have problems (memtest).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
SarahS93
l33t
Joined: 21 Nov 2013
Posts: 693
|
| Posted: Sun Dec 10, 2017 3:03 am Post subject: |
|
|
Memtest is a good idea. I have memtest86+ installed, what lines do i add by hand to my grub.cfg file (grub2) ?
Or is there any way to test the RAM at a running system?
Or how can i disable buffering into RAM by extacting with unrar?
Or is there a way to see i there goes anything wrong in the RAM by extracting files, so i know it and can try to extract it again new. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Sun Dec 10, 2017 6:08 am Post subject: |
|
|
There is sys-apps/memtester so you don't need to reboot, but it's not as thorough.
You can keep on uncompressing to the same disk and you get different checksums each time, another red flag.
You should be able to run memtest86+ the same way as any other Linux kernel with grub2. I would imagine grub-mkconfig should automatically pick up memtest86+ for you. Mine automatically used the linux16 directive to run it, the binary was installed to [BOOT]/memtest86plus/memtest .
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Akkara
Bodhisattva
Joined: 28 Mar 2006
Posts: 6702
Location: &akkara
|
| Posted: Sun Dec 10, 2017 8:10 am Post subject: |
|
|
| eccerr0r wrote: | | There is sys-apps/memtester so you don't need to reboot, but it's not as thorough. |
I've seen cases where sys-apps/memtester finds problems that memtest86+ misses. Probably has something to do with the fact that it runs on the live system while other things are going on, so it can spot rare failures that a pure pattern-based tester might not "tickle".
Running sys-apps/memtester + emerge makes a good pair if you suspect hardware problems. Ideally do it in /tmp or somewhere unimportant. I've had a machine hardlock while running that. It had been unnerving enough that I've since moved to ECC RAM. Never had a problem after that.
_________________
Many think that Dilbert is a comic. Unfortunately it is a documentary. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Sun Dec 10, 2017 10:38 am Post subject: |
|
|
I was more concerned with the memory that's locked into position by either the kernel or another program, and also it's a bit harder to map a failure to a physical address/specific module due to MMU maps (though it may not be that easy with memtest86+ either).
Either program will at least indicate existence of a problem location if it can access it.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Sun Dec 10, 2017 10:39 am Post subject: |
|
|
SarahS93,
| Code: | smartctl --all /dev/sdc
...
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8 |
That drive is scrap. It has 8 used sectors that it can no longer read.
If it could read them, even once, they would be relocated.
That's only the failed sectors it knows about too.
Check your warranty status.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
SarahS93
l33t
Joined: 21 Nov 2013
Posts: 693
|
| Posted: Sun Dec 10, 2017 7:46 pm Post subject: |
|
|
i know, the drive i very scrap.
is there a way to unrar into md5sum on the fly?
so i must not write the file first and read it than to become the md5sum |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Mon Dec 11, 2017 11:19 am Post subject: |
|
|
unrar supports the "t" command to use its internal checksum to test the rar file:
You could also use the p command
| Code: | | $ unrar p file.rar|md5sum |
though I don't know what it would do with an archive that contains more than one file...
I rarely if ever use rar, this is just a guess...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Wed Feb 07, 2018 1:48 am Post subject: |
|
|
Moved from Multimedia to Kernel & Hardware. Not Multimedia related and appears to be a hardware issue.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
|
Kernel & Hardware
