Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
strange things with md5sum
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Multimedia
View previous topic :: View next topic  
Author Message
SarahS93
Guru
Guru


Joined: 21 Nov 2013
Posts: 382

PostPosted: Sun Dec 10, 2017 1:22 am    Post subject: strange things with md5sum Reply with quote

in my system i have a disk only for downloads (sdc), in the past i download a lot of stuff to this drive and extract it (with unrar) to my data hdd (sdg).
i extract from sdc with unrar to my backup drive sdh too, and then i run md5sum on both files and become not the same checksum, how can that be possible?

with smartctl i check sdc, sdg and sdh
sdg and sdh are new drives, only two weeks old, all smartctl infos looks good.
but my download/temp drive sdc looks very ungly
Code:
smartctl --all /dev/sdc
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.12.12-gentoo] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate NAS HDD
Device Model:     ST4000VN000-1H4168
Serial Number:    XXXXXXXX
LU WWN Device Id: 5 000c50 067120e15
Firmware Version: SC44
User Capacity:    4.000.787.030.016 bytes [4,00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Sun Dec 10 02:15:54 2017 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  117) seconds.
Offline data collection
capabilities:                    (0x73) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 521) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x10bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   108   099   006    Pre-fail  Always       -       20486048
  3 Spin_Up_Time            0x0003   094   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       85
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   074   060   030    Pre-fail  Always       -       27766222
  9 Power_On_Hours          0x0032   066   066   000    Old_age   Always       -       30079
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       73
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   077   077   000    Old_age   Always       -       23
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   067   051   045    Old_age   Always       -       33 (Min/Max 21/33)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       30
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       85
194 Temperature_Celsius     0x0022   033   049   000    Old_age   Always       -       33 (0 18 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       8
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       8
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0

SMART Error Log Version: 1
ATA Error Count: 23 (device log contains only the most recent five errors)
        CR = Command Register [HEX]
        FR = Features Register [HEX]
        SC = Sector Count Register [HEX]
        SN = Sector Number Register [HEX]
        CL = Cylinder Low Register [HEX]
        CH = Cylinder High Register [HEX]
        DH = Device/Head Register [HEX]
        DC = Device Command Register [HEX]
        ER = Error register [HEX]
        ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 23 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  43d+05:50:41.678  READ FPDMA QUEUED
  e5 00 00 00 00 00 00 00  43d+05:50:41.678  CHECK POWER MODE
  ef 10 02 00 00 00 a0 00  43d+05:50:41.678  SET FEATURES [Enable SATA feature]
  27 00 00 00 00 00 e0 00  43d+05:50:41.678  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
  ec 00 00 00 00 00 a0 00  43d+05:50:41.677  IDENTIFY DEVICE

Error 22 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  43d+05:50:37.606  READ FPDMA QUEUED
  ef 10 02 00 00 00 a0 00  43d+05:50:37.606  SET FEATURES [Enable SATA feature]
  27 00 00 00 00 00 e0 00  43d+05:50:37.606  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
  ec 00 00 00 00 00 a0 00  43d+05:50:37.605  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 00  43d+05:50:37.605  SET FEATURES [Set transfer mode]

Error 21 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  43d+05:50:33.608  READ FPDMA QUEUED
  60 00 08 ff ff ff 4f 00  43d+05:50:33.608  READ FPDMA QUEUED
  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED
  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED
  60 00 08 ff ff ff 4f 00  43d+05:50:33.607  READ FPDMA QUEUED

Error 20 occurred at disk power-on lifetime: 25209 hours (1050 days + 9 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 00 ff ff ff 4f 00  43d+05:50:29.447  READ FPDMA QUEUED
  60 00 80 ff ff ff 4f 00  43d+05:50:29.446  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  43d+05:50:29.271  READ FPDMA QUEUED
  60 00 80 ff ff ff 4f 00  43d+05:50:29.270  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  43d+05:50:28.319  READ FPDMA QUEUED

Error 19 occurred at disk power-on lifetime: 25208 hours (1050 days + 8 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 08 ff ff ff 4f 00  43d+04:30:46.995  READ FPDMA QUEUED
  ef 10 02 00 00 00 a0 00  43d+04:30:46.994  SET FEATURES [Enable SATA feature]
  27 00 00 00 00 00 e0 00  43d+04:30:46.994  READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
  ec 00 00 00 00 00 a0 00  43d+04:30:46.994  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 00  43d+04:30:46.994  SET FEATURES [Set transfer mode]

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

does unrar check the files for corruption?

[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 6339
Location: almost Mile High in the USA

PostPosted: Sun Dec 10, 2017 1:50 am    Post subject: Reply with quote

Yes, rar does have checksums embedded during decompression. Most of the time, if there was a problem reading the file, unrar should flag this.

Usually if there is corruption read from the hard drive, a bad sector warning will pop up during read as the hard drive also has error detection, but the OS will be notified before passing the corrupt data to your application. It does look like your hard drive had problems reading a few sectors so you're rightly inclined to blame it, though it should have reported an i/o error to the application (unrar).

If it didn't flag any errors during decompression possibly it was corrupted in ram buffers, make sure your RAM doesn't have problems (memtest).
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
SarahS93
Guru
Guru


Joined: 21 Nov 2013
Posts: 382

PostPosted: Sun Dec 10, 2017 3:03 am    Post subject: Reply with quote

Memtest is a good idea. I have memtest86+ installed, what lines do i add by hand to my grub.cfg file (grub2) ?

Or is there any way to test the RAM at a running system?

Or how can i disable buffering into RAM by extacting with unrar?

Or is there a way to see i there goes anything wrong in the RAM by extracting files, so i know it and can try to extract it again new.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 6339
Location: almost Mile High in the USA

PostPosted: Sun Dec 10, 2017 6:08 am    Post subject: Reply with quote

There is sys-apps/memtester so you don't need to reboot, but it's not as thorough.

You can keep on uncompressing to the same disk and you get different checksums each time, another red flag.

You should be able to run memtest86+ the same way as any other Linux kernel with grub2. I would imagine grub-mkconfig should automatically pick up memtest86+ for you. Mine automatically used the linux16 directive to run it, the binary was installed to [BOOT]/memtest86plus/memtest .
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Akkara
Administrator
Administrator


Joined: 28 Mar 2006
Posts: 6364
Location: &akkara

PostPosted: Sun Dec 10, 2017 8:10 am    Post subject: Reply with quote

eccerr0r wrote:
There is sys-apps/memtester so you don't need to reboot, but it's not as thorough.

I've seen cases where sys-apps/memtester finds problems that memtest86+ misses. Probably has something to do with the fact that it runs on the live system while other things are going on, so it can spot rare failures that a pure pattern-based tester might not "tickle".

Running sys-apps/memtester + emerge makes a good pair if you suspect hardware problems. Ideally do it in /tmp or somewhere unimportant. I've had a machine hardlock while running that. It had been unnerving enough that I've since moved to ECC RAM. Never had a problem after that.
_________________
The reason there appears to be no god in the world, is because he's overwhelmed constantly pulling all those miracles that are needed to keep all the software we're using, mostly working.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 6339
Location: almost Mile High in the USA

PostPosted: Sun Dec 10, 2017 10:38 am    Post subject: Reply with quote

I was more concerned with the memory that's locked into position by either the kernel or another program, and also it's a bit harder to map a failure to a physical address/specific module due to MMU maps (though it may not be that easy with memtest86+ either).

Either program will at least indicate existence of a problem location if it can access it.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39261
Location: 56N 3W

PostPosted: Sun Dec 10, 2017 10:39 am    Post subject: Reply with quote

SarahS93,

Code:
smartctl --all /dev/sdc
...
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       8


That drive is scrap. It has 8 used sectors that it can no longer read.
If it could read them, even once, they would be relocated.
That's only the failed sectors it knows about too.

Check your warranty status.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
SarahS93
Guru
Guru


Joined: 21 Nov 2013
Posts: 382

PostPosted: Sun Dec 10, 2017 7:46 pm    Post subject: Reply with quote

i know, the drive i very scrap.
is there a way to unrar into md5sum on the fly?
so i must not write the file first and read it than to become the md5sum
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Multimedia All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum