Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Volume group input/output error after a while [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
vrghost
n00b
n00b


Joined: 03 Oct 2003
Posts: 48
Location: London

PostPosted: Wed Jul 24, 2013 2:19 pm    Post subject: Volume group input/output error after a while [SOLVED] Reply with quote

So made some updates, including kernel to 3.8.13

All is fine, and it seems to work quite well, except that after a while (believe it is after inactivity, going to say maybe about 10-30 or possibly 60 minutes, the rootvg or the sda disk seems to go offline, and I get Input/Output error on any files, directories after mount point and even if I do a vgscan each of the members of the rootvg reports Input/Output error. It seems like I can continue using the root filesystem, even if it is mounted on the same disk.

I booted the system on a live cd (knopix) and ran fsck on all disks, no real problems.
I can reboot the system and the disk seems perfectly happy for about whatever time it is, then after whatever the time is, it just starts giving me Input Output errors

If I boot it on 3.7.10 it seems perfectly happy though, so seems like it may be a kernel issue, but at a loss where to start and what might be worth looking at, so was wondering if anyone else had had the same issue.
Or if someone have any suggestions about where to start.

The disk is a solid state drive if that helps

Edit
The disk is a Crucial SSD M4


Last edited by vrghost on Mon Jul 29, 2013 5:00 pm; edited 1 time in total
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21431

PostPosted: Fri Jul 26, 2013 1:08 am    Post subject: Reply with quote

What vendor and model is the drive? What does dmesg show after the drive begins reporting errors?
Back to top
View user's profile Send private message
vrghost
n00b
n00b


Joined: 03 Oct 2003
Posts: 48
Location: London

PostPosted: Fri Jul 26, 2013 9:04 am    Post subject: Reply with quote

Thank you very much for your reply Hu

So, the two key pieces of hardware should be:
Code:
Motherboard:   Gigabyte Z68X-UD3H-B3 LGA1155 ATX
Offending disk: Crucial 256GB SATA 6Gb/s SSD M4


The motherboard have two controllers (mixture of lspci -v -k and -nn):
Code:
00:1f.2 SATA controller [0106]: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller [8086:1c02] (rev 05)
Subsystem: Giga-byte Technology Device b005
Kernel driver in use: ahci

09:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller [1b4b:9172] (rev 11)
Subsystem: Giga-byte Technology Device b000
Kernel driver in use: ahci


And the disk:
Code:
hdparm -i /dev/sda

/dev/sda:

 Model=M4-CT256M4SSD2, FwRev=0009, SerialNo=000000001144031DE83E
 Config={ Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=500118192
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-3,4,5,6,7

 * signifies the current active mode
Back to top
View user's profile Send private message
vrghost
n00b
n00b


Joined: 03 Oct 2003
Posts: 48
Location: London

PostPosted: Fri Jul 26, 2013 9:24 am    Post subject: Reply with quote

And now the dmesg part


Code:
[ 1473.148943] nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead.
[ 2657.691392] ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[ 2657.691394] ata2.00: failed command: FLUSH CACHE EXT
[ 2657.691397] ata2.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
         res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
[ 2657.691398] ata2.00: status: { DRDY }
[ 2657.691401] ata2: hard resetting link
[ 2667.731218] ata2: softreset failed (device not ready)
[ 2667.731220] ata2: hard resetting link
[ 2677.771126] ata2: softreset failed (device not ready)
[ 2677.771128] ata2: hard resetting link
[ 2688.320526] ata2: link is slow to respond, please be patient (ready=0)
[ 2712.780809] ata2: softreset failed (device not ready)
[ 2712.780815] ata2: limiting SATA link speed to 3.0 Gbps
[ 2712.780817] ata2: hard resetting link
[ 2717.977628] ata2: softreset failed (device not ready)
[ 2717.977633] ata2: reset failed, giving up
[ 2717.977636] ata2.00: disabled
[ 2717.977640] ata2.00: device reported invalid CHS sector 0
[ 2717.977649] ata2: EH complete
[ 2717.977690] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977692] sd 1:0:0:0: [sda] 
[ 2717.977694] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977697] sd 1:0:0:0: [sda] CDB:
[ 2717.977698] Read(10): 28 00 05 75 cf b0 00 00 08 00
[ 2717.977708] end_request: I/O error, dev sda, sector 91606960
[ 2717.977724] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977725] sd 1:0:0:0: [sda] 
[ 2717.977726] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977726] sd 1:0:0:0: [sda] CDB:
[ 2717.977727] Write(10): 2a 00 07 74 6e 68 00 00 08 00
[ 2717.977730] end_request: I/O error, dev sda, sector 125070952
[ 2717.977732] Buffer I/O error on device dm-5, logical block 35277
[ 2717.977736] EXT4-fs warning (device dm-5): ext4_end_bio:317: I/O error writing to inode 524321 (offset 3276800 size 4096 starting block 35277)
[ 2717.977749] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977750] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977751] sd 1:0:0:0: [sda] 
[ 2717.977760] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977760] sd 1:0:0:0: [sda] CDB:
[ 2717.977763] sd 1:0:0:0: [sda] 
[ 2717.977764] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977765] sd 1:0:0:0: [sda] CDB:
[ 2717.977761] Read(10):
[ 2717.977765] Write(10)
[ 2717.977766] :
[ 2717.977767]  2a
[ 2717.977767]  00
[ 2717.977768]  07
[ 2717.977768]  f4
[ 2717.977769]  33
[ 2717.977769]  98
[ 2717.977769]  00
[ 2717.977770]  00
[ 2717.977770]  08
[ 2717.977771]  00

[ 2717.977772]  28
[ 2717.977773] end_request: I/O error, dev sda, sector 133444504
[ 2717.977773] end_request: I/O error, dev sda, sector 133444504
[ 2717.977774]  00 05 75 cf b0 00 00 08 00
[ 2717.977778] end_request: I/O error, dev sda, sector 91606960
[ 2717.977780] Aborting journal on device dm-5-8.
[ 2717.977789] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977791] sd 1:0:0:0: [sda] 
[ 2717.977792] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977793] sd 1:0:0:0: [sda] CDB:
[ 2717.977794] Write(10): 2a 00 07 bf 00 58 00 00 08 00
[ 2717.977806] end_request: I/O error, dev sda, sector 129957976
[ 2717.977809] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977810] Buffer I/O error on device dm-5, logical block 646155
[ 2717.977812] EXT4-fs warning (device dm-5): ext4_end_bio:317: I/O error writing to inode 524319 (offset 0 size 4096 starting block 646155)
[ 2717.977813] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977814] sd 1:0:0:0: [sda] 
[ 2717.977815] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977816] sd 1:0:0:0: [sda] CDB:
[ 2717.977819] sd 1:0:0:0: [sda] 
[ 2717.977820] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977821] sd 1:0:0:0: [sda] CDB:
[ 2717.977816] Write(10): 2a
[ 2717.977821] Write(10)
[ 2717.977822] :
[ 2717.977823]  2a
[ 2717.977823]  00
[ 2717.977824]  07
[ 2717.977824]  f4
[ 2717.977825]  20
[ 2717.977825]  00
[ 2717.977825]  00
[ 2717.977826]  00
[ 2717.977826]  08
[ 2717.977827]  00

[ 2717.977828] end_request: I/O error, dev sda, sector 133439488
[ 2717.977829]  00
[ 2717.977829] Buffer I/O error on device dm-5, logical block 1081344
[ 2717.977830] lost page write due to I/O error on dm-5
[ 2717.977839]  07 70 20 00 00 00 08 00
[ 2717.977842] end_request: I/O error, dev sda, sector 124788736
[ 2717.977843] JBD2: Error -5 detected when updating journal superblock for dm-5-8.
[ 2717.977845] Buffer I/O error on device dm-5, logical block 0
[ 2717.977846] lost page write due to I/O error on dm-5
[ 2717.977852] EXT4-fs error (device dm-5): ext4_journal_start_sb:349: Detected aborted journal
[ 2717.977855] EXT4-fs (dm-5): Remounting filesystem read-only
[ 2717.977856] EXT4-fs (dm-5): previous I/O error to superblock detected
[ 2717.977866] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.977867] sd 1:0:0:0: [sda] 
[ 2717.977868] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.977869] sd 1:0:0:0: [sda] CDB:
[ 2717.977869] Write(10): 2a 00 07 70 20 00 00 00 08 00
[ 2717.977880] end_request: I/O error, dev sda, sector 124788736
[ 2717.977881] Buffer I/O error on device dm-5, logical block 0
[ 2717.977889] lost page write due to I/O error on dm-5
[ 2717.978180] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.978181] sd 1:0:0:0: [sda] 
[ 2717.978182] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.978183] sd 1:0:0:0: [sda] CDB:
[ 2717.978184] Read(10): 28 00 05 75 cf 50 00 00 20 00
[ 2717.978187] end_request: I/O error, dev sda, sector 91606864
[ 2717.978239] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.978241] sd 1:0:0:0: [sda] 
[ 2717.978242] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.978243] sd 1:0:0:0: [sda] CDB:
[ 2717.978243] Read(10): 28 00 05 75 cf 50 00 00 08 00
[ 2717.978400] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.978402] sd 1:0:0:0: [sda] 
[ 2717.978403] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.978404] sd 1:0:0:0: [sda] CDB:
[ 2717.978404] Read(10): 28 00 05 75 e4 70 00 00 20 00
[ 2717.978451] sd 1:0:0:0: [sda] Unhandled error code
[ 2717.978453] sd 1:0:0:0: [sda] 
[ 2717.978454] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2717.978455] sd 1:0:0:0: [sda] CDB:
[ 2717.978455] Read(10): 28 00 05 75 e4 70 00 00 08 00
[ 2719.297873] sd 1:0:0:0: [sda] Unhandled error code
[ 2719.297875] sd 1:0:0:0: [sda] 
[ 2719.297876] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2719.297877] sd 1:0:0:0: [sda] CDB:
[ 2719.297878] Write(10): 2a 00 02 14 46 70 00 00 08 00
[ 2719.297883] Buffer I/O error on device sda5, logical block 34510
[ 2719.297885] EXT4-fs warning (device sda5): ext4_end_bio:317: I/O error writing to inode 2833 (offset 0 size 4096 starting block 4360399)
[ 2719.315438] sd 1:0:0:0: [sda] Unhandled error code
[ 2719.315440] sd 1:0:0:0: [sda] 
[ 2719.315441] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2719.315442] sd 1:0:0:0: [sda] CDB:
[ 2719.315443] Write(10): 2a 00 02 14 47 70 00 00 08 00
[ 2719.315448] Buffer I/O error on device sda5, logical block 34542
[ 2719.315450] EXT4-fs warning (device sda5): ext4_end_bio:317: I/O error writing to inode 2823 (offset 0 size 4096 starting block 4360431)
[ 2719.315897] sd 1:0:0:0: [sda] Unhandled error code
[ 2719.315898] sd 1:0:0:0: [sda] 
[ 2719.315899] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2719.315900] sd 1:0:0:0: [sda] CDB:
[ 2719.315901] Write(10): 2a 00 02 14 47 88 00 00 08 00
[ 2719.315905] Buffer I/O error on device sda5, logical block 34545
[ 2719.315906] EXT4-fs warning (device sda5): ext4_end_bio:317: I/O error writing to inode 2832 (offset 0 size 4096 starting block 4360434)
[ 2723.526300] sd 1:0:0:0: [sda] Unhandled error code
[ 2723.526302] sd 1:0:0:0: [sda] 
[ 2723.526303] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2723.526304] sd 1:0:0:0: [sda] CDB:
[ 2723.526305] Write(10): 2a 00 02 3c 1b e0 00 00 70 00
[ 2723.526309] blk_update_request: 6 callbacks suppressed
[ 2723.526310] end_request: I/O error, dev sda, sector 37493728
[ 2723.526350] Aborting journal on device sda5-8.
[ 2723.526362] sd 1:0:0:0: [sda] Unhandled error code
[ 2723.526364] sd 1:0:0:0: [sda] 
[ 2723.526365] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2723.526365] sd 1:0:0:0: [sda] CDB:
[ 2723.526366] Write(10): 2a 00 02 3c 10 00 00 00 08 00
[ 2723.526370] end_request: I/O error, dev sda, sector 37490688
[ 2723.526371] Buffer I/O error on device sda5, logical block 360448
[ 2723.526372] lost page write due to I/O error on sda5
[ 2723.526406] JBD2: Error -5 detected when updating journal superblock for sda5-8.
[ 2723.542183] sd 1:0:0:0: [sda] Unhandled error code
[ 2723.542187] sd 1:0:0:0: [sda] 
[ 2723.542189] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2723.542191] sd 1:0:0:0: [sda] CDB:
[ 2723.542193] Write(10): 2a 00 04 34 55 78 00 00 10 00
[ 2723.542202] end_request: I/O error, dev sda, sector 70538616
[ 2723.542242] Aborting journal on device dm-2-8.
[ 2723.542256] sd 1:0:0:0: [sda] Unhandled error code
[ 2723.542258] sd 1:0:0:0: [sda] 
[ 2723.542258] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2723.542259] sd 1:0:0:0: [sda] CDB:
[ 2723.542260] Write(10): 2a 00 04 34 20 00 00 00 08 00
[ 2723.542264] end_request: I/O error, dev sda, sector 70524928
[ 2723.542265] Buffer I/O error on device dm-2, logical block 3702784
[ 2723.542266] lost page write due to I/O error on dm-2
[ 2723.542300] JBD2: Error -5 detected when updating journal superblock for dm-2-8.
[ 2753.198273] sd 1:0:0:0: [sda] Unhandled error code
[ 2753.198278] sd 1:0:0:0: [sda] 
[ 2753.198280] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2753.198282] sd 1:0:0:0: [sda] CDB:
[ 2753.198284] Write(10): 2a 00 02 10 10 00 00 00 08 00
[ 2753.198293] end_request: I/O error, dev sda, sector 34607104
[ 2753.198296] Buffer I/O error on device sda5, logical block 0
[ 2753.198297] lost page write due to I/O error on sda5
[ 2753.198344] EXT4-fs error (device sda5): ext4_journal_start_sb:349: Detected aborted journal
[ 2753.198347] EXT4-fs (sda5): Remounting filesystem read-only
[ 2753.198348] EXT4-fs (sda5): previous I/O error to superblock detected
[ 2753.198360] sd 1:0:0:0: [sda] Unhandled error code
[ 2753.198361] sd 1:0:0:0: [sda] 
[ 2753.198362] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2753.198363] sd 1:0:0:0: [sda] CDB:
[ 2753.198363] Write(10): 2a 00 02 10 10 00 00 00 08 00
[ 2753.198367] end_request: I/O error, dev sda, sector 34607104
[ 2753.198368] Buffer I/O error on device sda5, logical block 0
[ 2753.198369] lost page write due to I/O error on sda5
[ 2753.198404] EXT4-fs (sda5): ext4_da_writepages: jbd2_start: 13312 pages, ino 2681; err -30
[ 2765.018120] sd 1:0:0:0: [sda] Unhandled error code
[ 2765.018122] sd 1:0:0:0: [sda] 
[ 2765.018122] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2765.018123] sd 1:0:0:0: [sda] CDB:
[ 2765.018124] Read(10): 28 00 02 19 ae 00 00 00 20 00
[ 2765.018128] end_request: I/O error, dev sda, sector 35237376
[ 2765.018144] sd 1:0:0:0: [sda] Unhandled error code
[ 2765.018145] sd 1:0:0:0: [sda] 
[ 2765.018145] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2765.018146] sd 1:0:0:0: [sda] CDB:
[ 2765.018147] Read(10): 28 00 02 19 ae 00 00 00 08 00
[ 2765.018158] end_request: I/O error, dev sda, sector 35237376
[ 2765.018185] sd 1:0:0:0: [sda] Unhandled error code
[ 2765.018194] sd 1:0:0:0: [sda] 
[ 2765.018194] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 2765.018195] sd 1:0:0:0: [sda] CDB:
[ 2765.018196] Read(10): 28 00 02 19 ae 00 00 00 08 00
[ 2765.018199] end_request: I/O error, dev sda, sector 35237376
Back to top
View user's profile Send private message
vrghost
n00b
n00b


Joined: 03 Oct 2003
Posts: 48
Location: London

PostPosted: Fri Jul 26, 2013 10:45 am    Post subject: Reply with quote

I may have solved the issue.

After a lot of trawling around the internet I found a comment on a forum about a crucial M4 firmware issues, something about causing Windows machines to crash after an hour once the disk had been in use over 5640 hours or some random but specific number of hours.

So updated the firmware to 070H from 0009. The machine have been up for about an hour now and the problem have not come up, will not change it to Solved yet, but once its been up for about a day or two I'll call it solved (if it works)

hdparm on the disk now:
Code:
hdparm -i /dev/sda

/dev/sda:

 Model=M4-CT256M4SSD2, FwRev=070H, SerialNo=000000001144031DE83E
 Config={ Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=500118192
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-3,4,5,6,7


Link to the firmware on crucials home page:
http://www.crucial.com/uk/support/firmware.aspx
Back to top
View user's profile Send private message
vrghost
n00b
n00b


Joined: 03 Oct 2003
Posts: 48
Location: London

PostPosted: Mon Jul 29, 2013 5:01 pm    Post subject: Reply with quote

After patch update it has now been running for three days without any problem (no rebuilds or anything, just updated firmware)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum