Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Use hexedit to manipulate a huge disk dump? [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
miroR
l33t
l33t


Joined: 05 Mar 2008
Posts: 826

PostPosted: Wed Oct 28, 2015 5:02 pm    Post subject: Use hexedit to manipulate a huge disk dump? [SOLVED] Reply with quote

title: Use hexedit to manipulate a huge disk dump?
---
I have a query (or an unnecessary qualm: see in the very bottom).
Code:

gbn ~ # ls -l /mnt/sdg1/dd_Add/dd_F1028_g0n_DONT_USE/
total 69993144
-rw-r--r-- 1 root root         104 2015-10-28 09:20 dd_F1028_g0n.md5
-rw-r--r-- 1 root root   505709056 2015-10-28 08:56 F1028_g0n_sda2.dd
-rw-r--r-- 1 root root 71167246848 2015-10-28 10:44 F1028_g0n_sda3.dd
gbn ~ # ls -l /mnt/sdh1/dd_Add/dd_F1028_g0n_DONT_USE/
total 69993140
-rw-r--r-- 1 root root         104 2015-10-28 09:20 dd_F1028_g0n.md5
-rw-r--r-- 1 root root   505709056 2015-10-28 08:56 F1028_g0n_sda2.dd
-rw-r--r-- 1 root root 71167246848 2015-10-28 09:08 F1028_g0n_sda3.dd
gbn ~ #


These are two disk dumps. See my tips

Postfix smtp/TLS, Bkp/Cloning Mthd, Censorship/Intrusion
https://forums.gentoo.org/viewtopic-t-999436.html#7613044

on how to backup a system

(unless you think that RAID is the way to do system backup

(the same topic)
https://forums.gentoo.org/viewtopic-t-999436.html#7819984
)

These dumps are taken after a scare that I suffered (I'm too easily caught by panic when I can't easily see the reason why something bad for my system or threatening to my system happens, and that's hard to correct in my character ;-) )

RBAC policy for tcpdump
https://forums.grsecurity.net/viewtopic.php?f=5&t=4301
(further on it occurs pretty all of a sudden in the narrative)

I'm slowly figuring out what happened there, but what I usually do in such cases, which is my poor user's incident response, is I shut the system down, and back it up, that is [d]isk [d]umnp it (if the unix command dd has that meaning originally).

I'm slowly figuring out what happened there, and it should be all there, with time, but my question, here, is connected to the integrity of the files.

Firstly, the system froze on me, just as I was preparing a little post, the sixth for the "RBAC policy for tcpdump" topic on grsecurity forums above, with some corrections and explanations.

And, when a system freezes (I think it waa either some of the ~amd64 version of Wireshark or some of the kernels vmlinuz-4.2.3-hardened-r3-151018, vmlinuz-4.2.4-hardened-151026, vmlinuz-4.2.4-hardened-r1-151026, but for investigating on it I don't have the energy left; vmlinuz-4.2.4-hardened-r1 and net-analyzer/wireshark-1.12.8-r1 may be working, and not freezing for me now, so...), you can't surely shut down the system regularly, and once you do either physically, with the swith, restart or shut down, and take the disk out or in some other way, dd-dump it like the files in top of this post are dumps of that system of mine that froze on me...

And so once you dd-dump the disk of such a system which had frozen, it the partition that suffered the freezing were ext4, then you can't anymore mount them readonly anymore, which I like to do, if I need stuff from the system partition/disk to take out/backup or other.

Because the partition was not cleanly unmounted.

First, the above listing in the top, is the dd-dump of the one only system, but in two different places (two different disks attached on the USB3).

The smaller dumps (n the two different storages), this one in sdg1:
Code:

-rw-r--r-- 1 root root   505709056 2015-10-28 08:56 F1028_g0n_sda2.dd

and this one in sdh1:
Code:

-rw-r--r-- 1 root root   505709056 2015-10-28 08:56 F1028_g0n_sda2.dd

were almost at the same time (no seconds in the listing, but it was almost at the same time). They are the dumps of the boot partition.

The larger dumps (n the two different storages), this one in sdg1:
Code:

-rw-r--r-- 1 root root 71167246848 2015-10-28 10:44 F1028_g0n_sda3.dd

and this one in sdh1:
Code:

-rw-r--r-- 1 root root 71167246848 2015-10-28 09:08 F1028_g0n_sda3.dd

were originally almost at the same time as well. They are the dumps of the one / (root) partition containing /usr and /var and all really (so when I backup/clone I have simple task to do.

But now the one in sdg1 storage has timestamp which is some one and a half hour later.

That is because it had to be mounted read-write, and it was changed when mount called some other filesystem tricks/commands on it to fix it.

I tried to mount it readonly at first, like this:

Code:

gbn ~ # losetup /dev/loop1 /mnt/sdg1/dd_Add/dd_F1028_g0n_DONT_USE/F1028_g0n_sda2.dd
gbn ~ # losetup /dev/loop3 /mnt/sdg1/dd_Add/dd_F1028_g0n_DONT_USE/F1028_g0n_sda3.dd
gbn ~ # blockdev --setro /dev/loop1
gbn ~ # blockdev --getro /dev/loop1
1
gbn ~ # blockdev --setro /dev/loop3
gbn ~ # blockdev --getro /dev/loop3
1
gbn ~ # mount /dev/loop3 /mnt/g0n
g0n/   g0n-C/
gbn ~ # mount /dev/loop3 /mnt/g0n-C/
mount: /dev/loop3 is write-protected, mounting read-only
mount: wrong fs type, bad option, bad superblock on /dev/loop3,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
gbn ~ #


And surely that could not happen, for the reason explained above.

And so I simply had to mount it read-write.

But in doing so, it looses its integrity! It cannot be verified anymore that it is the exact same dump that was previously taken!

And you can see that I take the MD5 of the dumps (Sleuthkit

http://forum.sleuthkit.org/

uses MD5, no need to be cleverer than experts, MD5 is fine for huge files like these), and I can always verify them later, if I want or need to.

I don't need to try, I know that the MD5 does no longer verifies, by the virtue of that filesystem being changed when it was fixed at the time of mounting it read-write.

But I have the other dump, that, unless the disk fails exactly right now or soon by Murphy's law, verifies correctly in all probability.

My wish is to learn to use hexedit to revert the changes back to the previous state of that file in /mnt/sdg1, so that that file verifies too by MD5.

I will need to learn to use hexedit, and I have made the first steps successfully

SSL Decode & My Hard-Earned Advice for SPDY/HTTP2 in Firefox
https://forums.gentoo.org/viewtopic-t-1029408.html

to recover lots of my data from a disk that I overwrote

Recover partly overwritten luks volume?
https://forums.gentoo.org/viewtopic-t-1004014.html

So I want to take this as exercize. I know I could simply copy the file that verifies over into the other storage with the file that does not verify, but I want to be able to do this task quicker with hexedit.

Any advice (and if you have, pls. allow time for me to understand your advice; I don't always even understand good advice right away

(the same topic, with the uncenz-restored advice)
https://forums.gentoo.org/viewtopic-t-1004014.html#7724060

, I learn and work slowly ;-) so don't get angry if I make wrong conjectures, pls.)

E.g. I haven't really opened with hexedit a file as huge and these 67G files... I hope it won't crash on me...


Last edited by miroR on Wed Nov 25, 2015 7:51 am; edited 1 time in total
Back to top
View user's profile Send private message
Abraxas
l33t
l33t


Joined: 25 May 2003
Posts: 807

PostPosted: Fri Nov 06, 2015 3:56 pm    Post subject: Reply with quote

Use a hexeditor that support large files like WXHexeditor.
_________________
Time makes more converts than reason. - Thomas Paine
Travel is fatal to prejudice, bigotry, and narrow-mindedness, and many of our people need it sorely on these accounts. - Mark Twain
Back to top
View user's profile Send private message
miroR
l33t
l33t


Joined: 05 Mar 2008
Posts: 826

PostPosted: Sun Nov 22, 2015 11:21 pm    Post subject: Reply with quote

Abraxas wrote:
Use a hexeditor that support large files like WXHexeditor.

Really sorry for seeing only this late your reply.

And thank you! Next time I have an issue with huge files, I'll remember your advice (I had to forgo woriking on these huge files this time, too many things were on me, related to Genoo and not).

Regards!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum