View previous topic :: View next topic |
Author |
Message |
turtles Veteran
Joined: 31 Dec 2004 Posts: 1655
|
Posted: Mon Aug 12, 2013 12:11 am Post subject: Tools for reverse engineering data file formats? |
|
|
What tools have you used for reverse engineering a data file? Say a old database file from no longer supported database or OS? Or a possibly damaged file?
I have been looking at strings and mdb-tools.
okteta seems like it can use strings as well.
others?
Thanks! _________________ Donate to Gentoo |
|
Back to top |
|
|
Navar Guru
Joined: 20 Aug 2012 Posts: 353
|
Posted: Mon Aug 12, 2013 1:55 am Post subject: |
|
|
I hope you're doing this for fun, otherwise I don't envy you.
I'm going to assume this is entirely closed source, you have no way of obtaining any details whatsoever about the format (via other's work in open source code or published docs). I'd start there, with all details you can find out about the format in question, particularly things available in the year(s) used.
Otherwise, hexdump and any decent hexviewer/editor. You will want to determine the byte ordering. Guessing number stored format(s) can be a pain, particularly with floating point. Since it's old, character strings may be the easiest aspect.
A strong disassembler can be helpful if you have a closed executable that produces the format in question in trying to derive block structures, etc.
If the data is packed, encrypted or obscured in some other ways it can take a lot of time and patience. _________________ Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn. |
|
Back to top |
|
|
Yamakuzure Advocate
Joined: 21 Jun 2006 Posts: 2284 Location: Adendorf, Germany
|
Posted: Wed Aug 28, 2013 7:38 am Post subject: |
|
|
How come, people always assume criminal hacking activities when someone says: "reverse engineering"?
Actually, it is my job to crack ("reverse engineering" is not the proper term) proprietary file formats. I am doing this to be able to extract metadata and documents from closed source document management systems, because those systems lack proper exporting functionality (at least in a usable speed), and our customers want to move (migrate) this data from their old systems to a new system. So, this is perfectly legal, as we are not "hacking" anybody elses data or are selling hacking tools, knowledge or whatever. This could be WORMs, tapes, CDs, DVDs or databases.
However, if you hope for some magic bullet tools, there are none. The tools I am using are hexdump, less, strings, grep, sed, cat, cut, bash, perl and gcc. The first 7 to analyze files, the latter 3 to write extraction programs.
If you want to extract data out of a proprietary system, you must first learn how the data is stored, and hexdumps are the only secure way. The other tools are merely used to get "pointers" in the right direction.
Have fun!
Edith just realized: I forgot to mention app-editors/hexedit - I work on customer files and an editor is a dangerous tools there, but this editor is great for searching strings and hex number chains. _________________ Important German:- "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
- "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
|
|
Back to top |
|
|
Navar Guru
Joined: 20 Aug 2012 Posts: 353
|
Posted: Wed Aug 28, 2013 11:00 pm Post subject: |
|
|
Yamakuzure wrote: | How come, people always assume criminal hacking activities when someone says: "reverse engineering"? |
Whom are you referring to? All I said is it can be a giant P.I.A., nothing about any legalities. Nor will I make any claims on the legal repercussions of your employer's actions. Maybe in your locale things are more liberal and such actions don't run afoul of your courts.
As for some other presumptions, there's enough lawsuits ongoing for years now regarding Patriot Act, DMCA provisions and EULAs in the States and elsewhere to argue against your claim and that's where the global corporations have dictated today's terms, particularly on data (you can start with Sony, Apple or Microsoft). People assume because that's what the government, courts and media are telling them. _________________ Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|