Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Tools for reverse engineering data file formats?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
turtles
Veteran
Veteran


Joined: 31 Dec 2004
Posts: 1651

PostPosted: Mon Aug 12, 2013 12:11 am    Post subject: Tools for reverse engineering data file formats? Reply with quote

What tools have you used for reverse engineering a data file? Say a old database file from no longer supported database or OS? Or a possibly damaged file?

I have been looking at strings and mdb-tools.
okteta seems like it can use strings as well.

others?

Thanks!
_________________
Donate to Gentoo
Back to top
View user's profile Send private message
Navar
Guru
Guru


Joined: 20 Aug 2012
Posts: 353

PostPosted: Mon Aug 12, 2013 1:55 am    Post subject: Reply with quote

I hope you're doing this for fun, otherwise I don't envy you.

I'm going to assume this is entirely closed source, you have no way of obtaining any details whatsoever about the format (via other's work in open source code or published docs). I'd start there, with all details you can find out about the format in question, particularly things available in the year(s) used.

Otherwise, hexdump and any decent hexviewer/editor. You will want to determine the byte ordering. Guessing number stored format(s) can be a pain, particularly with floating point. Since it's old, character strings may be the easiest aspect.

A strong disassembler can be helpful if you have a closed executable that produces the format in question in trying to derive block structures, etc.

If the data is packed, encrypted or obscured in some other ways it can take a lot of time and patience.
_________________
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Wed Aug 28, 2013 7:38 am    Post subject: Reply with quote

How come, people always assume criminal hacking activities when someone says: "reverse engineering"?

Actually, it is my job to crack ("reverse engineering" is not the proper term) proprietary file formats. I am doing this to be able to extract metadata and documents from closed source document management systems, because those systems lack proper exporting functionality (at least in a usable speed), and our customers want to move (migrate) this data from their old systems to a new system. So, this is perfectly legal, as we are not "hacking" anybody elses data or are selling hacking tools, knowledge or whatever. This could be WORMs, tapes, CDs, DVDs or databases.

However, if you hope for some magic bullet tools, there are none. The tools I am using are hexdump, less, strings, grep, sed, cat, cut, bash, perl and gcc. The first 7 to analyze files, the latter 3 to write extraction programs.
If you want to extract data out of a proprietary system, you must first learn how the data is stored, and hexdumps are the only secure way. The other tools are merely used to get "pointers" in the right direction.

Have fun!

Edith just realized: I forgot to mention app-editors/hexedit - I work on customer files and an editor is a dangerous tools there, but this editor is great for searching strings and hex number chains.
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
Navar
Guru
Guru


Joined: 20 Aug 2012
Posts: 353

PostPosted: Wed Aug 28, 2013 11:00 pm    Post subject: Reply with quote

Yamakuzure wrote:
How come, people always assume criminal hacking activities when someone says: "reverse engineering"?


Whom are you referring to? All I said is it can be a giant P.I.A., nothing about any legalities. Nor will I make any claims on the legal repercussions of your employer's actions. Maybe in your locale things are more liberal and such actions don't run afoul of your courts.

As for some other presumptions, there's enough lawsuits ongoing for years now regarding Patriot Act, DMCA provisions and EULAs in the States and elsewhere to argue against your claim and that's where the global corporations have dictated today's terms, particularly on data (you can start with Sony, Apple or Microsoft). People assume because that's what the government, courts and media are telling them.
_________________
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum