When it comes to binary files ...

Question

I can't say I have a good grasp of the real differences between files written in text mode and files written in binary mode.
For (C) example:
struct data
int a, b;
} d;
d.a = 432;
d.b = 581924;
char s[14] = "hello world!\n";
FILE *fd = fopen("test.txt", "wt");
fwrite(&d, sizeof(data), 1, fd);
fwrite(s, sizeof(char), 14, fd);
fclose(fd);
fd = fopen ("test.bin", "wb");
fwrite(&d, sizeof(data), 1, fd);
fwrite(s, sizeof(char), 14, fd);
fclose(fd);
test.txt hexdump:
B0 01 00 00 24 E1 08 00 68 65 6C 6C 6F 20 77 6F 72 6C 64 21 0D 0A 00
test.bin hexdump:
B0 01 00 00 24 E1 08 00 68 65 6C 6C 6F 20 77 6F 72 6C 64 21 0A 00
What I know (and see):
strings containing '\n', when are written in text mode to a file, have the '\n' replaced by CR+LF / LF / CR depending on the platform; in binary mode it is left unchanged
binary files are used to represent the image of the memory used by a program; so if a chain of structures are wrritten in a known order in a binary file, that chain can be restored on the next startup of the program, if the reading takes place in the order specified so no parsing and type casting is needed
Am I right? Other than these things what else should I know?
The reason I want to clear these things out is because I have a file which contains the data collected and interpreted by a counter device from some sensors.
I believe this file is the binary footprint of the memory used to hold the recordings because I can't decipher the hexdump.
Now, I have a program to read this type of file and see the recordings and export them to a file in a human readable format (and no, it doesn't have a command line argument for this so that I can make a script). The thing is, I want to put all the recordings in a database and manipulate them further for all kinds of statistics and I don't want to waste time exporting whenever there is a data to be inserted.
I have no clue if what I'm saying has a bit of truth so I would be grateful if someone can correct or enlighten me on how I should approach this  ... is there a way to "parse" the binary file directly not knowing the types of the placeholders for the recordings?
Last edited by knob (2011-03-03 22:41:04)

Answer 1

You're right about what happens when a "text" file is written to disk. However, files ("text" or "binary") can contain anything, not necessarily a "memory footprint". (Edit: well, technically, anything you write to disk comes from memory... but it doesn't have to come from a struct or an array. It doesn't matter anyway...)
That said, as far as I understand it, your problem is that your "counter device" uses an undocumented file format and closed-source software. First step would be to ask the developer for documentation and source code.
If the developer doesn't want to give the required information or update the software as needed, you'll have to reverse-engineer the file format, either as previously suggested by "guessing" the contents, or by decompiling the executables that access the files.
Last edited by stqn (2011-03-04 17:36:59)