---------------------------------------------------------------------------- | The XTRE Format | | By Mario "HCl" Brito | ---------------------------------------------------------------------------- Introduction I first began working on this TRE when i was learning how to program in Assembly. I feel this format is way more complex than it really needs to be. I'm glad Origin came up with the Prophecy TRE format since, in my humble opinion, it brings back the simplicity of the Privateer format and strings as well. The way i see it, the greatest flaw of this format is allowing files to be stored without a string associated to it, since you don't know which file you're looking at (unless you remove the reference from the TRE and see the game complain when it doesn't find the file). When i have some time, i'll probably work on finding out the CRC routine for WC3/4: even if the CRC cannot be reversed (and i probably won't brute-force strings with more than 6 characters unless i have a good reason to do it), it should allow us to put files inside a XTRE. The info First of all, i'll give some info on the data types i'll use, so there's no confusion: byte = 1 byte word = 2 bytes dword = 4 bytes string = A space meant to hold a sequence of characters with a null (or \0) at the end. It's size is constant (pretty much like an array). The Data section will not be covered since it'll be raw file data referenced by the header. You can find the header description below. -> The Header This header is probably one of the most complex ones i've seen, from the different WC TRE's i've examined. I'll divide it in 3 sections to make things easier: - Section 0 (the beginning of the file) - data type | size | description ------------------------------------------------------------------------------ string | 4 | Holds the chars X,T,R and E (XTRE sig). | | There's no null at the end in this case. ----------------|---------------|--------------------------------------------- dword | 4 | It's always 0 ----------------|---------------|--------------------------------------------- dword | 4 | Offset of section 1 ----------------|---------------|--------------------------------------------- dword | 4 | Offset of section 2 ----------------|---------------|--------------------------------------------- dword | 4 | Offset of section 3 ----------------|---------------|--------------------------------------------- dword | 4 | Offset of data area ----------------|---------------|--------------------------------------------- These sections hold different info, as you will see. You will notice that not all files use all 3 sections. - Section 1 - Basically, this section of the header holds a CRC and an offset to another section. data type | size | description ------------------------------------------------------------------------------ dword | 4 | The CRC, the game searches for this. ----------------|---------------|--------------------------------------------- dword | 4 | The offset of a part of either section 2 or | | 3, which holds the rest of the info. ------------------------------------------------------------------------------ Be warned, you may sometimes find references where the CRC is 0 and the offset is 0xffffffff. I think that these "garbage" areas were files that were removed from the TRE (or even just from the header, who knows.. i wish i had done some more work on this). Also, be sure to notice that if the offset points somewhere to section 3, you can forget about the file having a string stored in the TRE. - Section 2 - Basically, this section of the header holds the file's string and an offset of section 3. It's interesting to see that very few files have strings associated to it (which makes life harder to us). data type | size | description ------------------------------------------------------------------------------ byte | 1 | The size of the string. ----------------|---------------|--------------------------------------------- string | varies | This string is the name of the file. There | | is no null at the end of the string. It's | | size is determined from the byte before it. ----------------|---------------|--------------------------------------------- dword | 4 | The offset of a part of section 3, which | | holds the rest of the info of this file. ------------------------------------------------------------------------------ - Section 3 - This section holds the info you got used to seeing on the other 2 TRE formats. Basically, it holds the offset of the TRE where this file starts and it's uncompressed length. data type | size | description ------------------------------------------------------------------------------ dword | 4 | The offset of the TRE where this file | | starts. ----------------|---------------|--------------------------------------------- dword | 4 | The uncompressed length of the file. | | An important note: i'd ignore the two most | | significant bits of the most significant | | byte, as they can mess up the size. ----------------|---------------|--------------------------------------------- When processing the sections, you can check which section you're on by comparing the offset where you are with the offsets on section 0. However, we're not done yet.. there are 2 (two!) uncompression routines! And there are even files that may not be compressed at all. Do you agree with me that this format is way more complex than it should be? Anyway, to figure out if the file we're processing is compressed, just check if offset+length = offset of the next file. If it is, then the file isn't compressed and you just have to dump it to a file. If the file is compressed, you have to unpack it. But with which routine? After all we have 2 uncompression routines... The solution i found is to check if the first 4 bits of first byte of the compression data is 0xe (in, short do an "if ((byte & 0xe0 ) == 0xe0) then use routine1, else use routine2"). This may not be the best solution, but it's the one i found and it seems to work well. You can find the source code of my XTRE extractor in the Information Area. Keep in mind that both uncompression routines were taken from WC3, so the credit for those goes to Origin. The XTRE extractor was taken from the TREMAN source code (which is getting messy). I didn't have time to implement the extraction of files that have strings associated, but you can do it with this info (i know, TREMAN's warning message says the exact opposite... i guess that's what happens when you program at 5am ;) ). -> Thanks go to Origin Systems INC for creating Wing Commander 3 and Wing Commander 4. The uncompression routines used to unpack the files stored inside a TRE were taken from wc3.exe, which was created by Origin: http://www.origin.ea.com Mario "HCl" Brito mbrito@student.dei.uc.pt