Analysis of the CASC filesystem

[caali]

Hello folks,

When I was bored I spent the last few days analysing Blizzard's new CASC (Content Addressable Storage Container) filesystem used in the WoD Alpha. I've found out a lot already - so when I found several threads over here discussing the format, I thought I'd share my knowledge

I put my knowledge into a standalone file extractor without any DLL injection hooks and it is currently able to extract about 420k files (>99% of all) without any differences from when I extracted them using the hooking method. However, I will not share this code with you (sorry ). What you'll get is all I know about the format, though - so it won't take any semi-experienced programmer more than a few hours to produce the same code. All you need is attached to this post.

Please feel free to distribute and improve my analysis - however I'd be happy to be mentioned in the credits...

Greetings, Caali


// Edit
- v1.1 release (thanks to TOM_RUS for his feedback):

• FileTable renamed to "encoding" file
• Manifest File renamed to "root" file
• Noted that BlizzHash is actually Bob Jenkins' hash
• Added information about the "download" and "install" files

- v1.2 release

• Analysis of the 'data/config/' folder

[zifron]

how do I even use it lol ?

[TOM_RUS]

"FileTable" file is actually called "encoding" file
"Manifest" file is actually called "root" file
There's also 2 more special files in data.XXX: "download" file and "install" file

"BlizzHash" is actually Bob Jenkins's hash (http://burtleburtle.net/bob/c/lookup3.c: hashlittle2: return 2 32-bit hash values)

[caali]

v1.1 has been released.

how do I even use it lol ?

Unless you know how to interpret it and code a program to read the files - you don't and it's not intended for you, sorry

"FileTable" file is actually called "encoding" file
"Manifest" file is actually called "root" file
There's also 2 more special files in data.XXX: "download" file and "install" file

"BlizzHash" is actually Bob Jenkins's hash (http://burtleburtle.net/bob/c/lookup3.c: hashlittle2: return 2 32-bit hash values)

Thanks! I suppose you found out the special file names through reverse engineering? All I re'd was the hashing function, so I just gave meaningful names to those files

[TOM_RUS]

Thanks! I suppose you found out the special file names through reverse engineering? All I re'd was the hashing function, so I just gave meaningful names to those files

Those files are referenced in "Build Configuration". It can be found in Data\config\80\6f\806f4fd265de05a9b328310fcc42eed0 or online.

[caali]

Ah I see, that makes sense. I guess that's also how to properly locate those files within the data.XXX files, instead of simply assuming they're at the beginning of data.000 like I did. I updated the specification with information about the 'data/config/' folder.

[raeef]

is there a way to get the full listfile

[TOM_RUS]

is there a way to get the full listfile

Bruteforce or hack blizzard.

[raeef]

Im trying to get the ogredraenorarmorer.m2 and thx for the big list you releast before are you going to update it

[CatsNimo]

Ah I see, that makes sense. I guess that's also how to properly locate those files within the data.XXX files, instead of simply assuming they're at the beginning of data.000 like I did. I updated the specification with information about the 'data/config/' folder.

The data.XXX files are dynamically generated from the real data files ('the archives') based on your locale and the demands of the client (since streaming is still an option). AFAIK, the index files are used to tell your installation which files that your specific set of data.XXX files contains.

Everything you need to know - all of the raw data files (before they become data.XXX), and the location of download, install, etc is obvious once you have the config files. They list the md5s of the files, which is all you need to know because Blizzard FTPs only use MD5s and not real filenames under NGDP.

Im trying to get the ogredraenorarmorer.m2 and thx for the big list you releast before are you going to update it

That file doesn't exist.

I've identified every single M2 file contained within the game now, and there are none with that name.

[raeef]

can you give me your list file plz

[miztook]

this is really helpful. thx!

[miztook]

could you please send me the attached files to my email? I cannot download them in China
[email protected]
i am the writer of WowModelExplorer, wow explorer | Free software downloads at SourceForge.net
hope to find something about WOD.
thanks in advance.

[raeef]

Zippyshare.com - 17719d1397581710-analysis-of-casc-filesystem-casc_filesystem_v1-2.zip can you download it from here

[TOM_RUS]

One important thing we missing now is ability to find encoding/root files in data.XXX without extracting everything. Pretty sure there's should be a way to find data index and offset by MD5 content hash using data from idx or index files.