File Extractor - Warlords of Draenor

[valyron]

Build 18935: added about 400 new file names and removed invalid names (496245 known, 5061 missing): download list file

Extended CASCExplorer by the ability to scan all files for file name strings. Is currently doing stupid byte reading and uses a simple state machine to detect file names, but can be easily extened for more sophisticated scanning. If relevant enough for you TOM_RUS, feel free to pull. Executable can be found here.

[TOM_RUS]

Build 18935: added about 400 new file names and removed invalid names (496245 known, 5061 missing): download list file

Extended CASCExplorer by the ability to scan all files for file name strings. Is currently doing stupid byte reading and uses a simple state machine to detect file names, but can be easily extened for more sophisticated scanning. If relevant enough for you TOM_RUS, feel free to pull. Executable can be found here.

Why don't you use my latest listfile? Only 3217 missing (2415 blp, 212 ogg, 1 wtf, 1 bls, 1 text, rest doesn't exist on CDN).

Edit: I've merged your's listfile with mine, now down to 3184 unknowns.

Edit2: Got 21 more .ogg names. Down to 3163 missing (build 18935).
Edit3: Build 18950: 505847/508528, 2681 missing.

[valyron]

Why don't you use my latest listfile?

Didn't realize that there was an updated list file in the v0.4-alpha release zip file. Would be easier if you would keep a copy in the github repository.

Edit: updated the listfile (right click on "Raw" and select "Save link as...") (including you file from github releases) for build 18950 with the file names I found reading the game files (504428 known, 4100 missing).

(The link color is really bad and links are not even underlined on this website... whoever is in charge, please change it.)

[akspa420]

With TOM_RUS's latest listfile, most of the unknowns look to be BLP textures. I'm not saying that all of them are BLP's, but most of them seem to be - and they seem to either be NPC's or character model textures, specifically face textures and body textures. There's also a few environmental and possibly post-fx textures in the mix. I'm using xnviewer to look at the files without changing the file extensions. At least then I can do some batch conversions of the files to weed out the blp's from the other unknown data. I suppose some sort of script could be written to check the magic in the file headers, but beyond that, name data I don't believe is in the files themselves :/ If only BLP's offered metadata like that in the files themselves.

[TOM_RUS]

With TOM_RUS's latest listfile, most of the unknowns look to be BLP textures. I'm not saying that all of them are BLP's, but most of them seem to be - and they seem to either be NPC's or character model textures, specifically face textures and body textures. There's also a few environmental and possibly post-fx textures in the mix. I'm using xnviewer to look at the files without changing the file extensions. At least then I can do some batch conversions of the files to weed out the blp's from the other unknown data. I suppose some sort of script could be written to check the magic in the file headers, but beyond that, name data I don't believe is in the files themselves :/ If only BLP's offered metadata like that in the files themselves.

There's 2023 blp, 113 ogg, 1 wtf, 1 bls and 1 "text" file unknown (for enUS locale). And 2681 total unknown files.

[akspa420]

I suspect that the wtf file is for Ashran - "worldport 1116, 1036.3, -141.223, -9.58112, 6.0732"
edit: I could check to see where this lines up with in terms of graveyards... from memory of how the xyz coords work, I'm guessing this is Ashran.

[TOM_RUS]

I suspect that the wtf file is for Ashran - "worldport 1116, 1036.3, -141.223, -9.58112, 6.0732"
edit: I could check to see where this lines up with in terms of graveyards... from memory of how the xyz coords work, I'm guessing this is Ashran.

Ashran is on map 1191. This is somewhere on Draenor (map 1116), in Shadowmoon 6.0.

[Trixiap]

There's 2023 blp, 113 ogg, 1 wtf, 1 bls and 1 "text" file unknown (for enUS locale). And 2681 total unknown files.

BLPs are probably hardcoded textures for NPC/Items/Mobs because you didn´t have them in listfiles.

[valyron]

I suppose some sort of script could be written to check the magic in the file headers

Yes I can! I added a menu option that scans the magic number of every unknown file and adds the correct extension if known. Now we can simple double click on unknown BLP's. Code can still be improved. Also merged TOM_RUS branch into mine and fixed a bug (and some more bugs) in the file scanner where the unknown files were not scanned for file names in them. But they hadn't any names anyway.

Download: CASCExplorer with Scanner including latest list file

Update: now really includes latest list file. Missed TOM_RUS's Edit3.

[Cromon]

For wdt/adt/wmo/etc:
you have:

{ new byte[] { 0x52, 0x45, 0x56, 0x4d }, "w__" }, // .wmo .wdl .wdt .adt

If you face MVER read the next chunk:
MHDR -> adt
MOHD -> wmo root
MOGP -> wmo group
MPHD -> wdt

if neither of them keep scanning chunks, if you find any of the following, its wdl:
MAOF, MARE, MAOC, MAHO

[valyron]

Hmm not sure.

All following files start with REVM, but...
World\wmo\Draenor\Arakkoa\6AK_Arakkoa_BigSpire_01.wmo does not contain any M***
World\wmo\Azeroth\Buildings\AltarOfStorms\AltarOfStorms.wmo does not contain any M***
World\Maps\AbyssalMaw\AbyssalMaw_29_29_obj0.adt does not contain any M***
World\Maps\AbyssalMaw\AbyssalMaw.wdt does not contain any M***

Haven't found any files wmo files with a M*** string in it, but I only checked a few. Do you have an example wmo that contains those byte sequences? I still think you are right about the chunks. But your list is not exhausting.

And I just recognized that since I only scan the enUS files I miss the esES, zhCN, etc. Fixing that in a second.
Edit: This is now fixed, pushed and released in executables. I quickly looked at the files and it seems that almost all unknown files have an extension now.

Edit 2: If it starts with Worldport or worldport I could rename it to wtf

[Cromon]

wmo: https://i.imgur.com/RqDBgzO.png MOHD
wmo: https://i.imgur.com/WHAJZNV.png MOHD
for the obj0 and tex0 you are right, those do not have the MHDR again, thats just in the root terrain file. obj0 contains MMDX and tex0 contains MTEX
wdt: https://i.imgur.com/rdgw0bm.png MPHD

[valyron]

Ahh I was reading big endian. Like a human being. Obviously, I'm not good at anagrams.
Edit: Okay I read some wowmodelviewer code like wmo.cpp. Now understand a lot more. Sorry for being stupid. Probably implementing that tomorrow.

[Vinitharya]

I don't mean to blunder into this thing, but since I have no coding experience, I need to ask directly: Do you think that all the music (I have the same datamining music goal as you valyron) can be found and extracted from the gamefiles now? I have a huge and well-organized WoW music gamerip collection, and all I can think to contribute is that same collection to those that care. Once I learn access to the files, I will be fine, but can someone with no understanding of the mechanics of the extractor still use it?

[valyron]

I don't mean to blunder into this thing, but since I have no coding experience, I need to ask directly: Do you think that all the music (I have the same datamining music goal as you valyron) can be found and extracted from the gamefiles now? I have a huge and well-organized WoW music gamerip collection, and all I can think to contribute is that same collection to those that care. Once I learn access to the files, I will be fine, but can someone with no understanding of the mechanics of the extractor still use it?

I think it is safe to say that all the music can now be found and extracted from the game files (it is in Sound\Music\Draenor). They could add more music before release, but I don't think so. Also I can say we know all currently existing music file names. There are some missing file names and also some missing sound files, but they are not music but short noises or foreign (non English) language speech.

You should be able to use the CASCExplorer (either Tom's version or my version) without any problems (you need Windows with .net Framework) on the beta files now and also on the prepatch files once the prepatch arrives.

My foremost goal is to be able to extract all music files since not all the game music is on the soundtracks (which I reached) and as a secondary goal, I want to make the magic accessible. That means I think everybody should be able to scan the game files for new file names or be able to detect the file type of unknown files. That does not mean that I do not appreciate Tom's work - on the contrary, I thank you Tom for implementing a working CASCExplorer. He is really good in programming and also seems to have some extar skills finding file names that are not in the game files. But even with an incomplete list file, you can now extract all mp3 and ogg files from the CASC file system.