Wow Armory Mining

[garoboldy]

I've not done any work with the armory from an xml stand point so I'm curious about a few things.

How is mass data being pulled currently for statistics and build data without knowing the player name. Is everything still being based around pulling the arena roster list and following it back through guild rosters, then to players? I'm interested in playing around with finding players that meet specific achievement criteria or gear, just for personal bored interest.

So far I've only seen specific toon search ability and not crawling based setups.

Anyone worked with this or know more in depth about the data mining source for crawling it.

FYI I only work in C# / Python anymore as it's what I do at work.

[Jadedknight]

I can give you a quick example, but to answer your question yes you have to start broad and narrow down to what you want manually through Blizzard's XML. I'll show you a small snippet from a project I am currently working on.

/* Set the user agent to get XML data from Blizzard */
ini_set("user_agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");

function get_battlegroups($link) {
    $url = "http://www.wowarmory.com/battlegroups.xml";
    $xml = simplexml_load_file($url);

    foreach ($xml->battlegroups->battlegroup as $battlegroup) {
        $ladder_url = mysqli_real_escape_string($link, $battlegroup['ladderUrl']);

        if (!mysqli_query($link, "INSERT INTO battlegroups (battlegroup_url) VALUES ('$ladder_url')")) {
            printf("Error: %s\n", mysqli_sqlstate($link));
        }
    }
} 

First of all you need to set a user agent to get the xml instead of formated html. Then the function basically grabs a list of every BG and saves it to a database.

Soon you will find that you are going to run into problems when you want to pull specific achievement/statistic data. To do this you need to append &c=# to the end of the achievement or statistic URL.

[ryx]

"Is everything still being based around pulling the arena roster list and following it back through guild rosters, then to players?"

Yes, you cant browse from high level except for arena rosters.

this makes crawlers flawed, as there are level80 guilds that simply dont have anybody in a arena team.