Issues sending movement packets

[ramey]

You will have problems with using WoWX's way in the current state it is - it will crash due to threading issues perhaps.

[amadmonk]

definitely not low labor, but it can go by smooth sometimes. I don't remember the last time i had to update the authentication sequence, for example. Many things do break, however, when parsing packets across multiple version. object_update is _Definitely_ the biggest pain in the ass packet of them all. Most packets (at least ones i actually use) don't change much at all.

Yeah, actually, I'm coming back around. The update packet wasn't as bad as I thought (it looks like just a few more bytes to eat in the spline processing section -- wth is that, anyway?), and all I have left to fix is is the SMSG_AURA_UPDATE[_ALL].

I need to learn how to write IDA scripts so I can re-find the packet handler initialization routine (which is my trampoline into the packet handlers per opcode) next point release. I think that's why I hate maintaining offsets so much; since I don't know IDA scripting (I can use it manually just fine, but I've never written an IDA script), I have to re-find every important offset manually, every release. It adds up... I also wish I could figure out how to safely use IDA to debug WoW (but I'm sure that it's in the active windows hash list) since there's no substitute for a live system.

Still, it looks like the packet sniffing isn't so bad. I'm interested in the fact that you have a headless bot, you mentioned. How on earth did you have the patience to reverse every single packet structure? Or are you ignoring some?

[BoogieManTM]

Yeah, actually, I'm coming back around. The update packet wasn't as bad as I thought (it looks like just a few more bytes to eat in the spline processing section -- wth is that, anyway?), and all I have left to fix is is the SMSG_AURA_UPDATE[_ALL].

I need to learn how to write IDA scripts so I can re-find the packet handler initialization routine (which is my trampoline into the packet handlers per opcode) next point release. I think that's why I hate maintaining offsets so much; since I don't know IDA scripting (I can use it manually just fine, but I've never written an IDA script), I have to re-find every important offset manually, every release. It adds up... I also wish I could figure out how to safely use IDA to debug WoW (but I'm sure that it's in the active windows hash list) since there's no substitute for a live system.

Still, it looks like the packet sniffing isn't so bad. I'm interested in the fact that you have a headless bot, you mentioned. How on earth did you have the patience to reverse every single packet structure? Or are you ignoring some?

Splines are NPC static waypoints. Not sure what the extra bits on the end are lately (they've been messing with splines alot recently).

I'm ignoring alot of packets that aren't necessary for normal operation. eventually I'd like to get them all in, but who knows if that will ever happen. I used to work on private servers, before blizzard came knockin'.. so I already knew much of the protocol before I ever started on the clientless bot. It was a logical direction to go in being my experience and the lack of other bots of this nature.

[amadmonk]

Splines are NPC static waypoints. Not sure what the extra bits on the end are lately (they've been messing with splines alot recently).

I'm ignoring alot of packets that aren't necessary for normal operation. eventually I'd like to get them all in, but who knows if that will ever happen. I used to work on private servers, before blizzard came knockin'.. so I already knew much of the protocol before I ever started on the clientless bot. It was a logical direction to go in being my experience and the lack of other bots of this nature.

Well, I gotta admit, a headless bot (coupled with some kind of UI, mostly a radar and some kind of status about what the bot's thinking/doing, of course) would rock for a lot of paranoia-specific reasons. But given the number of CMSG/SMSG request/response pairs, I'd be nervous about using a live server. Ignoring SMSG_WARDEN_DATA, for instance, doesn't seem wise...

[BoogieManTM]

Well, I gotta admit, a headless bot (coupled with some kind of UI, mostly a radar and some kind of status about what the bot's thinking/doing, of course) would rock for a lot of paranoia-specific reasons. But given the number of CMSG/SMSG request/response pairs, I'd be nervous about using a live server. Ignoring SMSG_WARDEN_DATA, for instance, doesn't seem wise...

Ignoring warden has never got me banned, it just gets you disconnected after a few minutes. Warden falls under those "necessary" packets to not ignore, however I'm sure they do flag you for not responding to warden packets properly (and definitely if you respond improperly). But that's what trials are for! Still never lost a full account because of my bot *knocks on wood*

I do deal a vast number of the packets, at least parsing them in some way or another and storing their info somewhere. wither or not it's used in bot logic or in other situations (via the GUI or something) is another matter.

Most of the vital stuff is taken care of. object updates, movement packets (including players and mobs, as they're different packets), aura updates, spell castings, etc.. all the stuff you need for a grind bot. I still have to get around to the nitty-gritty of doing auction houses, vendoring, mail, banking, etc.. but that should be trivial compared to the vital stuff.

[namreeb]

So your bot gets disconnected after a few minutes and then reconnects?

[BoogieManTM]

So your bot gets disconnected after a few minutes and then reconnects?

nope - it stays on until i screw a packet up and the server drops me :P which can be almost indefinitely. still working out the kinks in some of the packets, like movement for example. the server is very picky about movement packets, and if you screw up, bam.. disconnect.

[amadmonk]

Ignoring warden has never got me banned, it just gets you disconnected after a few minutes. Warden falls under those "necessary" packets to not ignore, however I'm sure they do flag you for not responding to warden packets properly (and definitely if you respond improperly). But that's what trials are for! Still never lost a full account because of my bot *knocks on wood*

I do deal a vast number of the packets, at least parsing them in some way or another and storing their info somewhere. wither or not it's used in bot logic or in other situations (via the GUI or something) is another matter.

Most of the vital stuff is taken care of. object updates, movement packets (including players and mobs, as they're different packets), aura updates, spell castings, etc.. all the stuff you need for a grind bot. I still have to get around to the nitty-gritty of doing auction houses, vendoring, mail, banking, etc.. but that should be trivial compared to the vital stuff.

The warden data, in particular, is interesting to me. I've never reversed Warden, so all I know about it is hearsay; from what I understand most of the modules hash blocks of memory and send the results back to the mothership. How on earth do you do this without WoW actually loaded?

I guess if you reversed all the various warden modules and pickled the data to represent a "known good" state, you could just basically report back a hash that said "everything's fine here. We're fine. How are you?" (sorry for the Star Wars reference...) but it seems that that would be a pretty fragile design on their part (if I were writing warden, I'd hash at least some of Warden itself, which would imply that you couldn't just use a pickled hash code since it would change depending upon loaded modules). (Edit: errr, never mind that last bit, since Blizzard would have an equally hard time verifying these hashes, unless they were easy to compute, which would make faking them easy, too.)

And now that I think about it, how on earth do you do movement without the binary helping you out by doing collision detection and so forth? Unless you've parsed all the map data out of the binary (or made your own nav meshes or waypoints or something like that), your bot would always just try to move in straight lines.

I guess at the root I'm just blown away at the amount of work you must have put into your bot to even allow you to stay online for a few seconds, let alone long enough to grind...

[lanman92]

Who needs collision? Not a big deal to not have trees or fences. Maybe the WMOs may matter, but others don't much.

[amadmonk]

Who needs collision? Not a big deal to not have trees or fences. Maybe the WMOs may matter, but others don't much.

I was thinking more world geometry collision. The more I think about this, the more I realize that it's impossible without a waypoint system (or nav mesh, or some other way of representing "valid" locations). Otherwise you bot would just try to walk right through a mountain or whatever (which the client might well let you do, but which should trigger anti-cheat mechanisms on Blizzard's part).

Edit: I know that you could just read the ADT files (those are the ones that contain ground height, right?) to figure out the Z portion of your coordinates. Or, just use some kind of nav mesh that included the Z coordinate. However, I guess that any bot that wants to move around in the world would need this, so... hmm.

[BoogieManTM]

I was thinking more world geometry collision. The more I think about this, the more I realize that it's impossible without a waypoint system (or nav mesh, or some other way of representing "valid" locations). Otherwise you bot would just try to walk right through a mountain or whatever (which the client might well let you do, but which should trigger anti-cheat mechanisms on Blizzard's part).

Edit: I know that you could just read the ADT files (those are the ones that contain ground height, right?) to figure out the Z portion of your coordinates. Or, just use some kind of nav mesh that included the Z coordinate. However, I guess that any bot that wants to move around in the world would need this, so... hmm.

That's what I'm doing at the moment, using the ADT's to verify my Z is correct. I'm also working on a fork of mpqnav (MPQNav Public Blog) to do the final nav mesh for collision, line of sight (something you guys forgot about :P) and pathfinding. It's definitely a big TODO on my list, as it's definitely necessary to do. atm though, i just walk through stuff - fences, trees, walls, etc.. noclip ftw!

[amadmonk]

That's what I'm doing at the moment, using the ADT's to verify my Z is correct. I'm also working on a fork of mpqnav (MPQNav Public Blog) to do the final nav mesh for collision, line of sight (something you guys forgot about :P) and pathfinding. It's definitely a big TODO on my list, as it's definitely necessary to do. atm though, i just walk through stuff - fences, trees, walls, etc.. noclip ftw!

LOL... I guess that makes sense (I'm no WoW movement guru). Doesn't it look odd to any players who happen to be around that your bot doesn't avoid trees/fences/whatever like a "human" would and instead (successfully!) runs right "through" them? I would be paranoid that that would trigger someone to /report me... Probably still much simpler than doing pathing, though.

I'll have to look up MPQNav; I'm lazy, and if someone has already done the work...

[amadmonk]

Oh, and this reminds me. For working with the MPQ's... do you guys find that it's just easier to extract all the files out of the MPQ's with each patch?

I was reading Spells.dbc out of the MPQ to get spell info, but it was sllloooooow (reading and parsing a compressed 20mb-ish stream). I realized that if I had it pre-extracted, since it uses fixed offsets, I could do some magic with a memory-mapped file to speed up reading by a factor of lots, but I'm not sure if this tactic (pre-extracting the MPQ's) is a valid one.

Thoughts?

[BoogieManTM]

Oh, and this reminds me. For working with the MPQ's... do you guys find that it's just easier to extract all the files out of the MPQ's with each patch?

I was reading Spells.dbc out of the MPQ to get spell info, but it was sllloooooow (reading and parsing a compressed 20mb-ish stream). I realized that if I had it pre-extracted, since it uses fixed offsets, I could do some magic with a memory-mapped file to speed up reading by a factor of lots, but I'm not sure if this tactic (pre-extracting the MPQ's) is a valid one.

Thoughts?

Streaming is pretty slow in every implementation I've seen. you are better off extracting, or at least keeping what you need in memory once you've streamed it from the MPQ. I'm going with the shared resources route.. load everything once, and have all bots reference that shared memory. It's slow to load up initially, but it's quick as hell and saves ram in the long run (especially if you plan on running a few dozen+ bots)

[kynox]

The warden data, in particular, is interesting to me. I've never reversed Warden, so all I know about it is hearsay; from what I understand most of the modules hash blocks of memory and send the results back to the mothership. How on earth do you do this without WoW actually loaded?

I guess if you reversed all the various warden modules and pickled the data to represent a "known good" state, you could just basically report back a hash that said "everything's fine here. We're fine. How are you?" (sorry for the Star Wars reference...) but it seems that that would be a pretty fragile design on their part (if I were writing warden, I'd hash at least some of Warden itself, which would imply that you couldn't just use a pickled hash code since it would change depending upon loaded modules). (Edit: errr, never mind that last bit, since Blizzard would have an equally hard time verifying these hashes, unless they were easy to compute, which would make faking them easy, too.)

And now that I think about it, how on earth do you do movement without the binary helping you out by doing collision detection and so forth? Unless you've parsed all the map data out of the binary (or made your own nav meshes or waypoints or something like that), your bot would always just try to move in straight lines.

I guess at the root I'm just blown away at the amount of work you must have put into your bot to even allow you to stay online for a few seconds, let alone long enough to grind...

Many sleepless nights, obsessively looking deeper and deeper into the hole that is Warden. That is how you emulate it.