World of Warcraft Pixel Botting - A Cross-Platform Odyssey

[Wontuse]

I've managed to develop an intricate pixel bot and will use this thread to do a write-up. Topics I'd like to cover in responses are listed below. Info will come shortly, but not all at once.

Short video of bot in action and key things it can do:
- Movement via auto-run and mouse drag
- Target acquisition via tab, + side looks for a greater field of view
- Will ignore Player targets, mobs already tagged
- Orientate to face target




Reading World State
- Lua addon
Architecture
- python
- pyautogui: system interactions; mouse, keyboard
- matplotlib, geopandas, numpy; pathing
Targeting
- acquisition
- tabbing
- looking left/right
- pulling
- turning to face target
- range checks
- identifying NPC mobs vs players
- identifying tagged mobs
- interacting with friendly NPCs
Looting
Navigation
- creating paths
- following paths
- no use of click to move
- using auto run
- getting unstuck
- dijkstras
- zone, continent, world
Logic
- simple conditional logic
- rely on game mechanics as much as possible
- Behavorial Tree, GOAP, etc
Detection
- obfuscating the addon

On average, each cycle of this bot is 0.3 seconds. Well below the GCD timer. That includes time to capture the pixels, interpret them, and complete an action whether that is a mouse drag, button press, or do nothing (in the case of consuming).

My first pass at this was actually an instance bot which worked extremely well. I botted instances as a resto druid and could AFK 95% of the run. I relied on checking interact distance of party members, and used the follow target mechanic against the nearest to me party member to stay close to the group.

With this pass of the bot I have farmed XP from 1 to currently mid 40s. It can average ~10k to ~15k XP/hr depending on level of mobs being killed.

This bot will do extremely well in instances as a healer class or ranged dps and in the open world as a melee class. Ranged would do great as well, but looting would be a big miss.

Pathing is done via previously recorded paths. Through additional data scrubbing to identify path intersections, converting to GeoJSON, and then importing to a graphing library I can apply Dijkstra's to find a path. This works well to go from the Spirit Healer to the my corpse, from the path I'm grinding to a repair vendor, to a flight master, etc. It's currently zone specific but w/ additional pathing "layers" I will be able to go across zones via flight paths, or continents via boat/zeppelin.

[Wontuse]

Reserved - Reading World State


Check out this thread for info. This is pretty much how I built the add-on I use to extract data. (How to build your own WoW Bot)

With this, you can extract everything you need about the game: Booleans, Integers, Decimals, (3 Char) Strings.

I do plan to make this add-on available, but first I'd like to basically dynamically obfuscate it. This would result in every copy being entirely unique and prevent it being identified through static analysis.

[Wontuse]

Architecture - Python

The bot is written in Python, despite the allure of compiled languages like C#. From cross-platform compatibility to an extensive library ecosystem, Python was ideal for bringing this bot to life.


Cross-Platform Flexibility

One of the primary factors influencing my decision to use Python was its outstanding cross-platform compatibility. I run the game on Linux, occasionally develop code on macOS, and don't have access to a Windows computer. Python, being a cross-platform language, seamlessly accommodated my diverse computing environment. Python made it possible without the need for complex workarounds like dual-boot setups or virtualizations. And the possibility of being able to make the bot generally available, sans a packaging nightmare.


Extensive Library Ecosystem

Python boasts an extensive library ecosystem that's second to none. This wealth of libraries played a pivotal role in creating something for a game as complex as a World of Warcraft, Python provides libraries to implement essential functionalities, such as mouse and keyboard control, and data analysis for pathfinding.

Python's rich library ecosystem is invaluable. Libraries like pyautogui facilitate precise control of the mouse and keyboard, allowing the bot to interact with the game seamlessly. When it came to pathfinding and navigation, I can rely on libraries like networkx and matplotlib for graphing and data analysis to optimize the bot's pathing algorithms.


Balancing Compiled vs. Interpreted Languages

While a compiled language like C# may seem like the obvious choice for creating a bot, Python's strengths in library availability and cross-platform compatibility tipped the scales in its favor. True, a compiled language might offer performance advantages, but Python's interpreter offers more flexibility and rapid development capabilities. Ultimately, Python released me from needing to rely on OS specific libraries for screenshots and control of the mouse and keyboard. Even to do this in .NET Core would have been near impossible on an OS other than Windows.

Python's interpreted nature also allows for quick development. I can test and refine bot features with ease, making the development process smoother and more efficient. Being able to make changes and test them all within a single combat cycle is gamechanging.

[Wontuse]

Reserved - Targeting

[Wontuse]

Reserved - Looting

[Wontuse]

Reserved - Navigation

Still finding time to work on a write up, but here's some visuals on how the bot does pathfinding. I'll write more later, but in short I pre-record all of the paths. So long as each path intersects at one point with another pre-recorded path, then we can find all the intersections, turn these paths into a graph, and apply Dijkstra's to acquire a path.


Visually, this is what pre-recorded paths would look like when plotted on WoW's grid system 0,0 to 100,100. Obviously not in straight lines, but for demonstration purposes:



Converting the paths to a graph would visually look as follows, where each node is an intersection between paths, or the start/end of a path and each edge is all the waypoints in between. Each path will in turn be reduced to a shorter path at every intersection.



To find a path, take the players current position, find the closest waypoint on a known path, and apply Dijkstra's to get to the destination. This obviously falls short if you're generally not within a mostly straight clear shot.


The bot works well enough to navigate around Stormwind. Even on crappy i5 laptops with integrated graphics. Not complaining though, I haven't bought a computer in 10 years as work provides me with one and on occasion they never make it back when parting ways.




Pathfinding is handled slightly differently when taking into consideration whether we are aimlessly grinding mobs, or have a destination to get to i.e. Corpse, Repair Vendor, etc. The open world is pretty easy to stay clear of things that will get us stuck, as such it has much greater freedom when it comes to taking time to turn and look for mobs, jump, the stuck/unstuck routine is different. Whereas in a place like Stormwind, there are tight turns, sconces to get stuck on. Pathing here truly is and can be focused on one thing and one thing only, getting to the destination. Especially considering movement speed can vary, the bot needs to react turns accordingly and take this into consideration.

[Wontuse]

Reserved - Bot Logic // Planning

[Wontuse]

Reserved - Detection

[Wontuse]

Reserved - Future Topic

[Wontuse]

Reserved - Another Future Topic

[Wontuse]

Reserved - Additional Future Topic

[developerer]

Really interested in the followup. I've been developing my own pixel bot for a couple months now, with pretty similar capabilities. Interested to see how you've handled certain "problems" of the pixel bot.

[Humbugz]

Will be very interesting to follow this

[developerer]

Sorry to say but this navigation is not good at all. Are you holding W and using the mouse to turn?? That's ludicrous. It also looks like you're overturning constantly :/

[Wontuse]

Sorry to say but this navigation is not good at all. Are you holding W and using the mouse to turn?? That's ludicrous. It also looks like you're overturning constantly :/

Curious how you handle pathing in your pixel bot you were messaging me privately about and asking tons of questions?

It's using Auto Run and cursor drags. 1.25 degrees of rotation is very close to 5 pixels of cursor drag. I will concede it's not exact, but it works well enough, even considering the game was running at 15 fps and the character was on a mount that increased speed by 60% following a path the bot assumes the character is moving at normal speed. In Stormwind with tons of obstacles nonetheless.

Pathing actually works extremely well at normal run speed, especially in open world. Perhaps the overturning you are referring to were the glancing looks? Those expand the target hit range to essentially be 180* of the characters forward direction instead of being limited to the 30* or so field of view you normally only get with tab targeting.

The bot employs no image recognition beyond the initial pixel capture, so the only way to orientate towards a target is calculated turns. Fortunately, ranged attacks have a wider cone than tab targeting does. I think ranged attacks are something like 60* cone of view. Thus if a target is found when glancing left/right, the bot will stop, rotation 45* in that direction and rely on game mechanics of ranged attacks to further orientate towards the target.

If being attacked from behind, it rotates in 120* increments. Thus, at most the character will only have to rotate 2 times to be correctly orientated towards a target.

Pathing is something I'm actively refining and improving upon to take into consideration speed increases, tight turns/u-turns, etc. This is really only an issue in towns where there are lots of obstacles and obstructions. I've mostly been able to correct this by orientating towards N nodes ahead of the next closest node based on movement speed. I have not considered or even put any thought into pathing in buildings/caves aside from NPCs that are only a few steps inside the entryway of a building.

At normal speed, look ahead 2 nodes







This actually creates something like a natural bezier curve around turns that looks 100% natural and undistinguishable from a normal character being controlled by a human. The only time pathing looks like a bot, is as it approaches the destination node. It does need to stop, orientate, run a few yards forward, re-orientate, take a few steps forward to get on top of the destination. I suppose it looks similar to click to move, so not 100% bottish.