A bot that presses keys based on images?

[Jetlaw]

There are Addons in World of Warcraft which can be configured and scripted to advise players on which buttons to press.

The primary addon I'm thinking of is Ovale.

Ovale Spell Priority - Addons - Curse

It has an in-game scripting language that allows the addon to determine which ability you should be using at any given time. In order to do this, it shows the icon of the ability.

It's highly customisable - You can make the script show autoattack if you're in autoattack range, earth shock if you're not in autoattack range and even lightning bolt if you're not in earth shock range.

I was wondering about the viability of a bot that basically emulates a human - You point it to the part of the screen where Ovale is and assign it ability icons to look for. It will then facerolls the relevant keybinding associated with the icon while the icon is displayed.

This would be useful for automating abilities in PvP and PvE combined with Ovale's extensive scripting.

It wouldn't be intended to control your character, just to control the used abilites.

Is there anything currently existing that could provide a function like this?

[AfterMidnight]

Nothing that already exists, but something could be made.
Allot of fishing bots in the old days would scan the screen for where the pixels go out of the ordinary (zoom to ocean spot, fishing bobber would stand out).

Now this is more to the extent of watching for white in a field of black, red in a field of green, yellow in a field of blue ect ect.
There's a searchengine where you can upload your image and it'll search corresponding images with amazing precision; if this could be scripted into a bot however..

Another option would be that you have a set of abilities and corresponding .MPQs changing the icons of each ability to a solid color (replacing the artworks) which would make it allot more plausible, but limit the options I'd figure.


tl;dr It doesn't exist but could be made

[b3vad]

i think AutoIt can find out pixel changes from your screen and send keys but it needs lots of scripting for each key
and another thing is you have to disable your global cool down (visibility) and anything that make changes in icon of spells
just a little problem how you can simulate movement and facing the enemy?

or you can just spam clicks on the addon interface!

[Jetlaw]

The intent is for the bot to automate abilities while you control your character, mainly for things like arenas or raiding.

Ovale can be configured to detect when many impairing debuffs are on your character too, so it could be configured so it suggests you to trinket at relevant times, use cooldowns at relevant times etc. to create a bot that only uses abilities - And one that doesn't hook into WoW in any way. It just grabs a section of the screen and sends relevant keypresses when the area of the screen matches a known image.

And the same for PvP. It can detect buffs and debuffs on you and your target, your own health, your target's health.

The only abilities it wouldn't be able to use are those where you target a location on the floor, for obvious reasons.

I guess I could say it would be more like a whack-a-mole bot, rather than a fully automated bot that plays your character.

I'm just trying to out if it's viable. The concept seems like it would be a safe and secure ability bot, the only direct interaction with WoW would be the keypresses it sends and an indirect interaction with scanning a screen area for a specific image.

[Kwipper]

I would love to see this developed. I do love Ovale.

[noodleman1]

Not trying to necro this thing, but I'm in the process of creating something like this, except for the fact that it clicks instead of sending keystrokes. (Why? Well, most keybound abilities can still be clicked with the mouse, however, mousebound actions such as healing can't always be executed through keystrokes. Also, it's nice to be able to chat while your bot's woking.)

Currently this tool is able to:

-Let you create a list of colors you need checked at certain coordinates
-Take action (using left, middle and right clicks, also supports modifiers) if a color is (not) found
-Wait until a cast has finished before proceeding
-Start looking at what to do next based on the priorities in the color list (this is basically rinse and repeat from the top, where you put the most important and common actions)
-Export and import profiles

It works really well, I am currently letting it monitor about 20 spots on my 70ish resto shaman. It automatically heals, refreshes my own watershield and casts riptide and earthshield on the tank once the buffs fade. It can also dispel.
The only thing I have to do is /follow another character or steer for a bit.

Screenie of one of my first profiles, without dispelling and featuring just two heals:



This could be quite powerful combined with Ovale, currently I'm just watching unitframes and PowerAuras.

[ntsmarkv]

Check out NoAddiction + Ovale, = godlike, and 98.2% what your looking for. Mine doesn;t click but presses 1 when needed, like stated in OP

[Saear]

noodleman1 any chance of releasing a copy of Qlick? What would it take to add in key press functions?

[noodleman1]

It's functional but the follow-function I'm implementing (which tries to get to your focus target if it's out of range and uses /follow) currently uses hardcoded variables (meaning that this feature can't be configured by users yet so this isn't quite ready for release), I'll add options for it soon. Haven't looked into hotkeys yet, it requires time rather than skill to add, so I might do it next week when I'm not as busy irl.

[noodleman1]

Got keys working already and added a proper cooldown/checking progress text (so you can see what the program's up to). Just need a GUI and some inputs for the follow thing now, which is still quite buggy tbh, you can just disable it though.