Writing Bots with Robot-js

[Frosttall]

But you a bad fish-botter if you position yourself incorrectly, you deserve to catch no fish.

Well my thoughts were rather long-term oriented and taking all use-cases into account instead of only fishing. Imagine you have to pick up objects on the floor - it would also require an interact with either TargetGUID and hotkey or mouse. So this problem would not only apply to fishing but to interaction with non-targetable objects in general.

[WiNiFiX]

Ye but then you have to do maths and geometry and trigonometry, thats confusing :P
But ye i know what you mean, i use lazy-raiders world2screen methods for all that LOS checking before clicks etc...

And kinda offtopic, but does Ownedcore have a discord channel?

[Torpedoes]

Was confused for the longest time because there's a robotjs and a robot-js.

Yeah, sorry about that. I had initially reserved robot-js on NPM before I even knew about robotjs' existence. That and Robot itself had been in development for quite a while before then. Although Robot has only been recently released, it has existed for a quite while, in fact, version 1 is still being used to power the Yeti-Bots applications. As for robotjs, it has a lot of the same functionality Keyboard and Mouse has but it's not really geared towards bot development since it's missing a lot of features that robot-js has.

A few recommendations, ARGB is too... difficult, to use. In the world of automation, a hex value of the color like 0xFFFFFF white would be much more user friendly. If you plan to stick with ARGB, perhaps provide a tool like AutoIt (AutoIt Window Info) that let's you extract PixelColor at mouse location and provide ARGB of it. Though many would prefer to provide a parameter value to get the hex value. As of right now I'm going to have to take the extra step to write a quick pixelcolor at mouse program, I think the website links to one provided as well, but gives the hex value instead.

Wait, what's the problem, I don't think I understand. Why is ARGB too difficult? And why are you using the color class, you can just manipulate the image data directly, as unsigned 32-bit integers. Here's an example:

const robot = require ("robot-js");

// Function to print numbers as hex
const printHex = v => console.log
	(v.toString (16).toUpperCase());

// If you need full-screen
// grabbing without params
robot.Screen.synchronize();

let image = robot.Image();
// Take screenshot of the screen
robot.Screen.grabScreen (image);

// Get the image pixel data
let data = image.getData();

// Get the last pixel in image
let n = image.getLength() - 1;

printHex (data[0]); // 0xAARRGGBB
printHex (data[1]); // 0xAARRGGBB
printHex (data[n]); // 0xAARRGGBB

printHex (data[0] & 0x00FFFFFF); // 0xRRGGBB
printHex (data[1] & 0x00FFFFFF); // 0xRRGGBB
printHex (data[n] & 0x00FFFFFF); // 0xRRGGBB

If you need to get the color at the position of the mouse, you could use a function like this:

const robot = require ("robot-js");

// Function to print numbers as hex
const printHex = v => console.log
	(v.toString (16).toUpperCase());

// Reuse same image instance
let image = robot.Image (1);
let data  = image.getData();

// Returns result as 0xRRGGBB
const getColorUnderMouse = () =>
{
	// Get the current mouse position
	const pos = robot.Mouse.getPos();

	// Get pixel color under mouse
	return robot.Screen.grabScreen
		(image, pos.x, pos.y, 1, 1) ?
		data[0] & 0x00FFFFFF : 0x0;
}

// Using GetPixel is slow because it has
// to make a native function call. Here,
// we're leveraging the fact that the
// image data buffers are reused so the
// data pointer will always be valid.

while (true) printHex (getColorUnderMouse());

I hope this has alleviated some of your concerns.

[StinkyTwitch]

How would I read NPC names out of the entry list? I keep trying:

name: memory.readString(entry + offsets.Entity.NPC.Name1, 80),

But getting strange results. Weird characters or no characters.

[Torpedoes]

How would I read NPC names out of the entry list? I keep trying:

name: memory.readString(entry + offsets.Entity.NPC.Name1, 80),

But getting strange results. Weird characters or no characters.

Hard to say but I'm pretty sure those examples still work without any changes or if there are changes, very minor ones to the memory offsets. You'd have to step through with a debugger and try and make sense of it. Use cheat engine to help you map out the data structures and make sure you're reading the correct values.

[murmir]

Hard to say but I'm pretty sure those examples still work without any changes or if there are changes, very minor ones to the memory offsets. You'd have to step through with a debugger and try and make sense of it. Use cheat engine to help you map out the data structures and make sure you're reading the correct values.

Having the same issue, it shows a load of weird characters - on OSX this is, also doesn't find the window / process, without going the long way around, i.e get active window etc.

Could you provide a working OSX/Mac example please? Would be super helpful.

[Torpedoes]

Having the same issue, it shows a load of weird characters - on OSX this is, also doesn't find the window / process, without going the long way around, i.e get active window etc.

Yeah OSX is a bit... different. I'm not sure how different though since I've never developed bots on that platform. But in theory, you should still be able to use the same principles from these examples to write a bot there. I'll try to write some examples when I have time but it might be a while.

[drooky]

+1 on a simple example on how to get it working on macOS / OSX.
That would be pretty awesome.

I was also wondering if this works for older versions of classic vanilla wow or only on current version?

Thanks for sharing!

[Torpedoes]

I was also wondering if this works for older versions of classic vanilla wow or only on current version?

Some data structures are quite different (or non-existent) in vanilla. Luckily the vanilla client has been reversed extensively so I'm sure you could adapt portions of the code to run on that version.

[tutrakan]

Some data structures are quite different (or non-existent) in vanilla. Luckily the vanilla client has been reversed extensively so I'm sure you could adapt portions of the code to run on that version.

I would like to take a look at this "extensively reversed vanilla client". Can I?

Edit:
I meaned .idb or .dbg. Sorry.

[murmir]

Yeah OSX is a bit... different. I'm not sure how different though since I've never developed bots on that platform. But in theory, you should still be able to use the same principles from these examples to write a bot there. I'll try to write some examples when I have time but it might be a while.

Ok cool, thanks, i'm a well seasoned JS dev (can link my github profile via PM ;p), not being able to connect to the process is the only thing holding me back really at the moment. The selectByFindProcess & Window functions in the example weren't selecting as getList for window or proc was not finding WoW at all. When I mashed it in to select by active window, the readStrings were returning unreadable characters - the offsets were correct.

I'd be happy to contribute to the JS side and help others, I just need to get started reading from the process ha. PM if you wana screenshare my mac etc if you don't have mac/osx

EDIT: its all coming back to me now! The find by process wouldn't work if wow was launched before running node script, would show only if you run node script then wow - but only occasionally worked, was weird.

[Torpedoes]

I would like to take a look at this "extensively reversed vanilla client". Can I?

I'm sure if you do a search online for 1.12.1 you'll find countless offsets and source code to reverse just about anything. For instance, ([WoW] 1.12.1.5875 Info Dump Thread)here's one I found on OwnedCore.

The selectByFindProcess & Window functions in the example weren't selecting as getList for window or proc was not finding WoW at all. When I mashed it in to select by active window, the readStrings were returning unreadable characters - the offsets were correct.

So the first thing that you must do is ensure you're running node with admin privileges. Next, start with the robot documentation and, more importantly, the node page. Play around with things like "robot.process.getList()" and then build a function similar to selectByFindProcess & window functions because those will be different on Mac due to the subtle differences. You will want to run with a debugger (or console.log) to find places where the code is failing.

[murmir]

I'm sure if you do a search online for 1.12.1 you'll find countless offsets and source code to reverse just about anything. For instance, ([WoW] 1.12.1.5875 Info Dump Thread)here's one I found on OwnedCore.



So the first thing that you must do is ensure you're running node with admin privileges. Next, start with the robot documentation and, more importantly, the node page. Play around with things like "robot.process.getList()" and then build a function similar to selectByFindProcess & window functions because those will be different on Mac due to the subtle differences. You will want to run with a debugger (or console.log) to find places where the code is failing.

Aye have done all that, am not a novice (am a node contributor), the problem is it can find the process fine,it just finds no windows when doing process.getWindows(), returns an empty array. findByProcess stops after trying to getWindows (because it doesn't find any windows) - and findByWindow fails because it also returns an empty array - the issue is with window finding always returning empty. AFAIK macs are limited to listing their own windows, or it only works using something like: Son of Grab

Would me spawning a process in node then using find by process window work there - wouldn't that be more risky as it's now a child of node? :confused:

[Torpedoes]

Aye have done all that, am not a novice (am a node contributor), the problem is it can find the process fine,it just finds no windows when doing process.getWindows(), returns an empty array. findByProcess stops after trying to getWindows (because it doesn't find any windows) - and findByWindow fails because it also returns an empty array - the issue is with window finding always returning empty. AFAIK macs are limited to listing their own windows, or it only works using something like: Son of Grab

Would me spawning a process in node then using find by process window work there - wouldn't that be more risky as it's now a child of node? :confused:

Robot doesn't let you spawn processes yet (and doing it through node isn't exactly what you're looking for). I'm a bit surprised that it doesn't work for you. I haven't tested this on Sierra but El Capitan works fine. Perhaps I'll have to do further investigation.

[murmir]

Robot doesn't let you spawn processes yet (and doing it through node isn't exactly what you're looking for). I'm a bit surprised that it doesn't work for you. I haven't tested this on Sierra but El Capitan works fine. Perhaps I'll have to do further investigation.

I was having the same issue before upgrading to Sierra - only upgraded to Sierra last night due to blizzard screwing up WoW on El Capitan (5 min load screens etc)

I can get around no window, spawning as a child_process in node kinda works - find by process now works, then I get modules, there's a ton of modules though not sure which one to use, tried the one named 'World of Warcraft' - then tried reading the game build but it returns an empty string, am using the latest 64bit offsets in your 7.1.5 thread, output below:

------------ MODULE -------------
modName World of Warcraft
modPath /Applications/World of Warcraft/World of Warcraft.app/Contents/MacOS/World of Warcraft
modBase 4294967296
modSize 0


------------ SEGMENTS -------------
name __PAGEZERO valid true segBase 0 segSize 4294967296
name __TEXT valid true segBase 4294967296 segSize 25460736
name __DATA valid true segBase 4320428032 segSize 8134656
name __LINKEDIT valid true segBase 4328562688 segSize 319488
containsAddress? 4311581900 false
GameBuild readString result:

Code for the above:

    console.log('');
    console.log('------------ MODULE -------------')
    console.log('modName', module.getName());
    console.log('modPath', module.getPath());
    console.log('modBase', module.getBase());
    console.log('modSize', module.getSize());
    console.log('');

    // Create a new memory object
    const memory = Memory(process);
    const offsets = Offsets.Offsets64;
    const segs = module.getSegments();

    console.log('');
    console.log('------------ SEGMENTS -------------')
    // Segs is an array
    segs.forEach(function (s) {
      console.log('name', s.name, 'valid', s.valid, 'segBase', s.base, 'segSize', s.size); // Load address
    });

    console.log('containsAddress?', module.getBase() + offsets.GameBuild, module.contains(module.getBase() + offsets.GameBuild));

    const build = memory.readString(module.getBase() + offsets.GameBuild, 6);

    console.log('GameBuild readString result:', build);

I have a feeling this is the wrong module, not sure which one to look at, there's several hundred:

World of Warcraft
AGL
AppleIntelHD5000GraphicsMTLDriver
GeForceMTLDriverWeb
AVFoundation
AVFAudio
Accelerate
vImage
libBLAS.dylib
libBNNS.dylib
libLAPACK.dylib
libLinearAlgebra.dylib
libQuadrature.dylib
libSparseBLAS.dylib
libvDSP.dylib
libvMisc.dylib
vecLib
AppKit
ApplicationServices
ATS
libFontParser.dylib
libFontRegistry.dylib
ColorSync
HIServices
LangAnalysis
PrintCore
QD
SpeechSynthesis
AudioToolbox
AudioUnit
CFNetwork
Carbon
CommonPanels
HIToolbox
Help
ImageCapture
Ink
OpenScripting
Print
SecurityHI
SpeechRecognition
Cocoa
CoreAudio
CoreBluetooth
CoreData
CoreDisplay
CoreFoundation
CoreGraphics
CoreImage
CoreMedia
CoreMediaIO
CoreServices
AE
CarbonCore
DictionaryServices
FSEvents
LaunchServices
Metadata
OSServices
SearchKit
SharedFileList
CoreText
CoreVideo
CoreWLAN
DiskArbitration
Foundation
GSS
IOBluetooth
IOKit
IOSurface
ImageIO
libGIF.dylib
libJP2.dylib
libJPEG.dylib
libPng.dylib
libRadiance.dylib
libTIFF.dylib
Kerberos
MediaAccessibility
MediaToolbox
Metal
NetFS
OpenCL
CFOpenDirectory
OpenDirectory
libCVMSPluginSupport.dylib
libCoreFSCache.dylib
libCoreVMClient.dylib
libGFXShared.dylib
libGL.dylib
libGLImage.dylib
libGLU.dylib
OpenGL
QuartzCore
ScriptingBridge
Security
SecurityFoundation
ServiceManagement
SystemConfiguration
VideoToolbox
AppContainer
AppSandbox
Apple80211
AppleFSCompression
AppleJPEG
AppleVA
Backup
ChunkingLibrary
CommonAuth
CoreAUC
CoreAVCHD
CoreEmoji
CoreServicesInternal
CoreSymbolication
CoreUI
CoreUtils
CoreWiFi
CrashReporterSupport
DataDetectorsCore
DebugSymbols
DesktopServicesPriv
FaceCore
libmetal_timestamp.dylib
GenerationalStorage
Heimdal
IOAccelerator
IOPresentment
IconServices
IntlPreferences
LanguageModeling
Mangrove
MetalPerformanceShaders
MultitouchSupport
NetAuth
PerformanceAnalysis
ProtocolBuffer
RemoteViewServices
SecCodeWrapper
Sharing
SkyLight
SpeechRecognitionCore
Symbolication
TCC
TextureIO
UIFoundation
XPCService
loginsupport
libCRFSuite.dylib
libChineseTokenizer.dylib
libDiagnosticMessagesClient.dylib
libFosl_dynamic.dylib
libMatch.1.dylib
libOpenScriptingUtil.dylib
libScreenReader.dylib
libSystem.B.dylib
libarchive.2.dylib
libate.dylib
libauto.dylib
libbsm.0.dylib
libbz2.1.0.dylib
libc++.1.dylib
libc++abi.dylib
libcmph.dylib
libcompression.dylib
libcoretls.dylib
libcoretls_cfhelpers.dylib
libcups.2.dylib
libdscsym.dylib
libenergytrace.dylib
libffi.dylib
libheimdal-asn1.dylib
libiconv.2.dylib
libicucore.A.dylib
liblangid.dylib
liblzma.5.dylib
libmarisa.dylib
libmecabra.dylib
libnetwork.dylib
libobjc.A.dylib
libpam.2.dylib
libpcap.A.dylib
libresolv.9.dylib
libsandbox.1.dylib
libspindump.dylib
libsqlite3.dylib
libxar.1.dylib
libxml2.2.dylib
libxslt.1.dylib
libz.1.dylib
libcache.dylib
libcommonCrypto.dylib
libcompiler_rt.dylib
libcopyfile.dylib
libcorecrypto.dylib
libdispatch.dylib
libdyld.dylib
libkeymgr.dylib
libkxld.dylib
liblaunch.dylib
libmacho.dylib
libquarantine.dylib
libremovefile.dylib
libsystem_asl.dylib
libsystem_blocks.dylib
libsystem_c.dylib
libsystem_configuration.dylib
libsystem_coreservices.dylib
libsystem_coretls.dylib
libsystem_dnssd.dylib
libsystem_info.dylib
libsystem_kernel.dylib
libsystem_m.dylib
libsystem_malloc.dylib
libsystem_network.dylib
libsystem_networkextension.dylib
libsystem_notify.dylib
libsystem_platform.dylib
libsystem_pthread.dylib
libsystem_sandbox.dylib
libsystem_secinit.dylib
libsystem_symptoms.dylib
libsystem_trace.dylib
libunwind.dylib
libxpc.dylib