EyeAuras - Automatic MP potions

[Xab3r]

Hi everyone,

Decided to drop this guide here as in the last 15 hours this already saved me do-not-even-know how many clicks due to how MP works in PoE2. Feel free to ask any questions here or in Discord

I'd want to show how to use the program I've been working on for the last 5 years. It is called EyeAuras and it is an automation framework which can be used for setting up everything from automatic potions all the way to full-blown bots written in C# leveraging machine learning models such as Yolo for real-time object detection and segmentation.

It has been used extensively by a private group in PoE1 as well - auto-potions, auto-guards, auto-vaal skill usage, follow-bots, aura-bots, automated pickup, rolling maps etc.

Guide


Important: EyeAuras operates on a freemium model, meaning you can access most features without registering. Something simple like auto-potions and even automated rotation could be done in the free version!
Feel free to give it a spin to see if it fits your needs.

Target users:
- those who can't code at all - you can still use macroses, auras, behavior trees and computer vision tools such as color search and image search. No coding there needed. Here is an example of a full-blown bot developed by one of our community members (namely DJKeterson) which using primarily BTs - Eye Auras

- those who can do a little bit of coding - all tools in EA support scripting. You can use C# to calculate something, draw overlay on the screen.
Here is a fishing bot for throne and liberty implemented by myself - Eye Auras
It uses ML model that I've trained using my other tool called YoloEase plus a single C# script which has all the fishing logic in it and controls when and what to capture/process.

- C# developers. In the end, EA can be used as just a framework providing a whole bunch of different utility tools such as different (~10 at this point) ways of simulating user inputs (software methods, drivers and a few hardware devices).
You can distribute whatever you've decided to implement using EA as a so-called packs - basically a combination of EA + all needed auras/scripts packed together. This has been battle-tested in Throne and Liberty already with the most popular pack (my fishing bot) having more than 12000 downloads in just a single month.

What else can I do with EyeAuras ?
For Throne and Liberty I've recorded a bunch of videos setting up automated rotations (not just potions, and entire combat rotation) and other things such as using Cloud gaming services for an extra layer of safety.
Unfortunately, due to NCSoft DMCAing videos, I had to take them down, but they are still available in media-guides section in our Discord.

Here is a link to one of the
Video here or in Discord thread



What is EyeAuras?

EyeAuras is a tool for creating and sharing pixel-based bots. For those familiar with World of Warcraft, you can think of EyeAuras as a supercharged version of WeakAuras, equipped with a machine-learning powered image search and a variety of user input emulation methods. It's been tested in games like Path of Exile, Lineage 2/M/W and many other and is now I think it is time to share it with the wider gaming community.

Features include:

• Real-time image capture (supports 4k60fps even with effects applied) - multiple methods, including full-screen capture
• A suite of effects to apply to captured images (denoise/blur/remove specific color/etc)
• Color Search: Matches the average color in a specific area against an expected color
• Image Search: Finds an image on the screen with a specified similarity
• ML Search: Uses a custom pre-trained Yolo ONNX machine learning model, GPU-accelerated
• Text recognition/matching: Performs OCR on the captured image and configures matching expressions
• Global hotkeys: Allows you to configure reactions to key/mouse presses and movement
• A variety of user input simulation types - SendInput, WinAPI messaging, several drivers(TetherScript, for example) and even support for Arduino-based USB keyboard emulator which you can make for yourself for 3$ from the nearest radio shop or AliExpress - I'll provide instructions and the firmware
• Multiple mouse movement smoothing algorithms
• Network messaging: Send and receive messages between EyeAuras instances - could be used to synchronize actions of multiple windows/bots running together
• Telegram messages: Send and receive messages via a Telegram channel
• Volume level detection: Listens for sound output from a specific process or the entire system
• Overlays: From basic "Some text/image on the screen" to "Real-time replica of another window/part of the window" to full-blown custom HTML/Razor/C# user interface
• Import/Export functionality via a short link
• And, for power users - a huge scripting subsystem that uses the latest version of C# and allows you to write the code which will have access to every single bit and function inside EyeAuras. You can write down the code for custom actions and even develop a custom UI that will be shared alongside your auras pack

How to get started?

• Download installer for a stable version here
• Install the program, it should start automatically afterwards, if it did not happen, start it manually by typing "Eye Auras" in Start menu
• Basically, you're good to go. I would highly recommend to check out videos on the channel to get a hang on concepts

Resources:

• EyeAuras website - landing page
• Downloads / changelog page - you can scroll there a bit to get a hang of what was recently added/changed
• Aura Library - this page focuses on providing a library of Auras which are publicly shared by some other users.
• YouTube videos on EyeAuras channel
• Just for fun, video which visualizes the development progress over time since 2019 when it all started - each sphere is a text file with the codehttps://www.youtube.com/watch?v=Lhu7oNWjRnQ
• Wiki - has some information about basic concepts of EyeAuras and short description of main triggers/actions
• Discord, my ID is Xab3r#3780 - feel free to join and ask any questions. Do not hesitate to ask something and you'll probably get the answer

[lostinthewoodslol]

Simple question, i have the bot, it's running, it seems like it's attached to POE 2 and im running the import for HP/MP.

Now how do you assign the hotkeys and thresholds for the pots?

p.s your video tutorial for POE is unavailable.

Thanks

[Xab3r]

Hm, double-checked and it seem to be available - new views, some comments, etc.
Just in case uploaded copy of the video to media-guides in discord as well

> Now how do you assign the hotkeys and thresholds for the pots?
That exact info is outlined in the video ) You select a region, set hotkey in the tree and that is it.

[Xab3r]

For god-knows what reason, it seems that for some users that YT video is not available despite everything being set to public - uploaded it to Discord and here is a direct link to the video

[Xab3r]

Added new video to the channel which shows how to:
- add HP tracking / auto-hp
- ensuring that bot presses buttons only while POE window is active




Just in case video won't be available for someone, uploaded it to Discord and cloud storage

[xxc2]

antivirus detects a virus

[Xab3r]

The program is sometimes detected by Windows Defender as malware, the specific name of detected malware could be totally random, I've seen more than a dozen different types at this point. All of them containted "!ml" at the end of the name, meaning that the detection was made by some kind of machine-learning algorithm, which is by its nature not precise.

Reason #1 - encryption
EyeAuras is using encrypted container to store its files that are needed for normal operation (DLLs, configuration, etc), that is why you can see exactly 1 file (EyeAuras.exe) and not more than 500 files that are needed for the program to work. Weight of the final executable is around 500mb, meaning a lot of space for potential false-positive signature detections.

Reason #2 - by its nature, the program is using a lot of "shady" techniques
Look at the program from antivirus perspective, here is what EA does:

1. intercepts/modifies user input - via HotkeyIsActive and system hooks - otherwise it would not be able to react so quickly on your keypresses
2. emulates user input using ~10 different techniques, some of which are loading drivers into memory for operation - for SendSequence
3. can capture the screen - for Capture triggers
4. can capture the sound - for Volume trigger
5. integrates with messengers such as Telegram and does network interaction with remote servers (e.g. for Network Message)
6. has built-in scripting system which is used to compile and load custom code on-the-fly
7. decrypts and loads some modules on the-fly with some code modifications in-between to change their signature

Despite the fact that these functions are active only when corresponding triggers/actions are used, they are still present in the executable and could be detected by antivirus programs, raising all kinds of alarms as from its perspective the program is extremely powerful and dangerous

What to do?
I do not have a good solution for this problem yet. There are only a few legit ways of counteracting Defender behavior and none of them will work for a program such as EyeAuras - they will only lead to easier detection by anticheats and that is it.
So, realistically, you have two options:

1. Do not use the program at all.
2. Accept risks and add the program to exclusion list in Defender, disabling its protection mechanisms for a specific folder where the program is started from.

To lessen risks, I have a few ideas such as publishing virus report (similar to virus total) made by some 3rd party site which will analyze contents of the program folder BEFORE virtualization/containerization. This will at least ensure that build server which built this specific version of the program is not infected by something. This won't affect Defender false-positives in any way though.
Running any kind of static analysis AFTER virtualization won't yield any meaningful results though

[GameHelper]

> To lessen risks, I have a few ideas such as publishing virus report (similar to virus total) made by some 3rd party site which will analyze contents of the program folder BEFORE virtualization/containerization. This will at least ensure that build server which built this specific version of the program is not infected by something

btw you can automate the uploading of such reports/build info/stuff on OC via the script i created some time ago.
Ref: https://www.ownedcore.com/forums/new...ml#post4450792 (Owned Core API access)
i am using it to auto-upload the gamehelper binary for POE-1 (not posting url so it's not considered a self advertisement).

[Xab3r]

>
btw you can automate the uploading of such reports/build info/stuff on OC via the script i created some time ago.
Ref: new...ml#post4450792

Thanks, good idea, will look into it after the initial poe-2 frenzy)

i am using it to auto-upload the gamehelper binary for POE-1 (not posting url so it's not considered a self advertisement).

I'll post the link for you Your helper is a great stuff and is definitely worth advertising - https://www.ownedcore.com/forums/mmo...exile-api.html

[Kintaro-Oe]

excellent tool - worth checking out.
i can also confirm that there is nothing wrong with it. false positives on "unknown" applications is pretty common and there is no way around it.
a small side note: visually auto potting could be challenging since being shocked changes the life bar appearance. poison as well i think.

[Xab3r]

excellent tool - worth checking out.
i can also confirm that there is nothing wrong with it. false positives on "unknown" applications is pretty common and there is no way around it.

Thank you

a small side note: visually auto potting could be challenging since being shocked changes the life bar appearance. poison as well i think.

Yeah, this could be solved by changing the approach - we've discussed this recently and it seems that the best way is to use _multiple_ color replacements in a row.
1) 1st replaces red with white
2) 2nd replaces green with white
3) 3rd replaces everything non-white with black


also I've found a bug in Replay functionality which would make picking green(poisoned) color much-much easier - will do the fix in alpha version. For now you can just replace all effects with this:
1) Select all text in the codebox below and copy it to clipboard (Ctrl+C)
2) Remove all current effects
3) Paste copied effects by pressing on this button

[
  {
    "AssemblyName": "EyeAuras.OpenCVAuras",
    "TypeName": "EyeAuras.OpenCVAuras.ImageEffects.ColorRangeReplacementImageEffectProperties",
    "Version": 1,
    "ConfigValue": {
      "MatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 255.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 255.0,
          "V3": 0.0
        }
      },
      "NonMatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 0.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 0.0,
          "V3": 0.0
        }
      },
      "ApplyMatchingColor": true,
      "ApplyNonMatchingColor": false,
      "Version": 1,
      "MinimumColor": {
        "Hue": 141.6,
        "Satuation": 73.5,
        "Value": 19.5,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 141.6,
          "V1": 73.5,
          "V2": 19.5,
          "V3": 0.0
        }
      },
      "MaximumColor": {
        "Hue": 180.0,
        "Satuation": 255.0,
        "Value": 255.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 180.0,
          "V1": 255.0,
          "V2": 255.0,
          "V3": 0.0
        }
      },
      "PickColorPreset": "Wide",
      "RegionOfInterest": "0, 0, 0, 0",
      "IsEnabled": true,
      "Id": "202412081041250HZ8KC0qlkat",
      "Bindings": []
    }
  },
  {
    "AssemblyName": "EyeAuras.OpenCVAuras",
    "TypeName": "EyeAuras.OpenCVAuras.ImageEffects.ColorRangeReplacementImageEffectProperties",
    "Version": 1,
    "ConfigValue": {
      "MatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 255.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 255.0,
          "V3": 0.0
        }
      },
      "NonMatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 0.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 0.0,
          "V3": 0.0
        }
      },
      "ApplyMatchingColor": true,
      "ApplyNonMatchingColor": false,
      "Version": 1,
      "MinimumColor": {
        "Hue": 32.0,
        "Satuation": 92.5,
        "Value": 0.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 32.0,
          "V1": 92.5,
          "V2": 0.0,
          "V3": 0.0
        }
      },
      "MaximumColor": {
        "Hue": 48.0,
        "Satuation": 255.0,
        "Value": 214.5,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 48.0,
          "V1": 255.0,
          "V2": 214.5,
          "V3": 0.0
        }
      },
      "PickColorPreset": "Wide",
      "RegionOfInterest": "0, 0, 0, 0",
      "IsEnabled": true,
      "Id": "202412100926231l0nOCM4tkvP",
      "Bindings": []
    }
  },
  {
    "AssemblyName": "EyeAuras.OpenCVAuras",
    "TypeName": "EyeAuras.OpenCVAuras.ImageEffects.ColorRangeReplacementImageEffectProperties",
    "Version": 1,
    "ConfigValue": {
      "MatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 255.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 255.0,
          "V3": 0.0
        }
      },
      "NonMatchingColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 0.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 0.0,
          "V3": 0.0
        }
      },
      "ApplyMatchingColor": false,
      "ApplyNonMatchingColor": true,
      "Version": 1,
      "MinimumColor": {
        "Hue": 0.0,
        "Satuation": 0.0,
        "Value": 127.5,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 0.0,
          "V2": 127.5,
          "V3": 0.0
        }
      },
      "MaximumColor": {
        "Hue": 0.0,
        "Satuation": 127.5,
        "Value": 255.0,
        "Dimension": 3,
        "MCvScalar": {
          "V0": 0.0,
          "V1": 127.5,
          "V2": 255.0,
          "V3": 0.0
        }
      },
      "PickColorPreset": "Wide",
      "RegionOfInterest": "0, 0, 0, 0",
      "IsEnabled": true,
      "Id": "20241210092630nsrOJ4H52jWD",
      "Bindings": []
    }
  }
]

Alternative would be to track background of HP bar above player's head.

[Xab3r]

Added new video:
- setup hotkey to teleport to hideout using /hideout
- setup interception conditions in your HotkeyIsActive to make key-binds work only while POE window is active



Just in case video won't be available for someone, uploaded it to Discord and cloud storage

[ravi372]

This is probably one of the most advanced automation tools I've seen for Windows. Great stuff, OP.

[magmia]

this works well mechanically, but how do i get the annoying yellow box around a selected window to dissapear? i can't find it anywhere in any setting related to capture, global settings or otherwise.

edit - seems like this is a windows 10 baked in feature for apps screencapping windows. does this tool not have a monitor capture i can use? this is highly disruptive and as far as i can tell in win10 you cannot remove this feature whatsoever. i'm guessing this app would need DXGI duplication or something similar - i've tried all the possible options of capture methods, none work

edit2 im literally retarded you need to apply the desktop duplication function to ALL methods. leaving this post here for others who come across the same problem i have.

thanks for the shout out of this app - it's neat.

[Xab3r]

this works well mechanically, but how do i get the annoying yellow box around a selected window to dissapear? i can't find it anywhere in any setting related to capture, global settings or otherwise.

edit - seems like this is a windows 10 baked in feature for apps screencapping windows. does this tool not have a monitor capture i can use? this is highly disruptive and as far as i can tell in win10 you cannot remove this feature whatsoever. i'm guessing this app would need DXGI duplication or something similar - i've tried all the possible options of capture methods, none work

edit2 im literally retarded you need to apply the desktop duplication function to ALL methods. leaving this post here for others who come across the same problem i have.

thanks for the shout out of this app - it's neat.

Yeah, as you've already correctly figured out, that yellow border is a feature of Windows 10 (disabled by default on Windows 11).
The simplest way of getting rid of it is changing Capture Method to something other than Windows Capture, usually SharedSurface or DesktopDuplication work just as well.
Also, keep in mind you do not have to change capture method in ALL auras 1-by-1, you can just set it on root folder level and it will be automatically applied to all new auras/bts you'll create. To apply it to all existing items click on Apply to all