Howto: Convert in-game to screen coordinates

[Vector0]

Here's code to convert in-game coordinates of an object as found by a memory reader, to a screen coordinate you can click on.

Note that there's sample code for this on another website. I tried it, and it kinda worked - but only if the object was near the center of the screen, the camera was zoomed out the right amount, etc. After failing to fix it, I just wrote my own from scratch. This code works perfectly under all conditions.

Code is in VB.NET. Plenty of comments are included to make it self-explanatory.

First, the public variables - along with what you should fill them with and where to get it:

  Public WowOrigin As Point ' set this to the upper-left corner of the client area
  Public WowSize As Point ' set this to the width and height of the client area
  Public WowCenter As Point ' set this to the screen coordinates of the center of the client area

  Public Const cCameraInfo As Integer = &HC76B48
  ' cCameraInfo+&h0 : X location (single-precision float)
  ' cCameraInfo+&h4 : Y location (single-precision float)
  ' cCameraInfo+&h8 : Z location (single-precision float)
  ' cCameraInfo+&h24: Facing (single-precision float)
  ' cCameraInfo+&h20: Tilt (single-precision float)
  '                   Take the Math.Asin() of this value to get proper tilt in radians.

  Public Structure strCamera
    ' Camera coordinates.
    Public X, Y, Z As Single
    ' The left-right rotation of the camera.  Range is from 0 to 2*PI.
    Public Facing
    ' The up-down rotation of the camera.  Range is from -0.5*PI to +0.5*PI.
    ' Note that the location I was able to find in memory actually ranges from -1 to +1.
    ' I'm not sure if I'm reading the correct address!  But taking the arcsine of the
    ' this value converts it to the proper range in radians.
    Public Tilt As Single
  End Structure
  Public Camera As strCamera ' set this to the camera location/rotation

And the function that does the conversion:

  Function GameCoordsToScreenCoords(ByVal X As Double, ByVal Y As Double, ByVal Z As Double) As Point
    ' Convert game coordinates to screen coordinates.
    ' Returns -1,-1 if not onscreen.

    ' The following transform was taken from Wikipedia's page on "3D Projection".
    '
    ' Setup was a little tricky since WoW and the transform coordinate and rotation
    ' systems were different, thus all the odd assignments.
    '
    ' point location
    Dim ax As Double = -X
    Dim ay As Double = -Z
    Dim az As Double = Y
    ' camera location
    Dim cx As Double = -Camera.X
    Dim cy As Double = -Camera.Z
    Dim cz As Double = Camera.Y
    ' camera rotation
    Dim Facing2 As Double = Camera.Facing
    If Facing2 > Math.PI Then Facing2 -= 2 * Math.PI
    Dim ox As Double = Camera.Tilt
    Dim oy As Double = -Facing2
    Dim oz As Double = 0
    ' Perform the transform to generate X,Y,Z of the point relative to the current
    ' camera position and rotation.
    Dim dx As Double, dy As Double, dz As Double
    dx = Math.Cos(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx)) - Math.Sin(oy) * (az - cz)
    dy = Math.Sin(ox) * (Math.Cos(oy) * (az - cz) + Math.Sin(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx))) + Math.Cos(ox) * (Math.Cos(oz) * (ay - cy) - Math.Sin(oz) * (ax - cx))
    dz = Math.Cos(ox) * (Math.Cos(oy) * (az - cz) + Math.Sin(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx))) - Math.Sin(ox) * (Math.Cos(oz) * (ay - cy) - Math.Sin(oz) * (ax - cx))
    ' Check to make sure it's not behind the camera.
    If dz <= 0 Then Return New Point(-1, -1)

    ' Project the point to a screen location.
    '
    ' This wasn't taken from Wiki, it came from somewhere else.  I forget where.
    ' FOV1 - The distance between the virtual eye, and the camera plane.
    '        If this is too large, you'll find the conversion works great at
    '        long distances, but falls short (towards the center of the screen)
    '        at short distances.  Not sure what this really is in WoW, but it's
    '        something small and 0.1 works just fine.
    Dim FOV1 As Double = 0.1
    ' FOV2 - Scaling to the screen.  This was found experimentally, at long (>20 yard)
    '        distance to avoid any potential errors from FOV1.  This must be changed
    '        if FOV1 changes!
    Dim FOV2 As Double = 7.4 * WowSize.X
    Dim SX As Integer = dx * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.X
    Dim SY As Integer = dy * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.Y
    ' Final check to make sure it's on-screen.
    If SX < WowOrigin.X Or SY < WowOrigin.Y Or _
      SX >= (WowOrigin.X + WowSize.X) Or SY >= (WowOrigin.Y + WowSize.Y) Then Return New Point(-1, -1)
    ' Return the result.
    Return New Point(SX, SY)
  End Function

Enjoy!

[UnknOwned]

Here's code to convert in-game coordinates of an object as found by a memory reader, to a screen coordinate you can click on.

Note that there's sample code for this on another website. I tried it, and it kinda worked - but only if the object was near the center of the screen, the camera was zoomed out the right amount, etc. After failing to fix it, I just wrote my own from scratch. This code works perfectly under all conditions.

Code is in VB.NET. Plenty of comments are included to make it self-explanatory.

First, the public variables - along with what you should fill them with and where to get it:

  Public WowOrigin As Point ' set this to the upper-left corner of the client area
  Public WowSize As Point ' set this to the width and height of the client area
  Public WowCenter As Point ' set this to the screen coordinates of the center of the client area

  Public Const cCameraInfo As Integer = &HC76B48
  ' cCameraInfo+&h0 : X location (single-precision float)
  ' cCameraInfo+&h4 : Y location (single-precision float)
  ' cCameraInfo+&h8 : Z location (single-precision float)
  ' cCameraInfo+&h24: Facing (single-precision float)
  ' cCameraInfo+&h20: Tilt (single-precision float)
  '                   Take the Math.Asin() of this value to get proper tilt in radians.

  Public Structure strCamera
    ' Camera coordinates.
    Public X, Y, Z As Single
    ' The left-right rotation of the camera.  Range is from 0 to 2*PI.
    Public Facing
    ' The up-down rotation of the camera.  Range is from -0.5*PI to +0.5*PI.
    ' Note that the location I was able to find in memory actually ranges from -1 to +1.
    ' I'm not sure if I'm reading the correct address!  But taking the arcsine of the
    ' this value converts it to the proper range in radians.
    Public Tilt As Single
  End Structure
  Public Camera As strCamera ' set this to the camera location/rotation

And the function that does the conversion:

  Function GameCoordsToScreenCoords(ByVal X As Double, ByVal Y As Double, ByVal Z As Double) As Point
    ' Convert game coordinates to screen coordinates.
    ' Returns -1,-1 if not onscreen.

    ' The following transform was taken from Wikipedia's page on "3D Projection".
    '
    ' Setup was a little tricky since WoW and the transform coordinate and rotation
    ' systems were different, thus all the odd assignments.
    '
    ' point location
    Dim ax As Double = -X
    Dim ay As Double = -Z
    Dim az As Double = Y
    ' camera location
    Dim cx As Double = -Camera.X
    Dim cy As Double = -Camera.Z
    Dim cz As Double = Camera.Y
    ' camera rotation
    Dim Facing2 As Double = Camera.Facing
    If Facing2 > Math.PI Then Facing2 -= 2 * Math.PI
    Dim ox As Double = Camera.Tilt
    Dim oy As Double = -Facing2
    Dim oz As Double = 0
    ' Perform the transform to generate X,Y,Z of the point relative to the current
    ' camera position and rotation.
    Dim dx As Double, dy As Double, dz As Double
    dx = Math.Cos(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx)) - Math.Sin(oy) * (az - cz)
    dy = Math.Sin(ox) * (Math.Cos(oy) * (az - cz) + Math.Sin(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx))) + Math.Cos(ox) * (Math.Cos(oz) * (ay - cy) - Math.Sin(oz) * (ax - cx))
    dz = Math.Cos(ox) * (Math.Cos(oy) * (az - cz) + Math.Sin(oy) * (Math.Sin(oz) * (ay - cy) + Math.Cos(oz) * (ax - cx))) - Math.Sin(ox) * (Math.Cos(oz) * (ay - cy) - Math.Sin(oz) * (ax - cx))
    ' Check to make sure it's not behind the camera.
    If dz <= 0 Then Return New Point(-1, -1)

    ' Project the point to a screen location.
    '
    ' This wasn't taken from Wiki, it came from somewhere else.  I forget where.
    ' FOV1 - The distance between the virtual eye, and the camera plane.
    '        If this is too large, you'll find the conversion works great at
    '        long distances, but falls short (towards the center of the screen)
    '        at short distances.  Not sure what this really is in WoW, but it's
    '        something small and 0.1 works just fine.
    Dim FOV1 As Double = 0.1
    ' FOV2 - Scaling to the screen.  This was found experimentally, at long (>20 yard)
    '        distance to avoid any potential errors from FOV1.  This must be changed
    '        if FOV1 changes!
    Dim FOV2 As Double = 7.4 * WowSize.X
    Dim SX As Integer = dx * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.X
    Dim SY As Integer = dy * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.Y
    ' Final check to make sure it's on-screen.
    If SX < WowOrigin.X Or SY < WowOrigin.Y Or _
      SX >= (WowOrigin.X + WowSize.X) Or SY >= (WowOrigin.Y + WowSize.Y) Then Return New Point(-1, -1)
    ' Return the result.
    Return New Point(SX, SY)
  End Function

Enjoy!

Haha, thoes calculations. They just stick to your face when you code this kind of stuff. Used it once in one of my fish bots, but found that calling the function for toggling the model was easier and avoided mouse control so i could do other stuff meanwhile.

But superb! Im sure many can use the math part of this post.

[Ermok]

I love you, this for 2008?
Can't you post the source.. though I should be fine.

I need to work out this memory editing/reading, starting with reading ;D
+2 rep

[Flos]

Nice post - thought about that too once, and I suppose it will work perfectly with a fishing bot etc

But my problem - queing for bg - remains: there are sometime ppl standing over the bg-person and if you click you select other characters...

But nice nonetheless, for farmbots etc... +rep

[Ermok]

I have a question, would you be able to create a simple application to read the memory and displaye some co-ords or something? i would be very grateful

[Ermok]

Also I am not sure how to compile this... QQ
Make me feel the noob. lmao
never had any problems before.
it is telling me that half the things aren't declared =/

[Vector0]

But my problem - queing for bg - remains: there are sometime ppl standing over the bg-person and if you click you select other characters...

You can verify the GUID of the object the mouse is hovering over by reading &hDD8B38, and see if it matches the GUID of the object you want. If not, you can try moving the mouse in a "search pattern" around the center coordinates, and see if you can find the right object.

However, if the object is completely covered, this won't work for you.

This may also be overkill. It sounds like you're targetting a specific NPC all the time, so will a simple /target macro work?

EDIT: /target macro doesn't work, still no way to interact. But Shift-V will bring up NPC nameplates, and the nameplate cannot be covered by a player. Using coordinates/GUID/mouse searching as above, you will be able to find and interact with your BG NPC.

[Vector0]

I have a question, would you be able to create a simple application to read the memory and displaye some co-ords or something? i would be very grateful

Maybe sometime in the future. There are examples already out there. Currently I'm focusing on posting information for which I have seen NO working examples.

This forum is my prime source of information, but here's two threads on other forums that particularly helped me understand and write my TLS memory object reader:

http://www.madx.dk/wowdev/forum/viewtopic.php?f=1&t=348
http://www.edgeofnowhere.cc/viewtopic.php?t=345914

Also I am not sure how to compile this... QQ
Make me feel the noob. lmao
never had any problems before.
it is telling me that half the things aren't declared =/

Put both blocks of code together in a module. I verified that it compiles with no unresolved references before I posted. Of course, you'll still have to fill in some variables with meaningful values before it will actually do anything useful.

[korknob]

hey vector, hoping you see this. anyway ive ported your function to c++ and cant get it to work right. ive even re-written it twice and spent about 6 hours tweaking it but still to no avail. any insight would be very appreciated.

point Convert3D2D(vect curVert)
{
	SetMouseOffsets(); // gets window client area
	UpdateCamera(); // gets camera position
	point ret;
	double ax = -curVert.x;
	double ay = -curVert.z;
	double az = curVert.y;
//    ' camera location
	double cx = -Camera.x;
	double cy = -Camera.z;
	double cz = Camera.y;
//    ' camera rotation
	double facing2 = Camera.facing;
    if (facing2 > PI)
		facing2 -= (2 * PI);
	double ox = Camera.tilt;
	double oy = -facing2;
	double oz = 0;
//    ' Perform the transform to generate X,Y,Z of the point relative to the current
//    ' camera position and rotation.
    double dx = cos(oy) * (sin(oz) * (ay - cy) + cos(oz) * (ax - cx)) - sin(oy) * (az - cz);
    double dy = sin(ox) * (cos(oy) * (az - cz) + sin(oy) * (sin(oz) * (ay - cy) + cos(oz) * (ax - cx))) + cos(ox) * (cos(oz) * (ay - cy) - sin(oz) * (ax - cx));
    double dz = cos(ox) * (cos(oy) * (az - cz) + sin(oy) * (sin(oz) * (ay - cy) + cos(oz) * (ax - cx))) - sin(ox) * (cos(oz) * (ay - cy) - sin(oz) * (ax - cx));
//    ' Check to make sure it's not behind the camera.
    if (dz <= 0)
	{
		ret.x = -1;
		ret.y = -1;
return ret;
	}
	
	double FOV1 = 0.1;
    double FOV2 = 7.4 * res_x;
//    Dim SX As Integer = dx * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.X
//    Dim SY As Integer = dy * (FOV1 / (dz + FOV1)) * FOV2 + WowCenter.Y
	int SX  = dx * (FOV1 / (dz + FOV1)) * FOV2 + (rect.left + (.5 * res_x));
	int SY  = dy * (FOV1 / (dz + FOV1)) * FOV2 + (rect.top + (.5 * res_y));
	if ((SX < rect.left) || (SY < rect.top) || (SX >= rect.right) || (SY >= rect.bottom))
	{
		ret.x = -1;
		ret.y = -1;
		return ret;
	}
	ret.x = SX;
	ret.y = SY;
    return ret;
}

edit: forgot to mention im using the following addresses

#define STATIC_CAMERA_X 0x00C7CC48
#define STATIC_CAMERA_Y STATIC_CAMERA_X+4
#define STATIC_CAMERA_Z STATIC_CAMERA_Y+4
#define STATIC_CAMERA_FACING STATIC_CAMERA_X+0x24
#define STATIC_CAMERA_TILT STATIC_CAMERA_X+0x20

[suicidity]

It looks something along the lines of what ISXWoW does.

Good work, +rep.

[ShoniShilent]

does anyone know if these addresses are still accurate now before i put alot of time into this portion?

thanks!

[korknob]

yeah my addresses are valid still but i couldnt get the code to work, i eventually said screw it and just have the mouse scan the screen until the GUID of what the mouse is on = desired_target_id. its slower but it actually works (decently fast, but i skip looting if im still in combat because i usually die by the time i get done looting and skinning. eventually ill make it so the bot kills everything within 20-30 yards then runs around looting mobs afterwards)

#define STATIC_MOUSE_GUID 0x00DDEC78
point ScanWithMouse(__int64 guid)
{
	SetMouseOffsets();
	SetForegroundWindow(hwnd);
	point ret;
	ret.x = -1;
	ret.y = -1;
	int min_x = (.25 * res_x) + x_offset;
	int max_x = (.75 * res_x) + x_offset;
	int min_y = (.25 * res_y) + y_offset;
	int max_y = (.75 * res_y) + y_offset;
	int x_interval = (max_x - min_x) / 8;
	int y_interval = (max_y - min_y) / 8;
	for (int x = min_x; x<max_x;x+=x_interval)
	{
		for (int y = min_y; y<max_y;y+=y_interval)
		{
			SetForegroundWindow(hwnd);
			SetCursorPos(x,y);
			sleep(50);
			if(CheckMouseGUID(guid))
			{
				ret.x = x;
				ret.y = y;
				return ret;
			}
			sleep(50);
		}
	}
	return ret;
}

bool CheckMouseGUID(__int64 guid)
{
	int read = 0;
	__int64 buf = 0;
	ReadProcessMemory(hProc, (LPCVOID)(STATIC_MOUSE_GUID), &buf, sizeof(__int64), (SIZE_T*)&read);
	if (buf == guid)
		return true;
	return false;
};

enjoy

[Shynd]

I can't get this working for me, either. As stated elsewhere, I'm no math genius, so most of this is all but meaningless to me unless I want to spend two hours single-stepping through the code (which I'd rather not do =p). It definitely changes the x/y output based on me moving or rotating left/right, but half the time it returns -1, -1 even when it's nearly right in the middle of my screen.

It's not really that big of a deal, seeing as I tore SelectUnit out of bobbysing's WoWX, but I'd still like to resolve it if at all possible.

[mrbrdo]

I've tried this also. However, i cannot find the Facing field. I use Kynox's description of camera in TLS, and this is how i get the properties, can't find the offset for facing:
Result.X = mem.GetSingle(pCamStruct + 0x;
Result.Y = mem.GetSingle(pCamStruct + 0xC);
Result.Z = mem.GetSingle(pCamStruct + 0x10);
Result.Tilt = arcsin(mem.GetSingle(pCamStruct + 0x1C));

[kynox]

Okay, this is probably going to be useless to most of you, i myself don't understand most of the math involved (except the basic trig).

bool CRenderer::WorldToScreen( idVec3 WorldLocation, idVec3& Screen )
{
    CCamera*    pCamera    =    NULL;

    __asm
    {
        MOV EAX, WorldFrame__GetCamera
        CALL EAX
        MOV pCamera, EAX
    }

    idVec3    Difference    = WorldLocation - pCamera->vecPos;
    if ( DotProduct ( Difference, pCamera->vecViewMatrix[0] ) < 0.f )
        return false;

    idVec3    View    = Difference * pCamera->vecViewMatrix.Inverse();
    idVec3    Camera    ( -View[1], -View[2], View[0] );

    if ( Camera.z > 0.f ) 
    {
        D3DVIEWPORT9 viewPort;
        m_pDevice->GetViewport( &viewPort );

        float    ScreenX    = viewPort.Width / 2.0f;
        float    ScreenY    = viewPort.Height / 2.0f;

        // Thanks pat0! Aspect ratio fix
        
        float    TmpX    = ScreenX / idMath::Tan ( ( (pCamera->fFov * 44.0f) / 2.0f ) * idMath::M_DEG2RAD );
        float    TmpY    = ScreenY / idMath::Tan ( ( (pCamera->fFov * 35.0f) / 2.0f ) * idMath::M_DEG2RAD );

        Screen.x    = ( Camera.x * TmpX / Camera.z ) + ScreenX;
        Screen.y    = ( Camera.y * TmpY / Camera.z ) + ScreenY;

        return true;
    }

    return false;
}

idMath::Tan is just atan
idMath::M_DEG2RAD is just 0.01745329251f

Now, the camera class

class CCamera
{
public:
    virtual void function0(); //
    virtual void function1(); //
    virtual void function2(); //
    virtual void function3(); //
    char unknown4[4]; //0x0004
    idVec3 vecPos; //0x0008  
    idMat3 vecViewMatrix; //0x0014  
    float ID02621370; //0x0038  
    float ID025E7F40; //0x003C  
    float fFov; //0x0040  
};//Size=0x0044(68)

idVec3 is just a 3D Vector (X,Y,Z)
idMat3 is just a 3x3 Matrix. (9 floats)

Now.. to obtain the camera pointer, you can either A) Call 0x006DD660 which will return the pointer or B) Read from [[0xDDEFF4]+0x732C] (Note, this WILL change. It is a HUUUGE class.