Finding map data in memory menu

User Tag List

Page 1 of 2 12 LastLast
Results 1 to 15 of 24
  1. #1
    Williamwillbera's Avatar Member
    Reputation
    2
    Join Date
    Jun 2019
    Posts
    7
    Thanks G/R
    5/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Finding map data in memory

    Hi!

    I understand that people do not want to share offsets about the game map, but maybe someone is willing to share some details on how the poe game map is represented in memory?
    For example, are walkable tiles still represented with a specific hex code?

    Thanks for any help

    Finding map data in memory
  2. #2
    pushedx's Avatar Contributor
    Reputation
    257
    Join Date
    Nov 2009
    Posts
    137
    Thanks G/R
    8/135
    Trade Feedback
    0 (0%)
    Mentioned
    12 Post(s)
    Tagged
    0 Thread(s)
    I don't feel terrain data is a secret anymore. Game has been getting reversed for over 8 years now and the system is still the same.

    3.10.1.4 (x64)

    First, constants to be aware of
    Code:
    int MapCellSizeI = 23; // map coords tiles are 23 x 23
    float WorldCellSizeF = 250.0f; // world coords tiles are 250 x 250
    
    float MapToWorldScalar = WorldCellSizeF / MapCellSizeF; // 10.869565f
    float WorldToMapScalar = MapCellSizeF / WorldCellSizeF; // 0.092f
    Next, getting the relevant data for reading terrain.
    You need to have the InGameState pointer already. If you need terrain data, I'm going to assume you already know how to get it, and if not, spend some time reversing the global state system, as it's a core system you need to code around.
    Code:
    InGameState + 0x378 = void* LocalData // Starting with the InGameState pointer, read a pointer from 0x378 to get the local data pointer
    
    LocalData + 0x610 = struct TerrainData (struct is 0x240 bytes) // read a struct with size 0x240 from localdata pointer + 0x610
    
    // From the aforementioned struct, the offsets relevant are:
    TerrainData[0x18] = long Cols
    TerrainData[0x20] = long Rows
    TerrainData[0xD8] = StdVector<byte> MeleeLayerPathfindingData
    TerrainData[0xF0] = StdVector<byte> RangedLayerPathfindingData
    TerrainData[0x108] = int BytesPerRow
    
    // Alternatively, you can just read each one using LocalData + 0x610 + offset, but I prefer the struct approach
    Also, in case you're not experienced with C++ containers (you'll need to learn them if you want to make use of a lot of things in memory), this is the underlying layout of a vector
    This is a C# struct, but you just want to have 3 type* pointers to read the array of data from First to Last, since the contiguous array in memory is from First to End, but Last marks where the valid data ends.
    Code:
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    internal struct StdVector
    {
    	public Native.Pointer First; // a custom wrapper for an IntPtr (8 bytes on x64)
    	public Native.Pointer Last;
    	public Native.Pointer End;
    }
    Now, the function that unpacks terrain data into walkable values. You'll end up with this logic if you reverse the terrain data processing system in the client.
    You can clean up and optimize things as you need, this code has a debugging friendly layout.
    Code:
    public static byte WalkableValue(byte[] data, int bytesPerRow, long c, long r)
    {
    	var offset = r * bytesPerRow + c / 2;
    	if (offset < 0 || offset >= data.Length)
    	{
    		throw new Exception(string.Format($"WalkableValue failed: ({c}, {r}) [{bytesPerRow}] => {offset}"));
    	}
    
    	byte b;
    	if ((c & 1) == 0)
    	{
    		b = (byte)(data[offset] & 0xF);
    	}
    	else
    	{
    		b = (byte)(data[offset] >> 4);
    	}
    	return b;
    }
    The data is upside down in memory, so to log it...
    Walkable values are 0 (unwalkable) to 5, where each number represents the max size of an object
    For example, player size is 2, so it can walk on any cell with a value of 2 or higher (so not 0 or 1)
    Code:
    for (r = Rows * MapCellSizeI - 1; r >= 0; --r)
    {
    	for (c = 0; c < Cols * MapCellSizeI; c++)
    	{
    		var b = WalkableValue(MeleeLayerPathfindingData, BytesPerRow, c, r);
    		var ch = b.ToString()[0];
    		if (b == 0)
    			ch = ' ';
    		sb.AppendFormat("{0}", ch);
    	}
    	sb.AppendLine();
    }
    That will yield something like this (Act 1 town): Screenshot - b528838eb38721ad669d48f70b4afa6f - Gyazo
    Note the map is rotated 45 degrees in game, so that's why I've rotated it to make it easier to verify

    If you zoom in around where the waypoint is: Screenshot - af426569773e64acc2ff4be36574656c - Gyazo

    Lastly, terrain data gets modified by various things at runtime, such as triggerable blockages (doors) as well as some other dynamic objects that only get spawned into the client when you're close enough to them in game. This means if you load into an area and build pathfinding data from the initial terrain data, the current state of the terrain data can be different as you move around, which leads to some annoying nav problems. It's a problem you'll have to solve or workaround to avoid stuck issues if you're doing a bot. You'll have more reversing to do to support that type of stuff though.

    That should be everything you need to get started at least. I'd highly recommend still debugging in x64dbg (make sure to spoof IsDebuggerPresent) to make your own sigs/layouts/offsets and better understand other relevant data in the client. Pathfinding is a real challenge in this game. Recast/Detour works, but requires some work to get up and going. You can probably find some help in the WoW section of this forum of people using it for another game if you're totally lost.

  3. Thanks Sychotix, Williamwillbera, TehCheat, Queuete, MrM000Cow, Stridemann, GameAssist, snowhawk, poeking99, p1-o2, Aoooooooo, GameHelper, houseradish, k1dnap (14 members gave Thanks to pushedx for this useful post)
  4. #3
    Williamwillbera's Avatar Member
    Reputation
    2
    Join Date
    Jun 2019
    Posts
    7
    Thanks G/R
    5/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    This is like an awesome christmas present. Cant wait to get home to play around with this. Thanks so much!

  5. #4
    GameHelper's Avatar ★ Elder ★ CoreCoins Purchaser
    Reputation
    2423
    Join Date
    Jun 2015
    Posts
    3,039
    Thanks G/R
    452/2168
    Trade Feedback
    0 (0%)
    Mentioned
    65 Post(s)
    Tagged
    1 Thread(s)
    Now, this is public, who wanna bet that they will change this in the next 1 or 2 releases.

  6. #5
    Williamwillbera's Avatar Member
    Reputation
    2
    Join Date
    Jun 2019
    Posts
    7
    Thanks G/R
    5/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    3.11.0e : The new offset for the terrain struct is 0x608 instead of 0x610. Everything else is the same as far as I can see.

  7. #6
    h42's Avatar Contributor CoreCoins Purchaser
    Reputation
    130
    Join Date
    Oct 2006
    Posts
    108
    Thanks G/R
    139/52
    Trade Feedback
    12 (100%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yeah, most leagues don't see any changes to this besides some small movement in relative offsets. I use a static hideout that I've saved, and then I search for the relevant info to automatically update at league start (width, dual pointers to data of expected size, etc)

    I was kind of expecting this to be changed given that it's public, but I guess with the other maphacks working just fine it's not a priority for them anyway.

  8. #7
    urgent2009's Avatar Member
    Reputation
    2
    Join Date
    Jan 2015
    Posts
    10
    Thanks G/R
    0/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I fail to find TerrainData struct at 0x610 or 0x608 in 3.11.1e

    Is it just me or did it change again?

  9. #8
    Sychotix's Avatar Moderator Authenticator enabled
    Reputation
    1415
    Join Date
    Apr 2006
    Posts
    3,941
    Thanks G/R
    285/571
    Trade Feedback
    1 (100%)
    Mentioned
    7 Post(s)
    Tagged
    0 Thread(s)
    Originally Posted by urgent2009 View Post
    I fail to find TerrainData struct at 0x610 or 0x608 in 3.11.1e

    Is it just me or did it change again?
    It normally changes every league. You'll probably have to relocate it.

  10. #9
    urgent2009's Avatar Member
    Reputation
    2
    Join Date
    Jan 2015
    Posts
    10
    Thanks G/R
    0/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally Posted by Sychotix View Post
    It normally changes every league. You'll probably have to relocate it.
    Would anyone mind telling a newbie, in very rough terms, what's the process behind doing so?

    Even the LocalData offset is now 0x500 and not 0x378 as mentioned above. And the only reason I know that is this.

    Do I just have to have a better understanding of the memory structures in PoE?
    Do I just have to get better at reversing?
    All of the above?

    I'm able to follow when someone explains in detail (duh) like in this post, or like in this (Learning to read memory from ExileAPI [Help]) post. I can iterate the NativeStateManager etc. But I have these huge gaps of knowledge and I get stuck and don't even know what to do to gain any progress.

    So I'm looking for any advice here.

  11. #10
    Sychotix's Avatar Moderator Authenticator enabled
    Reputation
    1415
    Join Date
    Apr 2006
    Posts
    3,941
    Thanks G/R
    285/571
    Trade Feedback
    1 (100%)
    Mentioned
    7 Post(s)
    Tagged
    0 Thread(s)
    Originally Posted by urgent2009 View Post
    Would anyone mind telling a newbie, in very rough terms, what's the process behind doing so?

    Even the LocalData offset is now 0x500 and not 0x378 as mentioned above. And the only reason I know that is this.

    Do I just have to have a better understanding of the memory structures in PoE?
    Do I just have to get better at reversing?
    All of the above?

    I'm able to follow when someone explains in detail (duh) like in this post, or like in this (Learning to read memory from ExileAPI [Help]) post. I can iterate the NativeStateManager etc. But I have these huge gaps of knowledge and I get stuck and don't even know what to do to gain any progress.

    So I'm looking for any advice here.
    The struct is in the first post Look around the area for something that matches that struct.

  12. #11
    pushedx's Avatar Contributor
    Reputation
    257
    Join Date
    Nov 2009
    Posts
    137
    Thanks G/R
    8/135
    Trade Feedback
    0 (0%)
    Mentioned
    12 Post(s)
    Tagged
    0 Thread(s)
    Originally Posted by urgent2009 View Post
    Would anyone mind telling a newbie, in very rough terms, what's the process behind doing so?

    Even the LocalData offset is now 0x500 and not 0x378 as mentioned above. And the only reason I know that is this.

    Do I just have to have a better understanding of the memory structures in PoE?
    Do I just have to get better at reversing?
    All of the above?

    I'm able to follow when someone explains in detail (duh) like in this post, or like in this (Learning to read memory from ExileAPI [Help]) post. I can iterate the NativeStateManager etc. But I have these huge gaps of knowledge and I get stuck and don't even know what to do to gain any progress.

    So I'm looking for any advice here.
    What I call "LocalData" is at 0x4F8 right now, and what was 0x378 before. The 0x500 you linked to in ExileApi, is what I called "InstanceInfo". Names can be anything you want, as everyone arrives at different names based on different things, but I always clarify since I use the same naming conventions when I talk about PoE stuff. There was an update some time after that initial post where InGameState got some extra data, and that resulted in the offset changing like you're seeing - pretty normal stuff in this game you have to work around.

    Everything you're asking comes down to just taking enough time to reverse/debug the client to understand what it's doing. For example, you know (because ExileApi and myself say so) a pointer to LocalData is stored at +0x4F8 of InGameState. Your question is basically "how do I know what offset the pointer is stored at"? The process of answering that question is going to be the same for just about anything you want to do: just figure out what the client is doing. In a sense, you could look at it as "getting better" at reversing, but in my opinion, getting better at reversing is basically just having more experience and time spent doing it.

    Of course, if you were starting from scratch, then simply knowing the pointer was being stored in InGameState would have been the first thing you needed to "discover". Those pointers are stored in multiple locations, so you'd just find a location that was the most accessible and easy to refer to over time, and then access it that way. Going through InGameState is very convenient, because the state pointers don't change once the states are created, so you can easily just do a few simple reads to get into the memory region you need to work with. If you then ask, how do you find InGameState or know some memory pointer is "InGameState", well that's something you'd discover as you reversed the game and started checking to see what data was held in various structures.

    At some point, the game has to allocate memory for that object, and then it saves the pointer to that memory into that specific memory location. For the LocalData/InstanceInfo pointers in InGameState, you (should) know they change each area. That means you can set a memory write hardware breakpoint for that address in a debugger (or maybe use CE if you're more familiar with that) to find what client code is actually saving the object. Once you track down that code, then it's just a matter of finding that code each update, and grabbing the offset from it. You should note though, due to the way C++ works, the offset you see in asm, might be shifted as a result of a the object being accessed through a derived class, which is why the object has a vtable in the first place (covered in the next paragraph). Making sense of things like that is just something you learn with experience.

    That's just one way of doing it. Another way involves understanding the basics of vtables in C++, more specifically, how MSVC generates them. If you look at the vtable of LocalData, and then search for all constants in the code section of the client, you will essentially find the constructor for the LocalData object. Once you know a function is a constructor, you can then check to see what calls it, and most of the time you should see some form of memory allocation, usually either on the heap or the stack, but can exist in an exe section for globals or even be part of another already allocated block of memory (as is the case for terrain data), that the function is called on. If you search for that code instead, you can then figure out what the memory layout of the object is, and then check pointers on patch changes to know if you have the right pointer still or not. Of course, this method only works for objects with vtables, not all objects you work with in the client will have that.

    The best way to avoid the two above methods, is to just reverse the entire InGameState layout itself. Once you understand it's memory layout, then you don't have to care about finding new offsets, because you'll indirectly already have the offset once you update that struct layout when it changes. This means if you already know how to find the InGameState pointer, then you don't have to have offset sigs for the struct itself, because you're just going to look at the memory layout each update and update it if it changed. This approach is more work in general, but doing it allows you to have less sigs and individual offsets as a whole, and it helps you understand the memory layout of more data than if you were to only take and use a few specific offsets. There's certain tools like reclass that help with this as well, but I'd always strongly suggest making your own tools for this stuff if you're in it for the long run.

    There's other things you can do, but those are the three main, basic ways you should at least be comfortable with. It doesn't matter what you do though, just that you create some process that you can consistently follow (preferably automated if it comes to sig searching) each client update to find whatever information you need. If the method you come up with breaks every client update, then you need to work on finding a better way so you have as little work as possible each update. Sometimes on big game updates, your methods of finding data will break, so having kept previous clients you've worked from and good notes is pretty important.

    Ultimately, what you have to keep in mind is, most of the stuff you see posted, comes from people who have just spent a lot of time with PoE, so there's no magic way to achieve the same level of understanding without also spending a lot of time on the game. In essence, what you're seeing is the end result of a lot of work that was done using a process that will vary person to person, and that process is essentially lost once a certain level of understanding is gained for the game. For example, my understanding of PoE dates back to 2012. Despite a lot of game changes over the years, there's still "enough" that is the same or similar, even when in x64, to where I can make sense of new client systems or find stuff without much trouble, generally speaking. It's still a bit of work, don't get me wrong, but there's a massive difference in my understanding of the game due to my time spent vs someone who might have only been working on stuff for a few months or even a few years, and there's no way to realistically close that gap in a short amount of time.

    Time spent reversing the game, and always working towards improving your understanding of the game (as opposed to just updating offsets to make something work) is what will help you fill in all your gaps. It's not easy reversing an entire game, and it doesn't matter how much time you spend on it, there's always going to be things you miss or didn't quite understand correctly the first time around. Using myself as an example, there's still new things I discover on a regular basis as I reverse the client that I just didn't know before, or I didn't make the connection between, so as a solo developer, trying to build up enough knowledge and experience single-handedly is quite an (impractical) endeavour, but there's no other way around it really. If there's something you want to do, you either have to learn how to do it yourself, find something someone else has already done you can use, or just give up on trying to do that exact thing and find a new way.

    Finally, building up broad experience with other games also helps give you different perspectives for approaching things. Despite how much time I have on PoE, I also had roughly the same amount of time years before across a ton of other games (none of which was bot specific, just reversing related). Reversing is a very experience driven field, that has a very high time cost to it, but if you don't have enough variety of experiences, and only stick to one thing, then it's going to be hard to grow and learn how to do different things. Some of this stuff might seem like magic, but there's very little magic to most of it. Rather, it's just the results of a lot of time and dedication working on something, so it's not inherently out of reach or impossible for anyone else to do or achieve for the most part.

    So to summarize the advice: you just need to spend more time on it, you need more general experience doing it, and lastly, you just have to be willing to struggle with it until you can put it all together, which might take place over many years and include lots of breaks in between. That's the reality I've seen at least over my entire time spent doing this stuff, maybe others have different advice.

  13. Thanks Sychotix, TehCheat, Stridemann, Williamwillbera, MrM000Cow, h42, GameAssist, NoobToken (8 members gave Thanks to pushedx for this useful post)
  14. #12
    MrM000Cow's Avatar Member
    Reputation
    2
    Join Date
    Mar 2007
    Posts
    5
    Thanks G/R
    9/1
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Terrain data offsets have changed slightly for 3.11.1f.

    Code:
    TerrainData[0x10] = long Cols
    TerrainData[0x18] = long Rows
    TerrainData[0xD0] = StdVector<byte> MeleeLayerPathfindingData
    TerrainData[0xE8] = StdVector<byte> RangedLayerPathfindingData
    TerrainData[0x100] = int BytesPerRow

  15. #13
    h42's Avatar Contributor CoreCoins Purchaser
    Reputation
    130
    Join Date
    Oct 2006
    Posts
    108
    Thanks G/R
    139/52
    Trade Feedback
    12 (100%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Does not look like any significant changes happened to this data with the asset restructuring patch, updating the LocalData pointer to be InGameState + 0x4D8 was the only change I needed to make personally. (-0x20)

  16. #14
    furukhai's Avatar Member
    Reputation
    1
    Join Date
    Oct 2020
    Posts
    3
    Thanks G/R
    0/0
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    perfect write @pushedx.
    Last edited by furukhai; 10-03-2020 at 07:39 PM.

  17. #15
    NoobToken's Avatar Member CoreCoins Purchaser
    Reputation
    8
    Join Date
    Nov 2010
    Posts
    52
    Thanks G/R
    11/9
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Possibly heightmap or tileinfo/tileid at 0x50?

    ```
    struct threeByteData {
    byte first;
    byte second;
    byte third;
    }

    Terrain[0x40] = long colPlusOne; //one bigger than cols
    Terrain[0x48] = long rowlPlusOne; //one bigger than rows
    TerrainData[0x50] = StdVector<threeByteData > heightMap; //size is colPlusOne * rowPlusOne * 3
    ```

    My guess is that col and row are 1 bigger here, because height might be interpolated from 4 tiles based on distance to them? I tried to make some sense from the values, but didn't have much luck.

    I drew the 3 byte data as [R][G][B] to a bitmap(amplified a bit to see the difference) and overlayed it over the walkable data:

    heightmap_gimped.png
    Last edited by NoobToken; 11-01-2020 at 03:43 AM.

  18. Thanks p1-o2 (1 members gave Thanks to NoobToken for this useful post)
Page 1 of 2 12 LastLast

Similar Threads

  1. Find mob data in memory
    By vivendi in forum GW2 Memory Editing
    Replies: 2
    Last Post: 01-18-2013, 07:36 AM
  2. Collision Data in Memory
    By matamore in forum WoW Memory Editing
    Replies: 13
    Last Post: 01-14-2012, 07:27 AM
  3. [Solution][3.3.0a][mac] Finding key bindings in memory
    By Tanaris4 in forum WoW Memory Editing
    Replies: 1
    Last Post: 01-29-2010, 11:56 AM
  4. [wow][mac] Help finding username/password in memory
    By Tanaris4 in forum WoW Memory Editing
    Replies: 14
    Last Post: 10-27-2009, 11:09 PM
  5. [Help Request] Find Cloud Objects in memory
    By boomingranny in forum WoW Memory Editing
    Replies: 5
    Last Post: 06-14-2009, 10:10 PM
All times are GMT -5. The time now is 03:38 AM. Powered by vBulletin® Version 4.2.3
Copyright © 2024 vBulletin Solutions, Inc. All rights reserved. User Alert System provided by Advanced User Tagging (Pro) - vBulletin Mods & Addons Copyright © 2024 DragonByte Technologies Ltd.
Digital Point modules: Sphinx-based search