[C#] Enigma.D3

[CrEEzz]

Meanwhile, I'm trying to figure out if there is something representing quest/bounty objective marker visible on minimap (I need to know position/scene_id of current objective ). I dumped all ACD but didn't find anything that might look like a marker (didn't check attributes tho). I looked through Quest class but no luck so far. Perhaps there is something in Sno,QuestObjectiveStep or somewhere among UI elements. Anyone can help me out with this?

[CrEEzz]

Hi, I think tomorrow I will be ready to release beta version of nav library. However Ocassionaly I get Exception on callstack:

 	Enigma.D3.dll!Enigma.ProcessMemory.ReadBytes(int address, byte[] buffer, int offset, int count) Line 82	C#
 	Enigma.D3.dll!Enigma.MemoryBase.ReadBytes(int address, int count) Line 77	C#
 	Enigma.D3.dll!Enigma.MemoryBase.Read<int>(int address) Line 41	C#
 	Enigma.D3.dll!Enigma.MemoryObject.Field<int>(int offset) Line 140 + 0x5e bytes	C#
 	Enigma.D3.dll!Enigma.D3.Sno.SerializeMemoryObject.Field<int>(int offset) Line 17 + 0xf bytes	C#
 	Enigma.D3.dll!Enigma.D3.Sno.Scene.x00_Id.get() Line 13 + 0x8 bytes	C#

SnoDefinition look like this:

-		sno_def	{123CBBC0 SnoDefinition`1}	Enigma.D3.SnoDefinition<Enigma.MemoryObject>
+		base	{123CBBC0 SnoDefinition`1}	Enigma.MemoryObject {Enigma.D3.SnoDefinition<Enigma.MemoryObject>}
		x00_Id	0xd3840bd0	int
		x04_CreationLoop	0x00000641	int
		x08_SnoGroupId	Scene	Enigma.D3.Enums.SnoGroupId
		x0C_Size	0x00004a64	int
+		x10_SnoItem	{8145D2F0 MemoryObject}	Enigma.MemoryObject

I use following method in SnoDefinition to convert it to Sno.Scene:

public S x10_SnoItemAs<S>() { return Dereference<S>(0x10); }

getting Sno.Scene like this:

Enigma.D3.Sno.Scene sno_scene = sno_def.x10_SnoItemAs<Enigma.D3.Sno.Scene>();

-		sno_scene	{8145D2F0 Scene}	Enigma.D3.Sno.Scene
-		base	{8145D2F0 Scene}	Enigma.D3.Sno.SerializeMemoryObject {Enigma.D3.Sno.Scene}
-		base	{8145D2F0 Scene}	Enigma.MemoryObject {Enigma.D3.Sno.Scene}
		_address	0x8145d2f0	int
+		_memory	{Enigma.ProcessMemory}	Enigma.MemoryBase {Enigma.ProcessMemory}
		_snapshotOffset	0x00000000	int
		Address	0x8145d2f0	int
+		Memory	{Enigma.ProcessMemory}	Enigma.MemoryBase {Enigma.ProcessMemory}
		Snapshot	null	byte[]
+		Static members		
		_baseAddress	null	int?
		BaseAddress	0x8145d2f0	int
+		x00_Id	'sno_scene.x00_Id' threw an exception of type 'System.ArgumentOutOfRangeException'	int {System.ArgumentOutOfRangeException}
+		x00C	'sno_scene.x00C' threw an exception of type 'System.ArgumentOutOfRangeException'	int {System.ArgumentOutOfRangeException}
+		x010_AABB	{8145D300 AABB}	Enigma.D3.Sno.Scene.AABB
+		x028_AABB	{8145D318 AABB}	Enigma.D3.Sno.Scene.AABB
+		x040_NavMeshDef	{8145D330 NavMeshDef}	Enigma.D3.Sno.Scene.NavMeshDef
+		x068_SerializeData	'sno_scene.x068_SerializeData' threw an exception of type 'System.ArgumentOutOfRangeException'	Enigma.D3.Sno.SerializeData {System.ArgumentOutOfRangeException}
+		x070_Snos	'sno_scene.x070_Snos' threw an exception of type 'System.ArgumentOutOfRangeException'	Enigma.D3.Sno.Sno[] {System.ArgumentOutOfRangeException}
+		x0A8_SerializeData	'sno_scene.x0A8_SerializeData' threw an exception of type 'System.ArgumentOutOfRangeException'	Enigma.D3.Sno.SerializeData {System.ArgumentOutOfRangeException}
+		x0B0_Snos	'sno_scene.x0B0_Snos' threw an exception of type 'System.ArgumentOutOfRangeException'	Enigma.D3.Sno.Sno[] {System.ArgumentOutOfRangeException}
...

Any idea what I'm doing wrong? I assume this specific Sno.Scene might be released as everything works 99% of the time and I get correct data from other SNOs. However in this case I have no idea how can i distinguish released released SnoDefinition from legit one. All above is from release version of Enigma.

EDIT: Why any attempt of reading from address over 0x80000000 (-214748364 is supposed to fail due to following code in ProcessMemory:

public override void ReadBytes(int address, byte[] buffer, int offset, int count)
{
	if (address < 0)
		throw new ArgumentOutOfRangeException("address");
        ...
}

EDIT 2: I guess thats why, nevermind xD

The virtual address space is divided into partitions as follows: The 2 GB partition in low memory (0x00000000 through 0x7FFFFFFF) is available to the process, and the 2 GB partition in high memory (0x80000000 through 0xFFFFFFFF) is reserved for the system.

EDIT 3: I resolved this issue by additional check

sno_def.x10_SnoItem.Address > 0

[enigma32]

Hi CrEEzz,

What is the actual exception you're getting? The line numbers tends to be quite inaccurate for release builds. If it's due to address < 0 then actually that part is wrong. D3 has the /LARGEADDRESSAWARE flag enabled meaning up to 3GB of RAM is available instead of the typical 2GB. IsValidAddress(address) should be used instead for address validation, I thought that had been changed but I must've forgotten to do it in ProcessMemory as well. SVN has been updated!

[CrEEzz]

Hi, Yeah lines are a bit messed up. Thx for the info. Perhaps you want to make IsValidAddress public and add to MemoryObject:

public bool HasValidAddress { get { return _memory != null ? _memory.IsValidAddress(Address) : false; } }

[enigma32]

Hi, Yeah lines are a bit messed up. Thx for the info. Perhaps you want to make IsValidAddress public and add to MemoryObject:

public bool HasValidAddress { get { return _memory != null ? _memory.IsValidAddress(Address) : false; } }

I don't see a practical use for that. If the address has gone bad since creation, then that page of memory has been released by the process, but it might very well have been allocated or reused for something else. Checking for this (on MemoryObject) before a read (which also would validate) is just going to cause race condition, you're never really sure a read will succeed either way, unless you freeze D3 all the time (which would be highly impractical), so it's better to be prepared that a random exception might be thrown at any given time due to D3 fiddling around with the memory that you're reading.

[CrEEzz]

You are right Enigma I handled it. However there is possible infinite loop on getting value of Attribute of invalid object. Code gets stuck on callstack:

>	Enigma.D3.dll!Enigma.ProcessMemory.ReadBytes(int address, byte[] buffer, int offset, int count) Line 111 + 0xe4 bytes	C#
 	Enigma.D3.dll!Enigma.MemoryBase.ReadBytes(int address, int count) Line 77	C#
 	Enigma.D3.dll!Enigma.MemoryBase.Read<int>(int address) Line 41	C#
 	Enigma.D3.dll!Enigma.MemoryObject.Field<int>(int offset) Line 142 + 0x5e bytes	C#
 	Enigma.D3.dll!Enigma.D3.Helpers.AttributeHelper.TryGetValue(Enigma.D3.Collections.Map<int,int> map, int key = -3735, out Enigma.D3.Memory.Pointer ptrValue = null) Line 81 + 0x10 bytes	C#
 	Enigma.D3.dll!Enigma.D3.Helpers.AttributeHelper.GetAttributeValuePtr(Enigma.D3.ActorCommonData acd, Enigma.D3.Enums.AttributeId attribId, int modifier) Line 65 + 0x36 bytes	C#
 	Enigma.D3.dll!Enigma.D3.Helpers.Attributes.SimpleAttribute<int>.GetValue(Enigma.D3.ActorCommonData acd, int modifier) Line 155	C#
 	Enigma.D3.dll!Enigma.D3.Helpers.Attribute<int>.GetValue(Enigma.D3.ActorCommonData acd) Line 105 + 0xa bytes	C#

exception Win32Exception is thrown, then catched and message is displayed but it seems like code gets stuck inside while loop in following function in AttributeHelper

private static bool TryGetValue(Map<int, int> map, int key, out Pointer ptrValue)
		{
			if (map != null)
			{
				var hash = key ^ (key >> 12);
				var entry = map.x10_Buckets[map.x00_Mask & hash];
				while (entry != null)
				{
					if (entry.x04_Key == key)
					{
						ptrValue = new Pointer(entry.Memory, entry.Address + 0x08);
						return true;
					}
					entry = entry.x00_Next;
				}
			}
			ptrValue = null;
			return false;
		}

[enigma32]

Yea I probably shouldn't swallow that exception and return random data. I've been working on a new library to handle memory reading in the future, that works a bit differently, but I won't make the transition in the public version before 2.1.0.

I would suggest you remove this part in ProcessMemory (line 110) or handle it as you see fit:
catch (Exception exception){
OnReadException(address, exception);
}

I'm keeping it as it is in the SVN version since I don't know how many other problems that might potentially generate.

[CrEEzz]

Yeah, I did something like this and seems to work just fine:

bool failed = false;
int numberOfBytesRead;
try
{
	NativeCalls++;
	if (Win32.ReadProcessMemory(
		_process.Handle,
		address,
		buffer,
		buffer.Length,
		out numberOfBytesRead))
	{
		ValidateNumberOfBytesRead(address, numberOfBytesRead, buffer.Length);
	}
	else
	{
		throw new Win32Exception();
	}
}
catch (Exception exception)
{
	OnReadException(address, exception);
        failed = true;
}

if (failed)
        throw new Win32Exception();

just to keep the message

[enigma32]

You can simplify to this:

catch (Exception exception)
{
      OnReadException(address, exception);
      throw; // rethrows the exception, stack and everything is retained
}

Not sure you would get a proper error message when throwing a 2nd Win32Exception, it calls Marshal.GetLastError to get the error code to translate, but that might be cleared when the first exception is created.

[CrEEzz]

Any ETA on update to 2.1?

[bastiflew]

Hi all, I will not update Enigma's code, but some updated offsets :

object manager : 0x01C51184
sno group : 0x1C4F154
Location : 0x1BACE7C

actor container : 0x928 / size : 0x454
acd container : 0x8C0
ui container : 0x99C
scene container : 0x968 / size : 0x6C4
fastattrib : 0x8b4
player index : 0x9AC
player data : 0x894 / size : 0xE6B0
quest manager : 0x8D0

actor struct changed a little bit, +4 @ 0x98

attributes : attrib link

I will add ofs which I forgot

[DarkAngelBGE]

Nice, thank you.

Still trying to figure how to apply these to my own work which uses the old ownedCore offsets in the form of

$ofs_objectmanager (well that one is obvious)
$ofs_ActorAtrib_Base
$ofs_LocalActor_ofs1
$ObjManACDOffset

etc.

Please keep them coming. Really appreciated.

[enigma32]

Any ETA on update to 2.1?

Much broken, much fixed, much weird stuff still to solve, I shouldn't have defined so darn many structures I guess I might have something tomorrow that works more or less. SNO reading is currently very broken, couldn't figure out how to help Dolphe with his FK.

btw, anyone know of a good diff method? :confused: Currently I generate signatures for previous patch, apply it to new IDA database, then hopefully it matches enough of the critical methods so I easily can compare them, but this is a very unstable method as it typically breaks on allocation size change or field offset change...

[bastiflew]

SNO reading : start at +0xC and you have an entry every 0x10.
SNOItem is still the same, but the struct size is 0x318

[enigma32]

SNO reading : start at +0xC and you have an entry every 0x10.
SNOItem is still the same, but the struct size is 0x318

Sorry I don't follow... +0x0C relative to what? Before I could get the SNO group (https://subversion.assembla.com/svn/...D3/SnoGroup.cs), enter the collection at x10, then enumerate all definitions (https://subversion.assembla.com/svn/...oDefinition.cs), but this, SnoDefinition changed and I don't really know how to get an item out of it anymore.

What would an SNOItem be? I'm guessing it's not the "SNO Item" from GameBalance as that would be 0x528 bytes.