DX11 EndScene Hook

[Ozius]

Hi, all!

I’m trying lua-injection via EndScene Hook. Everything works perfectly with dx9. But more and more people are using dx11. Cold you advise how to manage the EndScene Hook in case of dx11, please?

Thanks a lot for your answers!

[Bananenbrot]

Did you even try to google it? Dx11 has no endscene. Hook present instead.
Sent from my LG-E900 using Board Express

Edit: Wtf... now you know my phone by name

[suicidity]

Hook Present in the SwapChain, vtable index 8.

[zys924]

Mind sharing how to grab the "SwapChain"'s address? I know liittle about DXD11 data structure.

[adaephon]

I know liittle about DXD11 data structure.

That might be a good thing to start learning about then...

[_Mike]

Mind sharing how to grab the "SwapChain"'s address? I know liittle about DXD11 data structure.

Use the debug symbols.
CDXGISwapChainBase<IDXGISwapChain>::Present in dxgi.dll

[zys924]

Thanks Mike. That's a good idea. But I am having trouble in SymFromName() function which produces always 126 error. However, if I tried the kernel32.dll functions, like "CreateProcessW", then it will work properly. This is just the reason why I didn't use your debug symbol method before. Would you plz tell me why?

[streppel]

Thanks Mike. That's a good idea. But I am having trouble in SymFromName() function which produces always 126 error. However, if I tried the kernel32.dll functions, like "CreateProcessW", then it will work properly. This is just the reason why I didn't use your debug symbol method before. Would you plz tell me why?

Experienced a similar behaviour before,but didn't find out why,so if you find out it would be great if you'd tell me

What i did instead was hooking a method ingame that i know(or could google about), and run my GetProcAdress injected in the other process and get the returnvalue from this.

[_Mike]

Thanks Mike. That's a good idea. But I am having trouble in SymFromName() function which produces always 126 error. However, if I tried the kernel32.dll functions, like "CreateProcessW", then it will work properly. This is just the reason why I didn't use your debug symbol method before. Would you plz tell me why?

I think it's the template < > that messes up SymFromName. I don't know if there's any workaround for it, or if you have to rewrite the name some other way, but you can use the decorated name instead.

In my own hook I first call SymSetOptions without SYMOPT_UNDNAME and get the address of

/* public: virtual long __stdcall CDXGISwapChainBase<struct IDXGISwapChain>::Present(unsigned int,unsigned int) */
LPVOID address = _sym->GetAddressFromSymbol(L"?Present@?$CDXGISwapChainBase@UIDXGISwapChain@@@@UAGJII@Z");

and then call SymSetOptions again with SYMOPT_UNDNAME for everything else I need.

Oh, and by hooking both EndScene and Present, and with some API abstraction, you can have a generic renderer that works for both D3D9 and D3D11.

[zys924]

Still unsolved. Would you mind sharing your full debug symbol code? The same thing also happens to EndScene in DX9. Thanks a lot.

[_Mike]

Still unsolved. Would you mind sharing your full debug symbol code? The same thing also happens to EndScene in DX9. Thanks a lot.

param is a wide string to a directory path where dbghelp.dll and symsrv.dll is.

#include <Windows.h>
#include <DbgHelp.h>
#include <boost/filesystem.hpp>
#include <vector>
#include <sstream>

__declspec(dllexport) DWORD Initialize(LPVOID param)
{
	boost::filesystem::path basedir((wchar_t*)param);

	HMODULE dxgi = GetModuleHandle(L"dxgi");
	HMODULE d3d9 = GetModuleHandle(L"d3d9");

	if(!dxgi || !d3d9)
		return 1;

	HMODULE dbghlp = LoadLibrary((basedir / L"\\dbghelp.dll").c_str());
	HMODULE symsrv = LoadLibrary((basedir / L"\\symsrv.dll").c_str());

	if(!dbghlp || !symsrv)
		return 2;

	typedef DWORD (WINAPI* tSymSetOptions)(DWORD);
	typedef BOOL (WINAPI* tSymInitialize)(HANDLE, wchar_t const*, BOOL);
	typedef DWORD64 (WINAPI* tSymLoadModuleEx)(HANDLE, HANDLE, PCTSTR, PCTSTR, DWORD64, DWORD, PMODLOAD_DATA, DWORD);
	typedef BOOL (WINAPI* tSymFromName)(HANDLE, wchar_t const*, PSYMBOL_INFO);

	auto fSymSetOptions = reinterpret_cast<tSymSetOptions>(GetProcAddress(dbghlp, "SymSetOptions"));
	auto fSymInitialize = reinterpret_cast<tSymInitialize>(GetProcAddress(dbghlp, "SymInitializeW"));
	auto fSymLoadModuleEx = reinterpret_cast<tSymLoadModuleEx>(GetProcAddress(dbghlp, "SymLoadModuleExW"));
	auto fSymFromName = reinterpret_cast<tSymFromName>(GetProcAddress(dbghlp, "SymFromNameW"));

	if(!fSymSetOptions || !fSymInitialize || !fSymLoadModuleEx || !fSymFromName)
		return 3;

	// Use decorated names
	fSymSetOptions(SYMOPT_DEBUG | SYMOPT_DEFERRED_LOADS | /*SYMOPT_UNDNAME |*/ SYMOPT_FAVOR_COMPRESSED);
	
	if(!fSymInitialize(GetCurrentProcess(), NULL, FALSE))
		return 4;

	if( !fSymLoadModuleEx(GetCurrentProcess(), nullptr, L"dxgi", nullptr, (DWORD64)dxgi, 0, nullptr, 0) ||
		!fSymLoadModuleEx(GetCurrentProcess(), nullptr, L"d3d9", nullptr, (DWORD64)d3d9, 0, nullptr, 0))
		return 5;

	std::vector<char> buffer(sizeof(SYMBOL_INFO) + 255 * sizeof(wchar_t) + 1);
	PSYMBOL_INFO info = reinterpret_cast<PSYMBOL_INFO>(buffer.data());
	info->SizeOfStruct = sizeof(SYMBOL_INFO);
	info->MaxNameLen = sizeof(wchar_t) * 255 + 1;

	ULONG64 address;
	// D3D11 Present
	if(!fSymFromName(GetCurrentProcess(), L"?Present@?$CDXGISwapChainBase@UIDXGISwapChain@@@@UAGJII@Z", info))
		address = 0;
	else
		address = info->Address;

	std::wostringstream ss;
	ss << L"Present: 0x" << std::hex << address << std::endl;

	// Use undecorated names
	fSymSetOptions(SYMOPT_DEBUG | SYMOPT_DEFERRED_LOADS | SYMOPT_UNDNAME | SYMOPT_FAVOR_COMPRESSED);

	// D3D9 EndScene
	if(!fSymFromName(GetCurrentProcess(), L"CD3DBase::EndScene", info))
		address = 0;
	else
		address = info->Address;

	ss << L"EndScene: 0x" << std::hex << address << std::endl;

	MessageBoxW(NULL, ss.str().c_str(), L"Addresses", MB_OK | MB_ICONINFORMATION);

	return 0;
}

[zys924]

[DllImport("Kernel32")]
private static extern IntPtr LoadLibrary(String libName);
[DllImport("Kernel32")]
private static extern bool FreeLibrary(IntPtr handle);
[DllImport("Kernel32")]
private static extern IntPtr GetProcAddress(IntPtr handle, string funcName);
[DllImport("kernel32.dll")]
public static extern IntPtr GetModuleHandle(string lpModuleName);
[DllImport("kernel32.dll")]
private static extern IntPtr GetCurrentProcess();

[Flags()]
private enum SymLoadModuleOptions : uint
{
	SLMFLAG_NONE = 0x0,
	SLMFLAG_VIRTUAL = 0x1,
	SLMFLAG_ALT_INDEX = 0x2,
	SLMFLAG_NO_SYMBOLS = 0x4
}
[Flags()]
private enum SymInitOptions : uint
{
	SYMOPT_CASE_INSENSITIVE = 0x00000001,
	SYMOPT_UNDNAME = 0x00000002,
	SYMOPT_DEFERRED_LOADS = 0x00000004,
	SYMOPT_NO_CPP = 0x00000008,
	SYMOPT_LOAD_LINES = 0x00000010,
	SYMOPT_OMAP_FIND_NEAREST = 0x00000020,
	SYMOPT_LOAD_ANYTHING = 0x00000040,
	SYMOPT_IGNORE_CVREC = 0x00000080,
	SYMOPT_NO_UNQUALIFIED_LOADS = 0x00000100,
	SYMOPT_FAIL_CRITICAL_ERRORS = 0x00000200,
	SYMOPT_EXACT_SYMBOLS = 0x00000400,
	SYMOPT_ALLOW_ABSOLUTE_SYMBOLS = 0x00000800,
	SYMOPT_IGNORE_NT_SYMPATH = 0x00001000,
	SYMOPT_INCLUDE_32BIT_MODULES = 0x00002000,
	SYMOPT_PUBLICS_ONLY = 0x00004000,
	SYMOPT_NO_PUBLICS = 0x00008000,
	SYMOPT_AUTO_PUBLICS = 0x00010000,
	SYMOPT_NO_IMAGE_SEARCH = 0x00020000,
	SYMOPT_SECURE = 0x00040000,
	SYMOPT_NO_PROMPTS = 0x00080000,
	SYMOPT_OVERWRITE = 0x00100000,
	SYMOPT_IGNORE_IMAGEDIR = 0x00200000,
	SYMOPT_FLAT_DIRECTORY = 0x00400000,
	SYMOPT_FAVOR_COMPRESSED = 0x00800000,
	SYMOPT_ALLOW_ZERO_ADDRESS = 0x01000000,
	SYMOPT_DISABLE_SYMSRV_AUTODETECT = 0x02000000,
	SYMOPT_DEBUG = 0x80000000
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
private unsafe struct SYMBOL_INFO
{
	public uint SizeOfStruct;
	public uint TypeIndex;        // Type Index of symbol
	public fixed UInt64 Reserved[2];
	public uint Index;
	public uint Size;
	public UInt64 ModBase;          // Base Address of module containing this symbol
	public uint Flags;
	public UInt64 Value;            // Value of symbol, ValuePresent should be 1
	public UInt64 Address;          // Address of symbol including base address of module
	public uint Register;         // register holding value or pointer to value
	public uint Scope;            // scope of the symbol
	public uint Tag;              // pdb classification
	public uint NameLen;          // Actual length of name
	public uint MaxNameLen;
	public fixed char Name[1];          // Name of symbol
}
private delegate uint SymSetOptionsDelegate(SymInitOptions initOptions);
private static SymSetOptionsDelegate SymSetOptions = null;
private delegate bool SymInitializeDelegate(IntPtr hProc, IntPtr pSearchPath, bool invade);
private static SymInitializeDelegate SymInitialize = null;
private delegate ulong SymLoadModuleExDelegate(IntPtr hProc, IntPtr hFile, IntPtr pImageName, IntPtr pModuleName, ulong DllBase, uint dllSize, IntPtr pModLoadData, SymLoadModuleOptions flags);
private static SymLoadModuleExDelegate SymLoadModuleEx = null;
private delegate bool SymFromNameDelegate(IntPtr hProc, IntPtr name, IntPtr symInfo);
private static SymFromNameDelegate SymFromName = null;

private static void GetDirectXHookPoint()
{
	IntPtr hD3D9 = GetModuleHandle("d3d9");
	if (hD3D9 == IntPtr.Zero)
	{
		m_Logger2.Log("Unable to find DirectX handle.");
		return;
	}
	IntPtr hDBGHelp = LoadLibrary("dbghelp.dll");
	IntPtr hSymSrv = LoadLibrary("symsrv.dll");
	if (hDBGHelp == IntPtr.Zero || hSymSrv == IntPtr.Zero)
	{
		m_Logger2.Log("Unable to load debug library files.");
		return;
	}
	SymSetOptions = Marshal.GetDelegateForFunctionPointer(GetProcAddress(hDBGHelp, "SymSetOptions"), typeof(SymSetOptionsDelegate)) as SymSetOptionsDelegate;
	SymInitialize = Marshal.GetDelegateForFunctionPointer(GetProcAddress(hDBGHelp, "SymInitialize"), typeof(SymInitializeDelegate)) as SymInitializeDelegate;
	SymLoadModuleEx = Marshal.GetDelegateForFunctionPointer(GetProcAddress(hDBGHelp, "SymLoadModuleEx"), typeof(SymLoadModuleExDelegate)) as SymLoadModuleExDelegate;
	SymFromName = Marshal.GetDelegateForFunctionPointer(GetProcAddress(hDBGHelp, "SymFromName"), typeof(SymFromNameDelegate)) as SymFromNameDelegate;
	if (SymSetOptions == null || SymInitialize == null || SymLoadModuleEx == null || SymFromName == null)
	{
		m_Logger2.Log("Unable to marshal debug APIs.");
		return;
	}
	SymSetOptions(SymInitOptions.SYMOPT_DEBUG | SymInitOptions.SYMOPT_DEFERRED_LOADS | SymInitOptions.SYMOPT_UNDNAME | SymInitOptions.SYMOPT_FAVOR_COMPRESSED);
	if (!SymInitialize(GetCurrentProcess(), IntPtr.Zero, false))
	{
		m_Logger2.Log("Unable to SymInitialize().");
		return;
	}
	IntPtr pImageFile = Marshal.StringToHGlobalAnsi("d3d9");
	if (SymLoadModuleEx(GetCurrentProcess(), IntPtr.Zero, pImageFile, IntPtr.Zero, (ulong)hD3D9, 0, IntPtr.Zero, SymLoadModuleOptions.SLMFLAG_NONE) != (ulong)hD3D9)
	{
		m_Logger2.Log("Unable to SymLoadModuleEx().");
		return;
	}
	Marshal.FreeHGlobal(pImageFile);
	string symbol = "CD3DBase::EndScene";
	unsafe
	{
		IntPtr ptrString = Marshal.StringToHGlobalAnsi(symbol);
		IntPtr ptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(SYMBOL_INFO)) + 255 + 1);
		SYMBOL_INFO* pInfo = (SYMBOL_INFO*)ptr;
		pInfo->MaxNameLen = 255 + 1;
		pInfo->SizeOfStruct = (uint)Marshal.SizeOf(typeof(SYMBOL_INFO));
		if (!SymFromName(GetCurrentProcess(), ptrString, ptr))
		{
			Marshal.FreeHGlobal(ptrString);
			Marshal.FreeHGlobal(ptr);
			m_Logger2.Log("Unable to SymFromName()." + Marshal.GetLastWin32Error().ToString());
			return;
		}
		Marshal.FreeHGlobal(ptrString);
		Marshal.FreeHGlobal(ptr);
		m_Logger0.Log("EndScene() is @ 0x{0:X}", pInfo->Address);
	}
}

I did almost exactly the same version of your code but in C# language. I am in-process and error jumps out as "Unable to SymFromName().126". 126 error still exists. Any ideas?

[_Mike]

Your code works for me. Make sure you are using matching versions of the dbghelp and symsrv dlls.
Wow supplies it's own dbghelp, and there is also one in system32/syswow64, so you might be loading one of those instead since you aren't using absolute paths.

[zys924]

Loading these dlls from the WoW dir seems still not working...

IntPtr hDBGHelp = LoadLibrary(Environment.CurrentDirectory + "\\dbghelp.dll");
IntPtr hSymSrv = LoadLibrary(Environment.CurrentDirectory + "\\symsrv.dll");

[lanman92]

AppDomain.CurrentDomain.BaseDirectory will give you the folder where your .NET module is at. Put your DLL's in there and load them from that absolute path with the DLL filename Path.Combine'd onto the end.