How I hooked EndScene menu

Shout-Out

User Tag List

Results 1 to 4 of 4
  1. #1
    Sillyboy72's Avatar Member
    Reputation
    13
    Join Date
    Jan 2009
    Posts
    66
    Thanks G/R
    0/0
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    How I hooked EndScene

    I have been going through the learning curve of reverse engineering. And since you fine folks have been so helpful, I wanted to share how I went through the process of finding where to hook direct3d's EndScene.

    This is likely only interesting to n00bs (and people who want to poke fun ).

    (btw, minor note that injecting threads into wow.exe and letting them run willy nilly is a recipe for disaster. doing my work in endscene made things so much better).

    This is actually really straight forward, but being a total n00b... it took me a while get wrap my head around it.

    Fundamentally, the only way an app does direct3d is to create the initial direct3d pointer via the api Direct3DCreate9. This returns a IDirect3D9 pointer and from there apps call IDirect3D9::CreateDevice, and it is that device pointer we are interested in (EndScene() hangs off it).

    So, step 1... pull up ida and search for Direct3DCreate9. bingo, bongo, bungo. You quickly find a routine that takes 2 params... the hModule for d3d9.dll and the device pointer. So, we ask ourselves who calls this bad boy... and where do they store that pointer.

    So, I look up the xrefs, there are 4 or so callers, and I get lost, and cry. Hey, I'm a n00b what did you expect?

    Okay, take a step back.. what do I actually want? not the IDirect3D9 pointer, I want the IDirect3DDevice9 ... that is returned from IDirect3D9::CreateDevice.... how the hells do I find that?

    Well, I could debug wow... and set a breakpoint, and see who is calling it. That sounds fun. So, I actually write some test code that calls these functions...

    My code, with disasm:
    Code:
    	IDirect3D9 *pD3D = Direct3DCreate9(D3D_SDK_VERSION);
    0040185A  push        20h  
    0040185C  call        Direct3DCreate9 (4031C0h) 
    00401861  mov         dword ptr [pD3D],eax 
    	pD3D->CreateDevice(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, NULL, 3, NULL, NULL);
    00401864  push        0    
    00401866  push        0    
    00401868  push        3    
    0040186A  push        0    
    0040186C  push        1    
    0040186E  push        0    
    00401870  mov         eax,dword ptr [pD3D] 
    00401873  mov         ecx,dword ptr [eax] 
    00401875  mov         edx,dword ptr [pD3D] 
    00401878  push        edx  
    00401879  mov         eax,dword ptr [ecx+40h] 
    0040187C  call        eax
    Well, okay, we can see that 0x40 off from start of vtable is the pointer to CreateDevice. cool'ish... lets just run it and step into the "call eax"
    Code:
    6FD3B60A  mov         edi,edi
    The only thing interesting here is the address -- 6FD3B60A. If I dig around a bit, my debugger tells me where d3d9 is actually loaded:
    Code:
    d3d9.dll	6FCF0000-6FEAA000
    And 6FD3B60A minus 6FCF0000 = 4B60A for our offset from start of module.

    So... wherever wow.exe loads d3d9.dll... 0x4b60a from there is where we want to set our breakpoint. When I did a test run... the module got loaded at the same exact address. Neat. (always that way? dunno).

    so, break point at 6FD3B60A, and let wow.exe rip.

    I am using windbg, here is the stack after we hit the breakpoint:
    Code:
    0:000> k
    *** ERROR: Module load completed but symbols could not be loaded for Wow.exe
    ChildEBP RetAddr  
    0019fabc 005de9f5 d3d9!CEnum::CreateDevice
    WARNING: Stack unwind information not available. Following frames may be wrong.
    0019fb68 005dfb2a Wow+0x1de9f5
    0019fbc4 005d1ad7 Wow+0x1dfb2a
    0019fbd0 005dfdcb Wow+0x1d1ad7
    ...
    Okay, looks like we will be returning to 005de9f5 when we return from CreateDevice. Let's pull that up in IDA.

    Code:
    .text:005DE9D0                 lea     edi, [esi+3860h]
    .text:005DE9D6                 push    edi             ; IDirect3DDevice9 ** ppReturnedDeviceInterface
    .text:005DE9D7                 lea     edx, [ebp+Dst]
    .text:005DE9DA                 push    edx             ; D3DPRESENT_PARAMETERS * pPresentationParameters,
    .text:005DE9DB                 mov     edx, [esi+384Ch]
    .text:005DE9E1                 sbb     ebx, ebx
    .text:005DE9E3                 and     ebx, 30h
    .text:005DE9E6                 add     ebx, 22h
    .text:005DE9E9                 push    ebx             ; DWORD BehaviorFlags,
    .text:005DE9EA                 push    edx             ; HWND hFocusWindow,
    .text:005DE9EB                 push    1               ; D3DDEVTYPE DeviceType  (HAL)
    .text:005DE9ED                 push    0               ; UINT Adapter
    .text:005DE9EF                 push    eax
    .text:005DE9F0                 mov     eax, [ecx+40h]
    .text:005DE9F3                 call    eax
    w00t!

    I added param commenting, but this is clearly goodness. The pointer is stored at 0x384c away from esi, ... a bit further back in the code you will see that esi just came from ecx (this pointer), which I call just "CGame". Same pointer that is referenced some 2860 times in wow.exe

    So, something like:
    Code:
    struct CGame
    {
       char WhoKnows(0x3860);
       IDirect3DDevice9 *pDevice;
    };
    And you are pretty well in business. (from there, I just detour vtable[42] to "MyEndScene").

    -Silly

    How I hooked EndScene
  2. #2
    Cypher's Avatar Kynox's Sister's Pimp
    Reputation
    1358
    Join Date
    Apr 2006
    Posts
    5,368
    Thanks G/R
    0/6
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You don't actually need to go to all the work of finding the device pointers etc. You can just hook CreateDevice using a regular API hook if you're injecting when the process is created. Otherwise I believe theres an API you can call that will return the current device pointer.

    Either way, there are better ways to hook the D3D APIs that are fully generic (ie will work on any game without modification).

  3. #3
    kynox's Avatar Member
    Reputation
    830
    Join Date
    Dec 2006
    Posts
    888
    Thanks G/R
    0/0
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I prefer this method, you can inject at any time. (Unless you find that special API to find the current device).

  4. #4
    Cypher's Avatar Kynox's Sister's Pimp
    Reputation
    1358
    Join Date
    Apr 2006
    Posts
    5,368
    Thanks G/R
    0/6
    Trade Feedback
    0 (0%)
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally Posted by kynox View Post
    I prefer this method, you can inject at any time. (Unless you find that special API to find the current device).
    The great thing about API hooking though is that not only do you get a generic way to get the device pointer, you ALSO get a generic way to get the window handle so you can hook the window's message callback to forward inputs etc.

    I assume you're also finding a pointer to the window handle or message callback on a game-by-game basis?

    **** that. I'll stick with process-creation. I'd much rather have a generic system that requires 0-updates than have to track down that shit for every game I feel like injecting into. Then again, you're probably less lazy than me. Each to their own.

    P.S. Can't Fraps inject itself at runtime? Can't remember.... If so, that might be worth looking into to see how it gets the device pointer.

Similar Threads

  1. [C#] Need help with synchronization mutexes in hook EndScene
    By Ferroks in forum Diablo 3 Memory Editing
    Replies: 1
    Last Post: 08-09-2014, 06:05 AM
  2. C# DLL Injection and Hooking Endscene with WhiteMagic
    By rlane187 in forum WoW Memory Editing
    Replies: 6
    Last Post: 01-26-2012, 01:27 PM
  3. Replies: 0
    Last Post: 07-26-2010, 10:19 AM
  4. Finding the Direct3D VMT table to hook EndScene
    By revertlife in forum WoW Memory Editing
    Replies: 14
    Last Post: 07-07-2010, 07:09 PM
  5. How to Make EndScene Hook work?
    By misz in forum WoW Memory Editing
    Replies: 3
    Last Post: 02-23-2010, 08:36 PM
All times are GMT -5. The time now is 11:40 PM. Powered by vBulletin® Version 4.2.3
Copyright © 2025 vBulletin Solutions, Inc. All rights reserved. User Alert System provided by Advanced User Tagging (Pro) - vBulletin Mods & Addons Copyright © 2025 DragonByte Technologies Ltd.
Google Authenticator verification provided by Two-Factor Authentication (Free) - vBulletin Mods & Addons Copyright © 2025 DragonByte Technologies Ltd.
Digital Point modules: Sphinx-based search