FrameScript_RegisterFunction Fail

[Harland]

It's embarrassing posting this but it's been driving me nuts for the past two nights. I can't seem to figure out why my RegisterFunctionHandler never hits 0x004181B0 on a hardware breakpoint in Olly.



As you can see I'm using the current(3.3.5a) FrameScript_RegisterFunction address at 0x004181B0 with Apoc's Lua Class as per ->.

To be sure OllyDBG was not being a dill (or me), I HWBP'd the FrameScript_Execute (aka DoString) at 0x00819210, and indeed when I invoke my DoStringHandler, control jumps to OllyDbg.

Running a trace on the instantiation of the Lua static class it is certainly being called by my EndScene Pulse detour.

IDA with HexRays shows the function prototype for FrameScript_RegisterFunction has not changed.

What have a I clearly missed that is causing my RegisterCommandHandler to fail?

C# Code (Courtesy of Apoc):

Magic _Magic;
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate uint LuaRegisterCommand(string szName, IntPtr pFunc);

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate int RegisteredLuaCommandHandler(IntPtr pLuaState);

static Lua()
{
    _Magic = new Magic();

    Process.EnterDebugMode();

    RegisterCommandHandler = _Magic.RegisterDelegate<LuaRegisterCommand>(0x004181B0);
            
    try
    {
         RegisterCommand("MyInput", CommandParser);
         System.Windows.Forms.MessageBox.Show("Registered");
     }
     catch (Exception e)
     {
         System.Windows.Forms.MessageBox.Show("RegisterCommandFail"+e.Message);
     }
}

public static void RegisterCommand(string commandName, RegisteredLuaCommandHandler handler)
{
      RegisterCommandHandler(commandName, WriteLuaCallback(Marshal.GetFunctionPointerForDelegate(handler)));
      return;
}

Some code has been removed for brevity. WriteLuaCallback has been omitted; all you need to know is an IntPtr is returned that points to a Jump within the code segment.

Just so you know a MessageBox appears displaying "Registered" indicating RegisterCommand at least did not raise an exception. Also rendering in Wow halts during the time the MessageBox is active (ie. in process in Wow's main thread).

Ohwell let me know if anyone has a hint to where it may be failing.

EDIT: Sorry during my rush to go to the movies last night (BTW Inception = AWESOME. Go see it!), I accidentally hit back on the browser and some of my post had been deleted.

[caytchen]

No idea why you don't hit that hardware breakpoint, but for starters your FrameScript_RegisterFunction seems off. Real one is at 0x00817F90. Prototype is

void __cdecl FrameScript_RegisterFunction(char* functionName, int (__cdecl* function)(void*/*LuaState*/))

but I guess thats in your code already.

[Harland]

No idea why you don't hit that hardware breakpoint, but for starters your FrameScript_RegisterFunction seems off. Real one is at 0x00817F90. Prototype is

void __cdecl FrameScript_RegisterFunction(char* functionName, int (__cdecl* function)(void*/*LuaState*/))

but I guess thats in your code already.

Correct I have a delegate that handles that, I'll have to do a closer inspection tonight. Although I'm sure Starcraft II is going to distract me.