GreyMagic - The best of both worlds, and then some

[Valediction]

@Apoc, is it possible to make this skip entering debug mode when using InProcessMemoryReader? I can comment out "Process.EnterDebugMode();" in MemoryBase.cs but I'm sure there has to be a better way.

Thanks

Funny that today I have come with exactly the same need you're describing. Since I'm mostly using InProcessMemoryReader for pointer-like reading/writing while injected in the process, those privileges (at least for now) aren't needed. In fact, I'm not sure I can get debug privileges when injected, at least with WoW launched in regular mode... for the time being I just added a flag to the constructor (of MemoryBase) that skips all the part that requires privileges and I set it when building an InProcessMemoryReader, but doesn't seem elegant at all.

[TOM_RUS]

I'm not even sure why you even need debug privileges for reading memory. Everything should work without them.

[Valediction]

I'm not even sure why you even need debug privileges for reading memory. Everything should work without them.

That's the point, since debug privileges are needed starting right from the base class, maybe that part should be moved somewhere else (if I'm not missing anything which actually requires those privileges throughout the whole class hierarchy).

[GRB]

Kinda stupid question, but after 4+ hours of search i cant find it.

Is there anyway to open a process using this greymagic? or it must be done using System.Diagnostics?

Ex: for blackmagic u just go

blackmagic somename = new blackmagic();
somename.open bla bla bla

i know grey dont use the **** *** = new *****();
and ofc i could open the process normal way, but the question is, if theres anyway to do it using greymagic.

i noticed that hbrelogger have a import.cs file on greymagic source code, and have the open process through the safememoryhandle, but after compiling grey, that functions is not there, as all other imports.

Whats the diference from MemoryBase.ReadByte and the ExternalProcess one?

[Apoc]

I'm not even sure why you even need debug privileges for reading memory. Everything should work without them.

Debug privs are mostly for... well... debugging. If you don't need them, you can comment them out. It's mostly useful for out of proc stuff where RPM will throw random errors because you're reading on page boundaries and the like.

[Hyru]

GreyMagic.MemoryBase:

public IntPtr GetRelative(IntPtr absolute) {
    return ImageBase - (int) absolute;
}

If your absolute address is 0x1040 and your ImageBase is 0x1000, your relative address to the image base would be 0x40 or absolute minus ImageBase.

[xalcon]

He asked for the difference between MemoryBase.ReadByte() and ExternalProcessReader.ReadByte(), not what the MemoryBase might be...
The MemoryBase.ReadByte() is a abstract method, just a prototype which then gets overridden by the ExternalProcessReader... so... no difference :P the first "just can't be used" :P

[Apoc]

I've made some slight changes recently to this lib.

Firstly, you can now create generic structures. (AKA; templated types. Mostly useful for STL containers and the like)

Replace MarshalCache.cs with:

using System;using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;
using System.Runtime.InteropServices;
using System.Text;


namespace Loki.MemoryManagement
{
    public static class MarshalCache<T>
    {
        /// <summary> The size of the Type </summary>
        public static int Size;


        /// <summary> The real, underlying type. </summary>
        public static Type RealType;


        /// <summary> The type code </summary>
        public static TypeCode TypeCode;


        /// <summary> True if this type requires the Marshaler to map variables. (No direct pointer dereferencing) </summary>
        public static bool TypeRequiresMarshal;


        private static int GetSizeOf(Type t)
        {
            if (!t.IsGenericType)
                return Marshal.SizeOf(t);
            
            // So, chances are, we're using generic sub-types.
            // This is a good, and bad thing.
            // Good for STL implementations, bad for most everything else.
            // But for the sake of completeness, lets make this work.


            int totalSize = 0;


            foreach (var field in t.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance))
            {
                // Recursive. We want to allow ourselves to dive back into this function if we need to!
                totalSize += GetSizeOf(field.FieldType);
            }
            return totalSize;
        }


        static MarshalCache()
        {
            TypeCode = Type.GetTypeCode(typeof(T));


            // Bools = 1 char.
            if (typeof(T) == typeof(bool))
            {
                Size = 1;
                RealType = typeof(T);
            }
            else if (typeof(T).IsEnum)
            {
                var underlying = typeof(T).GetEnumUnderlyingType();
                Size = GetSizeOf(underlying);
                RealType = underlying;
                TypeCode = Type.GetTypeCode(underlying);
            }
            else
            {
                Size = GetSizeOf(typeof (T));
                RealType = typeof(T);
            }


            // Basically, if any members of the type have a MarshalAs attrib, then we can't just pointer deref. :(
            // This literally means any kind of MarshalAs. Strings, arrays, custom type sizes, etc.
            // Ideally, we want to avoid the Marshaler as much as possible. It causes a lot of overhead, and for a memory reading
            // lib where we need the best speed possible, we do things manually when possible!
            TypeRequiresMarshal = RealType.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic).Any(m => m.GetCustomAttributes(typeof(MarshalAsAttribute), true).Any());


            // Generate a method to get the address of a generic type. We'll be using this for RtlMoveMemory later for much faster structure reads.
            var method = new DynamicMethod(
                string.Format("GetPinnedPtr<{0}>", typeof(T).FullName.Replace(".", "<>")), typeof(void*), new[] { typeof(T).MakeByRefType() },
                typeof(MarshalCache<>).Module);
            ILGenerator generator = method.GetILGenerator();
            generator.Emit(OpCodes.Ldarg_0);
            generator.Emit(OpCodes.Conv_U);
            generator.Emit(OpCodes.Ret);
            GetUnsafePtr = (GetUnsafePtrDelegate)method.CreateDelegate(typeof(GetUnsafePtrDelegate));
        }




        internal unsafe delegate void* GetUnsafePtrDelegate(ref T value);


        internal static readonly GetUnsafePtrDelegate GetUnsafePtr;
    }
}

[dms51]

Thx Apoc ! Seems to be a very useful lib you share with us!

[Hyru]

Converted over from WhiteMagic and my home grown out-of-process memory manager to a GreyMagic refactor about a week ago. Happy with the results so far.

[Hyru]

He asked for the difference between MemoryBase.ReadByte() and ExternalProcessReader.ReadByte(), not what the MemoryBase might be...
The MemoryBase.ReadByte() is a abstract method, just a prototype which then gets overridden by the ExternalProcessReader... so... no difference :P the first "just can't be used" :P

I wasn't responding to GRB. I was stating that the method implementation for getting the relative address given the absolute was incorrect. Not a big deal as it'll be obvious the first time you call it.

[Apoc]

More changes!

This one is to fix the "TypeRequiresMarshal" issue where a sub-struct requires marshaling, but the main struct doesn't. It'll now just iterate over the sub-structs and see if something needs to be marshaled, then mark the parent type as requiring marshaling.

    public static class MarshalCache<T>    {
        /// <summary> The size of the Type </summary>
        public static int Size;


        /// <summary> The real, underlying type. </summary>
        public static Type RealType;


        /// <summary> The type code </summary>
        public static TypeCode TypeCode;


        /// <summary> True if this type requires the Marshaler to map variables. (No direct pointer dereferencing) </summary>
        public static bool TypeRequiresMarshal;


        private static int GetSizeOf(Type t)
        {
            try
            {
                // Note: This is in a try/catch for a reason.


                // A structure doesn't have to be marked as generic, to have generic types INSIDE of it.
                // Marshal.SizeOf will toss an exception when it can't find a size due to a generic type inside it.
                // Also... this just makes sure we can handle any other shenanigans the marshaler does.
                return Marshal.SizeOf(t);
            }
            catch
            {
                // So, chances are, we're using generic sub-types.
                // This is a good, and bad thing.
                // Good for STL implementations, bad for most everything else.
                // But for the sake of completeness, lets make this work.


                int totalSize = 0;


                foreach (var field in t.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance))
                {
                    // Check if its a fixed-size-buffer. Eg; fixed byte Pad[50];
                    var attr = field.GetCustomAttributes(typeof (FixedBufferAttribute), false);
                    if (attr.Length > 0)
                    {
                        var fba = attr[0] as FixedBufferAttribute;
                        totalSize += GetSizeOf(fba.ElementType) * fba.Length;
                    }
                    


                    // Recursive. We want to allow ourselves to dive back into this function if we need to!
                    totalSize += GetSizeOf(field.FieldType);
                }
                return totalSize;
            }
        }


        static bool RequiresMarshal(Type t)
        {
            foreach (var fieldInfo in t.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
            {
                var requires = fieldInfo.GetCustomAttributes(typeof (MarshalAsAttribute), true).Any();


                if (requires)
                {
                    Debug.WriteLine(fieldInfo.FieldType.Name + " requires marshaling.");
                    return true;
                }


                // Nope
                if (t == typeof (IntPtr) || t == typeof (string))
                    continue;


                // If it's a custom object, then check it separately for marshaling requirements.
                if (Type.GetTypeCode(t) == TypeCode.Object)
                    requires |= RequiresMarshal(fieldInfo.FieldType);


                // if anything requires a marshal, period, no matter where/what it is.
                // just return true. Hop out of this func as early as possible.
                if (requires)
                {
                    Debug.WriteLine(fieldInfo.FieldType.Name + " requires marshaling.");
                    return true;
                }
            }
            return false;
        }


        static MarshalCache()
        {
            TypeCode = Type.GetTypeCode(typeof(T));


            // Bools = 1 char.
            if (typeof(T) == typeof(bool))
            {
                Size = 1;
                RealType = typeof(T);
            }
            else if (typeof(T).IsEnum)
            {
                var underlying = typeof(T).GetEnumUnderlyingType();
                Size = GetSizeOf(underlying);
                RealType = underlying;
                TypeCode = Type.GetTypeCode(underlying);
            }
            else
            {
                Size = GetSizeOf(typeof (T));
                RealType = typeof(T);
            }


            // Basically, if any members of the type have a MarshalAs attrib, then we can't just pointer deref. :(
            // This literally means any kind of MarshalAs. Strings, arrays, custom type sizes, etc.
            // Ideally, we want to avoid the Marshaler as much as possible. It causes a lot of overhead, and for a memory reading
            // lib where we need the best speed possible, we do things manually when possible!
            TypeRequiresMarshal = RequiresMarshal(RealType);
            //Debug.WriteLine("Type " + typeof(T).Name + " requires marshaling: " + TypeRequiresMarshal);


            // Generate a method to get the address of a generic type. We'll be using this for RtlMoveMemory later for much faster structure reads.
            var method = new DynamicMethod(
                string.Format("GetPinnedPtr<{0}>", typeof(T).FullName.Replace(".", "<>")), typeof(void*), new[] { typeof(T).MakeByRefType() },
                typeof(MarshalCache<>).Module);
            ILGenerator generator = method.GetILGenerator();
            generator.Emit(OpCodes.Ldarg_0);
            generator.Emit(OpCodes.Conv_U);
            generator.Emit(OpCodes.Ret);
            GetUnsafePtr = (GetUnsafePtrDelegate)method.CreateDelegate(typeof(GetUnsafePtrDelegate));
        }




        internal unsafe delegate void* GetUnsafePtrDelegate(ref T value);


        internal static readonly GetUnsafePtrDelegate GetUnsafePtr;
    }

Edit; another fix for fixed size buffers in structs.

You can now do *anything* with your structures (generics included) and be able to read them! (Writing... not so much )

[GRB]

With lastest apoc changes im getting the follow error.

"The type or namespace name 'FixedBufferAttribute' could not be found (are you missing a using directive or an assembly reference?)"

[TOM_RUS]

With lastest apoc changes im getting the follow error.

"The type or namespace name 'FixedBufferAttribute' could not be found (are you missing a using directive or an assembly reference?)"

How about adding missing using statement?

[GRB]

How about adding missing using statement?

lol, im sleeping here.. +rep for you