[C#] Converting array of bytes to array of structs?

[xalcon]

Hi there,
currently I'm working on my wow world parser (nav mesh generation :3) and I'm realy far now. But I've a little problem with speed. The parser needs the most time for parsing the vertices, indices and normal vectors from the wmo groupfiles.

I have a stream which I read with this method:
Indices:

this.indices = new ushort[polys * 3];
for (int i = 0; i < polys * 3; i++)
{
	this.indices[i] = chunk.ReadStruct<ushort>();
}

Normals and Vertices

this.vertices = new Vector3D[chunk.Length / 3];
for (int i = 0; i < chunk.Length / 3; i++)
{
	this.vertices[i] = chunk.ReadStruct<Vector3D>();
}

ReadSturct<T>():

public static T ReadStruct<T>(this Stream s) where T : struct
{
	var sz = Marshal.SizeOf(typeof(T));
	var buf = new byte[sz];
	s.Read(buf, 0, sz);
	var handle = GCHandle.Alloc(buf, GCHandleType.Pinned);
	var t = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
	handle.Free();
	return t;
}

which gets painfully slow, when reading big wmoRoot files (and their associated groups) like Ogrimmar2.wmo
So I made a performance check using visual studios cpu sampling feature:

Parsing Normals: 30%
Parsing Vertices: 29.8%
Parsing Indices: 24.3%

I've found a way to read chunks of primitive arrays like ushort[]. After using the code below, I was able to reduce the workload on the index parsing from 24.3% to around 0.1%!!

public static TDest[] ConvertArray<TSource, TDest>(TSource[] source)
	where TSource : struct
	where TDest : struct
{

	if (source == null)
		throw new ArgumentNullException("source");

	var sourceType = typeof(TSource);
	var destType = typeof(TDest);

	var sourceByteSize = Buffer.ByteLength(source);
	var destTypeSize = Marshal.SizeOf(destType);
	if (sourceByteSize % destTypeSize != 0)
		throw new Exception(
			"The source array is " + sourceByteSize + " bytes, which can " +
			"not be transfered to chunks of " + destTypeSize + ", the size " +
			"of type " + typeof(TDest).Name + ". Change destination type or " +
			"pad the source array with additional values."
		);

	var destCount = sourceByteSize / destTypeSize;
	var destArray = new TDest[destCount];

	Buffer.BlockCopy(source, 0, destArray, 0, sourceByteSize);

	return destArray;
}

The problem is, I can't use the ConvertArray() method to convert byte arrays to arrays of struct X (like Vector3D), since Buffer.BlockCopy() is not able to copy into a struct. Is there any solution to speed up my vertex and normal parsing, too?

[Apoc]

Have you tried casting from a fixed byte* to the struct? Eg;

fixed (byte* pBuffer = bytes)
for(int i = 0; i < count; i++)
    ret[i] = ((StructureType*)pBuffer)[i];

[xalcon]

You are my hero, apoc.

Before: 8.65 seconds
After: 0.94 seconds

That is realy awesome Thank you!

Sadly, VS don't like to give me an unmanaged pointer to T

public static unsafe T[] Convert<T>(byte[] buffer)
{
	T[] v = new T[buffer.Length / (Marshal.SizeOf(typeof(T)))];
	fixed (byte* pBuffer = buffer)
	{
		for (int i = 0; i < v.Length; i++)
			v[i] = ((T*)pBuffer)[i];
	}
	return v;
}

But anyway, changing T to a specific type works and since I don't have so much different structures I need to parse fast, this is working realy well too!
Here is some working code:

public static unsafe Vector3D[] GetVertices(byte[] buffer)
{
	Vector3D[] vecs = new Vector3D[buffer.Length / (3*4)];
	fixed (byte* pBuffer = buffer)
	{
		for (int i = 0; i < vecs.Length; i++)
			vecs[i] = ((Vector3D*)pBuffer)[i];
	}
	return vecs;
}

// Edit:
I've changed another part to this converting aproach and... now I only need 0.090 seconds O__O to load Ogrimmar2.wmo... and the most time I spend on removing unimportant polygons (Flags & 4 && Material == 0xFF). Just 42 ms for parsing all WMO related stuff... compared to the 8.6 seconds before... its ridiculous

*Hug apoc*

[Apoc]

You are my hero, apoc.

Before: 8.65 seconds
After: 0.94 seconds

That is realy awesome Thank you!

Sadly, VS don't like to give me an unmanaged pointer to T

public static unsafe T[] Convert<T>(byte[] buffer)
{
    T[] v = new T[buffer.Length / (Marshal.SizeOf(typeof(T)))];
    fixed (byte* pBuffer = buffer)
    {
        for (int i = 0; i < v.Length; i++)
            v[i] = ((T*)pBuffer)[i];
    }
    return v;
}

There are ways to get around that, but you'll need some more stuff that is probably not worth it for the stuff you're doing.