WoW protocol objects synchronization

[nerexis]

I'm looking for information about WoW protocol, and gather some if it here, mostly for version 2.4.3 (3.3.5 should also be ok), but I think general method didn't change so far in later versions, so someone may know something about it. I've also seen the 'clientless' program called PseuWoW, and it's source code helped me very much, but still I have problem when creating my "ObjectMgr" for packet only based client.

I wanna know which and how packets are designed which synchronize object data (creatures, players, npc, items, bags etc.) between server and client.

What I know so far, the code in private server like TrinityCore which synchronizes data is defined in Object.cpp, which is creating part of whole packet called Update Block, where the full packet can contain many of these blocks, function:

void Object::BuildCreateUpdateBlockForPlayer(UpdateData *data, Player *target) const

The 'UpdateData' packets are build in blocks, first bytes of block are built this way:

ByteBuffer buf(500);
    buf << uint8(updatetype);
    //buf.append(GetPackGUID());    
    buf << uint8(0xFF) << GetGUID();
    buf << uint8(m_objectTypeId);

later to this block, information about object movement (position etc.) is added.

Possible error: in PseuWoW code which reads packet data, reads it using function readPackGUID() from ByteBuffer, and half of GUID read from packet is wrong.
The server also has function to add guid to bufer in 'packed version' but for some reason in Trinity it is not used (commented as you can see above).
I don't know what is difference when reading guid using 'packed' function.
Why before GUID is added uint8(0xFF)?
So maybe this is problem with incorrect GUID when reading on client side.

PseuWoW, Function:

_HandleUpdateObjectOpcode(WorldPacket& recvPacket)

in UpdateData.cpp

case UPDATETYPE_CREATE_OBJECT2: // will be sent when our very own character is created
		case UPDATETYPE_CREATE_OBJECT: // will be sent on any other object creation
		{

			// added by me, didnt solve it
			uint8 dummy;
			//recvPacket >> dummy; //0xFF - didnt solve it

			uguid = recvPacket.readPackGUID();
			//recvPacket >> uguid;


			uint8 objtypeid;
			recvPacket >> objtypeid;
			logdebug("Create Object type %u with guid "I64FMT, objtypeid, uguid);
...

ByteBuffer.cpp

        uint64 readPackGUID()
        {
            uint64 guid = 0;
            uint8 guidmark = 0;
            (*this) >> guidmark;

            for(int i = 0; i < 8; ++i)
            {
                if(guidmark & (uint8(1) << i))
                {
                    uint8 bit;
                    (*this) >> bit;
                    guid |= (uint64(bit) << (i * 8));
                }
            }

            return guid;
        }

...

Functions for building Movement part and Values part of update block.

Functions from TrinityCore 2.4.3:

void Object::BuildMovementUpdate(ByteBuffer * data, uint8 updateFlags) const

builds data.
Full: [C++] void Object::BuildMovementUpdate(ByteBuffer * data, uint8 updateFlags) const { - Pastebin.com

Later, information about object stats are added (UpdateFields) to the buffer, and here I have problem. , function:

void Object::BuildValuesUpdate(uint8 updatetype, ByteBuffer * data, UpdateMask *updateMask, Player *target) const

full code: [C++] void Object::BuildValuesUpdate(uint8 updatetype, ByteBuffer * data, UpdateMask * - Pastebin.com

When I try to read this data something wrong happens.
Actually I've checked and using some code from PseuWoW but still it doesn't work.
First of all GUID is not read correctly (at start of update block) and probably because of that, later the update mask of block and object stats values are read incorrectly, however movement info, which is before the values info (but also needs better check...), is read ok.

The code which builds whole packet sent to client is defined in TrinityCore in UpdateData.cpp:
[C++] bool UpdateData::BuildPacket(WorldPacket *packet, bool hasTransport) { //B - Pastebin.com

I don't understand fully the first part about UPDATETYPE_OUT_OF_RANGE_OBJECTS.
Later, it is pretty easy, if packet has size more than >50, data is compressed using zlib and sent as opcode SMSG_COMPRESSED_UPDATE_OBJECT, if not SMSG_UPDATE_OBJECT, haven't got problem with that.

Code from PseuWoW, UpdateData.cpp reading data from packets, all 3 functions for reading first part of packet, then movement updates, then object values.
[C++] void WoWWorldServerClient::_HandleUpdateObjectOpcode(WorldPacket& recvPacket) { - Pastebin.com


What else I need to know: when I enter the game with my client based on packets, my client can receive packet about creation of new object, but I can't about objects already in place, where player spawns in world, after logging in. The server doesn't send packets about creation of these objects, but only update packets for them, without whole info (or am I incorrect?).
So why is that, and how I can force server to send me all information about near objects.

Also If some information posted by me is incorrect or more information i s needed, please, tell me. I'm working on it all the time currently, and some information here may change.

[nerexis]

Already fixed most of above problems, about reading movement info and values info from packet.

Code handling of movement update in PseuWoW was bugged and also as I expected, the packed GUID thing (should be read as unpacked, and before that read that 0xFF). Probably because even if they supported versions form classic wow to partly wotlk, they missed some checks and 2.4.3 version was bugged in latest commit. However may work of ouf box with 3.3.5 idk.

Still don't know what to do to force server to send me packets with information about objects (npcs etc.) near player. I get packet with that information only, when object is created when player is already in that place. So if for example guard is standing in front of Orgrimmar, and I log in, I don't get info about that object. How to force server to send me information about local objects after I enter the game.


UPDATE:
I forgot to set socket option TcpNoDelay to true, and when I did that, server is sending info about local objects
However something wrong is with buffering packets probably in my client or server.
Currently I'm sending many packets very fast like CMSG_CREATURE_QUERY to get creature information like name etc. when they are initialized.
Server is only responding maybe to one packet. Maybe it's kind of anti DDOS feature or something.

[andy012345]

Just look through the logs of the popular emulators who have had this solved for years.

The 0xFF sounds like it's just another packed GUID, 0xFF followed by the GUID is essentially a packed guid saying every byte has been included.

[nerexis]

Partially fixed packets issue, just changed socket "Select" function sleep time to 0, but socket is blocking now which isn't good.
If I don't use Select as blocking, most of packets aren't captured, I don't know why.
I'm using this library for my tcp sockets: C++ Sockets Library: Unix & Windows networking made easy
I think maybe packets are buffered or something, have to check it.


UPDATE: missing packets issue is fixed. World packets were sometimes delayed, now I can read every packet.