Packet reassembly for the guys who is using sharppcap

[snoke]

Hi one of the biggest problem i have had while trying to create a winpcap sniffer of wow packets is that with winpcap the packets might come in out of order. so maybe someone here might be interested in it.

i have created a simple class , not really production ready since its almost at prototype stage now ( so dont bug me how about how i dont check for null values etc ) . but anywhoos it reassembles the packets so that everything comes in the correct order. this is extremely important in wow because of the encryption , you cant just miss one packet , if you do rest of the packets will be a mess.

The sniffer class is a generic one which could sniff any data in correct order in "theory"

An example how i sniff wow packets below ( PacketMgr is the class that actually decrypts the wow data and parses it as it is coming in ), i wont show how you actually decrypt the packets , that is up to the reader , but a tip the mangos 3.1 branch is a good starting point , so is the java snifftzt app which shows you how to decrypt and parse the packets.

   public class SnifferWorld : Sniffer
    {
        static readonly PacketMgr packetmgr = Singleton<PacketMgr>.Instance;

        public SnifferWorld()
        {
            AddListenHostConnectionInfo("80.239.149.99",3724,"EU-Magtheridon"); // Magtheridon eu
            AddListenHostConnectionInfo("80.239.149.103", 3724,"EU-The Maelstrom"); // The Maelstrom eu
            AddListenHostConnectionInfo("80.239.233.88", 3724, "EU-Stormscale");
        }
        public override void Connected(Connection conn)
        {
            packetmgr.Reset(conn.Identifier); // reset key info and logs.
        }
 
        public override void AddClientData(byte[] buffer)
        {
            var data = new byte[buffer.Length];
            buffer.CopyTo(data, 0);
            packetmgr.AddTcpPacket(PacketType.Client, data);
            
        }
        public override void AddHostData(byte[] buffer)
        {
            var data = new byte[buffer.Length];
            buffer.CopyTo(data, 0);
            packetmgr.AddTcpPacket(PacketType.Server, data);
            
        }



    }

Here is the base class , the AssemblePacket method is the one responsible for putting tcp data in the correct order.

using System;
using System.Collections.Generic;
using SharpPcap;
using SharpPcap.Packets;
using System.Net;


namespace Proxy
{
    // class used for identifing host/port combos we should listen for.
    public class ListeHostConnectionInfo
    {
        public string IP;
        public int Port;
        public string Identifier;
        public ListeHostConnectionInfo(string ip, int port,string identifier)
        {
            IP = ip;
            Port = port;
            Identifier = identifier;
        }
    }

    // each successfull 3way tcp connection gets a new Connection
    // this is removed when Fin is received and connection is closed.
    public class Connection
    {
        public long ClientAddress; // client initiating the connection
        public int ClientPort;

        public long HostAddress;   // host receiving the connection
        public int HostPort;

        public long ClientSyn;    // starting syn sent from client
        public long HostSyn;      // starting syn sent from host;

        public long NextClientSeq;
        public long NextHostSeq;

        public bool HostClosed;
        public bool ClientClosed;

        public string Identifier = "";
        public bool ThreeWayCompleted = false; // three way connection is completed

        // Fragments , used when we get newer packets that expected.
        // so we need to wait for expected before adding them.
        public SortedDictionary<long, TCPPacket> HostFragments = new SortedDictionary<long, TCPPacket>();
        public SortedDictionary<long, TCPPacket> ClientFragments = new SortedDictionary<long, TCPPacket>();

        // returns client ip:client port as a string
        public string GetClientAddressPort()
        {
            return string.Format("{0}:{1}", new IPAddress(ClientAddress).ToString(), ClientPort);
        }
        // returns host ip:host port as a string
        public string GetHostAddressPort()
        {
            return string.Format("{0}:{1}", new IPAddress(HostAddress).ToString(), HostPort);
        }

        // packet is from host
        public bool IsFromHost(TCPPacket tcp)
        {
            return ClientAddress == tcp.DestinationAddress.Address &&
            ClientPort == tcp.DestinationPort &&
            HostAddress == tcp.SourceAddress.Address &&
            HostPort == tcp.SourcePort;
        }
        // packet is from client
        public bool IsFromClient(TCPPacket tcp)
        {
            return ClientAddress == tcp.SourceAddress.Address &&
            ClientPort == tcp.SourcePort &&
            HostAddress == tcp.DestinationAddress.Address &&
            HostPort == tcp.DestinationPort;
        }


        public Connection(long clientAddress, int clientPort, long hostAddress, int hostPort, long clientSyn)
        {
            this.ClientAddress = clientAddress;
            this.ClientPort = clientPort;
            this.HostAddress = hostAddress;
            this.HostPort = hostPort;
            this.ClientSyn = clientSyn;
        }

    }

    
    // Base sniffer class.
    // this handles all the actuall packet capture
    // and reassembles the data into a tcp stream.

    // derive from this and overload the AddClientData , AddHostData etc to receive traffic.
    // use AddListenHostConnectionInfo to tell sniffer what ip/ports you are interested in.
    public class Sniffer
    {
        // what ip's and port we should listen for.
        // added from subclasses.
        readonly List<ListeHostConnectionInfo> _listenHostConnectionInfos = new List<ListeHostConnectionInfo>();
        
        // list of Connections we have seen do a three way handshake
        // connection will be valid untill we have seen a packet with Fin from both parties.
        public List<Connection> Connections = new List<Connection>();

        // on valid client data arriving.
        public virtual void AddClientData(byte[] buffer){}

        // on valid host data arriving. 
        public virtual void AddHostData(byte[] buffer){}

        // when connected
        public virtual void Connected(Connection conn){}

        // when disconnected
        public virtual void Disconnected(Connection conn){}
        
        // main capture loop will run forever
        public void Run()
        {

            /* Retrieve the device list */
            var devices = SharpPcap.Pcap.GetAllDevices();

            /*If no device exists, print error */
            if (devices.Count < 1)
            {
                Console.WriteLine("No device found on this machine");
                return;
            }

            Console.WriteLine();
            Console.WriteLine("The following devices are available on this machine:");
            Console.WriteLine("----------------------------------------------------");
            Console.WriteLine();

            int i = 0;

            /* Scan the list printing every entry */
            foreach (PcapDevice dev in devices)
            {
                /* Description */
                Console.WriteLine("{0}) {1} {2}", i, dev.Name, dev.Description);
                i++;
            }

            Console.WriteLine();
            Console.Write("-- Please choose a device to capture: ");
            i = int.Parse(Console.ReadLine());

            var device = devices[i];

            //Register our handler function to the 'packet arrival' event
            device.OnPacketArrival += DevicePcapOnPacketArrival;

            //Open the device for capturing
            //true -- means promiscuous mode
            //10-- means a read wait of 10ms
            device.Open(true, 10);

            Console.WriteLine();
            Console.WriteLine("-- Listenning on {0}, hit 'Enter' to stop...",
                device.Description);

            //Start the capturing process
            device.StartCapture();

            //Wait for 'Enter' from the user.
            string line = "";
            while ( line.IndexOf("exit") == -1 )
            {
                line = Console.ReadLine();
            }

            //Stop the capturing process
            device.StopCapture();

            Console.WriteLine("-- Capture stopped.");

            //Close the pcap device
            device.Close();
        }


        // pcap event
        private void DevicePcapOnPacketArrival(object sender, PcapCaptureEventArgs e)
        {
            if (e.Packet is TCPPacket)
            {
                AssemblePacket(e.Packet as TCPPacket); 
            }
        }

        // add ip,ports which should be listened for. also a helper identifier which can be used
        // to get a name for the connection.
        protected void AddListenHostConnectionInfo(string ip, int port,string identifier)
        {
            _listenHostConnectionInfos.Add(new ListeHostConnectionInfo(ip, port, identifier));
        }

        // simple assembling of packets.
        // missing lots, but works.
        private void AssemblePacket(TCPPacket tcp)
        {
            string identifier = ""; //
            // check if we should listen for this host ip and port.
            bool found = false;
            foreach (var info in _listenHostConnectionInfos)
            {
                if ((tcp.SourceAddress.ToString() == info.IP && tcp.SourcePort == info.Port) ||
                    (tcp.DestinationAddress.ToString() == info.IP && tcp.DestinationPort == info.Port))
                {
                    identifier = info.Identifier;
                    found = true;
                    break;
                }
            }
            if (!found) return;

            if (tcp.Syn && tcp.PayloadDataLength == 0) // 3 way connection establishing. for ISN
            {
                // find connection if available
                if (tcp.Ack)
                {
                    foreach (var conn in Connections)
                    {
                        if (!conn.IsFromHost(tcp)) continue;
                        conn.HostSyn = tcp.SequenceNumber;
                        conn.NextClientSeq = tcp.AcknowledgmentNumber; // Steap 2 of 3 way done.
                    }
                }
                else // step 1 of 3 way
                {
                    var conn = new Connection(tcp.SourceAddress.Address, tcp.SourcePort, tcp.DestinationAddress.Address, tcp.DestinationPort, tcp.SequenceNumber);
                    Connections.Add(conn);
                }
            }
            else if (tcp.Fin) // connection closing
            {
                Connection tmpConn = null;
                foreach (var conn in Connections)
                {
                    if (conn.IsFromClient(tcp))
                    {
                        conn.ClientClosed = true;
                        if (conn.HostClosed)
                            tmpConn = conn;
                    }
                    else if (conn.IsFromHost(tcp))
                    {
                        conn.HostClosed = true; ;
                        if (conn.ClientClosed)
                            tmpConn = conn;
                    }
                }
                if (tmpConn != null)
                {
                    Disconnected(tmpConn);
                    Console.WriteLine("Connection: {0}->{1} closed, removing conn object", tmpConn.GetClientAddressPort(), tmpConn.GetHostAddressPort());
                    Connections.Remove(tmpConn);
                }
            }
            else // Data
            {
                foreach (var conn in Connections)
                {
                    // from client
                    if (conn.IsFromClient(tcp))
                    {
                        if (tcp.SequenceNumber < conn.NextClientSeq) // old packet
                        {
                            // just drop these for now
                            return;
                        }
                        
                        if (tcp.SequenceNumber > conn.NextClientSeq) // out of order data
                        {
                            if (!conn.ClientFragments.ContainsKey(tcp.SequenceNumber))
                            {
                                conn.ClientFragments.Add(tcp.SequenceNumber, tcp);
                            }
                            else // expect new data to be better?
                            {
                                conn.ClientFragments[tcp.SequenceNumber] = tcp;
                            }
                        }
                        else
                        {
                            while (tcp.SequenceNumber == conn.NextClientSeq)
                            {
                                if (!conn.ThreeWayCompleted) // 3 way handshake is a success and a connection is up and running.
                                {
                                    Console.WriteLine("Connection between: {0} and {1} established", conn.GetClientAddressPort(), conn.GetHostAddressPort());
                                    conn.ThreeWayCompleted = true;
                                    conn.NextHostSeq = tcp.AcknowledgmentNumber;
                                    conn.Identifier = identifier;
                                    Connected(conn);
                                    break;
                                }
                                conn.ClientFragments.Remove(tcp.SequenceNumber); // remove fragment
                                if (tcp.PayloadDataLength == 0)
                                    break;
                                conn.NextClientSeq = conn.NextClientSeq + tcp.PayloadDataLength;
                                // data should be valid here.
                                AddClientData(tcp.Data);

                                if (conn.ClientFragments.ContainsKey(conn.NextClientSeq)) // check if we have newer fragments which will now fit.
                                {
                                    tcp = conn.ClientFragments[conn.NextClientSeq];
                                }
                                else
                                    break;
                            }
                        }

                    }
                    // from host
                    else if (conn.IsFromHost(tcp))
                    {
                        if (tcp.SequenceNumber < conn.NextHostSeq) // old packet
                        {
                            // just drop these for now
                            return;
                        }
                        if (tcp.SequenceNumber > conn.NextHostSeq) // newer out of order data
                        {
                            if (!conn.HostFragments.ContainsKey(tcp.SequenceNumber))
                            {
                                conn.HostFragments.Add(tcp.SequenceNumber, tcp);
                            }
                            else
                            {
                                conn.HostFragments[tcp.SequenceNumber] = tcp;
                            }
                        }
                        else // 
                        {
                            while (tcp.SequenceNumber == conn.NextHostSeq) // on time
                            {
                                conn.HostFragments.Remove(tcp.SequenceNumber); // remove fragment

                                if (tcp.PayloadDataLength == 0)
                                    break;
                                conn.NextHostSeq = conn.NextHostSeq + tcp.PayloadDataLength;

                                // data should be valid here
                                AddHostData(tcp.Data);
                                if (conn.HostFragments.ContainsKey(conn.NextHostSeq)) // check if we have newer fragments which will now fit.
                                {
                                    tcp = conn.HostFragments[conn.NextHostSeq];
                                }
                                else
                                    break;

                            }
                        }
                    }
                }
            }
        }
    }
}

[amadmonk]

This is super useful. I use a (mostly) out-of-process bot, and without packet sniffing this really limits some of the data I can notify on. Your code, and a bit of research, should get me most of what I need.

Now if I could just figure out a way to set my character's target without running a LUA function, I'd be a happy camper...

[Shynd]

Set a keybinding to target last target, modify the last target in memory, then hit the keybind! Yes!

[amadmonk]

That's an incredibly simple and easy idea. Dankeschon!

[amadmonk]

Thanks again, snoke. With your leads I've got a complete packet assembler and decryptor working now. Mixing it up with the source codes from Mangos, I've got a way to get all the info I need from WoW without needing to maintain a single offset from patch to patch (well, except for the offset to the session key in memory... okay, one...) Plus, since it's packet-driven, it's inherently event-driven... no more polling memory!

My OOP bot is coming along nicely.

OT: now that I can see all its chatter... Warden is a chatty little beast, isn't it? I'm tempted to capture the Warden traffic stream and isolate it from the rest just to watch it at work...

[MikeDotNet]

Nice stuff, I might implement that eventually if I ever bother finding the session key (wich I would need to do someday).

Btw, if you wan't to work on warden, feel free to drop me a PM I would need to work onto wow warden a little eventually (it shouldn't be that different from d2/sc/w3 warden).

[kynox]

Nice stuff, I might implement that eventually if I ever bother finding the session key (wich I would need to do someday).

Btw, if you wan't to work on warden, feel free to drop me a PM I would need to work onto wow warden a little eventually (it shouldn't be that different from d2/sc/w3 warden).

The packet structure is exactly the same.