1) Client sends request to server:

MY HUFFMAN:
Preparing to send [unencoded]: 0 50 46 48 45 97 108 112 104 97 0 112 97 115 115 119 111 114 100 0 5 191 48 50 98 51 99 102 56 53 97 49 97 101 100 57 55 57 52 55 101 53 49 98 52 54 100 54 98 102 53 49 56 99 0
Sending (out 56) [encoded]: 255 0 50 46 48 45 97 108 112 104 97 0 112 97 115 115 119 111 114 100 0 5 191 48 50 98 51 99 102 56 53 97 49 97 101 100 57 55 57 52 55 101 53 49 98 52 54 100 54 98 102 53 49 56 99 0


CURRENT HUFFMAN:
Preparing to send [unencoded]: 0 50 46 48 45 97 108 112 104 97 0 112 97 115 115 119 111 114 100 0 5 58 48 50 98 51 99 102 56 53 97 49 97 101 100 57 55 57 52 55 101 53 49 98 52 54 100 54 98 102 53 49 56 99 0
Sending (out 56) [encoded]: 255 0 50 46 48 45 97 108 112 104 97 0 112 97 115 115 119 111 114 100 0 5 58 48 50 98 51 99 102 56 53 97 49 97 101 100 57 55 57 52 55 101 53 49 98 52 54 100 54 98 102 53 49 56 99 0

=================================================================================================
**ALL GOOD SO FAR**
The unencoded has some slightly different bytes, unknown how significant this is, will test later
=================================================================================================


2) Server responds

MY HUFFMAN:
Server to client response
Reading (10 bytes) [encoded]: 7 35 73 242 96 238 161 180 100 0
Reading [decoded]: 3 0 0 0 0 0 77 65 80 48 49 0


CURRENT HUFFMAN:
Server to client response
Reading (10 bytes) [encoded]: 7 68 146 132 139 19 73 79 151 0
Reading [decoded]: 3 0 0 0 0 0 77 65 80 48 49 0

====================
**ALL GOOD SO FAR**
Both say in the console:
"Connected!"
"Authenticating level..."
====================



3) Client responds again:

MY HUFFMAN:
CLIENT:
Preparing to send [unencoded]: 1 0 97 102 54 50 48 101 53 100 54 51 50 54 100 97 100 56 54 99 56 55 54 97 49 99 98 97 101 99 49 57 97 100 0 53 54 56 48 48 54 50 51 49 98 51 50 100 52 48 50 57 52 49 100 50 50 100 54 99 100 48 56 52 51 51 54 0 57 54 49 97 97 102 52 48 55 49 57 101 55 48 53 48 56 57 98 102 102 55 101 100 49 48 98 101 52 57 100 101 0 97 51 99 98 98 102 48 99 56 100 52 48 48 53 57 56 50 50 50 49 100 102 52 102 97 102 53 49 53 50 52 55 0 0
Sending (out 136) [encoded]: 255 1 0 97 102 54 50 48 101 53 100 54 51 50 54 100 97 100 56 54 99 56 55 54 97 49 99 98 97 101 99 49 57 97 100 0 53 54 56 48 48 54 50 51 49 98 51 50 100 52 48 50 57 52 49 100 50 50 100 54 99 100 48 56 52 51 51 54 0 57 54 49 97 97 102 52 48 55 49 57 101 55 48 53 48 56 57 98 102 102 55 101 100 49 48 98 101 52 57 100 101 0 97 51 99 98 98 102 48 99 56 100 52 48 48 53 57 56 50 50 50 49 100 102 52 102 97 102 53 49 53 50 52 55 0 0

Reading (5 bytes) [encoded]: 1 227 148 228 20
/* BOOM client error */
// Seems to crash when this is decoded
// Decoding analysis on its own yields that my decoder translates it to: 3 1 0 0 0 1 0
// This is what is expected, so it seems to crash with the same code that it gets from the current huffman... which makes no sense

// Apparently sends out something here for some reason? Is this a disconnect?
Preparing to send [unencoded]: 4
Sending (out 2) [encoded]: 255 4



SERVER:
Preparing to send [unencoded]: 3 1 0 0 0 1 0
Sending (out 5) [encoded]: 1 227 148 228 20
// Shows that there is proof of it being sent compressed properly

Reading (2 bytes) [encoded]: 255 4
Reading [decoded]: 4
127.0.0.1:10667 disconnected.
// This is probably the quit message


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




CURRENT HUFFMAN:
CLIENT:
Preparing to send [unencoded]: 1 0 97 102 54 50 48 101 53 100 54 51 50 54 100 97 100 56 54 99 56 55 54 97 49 99 98 97 101 99 49 57 97 100 0 53 54 56 48 48 54 50 51 49 98 51 50 100 52 48 50 57 52 49 100 50 50 100 54 99 100 48 56 52 51 51 54 0 57 54 49 97 97 102 52 48 55 49 57 101 55 48 53 48 56 57 98 102 102 55 101 100 49 48 98 101 52 57 100 101 0 97 51 99 98 98 102 48 99 56 100 52 48 48 53 57 56 50 50 50 49 100 102 52 102 97 102 53 49 53 50 52 55 0 0
Sending (out 136) [encoded]: 255 1 0 97 102 54 50 48 101 53 100 54 51 50 54 100 97 100 56 54 99 56 55 54 97 49 99 98 97 101 99 49 57 97 100 0 53 54 56 48 48 54 50 51 49 98 51 50 100 52 48 50 57 52 49 100 50 50 100 54 99 100 48 56 52 51 51 54 0 57 54 49 97 97 102 52 48 55 49 57 101 55 48 53 48 56 57 98 102 102 55 101 100 49 48 98 101 52 57 100 101 0 97 51 99 98 98 102 48 99 56 100 52 48 48 53 57 56 50 50 50 49 100 102 52 102 97 102 53 49 53 50 52 55 0 0

Reading (5 bytes) [encoded]: 3 100 151 180 11
Reading [decoded]: 3 1 0 0 0 1 0
Level authenticated!



SERVER:
Preparing to send [unencoded]: 3 1 0 0 0 1 0
Sending (out 5) [encoded]: 3 100 151 180 11

Quote
For example, there is a reliable packet buffer which could probably be streamed rather than compressed individually. Surely lz4 should win out over huffman if that can work.

Quote
Is your implementation using the same table as the old one? If not, this would mean that servers using the new implementation (i.e. 3.x+) cannot tell clients clients using the old implementation (2.0) that their version is outdated, right?

Quote
If this is the case, we first have to decide on a new network header. We should only completely break backwards compatibility once. The new header surely needs to contain the packet number. From what I understood, this is a requirement to make an lz4 stream possible.

Quote from Watermelon
If we realize its an old client

Quote from Watermelon
Do you think its worth pursuing lz4 still?

Quote from Watermelon

Also, any suggestions on what you had in mind for the header?
Ex: Single byte?
Bitfields?

Quote
How can we find out that it's on old client?

bool isNewClient = (header != 0xFF) && (!!(header & 0x70));

Quote
Yes, the stream should automatically adjust the compression to the data. Thus, it should be much more efficient than any fixed table and we don't have to reinvent the wheel, since lz4 is designed to compress network traffic.

If I understood Dusk correctly when we talked about lz4 last, one of the main problems was that we didn't have a proper network header that shows the packet number before decompressing the packet.

Mandatory contents of the header are the compression type and the packet number (to keep track of lost packets). How exactly they are encoded and if additional things are needed is up for discussion.

// Let header = data[0]
bool oldClient = false; // Assume by default it's a new client.

if (header == 0xFF)
    oldClient = true
    rawData = 1 .. n
else if (header <= 7)
    oldClient = true
    oldHuffmanDecode...
else if (header & 0x20)
    huffmanDecode...
else if (header & 0x40)
    switch (header & 0x3F)
    case 0:
        LZ4...
    case 1:
        methodX...
    case 2:
        methodY...
    default:
        *corrupt data detected*
else
    *corrupt data detected*