About a year ago I (netfinity) started working on an implementation to exchange incomplete parts between clients by advertising completed AICH blocks when there was no other parts the remote client needed. This feature was later picked up and polished by the NEO mod. However, I always felt the feature too much of a compromise. So, I started looking at how the hybrids do it and decided that it would be a good foundation for my project.
Below I present the findings about the protocol that have been done so far. With reservation for errors, ofcourse!
Crumbs Protocol Revision 1 [pr=1]
In the Crumbs protocol, the file is divided into 475kB chunks instead of the normal 9500kB chunks. This is reflected on the part status vector that will become ~20 times the size due to this.
OPCODES said:
This opcode is used to request a Crumb hashset from a client.
OP_CRUMBSETANS - 0x68 <File Hash>[<has part hash set>[<Part hash set>]]<has crumb hash set>[<Crumb Hash set>]
This is the reply to OP_CRUMBSETREQ and contains a list of hashes for each Crumb in the file. A Crumb hash is the first 8 bytes in the MD4 hash of the Crumb. It also contains the part hash set if the file has more than one part.
OP_CRUMBCOMPLETE - 0x6A <File Hash><Crumb Index> [This op-code can be ignored, but is not recommended]
This opcode is sent to all connected clients downloading the specific file when completing a Crumb. I.e its used to advertise that a new Crumb has become available.
Hello TAGS said:
This tag is often called the Horde tag, but I haven't noticed that the tag has any relation to the Horde protocol. Instead it is used to inform the remote client that we support part vectors using Crumbs as the chunk size.
Extended OPCODES said:
This opcode would be sent when a remote client ask for the specified file and we haven't sent this info before. The hashed hashsets are the ones we currently beleive are true. The benefit of knowing every hashset, trusted by other clients, is that we can with quite certanity know that we can trust a hashset if no other hashset have been published. I.e if there is one good source, there must be atleast one good hashset.
SCT Protocol Revision 2 [pr=2]
As eMule is not using crumb hashes and already have the AICH hashing mechanism, a modified version of the protocol is in place.
In this version, all the above in Crumbs Protocol Revision 1 has to be supported. What differs is that a new set of chunk sizes can be used. The new sizes are 180kB, 360kB, 720kB, 1440kB, 2880kB, 5760kB and "the entire file". The last is a part status vector with only one bit which is set to 0 and is used to signify that the source doesn't yet have any parts to share. Note, that all chunks except for "the entire file" are truncated at part boundary as in AICH. The op-codes in revision 1 is not used when using these chunk sizes.
I hereby declare this the official thread for the implentation of smaller chunks in eMule!
/netfinity
This post has been edited by netfinity: 24 August 2013 - 06:02 PM