Improvement Of Emule-Kad Lookup Performance packet loss; inconsistency of round-trip UDP ports
#1
Posted 31 August 2011 - 03:13 PM
To solve the problems above, We have made a patch on eMule 0.50a. For more detailed descriptions and a full patch, please Email me.
#2
Posted 31 August 2011 - 03:42 PM
This post has been edited by Stulle: 31 August 2011 - 03:43 PM
[Imagine there was a sarcasm meter right here!]
No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
#3
Posted 31 August 2011 - 04:26 PM
Extended signature: click.
#4
Posted 01 September 2011 - 02:23 AM
Stulle, on 31 August 2011 - 03:42 PM, said:
#5
Posted 01 September 2011 - 02:27 AM
#6
Posted 01 September 2011 - 08:26 AM
liubingshuang, on 01 September 2011 - 04:23 AM, said:
Stulle, on 31 August 2011 - 03:42 PM, said:
A place to upload your patch? Maybe a dropbox account or a personal wiki?
Bear in mind that official developers are quite busy, instead all developers on emule project forum can analyze your patch more quickly.
Extended signature: click.
#7
Posted 01 September 2011 - 09:59 AM
pier4r, on 01 September 2011 - 08:26 AM, said:
liubingshuang, on 01 September 2011 - 04:23 AM, said:
Stulle, on 31 August 2011 - 03:42 PM, said:
A place to upload your patch? Maybe a dropbox account or a personal wiki?
Bear in mind that official developers are quite busy, instead all developers on emule project forum can analyze your patch more quickly.
#8
Posted 01 September 2011 - 11:08 AM
liubingshuang, on 01 September 2011 - 01:59 PM, said:
Then, why you posted it here, while it's clearly written in Code Snippets' subtitle: To post validated fixes & important addons (only!)?
#9
Posted 01 September 2011 - 11:33 AM
fox88, on 01 September 2011 - 01:08 PM, said:
Ok, I'm not sure, but I can't remember it was posted in Code Snippets yesterday but only in the eMule Development section, so I think it's a slight chance the topic has been moved.
Well, memory is what it is, but I have strange feeling about it though...
This post has been edited by Nissenice: 01 September 2011 - 11:34 AM
#10
Posted 01 September 2011 - 11:36 AM
[Imagine there was a sarcasm meter right here!]
No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
#11
Posted 01 September 2011 - 03:15 PM
liubingshuang, on 01 September 2011 - 11:59 AM, said:
Ok then, i don't agree with your protocol of sharing knowledge but anyway i can't say "i'm right" or "you must do this!". I hope that sooner or later your work will be available to emule community .
Extended signature: click.
#12
Posted 02 September 2011 - 02:23 AM
#14
Posted 02 September 2011 - 07:14 AM
come on liubingshuang, now it's just getting weird! you were speaking about a detailed description and some code. post an abstract here and upload the files to a file hoster anyone with basic english skills can use without having to use his personal crystal ball.
This post has been edited by Stulle: 02 September 2011 - 07:16 AM
[Imagine there was a sarcasm meter right here!]
No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
#15
Posted 02 September 2011 - 07:29 AM
Hope it works. The file is named eMule_srchybrid.patch.
This post has been edited by Nissenice: 02 September 2011 - 07:33 AM
#16
Posted 02 September 2011 - 05:04 PM
Stulle, on 02 September 2011 - 07:14 AM, said:
come on liubingshuang, now it's just getting weird! you were speaking about a detailed description and some code. post an abstract here and upload the files to a file hoster anyone with basic english skills can use without having to use his personal crystal ball.
#17
Posted 02 September 2011 - 05:14 PM
[Imagine there was a sarcasm meter right here!]
No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
#18
Posted 02 September 2011 - 06:39 PM
come ottenere aiuto italian guides - guide della sezione italiana
italian support - sezione italiana scaricare la lista server
ottenere id alto impostare le porte nel router
recuperare file corrotti i filtri ip
Sembra talco ma non č serve a darti l'allegrIa! Se lo lanci e poi lo respiri ti dā subito l'allegrIa!
#19
Posted 02 September 2011 - 08:04 PM
diff -urB srchybrid/ClientUDPSocket.cpp srchybrid_withpatch/ClientUDPSocket.cpp --- srchybrid/ClientUDPSocket.cpp 2009-02-22 19:36:56.000000000 +0800 +++ srchybrid_withpatch/ClientUDPSocket.cpp 2011-08-31 10:48:16.000000000 +0800 @@ -62,7 +62,8 @@ POSITION pos = controlpacket_queue.GetHeadPosition(); while (pos){ UDPPack* p = controlpacket_queue.GetNext(pos); - delete p->packet; + if(!p->bKad)//delete kad packet in packet resend method + delete p->packet; delete p; } } @@ -523,7 +524,8 @@ sentBytes += nLen; // ZZ:UploadBandWithThrottler (UDP) controlpacket_queue.RemoveHead(); - delete cur_packet->packet; + if(!cur_packet->bKad)//delete kad packet in packet resend method + delete cur_packet->packet; delete cur_packet; } delete[] sendbuffer; @@ -531,7 +533,8 @@ else { controlpacket_queue.RemoveHead(); - delete cur_packet->packet; + if(!cur_packet->bKad)//delete kad packet in packet resend method + delete cur_packet->packet; delete cur_packet; } } Only in srchybrid_withpatch/: ClientUDPSocket.cpp.bak diff -urB srchybrid/kademlia/kademlia/Kademlia.cpp srchybrid_withpatch/kademlia/kademlia/Kademlia.cpp --- srchybrid/kademlia/kademlia/Kademlia.cpp 2010-03-10 18:57:44.000000000 +0800 +++ srchybrid_withpatch/kademlia/kademlia/Kademlia.cpp 2011-08-31 10:48:18.000000000 +0800 @@ -315,8 +315,10 @@ delete pContact; } - if (GetUDPListener() != NULL) + if (GetUDPListener() != NULL){ GetUDPListener()->ExpireClientSearch(); // function does only one compare in most cases, so no real need for a timer + GetUDPListener()->ResendPackets();//resend no reply packets here + } } void CKademlia::AddEvent(CRoutingZone *pZone) Only in srchybrid_withpatch/kademlia/kademlia: Kademlia.cpp.bak diff -urB srchybrid/kademlia/kademlia/Search.cpp srchybrid_withpatch/kademlia/kademlia/Search.cpp --- srchybrid/kademlia/kademlia/Search.cpp 2010-04-05 12:39:52.000000000 +0800 +++ srchybrid_withpatch/kademlia/kademlia/Search.cpp 2011-08-31 10:48:20.000000000 +0800 @@ -361,15 +361,28 @@ //Find contact that is responding. CUInt128 uFromDistance((ULONG)0); CContact* pFromContact = NULL; + CUInt128 uWeakFromDistance((ULONG)0); + CContact* pWeakFromContact = NULL; for (ContactMap::const_iterator itContactMap = m_mapTried.begin(); itContactMap != m_mapTried.end(); ++itContactMap) { CContact* pTmpContact = itContactMap->second; - if ((pTmpContact->GetIPAddress() == uFromIP) && (pTmpContact->GetUDPPort() == uFromPort)) + if ((pTmpContact->GetIPAddress() == uFromIP) && (pTmpContact->GetUDPPort() == uFromPort))//first find ip and port { uFromDistance = itContactMap->first; pFromContact = pTmpContact; break; } + //if there are more than one contact having the same ip in m_mapTried, we chose the farthest one. + if(pTmpContact->GetIPAddress() == uFromIP && m_mapResponded.count(itContactMap->first) == 0)//if not find, find only ip + { + uWeakFromDistance = itContactMap->first; + pWeakFromContact = pTmpContact; + } + } + if(pFromContact == NULL) + { + uFromDistance = uWeakFromDistance; + pFromContact = pWeakFromContact; } // Make sure the node is not sending more results than we requested, which is not only a protocol vialoation @@ -400,7 +413,8 @@ // Add contacts to the History for GUI purposes for (ContactList::iterator itContactList = plistResults->begin(); itContactList != plistResults->end(); ++itContactList) { - CUInt128 uDistance(((CContact*)*itContactList)->GetClientID().Xor(m_uTarget)); + //CUInt128 uDistance(((CContact*)*itContactList)->GetClientID().Xor(m_uTarget));//there modfy (*itContactList)'s clientID by mistake + CUInt128 uDistance(CUInt128(((CContact*)*itContactList)->GetClientID()).Xor(m_uTarget)); m_pLookupHistory->ContactReceived(*itContactList, pFromContact, uDistance, uDistance < uFromDistance, true); } theApp.emuledlg->kademliawnd->UpdateSearchGraph(m_pLookupHistory); Only in srchybrid_withpatch/kademlia/kademlia: Search.cpp.bak diff -urB srchybrid/kademlia/net/KademliaUDPListener.cpp srchybrid_withpatch/kademlia/net/KademliaUDPListener.cpp --- srchybrid/kademlia/net/KademliaUDPListener.cpp 2010-04-05 09:47:30.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/KademliaUDPListener.cpp 2011-08-31 10:48:18.000000000 +0800 @@ -42,8 +42,8 @@ #include "../routing/RoutingZone.h" #include "../io/ByteIO.h" #include "../../emule.h" -#include "../../ClientUDPSocket.h" -#include "../../Packets.h" +//#include "../../ClientUDPSocket.h" +//#include "../../Packets.h" #include "../../emuledlg.h" #include "../../KadContactListCtrl.h" #include "../../kademliawnd.h" @@ -1172,6 +1172,11 @@ strError.Format(_T("***NOTE: Received wrong size (%u) packet in %hs"), uLenPacket, __FUNCTION__); throw strError; } + else if (!IsOnOutTrackList(uIP, KADEMLIA_SEARCH_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } // What search does this relate to CByteIO byteIO(pbyPacketData, uLenPacket); @@ -1212,6 +1217,12 @@ // Used in Kad2.0 only void CKademliaUDPListener::Process_KADEMLIA2_SEARCH_RES (const byte *pbyPacketData, uint32 uLenPacket, CKadUDPKey /*senderUDPKey*/, uint32 uIP, uint16 uUDPPort) { + //there may be more than one response packet + if (!IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_KEY_REQ, true) && !IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_SOURCE_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } CByteIO byteIO(pbyPacketData, uLenPacket); // Who sent this packet. @@ -1592,6 +1603,11 @@ // Used only by Kad2.0 void CKademliaUDPListener::Process_KADEMLIA2_SEARCH_NOTES_REQ (const byte *pbyPacketData, uint32 uLenPacket, uint32 uIP, uint16 uUDPPort, CKadUDPKey senderUDPKey) { + if (!IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_NOTES_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } CSafeMemFile fileIO(pbyPacketData, uLenPacket); CUInt128 uTarget; fileIO.ReadUInt128(&uTarget); @@ -2046,7 +2062,6 @@ ASSERT(0); return; } - AddTrackedOutPacket(uDestinationHost, pbyData[1]); Packet* pPacket = new Packet(OP_KADEMLIAHEADER); pPacket->opcode = pbyData[1]; pPacket->pBuffer = new char[uLenData+8]; @@ -2055,6 +2070,10 @@ if( uLenData > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, pbyData[1]); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); @@ -2062,7 +2081,6 @@ void CKademliaUDPListener::SendPacket(const byte *pbyData, uint32 uLenData, byte byOpcode, uint32 uDestinationHost, uint16 uDestinationPort, CKadUDPKey targetUDPKey, const CUInt128* uCryptTargetID) { - AddTrackedOutPacket(uDestinationHost, byOpcode); Packet* pPacket = new Packet(OP_KADEMLIAHEADER); pPacket->opcode = byOpcode; pPacket->pBuffer = new char[uLenData]; @@ -2071,6 +2089,10 @@ if( uLenData > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, byOpcode); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); @@ -2078,12 +2100,15 @@ void CKademliaUDPListener::SendPacket(CSafeMemFile *pbyData, byte byOpcode, uint32 uDestinationHost, uint16 uDestinationPort, CKadUDPKey targetUDPKey, const CUInt128* uCryptTargetID) { - AddTrackedOutPacket(uDestinationHost, byOpcode); Packet* pPacket = new Packet(pbyData, OP_KADEMLIAHEADER); pPacket->opcode = byOpcode; if( pPacket->size > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, byOpcode); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); Only in srchybrid_withpatch/kademlia/net: KademliaUDPListener.cpp.bak diff -urB srchybrid/kademlia/net/PacketTracking.cpp srchybrid_withpatch/kademlia/net/PacketTracking.cpp --- srchybrid/kademlia/net/PacketTracking.cpp 2010-03-09 23:33:04.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/PacketTracking.cpp 2011-08-31 10:48:20.000000000 +0800 @@ -37,24 +37,61 @@ } CPacketTracking::~CPacketTracking(){ + for (POSITION pos = listTrackedRequests.GetTailPosition(); pos != NULL; listTrackedRequests.GetPrev(pos)){ + delete listTrackedRequests.GetAt(pos).pPacket; + delete[] listTrackedRequests.GetAt(pos).pachTargetClientHashORKadID; + } m_mapTrackPacketsIn.RemoveAll(); while (!m_liTrackPacketsIn.IsEmpty()) delete m_liTrackPacketsIn.RemoveHead(); } -void CPacketTracking::AddTrackedOutPacket(uint32 dwIP, uint8 byOpcode){ +bool CPacketTracking::AddTrackedOutPacket(uint32 dwIP, uint16 nPort, const uchar* pachTargetClientHashORKadID, uint32 nReceiverVerifyKey, Packet* packet, uint8 byOpcode){ // this tracklist tacks _outgoing_ request packets, to make sure incoming answer packets were requested // only track packets which we actually check for later - if (!IsTrackedOutListRequestPacket(byOpcode)) - return; - TrackPackets_Struct sTrack = {dwIP, ::GetTickCount(), byOpcode}; + if (!IsTrackedOutListRequestPacket(byOpcode)){ + delete packet; + return false; + } + uchar* pachTargetClientHashORKadIDTmp = NULL; + if( pachTargetClientHashORKadID != NULL){ + pachTargetClientHashORKadIDTmp = new uchar[32]; + memcpy(pachTargetClientHashORKadIDTmp, pachTargetClientHashORKadID, 32); + } + TrackPackets_Struct sTrack = {packet, dwIP, nPort, pachTargetClientHashORKadIDTmp, nReceiverVerifyKey, ::GetTickCount(), byOpcode, PACKET_RESEND_TIMES, false}; listTrackedRequests.AddHead(sTrack); - while (!listTrackedRequests.IsEmpty()){ - if (::GetTickCount() - listTrackedRequests.GetTail().dwInserted > SEC2MS(180)) - listTrackedRequests.RemoveTail(); - else + //while (!listTrackedRequests.IsEmpty()){ + // if (::GetTickCount() - listTrackedRequests.GetTail().dwInserted > SEC2MS(180)) + // listTrackedRequests.RemoveTail(); + // else + // break; + //} + return true; +} + +void CPacketTracking::ResendPackets(){ + CList<TrackPackets_Struct> listTrackedRequestsTmp; + for (POSITION pos = listTrackedRequests.GetTailPosition(); pos != NULL; listTrackedRequests.GetPrev(pos)){ + TrackPackets_Struct& sTrack = listTrackedRequests.GetAt(pos); + if (::GetTickCount() - sTrack.dwInserted > SEC2MS(180)){//clean expired packet after 180 seconds + delete sTrack.pPacket; + delete[] sTrack.pachTargetClientHashORKadID; + listTrackedRequests.RemoveAt(pos); + }else if(::GetTickCount() - sTrack.dwInserted >= PACKET_RESEND_INTERVAL){//resend packet + if(sTrack.nResendTimes > 0 && !sTrack.bRecv){ + theApp.clientudp->SendPacket(sTrack.pPacket, ntohl(sTrack.dwIP), sTrack.nPort, true + , sTrack.pachTargetClientHashORKadID, true , sTrack.nReceiverVerifyKey); + sTrack.nResendTimes--; + sTrack.dwInserted = ::GetTickCount(); + listTrackedRequestsTmp.AddHead(sTrack); + listTrackedRequests.RemoveAt(pos); + } + }else//these packets in front of this are not expired break; } + + listTrackedRequests.AddHead(&listTrackedRequestsTmp); + listTrackedRequestsTmp.RemoveAll(); } bool CPacketTracking::IsTrackedOutListRequestPacket(uint8 byOpcode) const @@ -88,8 +125,14 @@ #endif for (POSITION pos = listTrackedRequests.GetHeadPosition(); pos != NULL; listTrackedRequests.GetNext(pos)){ if (listTrackedRequests.GetAt(pos).dwIP == dwIP && listTrackedRequests.GetAt(pos).byOpcode == byOpcode && ::GetTickCount() - listTrackedRequests.GetAt(pos).dwInserted < SEC2MS(180)){ - if (!bDontRemove) + if (!bDontRemove){ + delete listTrackedRequests.GetAt(pos).pPacket; + delete[] listTrackedRequests.GetAt(pos).pachTargetClientHashORKadID; listTrackedRequests.RemoveAt(pos); + } + else{ + listTrackedRequests.GetAt(pos).bRecv = true; + } return true; } } Only in srchybrid_withpatch/kademlia/net: PacketTracking.cpp.bak diff -urB srchybrid/kademlia/net/PacketTracking.h srchybrid_withpatch/kademlia/net/PacketTracking.h --- srchybrid/kademlia/net/PacketTracking.h 2008-08-30 12:12:52.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/PacketTracking.h 2011-08-31 10:48:20.000000000 +0800 @@ -17,13 +17,23 @@ #pragma once #include "../utils/UInt128.h" +#include "../../ClientUDPSocket.h" +#include "../../Packets.h" namespace Kademlia { +#define PACKET_RESEND_TIMES 3 +#define PACKET_RESEND_INTERVAL 2000 //ms struct TrackPackets_Struct{ + Packet* pPacket; uint32 dwIP; + uint16 nPort; + uchar* pachTargetClientHashORKadID; + uint32 nReceiverVerifyKey; uint32 dwInserted; uint8 byOpcode; + uint8 nResendTimes; + bool bRecv;//process multi-response packets }; struct TrackChallenge_Struct{ @@ -53,10 +63,12 @@ { public: CPacketTracking(); + void ResendPackets(); virtual ~CPacketTracking(); protected: - void AddTrackedOutPacket(uint32 dwIP, uint8 byOpcode); + //if return false, the opcode is wrong, so dont need send this packet + bool AddTrackedOutPacket(uint32 dwIP, uint16 nPort, const uchar* pachTargetClientHashORKadID, uint32 nReceiverVerifyKey, Packet* packet, uint8 byOpcode); bool IsOnOutTrackList(uint32 dwIP, uint8 byOpcode, bool bDontRemove = false); bool InTrackListIsAllowedPacket(uint32 uIP, uint8 byOpcode, bool bValidReceiverkey); void InTrackListCleanup(); Only in srchybrid_withpatch/kademlia/net: PacketTracking.h.bak
#20
Posted 03 September 2011 - 07:20 AM
sonoro, on 02 September 2011 - 08:04 PM, said:
diff -urB srchybrid/ClientUDPSocket.cpp srchybrid_withpatch/ClientUDPSocket.cpp --- srchybrid/ClientUDPSocket.cpp 2009-02-22 19:36:56.000000000 +0800 +++ srchybrid_withpatch/ClientUDPSocket.cpp 2011-08-31 10:48:16.000000000 +0800 @@ -62,7 +62,8 @@ POSITION pos = controlpacket_queue.GetHeadPosition(); while (pos){ UDPPack* p = controlpacket_queue.GetNext(pos); - delete p->packet; + if(!p->bKad)//delete kad packet in packet resend method + delete p->packet; delete p; } } @@ -523,7 +524,8 @@ sentBytes += nLen; // ZZ:UploadBandWithThrottler (UDP) controlpacket_queue.RemoveHead(); - delete cur_packet->packet; + if(!cur_packet->bKad)//delete kad packet in packet resend method + delete cur_packet->packet; delete cur_packet; } delete[] sendbuffer; @@ -531,7 +533,8 @@ else { controlpacket_queue.RemoveHead(); - delete cur_packet->packet; + if(!cur_packet->bKad)//delete kad packet in packet resend method + delete cur_packet->packet; delete cur_packet; } } Only in srchybrid_withpatch/: ClientUDPSocket.cpp.bak diff -urB srchybrid/kademlia/kademlia/Kademlia.cpp srchybrid_withpatch/kademlia/kademlia/Kademlia.cpp --- srchybrid/kademlia/kademlia/Kademlia.cpp 2010-03-10 18:57:44.000000000 +0800 +++ srchybrid_withpatch/kademlia/kademlia/Kademlia.cpp 2011-08-31 10:48:18.000000000 +0800 @@ -315,8 +315,10 @@ delete pContact; } - if (GetUDPListener() != NULL) + if (GetUDPListener() != NULL){ GetUDPListener()->ExpireClientSearch(); // function does only one compare in most cases, so no real need for a timer + GetUDPListener()->ResendPackets();//resend no reply packets here + } } void CKademlia::AddEvent(CRoutingZone *pZone) Only in srchybrid_withpatch/kademlia/kademlia: Kademlia.cpp.bak diff -urB srchybrid/kademlia/kademlia/Search.cpp srchybrid_withpatch/kademlia/kademlia/Search.cpp --- srchybrid/kademlia/kademlia/Search.cpp 2010-04-05 12:39:52.000000000 +0800 +++ srchybrid_withpatch/kademlia/kademlia/Search.cpp 2011-08-31 10:48:20.000000000 +0800 @@ -361,15 +361,28 @@ //Find contact that is responding. CUInt128 uFromDistance((ULONG)0); CContact* pFromContact = NULL; + CUInt128 uWeakFromDistance((ULONG)0); + CContact* pWeakFromContact = NULL; for (ContactMap::const_iterator itContactMap = m_mapTried.begin(); itContactMap != m_mapTried.end(); ++itContactMap) { CContact* pTmpContact = itContactMap->second; - if ((pTmpContact->GetIPAddress() == uFromIP) && (pTmpContact->GetUDPPort() == uFromPort)) + if ((pTmpContact->GetIPAddress() == uFromIP) && (pTmpContact->GetUDPPort() == uFromPort))//first find ip and port { uFromDistance = itContactMap->first; pFromContact = pTmpContact; break; } + //if there are more than one contact having the same ip in m_mapTried, we chose the farthest one. + if(pTmpContact->GetIPAddress() == uFromIP && m_mapResponded.count(itContactMap->first) == 0)//if not find, find only ip + { + uWeakFromDistance = itContactMap->first; + pWeakFromContact = pTmpContact; + } + } + if(pFromContact == NULL) + { + uFromDistance = uWeakFromDistance; + pFromContact = pWeakFromContact; } // Make sure the node is not sending more results than we requested, which is not only a protocol vialoation @@ -400,7 +413,8 @@ // Add contacts to the History for GUI purposes for (ContactList::iterator itContactList = plistResults->begin(); itContactList != plistResults->end(); ++itContactList) { - CUInt128 uDistance(((CContact*)*itContactList)->GetClientID().Xor(m_uTarget)); + //CUInt128 uDistance(((CContact*)*itContactList)->GetClientID().Xor(m_uTarget));//there modfy (*itContactList)'s clientID by mistake + CUInt128 uDistance(CUInt128(((CContact*)*itContactList)->GetClientID()).Xor(m_uTarget)); m_pLookupHistory->ContactReceived(*itContactList, pFromContact, uDistance, uDistance < uFromDistance, true); } theApp.emuledlg->kademliawnd->UpdateSearchGraph(m_pLookupHistory); Only in srchybrid_withpatch/kademlia/kademlia: Search.cpp.bak diff -urB srchybrid/kademlia/net/KademliaUDPListener.cpp srchybrid_withpatch/kademlia/net/KademliaUDPListener.cpp --- srchybrid/kademlia/net/KademliaUDPListener.cpp 2010-04-05 09:47:30.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/KademliaUDPListener.cpp 2011-08-31 10:48:18.000000000 +0800 @@ -42,8 +42,8 @@ #include "../routing/RoutingZone.h" #include "../io/ByteIO.h" #include "../../emule.h" -#include "../../ClientUDPSocket.h" -#include "../../Packets.h" +//#include "../../ClientUDPSocket.h" +//#include "../../Packets.h" #include "../../emuledlg.h" #include "../../KadContactListCtrl.h" #include "../../kademliawnd.h" @@ -1172,6 +1172,11 @@ strError.Format(_T("***NOTE: Received wrong size (%u) packet in %hs"), uLenPacket, __FUNCTION__); throw strError; } + else if (!IsOnOutTrackList(uIP, KADEMLIA_SEARCH_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } // What search does this relate to CByteIO byteIO(pbyPacketData, uLenPacket); @@ -1212,6 +1217,12 @@ // Used in Kad2.0 only void CKademliaUDPListener::Process_KADEMLIA2_SEARCH_RES (const byte *pbyPacketData, uint32 uLenPacket, CKadUDPKey /*senderUDPKey*/, uint32 uIP, uint16 uUDPPort) { + //there may be more than one response packet + if (!IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_KEY_REQ, true) && !IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_SOURCE_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } CByteIO byteIO(pbyPacketData, uLenPacket); // Who sent this packet. @@ -1592,6 +1603,11 @@ // Used only by Kad2.0 void CKademliaUDPListener::Process_KADEMLIA2_SEARCH_NOTES_REQ (const byte *pbyPacketData, uint32 uLenPacket, uint32 uIP, uint16 uUDPPort, CKadUDPKey senderUDPKey) { + if (!IsOnOutTrackList(uIP, KADEMLIA2_SEARCH_NOTES_REQ, true)){ + CString strError; + strError.Format(_T("***NOTE: Received unrequested response packet, size (%u) in %hs"), uLenPacket, __FUNCTION__); + throw strError; + } CSafeMemFile fileIO(pbyPacketData, uLenPacket); CUInt128 uTarget; fileIO.ReadUInt128(&uTarget); @@ -2046,7 +2062,6 @@ ASSERT(0); return; } - AddTrackedOutPacket(uDestinationHost, pbyData[1]); Packet* pPacket = new Packet(OP_KADEMLIAHEADER); pPacket->opcode = pbyData[1]; pPacket->pBuffer = new char[uLenData+8]; @@ -2055,6 +2070,10 @@ if( uLenData > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, pbyData[1]); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); @@ -2062,7 +2081,6 @@ void CKademliaUDPListener::SendPacket(const byte *pbyData, uint32 uLenData, byte byOpcode, uint32 uDestinationHost, uint16 uDestinationPort, CKadUDPKey targetUDPKey, const CUInt128* uCryptTargetID) { - AddTrackedOutPacket(uDestinationHost, byOpcode); Packet* pPacket = new Packet(OP_KADEMLIAHEADER); pPacket->opcode = byOpcode; pPacket->pBuffer = new char[uLenData]; @@ -2071,6 +2089,10 @@ if( uLenData > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, byOpcode); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); @@ -2078,12 +2100,15 @@ void CKademliaUDPListener::SendPacket(CSafeMemFile *pbyData, byte byOpcode, uint32 uDestinationHost, uint16 uDestinationPort, CKadUDPKey targetUDPKey, const CUInt128* uCryptTargetID) { - AddTrackedOutPacket(uDestinationHost, byOpcode); Packet* pPacket = new Packet(pbyData, OP_KADEMLIAHEADER); pPacket->opcode = byOpcode; if( pPacket->size > 200 ) pPacket->PackPacket(); theStats.AddUpDataOverheadKad(pPacket->size); + bool bAdd = true; + bAdd = AddTrackedOutPacket(uDestinationHost, uDestinationPort, ((const uchar*) (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL), targetUDPKey.GetKeyValue(theApp.GetPublicIP(false)), pPacket, byOpcode); + if(!bAdd) + return; theApp.clientudp->SendPacket(pPacket, ntohl(uDestinationHost), uDestinationPort, true , (uCryptTargetID != NULL) ? uCryptTargetID->GetData() : NULL , true , targetUDPKey.GetKeyValue(theApp.GetPublicIP(false))); Only in srchybrid_withpatch/kademlia/net: KademliaUDPListener.cpp.bak diff -urB srchybrid/kademlia/net/PacketTracking.cpp srchybrid_withpatch/kademlia/net/PacketTracking.cpp --- srchybrid/kademlia/net/PacketTracking.cpp 2010-03-09 23:33:04.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/PacketTracking.cpp 2011-08-31 10:48:20.000000000 +0800 @@ -37,24 +37,61 @@ } CPacketTracking::~CPacketTracking(){ + for (POSITION pos = listTrackedRequests.GetTailPosition(); pos != NULL; listTrackedRequests.GetPrev(pos)){ + delete listTrackedRequests.GetAt(pos).pPacket; + delete[] listTrackedRequests.GetAt(pos).pachTargetClientHashORKadID; + } m_mapTrackPacketsIn.RemoveAll(); while (!m_liTrackPacketsIn.IsEmpty()) delete m_liTrackPacketsIn.RemoveHead(); } -void CPacketTracking::AddTrackedOutPacket(uint32 dwIP, uint8 byOpcode){ +bool CPacketTracking::AddTrackedOutPacket(uint32 dwIP, uint16 nPort, const uchar* pachTargetClientHashORKadID, uint32 nReceiverVerifyKey, Packet* packet, uint8 byOpcode){ // this tracklist tacks _outgoing_ request packets, to make sure incoming answer packets were requested // only track packets which we actually check for later - if (!IsTrackedOutListRequestPacket(byOpcode)) - return; - TrackPackets_Struct sTrack = {dwIP, ::GetTickCount(), byOpcode}; + if (!IsTrackedOutListRequestPacket(byOpcode)){ + delete packet; + return false; + } + uchar* pachTargetClientHashORKadIDTmp = NULL; + if( pachTargetClientHashORKadID != NULL){ + pachTargetClientHashORKadIDTmp = new uchar[32]; + memcpy(pachTargetClientHashORKadIDTmp, pachTargetClientHashORKadID, 32); + } + TrackPackets_Struct sTrack = {packet, dwIP, nPort, pachTargetClientHashORKadIDTmp, nReceiverVerifyKey, ::GetTickCount(), byOpcode, PACKET_RESEND_TIMES, false}; listTrackedRequests.AddHead(sTrack); - while (!listTrackedRequests.IsEmpty()){ - if (::GetTickCount() - listTrackedRequests.GetTail().dwInserted > SEC2MS(180)) - listTrackedRequests.RemoveTail(); - else + //while (!listTrackedRequests.IsEmpty()){ + // if (::GetTickCount() - listTrackedRequests.GetTail().dwInserted > SEC2MS(180)) + // listTrackedRequests.RemoveTail(); + // else + // break; + //} + return true; +} + +void CPacketTracking::ResendPackets(){ + CList<TrackPackets_Struct> listTrackedRequestsTmp; + for (POSITION pos = listTrackedRequests.GetTailPosition(); pos != NULL; listTrackedRequests.GetPrev(pos)){ + TrackPackets_Struct& sTrack = listTrackedRequests.GetAt(pos); + if (::GetTickCount() - sTrack.dwInserted > SEC2MS(180)){//clean expired packet after 180 seconds + delete sTrack.pPacket; + delete[] sTrack.pachTargetClientHashORKadID; + listTrackedRequests.RemoveAt(pos); + }else if(::GetTickCount() - sTrack.dwInserted >= PACKET_RESEND_INTERVAL){//resend packet + if(sTrack.nResendTimes > 0 && !sTrack.bRecv){ + theApp.clientudp->SendPacket(sTrack.pPacket, ntohl(sTrack.dwIP), sTrack.nPort, true + , sTrack.pachTargetClientHashORKadID, true , sTrack.nReceiverVerifyKey); + sTrack.nResendTimes--; + sTrack.dwInserted = ::GetTickCount(); + listTrackedRequestsTmp.AddHead(sTrack); + listTrackedRequests.RemoveAt(pos); + } + }else//these packets in front of this are not expired break; } + + listTrackedRequests.AddHead(&listTrackedRequestsTmp); + listTrackedRequestsTmp.RemoveAll(); } bool CPacketTracking::IsTrackedOutListRequestPacket(uint8 byOpcode) const @@ -88,8 +125,14 @@ #endif for (POSITION pos = listTrackedRequests.GetHeadPosition(); pos != NULL; listTrackedRequests.GetNext(pos)){ if (listTrackedRequests.GetAt(pos).dwIP == dwIP && listTrackedRequests.GetAt(pos).byOpcode == byOpcode && ::GetTickCount() - listTrackedRequests.GetAt(pos).dwInserted < SEC2MS(180)){ - if (!bDontRemove) + if (!bDontRemove){ + delete listTrackedRequests.GetAt(pos).pPacket; + delete[] listTrackedRequests.GetAt(pos).pachTargetClientHashORKadID; listTrackedRequests.RemoveAt(pos); + } + else{ + listTrackedRequests.GetAt(pos).bRecv = true; + } return true; } } Only in srchybrid_withpatch/kademlia/net: PacketTracking.cpp.bak diff -urB srchybrid/kademlia/net/PacketTracking.h srchybrid_withpatch/kademlia/net/PacketTracking.h --- srchybrid/kademlia/net/PacketTracking.h 2008-08-30 12:12:52.000000000 +0800 +++ srchybrid_withpatch/kademlia/net/PacketTracking.h 2011-08-31 10:48:20.000000000 +0800 @@ -17,13 +17,23 @@ #pragma once #include "../utils/UInt128.h" +#include "../../ClientUDPSocket.h" +#include "../../Packets.h" namespace Kademlia { +#define PACKET_RESEND_TIMES 3 +#define PACKET_RESEND_INTERVAL 2000 //ms struct TrackPackets_Struct{ + Packet* pPacket; uint32 dwIP; + uint16 nPort; + uchar* pachTargetClientHashORKadID; + uint32 nReceiverVerifyKey; uint32 dwInserted; uint8 byOpcode; + uint8 nResendTimes; + bool bRecv;//process multi-response packets }; struct TrackChallenge_Struct{ @@ -53,10 +63,12 @@ { public: CPacketTracking(); + void ResendPackets(); virtual ~CPacketTracking(); protected: - void AddTrackedOutPacket(uint32 dwIP, uint8 byOpcode); + //if return false, the opcode is wrong, so dont need send this packet + bool AddTrackedOutPacket(uint32 dwIP, uint16 nPort, const uchar* pachTargetClientHashORKadID, uint32 nReceiverVerifyKey, Packet* packet, uint8 byOpcode); bool IsOnOutTrackList(uint32 dwIP, uint8 byOpcode, bool bDontRemove = false); bool InTrackListIsAllowedPacket(uint32 uIP, uint8 byOpcode, bool bValidReceiverkey); void InTrackListCleanup(); Only in srchybrid_withpatch/kademlia/net: PacketTracking.h.bak