The problem is well known, at the end of a part, blocks that have been requested of a slower client can become locked. A faster client may have them and be ready to transfer them, but can't be asked because they have already be 'allocated to'/'requested from' the slower source.
Some points of reference:
Slow Download Ending Solution
Faster 'endgame', Increase the speed at end of downloads (Dazzle)
Dynamic Block Requests, A way to significantly speed up eMule (Netfinity)
To partly deal with this issue in 46c the official eMule has:
void CUpDownClient::SendBlockRequests() said:
int blockCount = 3;
if(IsEmuleClient() && m_byCompatibleClient==0 && reqfile->GetFileSize()-reqfile->GetCompletedSize() <= PARTSIZE*4) {
// if there's less than two chunks left, request fewer blocks for
// slow downloads, so they don't lock blocks from faster clients.
// Only trust eMule clients to be able to handle less blocks than three
if(GetDownloadDatarate() < 600) {
blockCount = 1;
} else if(GetDownloadDatarate() < 1200) {
blockCount = 2;
}
}
Of this solution with reference to the alternatives ZZ wrote:
zz, on Oct 22 2005, 10:03 AM, said:
The small change I did simply decides if 1, 2 or 3 blocks will be requested.
/zz
So, as I was already trying to tweak the Dazzle faster endgame code I thought I'd also have a look at this piece of code. My solution does not kick out connections,(though using it with a version of Dazzle's code is my final aim.), and does not request blocks smaller than 180Kb.
Unlike the official code my code works at the end of every part where there are mulitple sources and works on relative speed, because the problem is always there, the very slow source scenario is just the more noticeable manifestation.
In DownloadClient.cpp CUpDownClient::SendBlockRequests()
void CUpDownClient::SendBlockRequests() said:
int blockCount = 3;
if(IsEmuleClient() && m_byCompatibleClient==0 && reqfile->ReduceAllocatedBlocks(this ,this->m_lastPartAsked)){
//near end of part with multiply sources, share the blocks out, slower sources have blockcount reduced first.
blockCount = 1;
}
In Partfile.h
Partfile.h said:
bool ReduceAllocatedBlocks(CUpDownClient* calling_source,uint16 m_PartAsked);
In Partfile.cpp
bool CPartFile::ReduceAllocatedBlocks(CUpDownClient* calling_source, on uint16 m_PartAsked) *added*, said:
for part' would complete the part minus those three blocks.This helps prevent a slow source locking blocks at the end of a part
causing sources which may not have any other parts we need to stop transfering. It also helps maintain a higher cumulative download
speed at the end of file and sometimes,(where a source has no other parts we need); at the end of part.*/
bool CPartFile::ReduceAllocatedBlocks(CUpDownClient* calling_source,uint16 m_PartAsked)
{
// no 'part last asked for'
if(m_PartAsked == 0xffff) return false;
//Quick Check, if there is only one source overall return no 'part last asked for'
if(m_downloadingSourceList.GetCount()<2)
{//if this source is the only source don't let it lock blocks if its too slow
if(calling_source->GetDownloadDatarate()<784) return true;
else return false;
}
uint16 sourcecount = 0;
uint32 remainingdata = 0;
uint32 otherstransferrate = 1;// a value of 1 to prevent 'divide by zero' (saves 1 + otherstransferrate later)
const uint32 threeblocks = EMBLOCKSIZE * 3;
/*Calculate total transfering sources for this part and overall transfer speed.*/
for(POSITION pos = m_downloadingSourceList.GetHeadPosition(); pos != NULL;)
{
//get next downloading source
CUpDownClient* cur_src = srclist.GetNext(pos);
//is this source sending data for the same part as the calling source but is not the calling source
if(cur_src->m_lastPartAsked==m_PartAsked && cur_src!=calling_source)
{
sourcecount++;//increment sources for this part
otherstransferrate += cur_src->GetDownloadDatarate();//add this clients transfer rate to the total
}
}
//if calling source is the only currently transfering source for this part use fixed reference
if(sourcecount < 1)
{//if this source is the only source don't let it lock blocks if its too slow
if(calling_source->GetDownloadDatarate()<784) return true;
else return false;
}
/* Calculate how much of the part is left in bytes (based on code from GetNextRequestedBlock() )*/
// Offsets of 'this' chunk
const uint32 uStart = m_PartAsked * PARTSIZE;
const uint32 uEnd = (GetFileSize() - 1 < (uStart + PARTSIZE - 1)) ?
GetFileSize() - 1 : (uStart + PARTSIZE - 1);
ASSERT( uStart <= uEnd );
if(uStart >= uEnd) return false;
//gets bytes remaining
for(POSITION pos = gaplist.GetHeadPosition(); pos != NULL; )
{
const Gap_Struct* cur_gap = gaplist.GetNext(pos);
//Check if Gap is into the limit
if(cur_gap->start < uStart)
{
if(cur_gap->end > uStart && cur_gap->end < uEnd) remainingdata += cur_gap->end - uStart + 1;
else if(cur_gap->end >= uEnd) return false;
} else
if(cur_gap->start <= uEnd)
{
if(cur_gap->end < uEnd) remainingdata += cur_gap->end - cur_gap->start + 1;
else remainingdata += uEnd - cur_gap->start + 1;
}
}
if(threeblocks > remainingdata) return true;//keep up combined download speed for as long as possible
else remainingdata -= threeblocks; //remainingdata to equal the remaining data after this client gets allocated 3 blocks
//if(3 * EMBLOCKSIZE / 'calling source speed' >= 'remaining incomplete part size - 3 blocks'/'total other sources transfer rate') return true
if((uint32)(threeblocks/(1 + calling_source->GetDownloadDatarate())) >= (uint32)(remainingdata/otherstransferrate)) return true;
return false;
}
I know it has been pointed out by people who know a great deal more than I do that Netfinity's solution is the best approach.
However I hope this code may present a simpler idea whilst still bringing about an improvement in performance. Thanks for reading!
edit: 19-11-2005 Bug fix in ReduceAllocatedBlocks() + add fix speed comparisons for single sources.
edit 25-11-2005 changed last if condition to >=
This post has been edited by BlueSonicBoy: 25 November 2005 - 05:31 PM