[Kutfiusz]

Hi
everybody must have noticed that emule can be very slow at the end of each download, especially at 99%.
Having experience only as a regular user, not seeing the code, I think that's only when I'm downloading a particular file from multiple clients. I guess emule assigns every client a frame (part of a file) and every client sends his part of file to me. Problem appears when there is no frames left to assign to client. Client is disconected. Because fastest clients ask for new frames more often then slow ones there is very high probability that the fastest clients are the first disconected ones and the last frame is downloaded from the slowest client. For example (very common situation) I dowload file archive.zip from 3 clients. One uploads to me with 80Kb/s, another 5Kb/s and the last 100B/s. The file dowloads fast, but the last frame is downloaded with 100B/s (from slowest client) and sometimes it takes 50% of total file download time.
I see 2 solutions. One is smaller frame size, but this will lead to more control data needed and lower transfers.
Another solution is more subtle. When emule cannot assign a frame to client (there are no free frames) it checks which client is the slowest one, "kicks" kim, and gives the last frame (part) to the actually fastest free client. In other words, Emule kicks the slowest client and lets the faster one to finish frame download.
Tell me what do you think?
greetings

[PluG]

Take a look at dazzle's -> Faster 'endgame'

Also netfinity's 'dynamic block requests' aims to resolve this senario too. But so far the devs haven't been inclined to implement such a thing in official.

[Kutfiusz]

thanks for replying, sorry but we didn't find this info.
But the question remains, why do devs won't implement it if it is such an easy problem to solve and gives a boost to the software.

[PluG]

Kutfiusz, on Oct 20 2005, 12:55 PM, said:

thanks for replying, sorry but we didn't find this info.

You didn't need to find it, the underlined^ Faster 'endgame' above is a link, just click it.

Quote

But the question remains, why do devs won't implement it if it is such an easy problem to solve and gives a boost to the software.

It's an old problem which is frequently mentioned, I think zz has responed to it a couple of times, but I guess the official devs don't see it as a big issue. You get your file eventually, whether it's in 10 minutes or 10 hours.

[Kutfiusz]

PluG, on Oct 20 2005, 02:09 PM, said:

Kutfiusz, on Oct 20 2005, 12:55 PM, said:

thanks for replying, sorry but we didn't find this info.

You didn't need to find it, the underlined^ Faster 'endgame' above is a link, just click it.

well, I meant i haven't found it before I asked my question..
thx to you too

[Andu]

PluG, on Oct 20 2005, 03:09 PM, said:

Kutfiusz, on Oct 20 2005, 12:55 PM, said:

thanks for replying, sorry but we didn't find this info.

You didn't need to find it, the underlined^ Faster 'endgame' above is a link, just click it.

Quote

But the question remains, why do devs won't implement it if it is such an easy problem to solve and gives a boost to the software.

It's an old problem which is frequently mentioned, I think zz has responed to it a couple of times, but I guess the official devs don't see it as a big issue. You get your file eventually, whether it's in 10 minutes or 10 hours.

Erm zz has already implemented his own version of the faster endgame in vanilla eMule iirc. Should have happened in 0.46c I think. I'm certain it was in one of the 0.46x clients.

[PluG]

Andu, on Oct 20 2005, 05:28 PM, said:

Erm zz has already implemented his own version of the faster endgame in vanilla eMule iirc. Should have happened in 0.46c I think. I'm certain it was in one of the 0.46x clients.

My mistake, your right, just checked the changelog for 46c.

Quote

----------------------
- June, 28. 2005    -
----------------------
zz: When downloading from a really slow source, request less of the file at a time. This locks less of the file, so if there's a faster source available we have a better chance of downloading from it even when the file is almost completely downloaded.

[zz]

I have further improvements to it in my local build, but I doubt we'll insert it in the next standard eMule. I don't want to do risky stuff just to get the few last Kbytes faster.

It will be part of the next ZZUL release though.

/zz

This post has been edited by zz: 20 October 2005 - 08:43 PM

[MasterJunior]

isnt that dazzel mod a bad 1 though? it has some features that arnt good for the network.

[Andu]

zz, on Oct 20 2005, 10:42 PM, said:

I have further improvements to it in my local build, but I doubt we'll insert it in the next standard eMule. I don't want to do risky stuff just to get the few last Kbytes faster.

It will be part of the next ZZUL release though.

/zz

Are you implying that the next release of the official client isn't too far off or is it just my wild imagination?

[PluG]

MasterJunior, on Oct 20 2005, 09:05 PM, said:

isnt that dazzel mod a bad 1 though? it has some features that arnt good for the network.

The mod is, as far as the reports around here go, tho' I never looked at it myself. But the Faster 'endgame' code is in the snippets section as I linked too....

Tho' all that's probably obsolette anyway, with zz's version implemented and under further developement, guess we just have to wait and see what nice surprises the future brings

[MasterJunior]

idk, i tried it out and its baaaaaaaaad, if i were a admin around here id remove its name from every post

[leuk_he]

MasterJunior, on Oct 20 2005, 11:05 PM, said:

isnt that dazzel mod a bad 1 though? it has some features that arnt good for the network.

dassel is indeed a very bad mod(0-uploading possiblity) , but nothing illegal.

All the dazzle end-game does is dropping slow sources, That is not the most efficient way since you still ahve to wait if a faster source is availble.

046c did an more generic improvement to the download code. that not only improves the laast chunk, but also might be an impreovement for all chunks.

[zz]

Andu, on Oct 20 2005, 11:11 PM, said:

Are you implying that the next release of the official client isn't too far off or is it just my wild imagination?

No, I'm not implying anything. Just that I won't put that code in the standard source, so it won't be part of a release whenever it may happen.

If it works out ok in a few ZZUL releases, I might feel safe enough to advocate for it in the standard eMule release, but we'll have to see.

/zz

[SlugFiller]

When I saw that in the changelog, I though they adapted netfinity's patch(hence saving me the trouble). But from the posts I'm reading, it seems to be inferior.
If a superior patch is, indeed, available, why not use it?
(Hey, first time I get to ask that about somebody else's patch)

[zz]

I don't remember the particulars about Netfinity's patch; but we didn't want to kick out connections, and didn't want request blocks smaller than the standard 180KBytes. Maybe that patch used one of those methods? It's too long ago to remember.

The small change I did simply decides if 1, 2 or 3 blocks will be requested.

/zz

[Andu]

Well it surely requested blocks smaller than 180 kB if necessary. Not sure about the first thing.

So I guess netfinity's patch still is a bit more advanced than that in the official client.

[SlugFiller]

There were two patchs to solve this.
First, Dazzle created a patch that kicks sources.
Then, NetFinity created one that asks for smaller blocks.

To me, NetFinity's solution is better, but either works as a solution to the problem at hand.

I don't see how there can be a solution without one of the two. Even before the appearance of these two patchs, I already knew those were the only two thinkable possible solutions, and I did discuss them in theory a bit.
It was actually somewhat funny when both variations appeared shortly after one another.

Well, actually, a third solution I thought of was requesting overlapping blocks when no free ones are left, and "may the faster uploader win", though that's somewhat wasteful with bandwidth.

[Andu]

Hm netfinity's patch was implemented into his mod significantly earlier than dazzle's patch iirc.

[SlugFiller]

Hmm... Yes.
I'm going over the threads and indeed NetFinity's snippet is dated Nov 11 2004, while Dazzle's is Apr 29 2005. I just didn't spot NetFinity's patch until tHeWiZaRdOfDoS brought the thread back from the dead in May 31 2005, a month after Dazzle's patch was posted.

Here's a twist, though, NetFinity's patch has already been around for over a year, and with all the eMule versions that's come and gone, it still isn't in the official.
And there I thought Check Diskspace and SafeHash took their sweet time.