[DatHebIkWeer]

Double download at the end of the last chunk.

It sometimes happens that 2 or more processes download the same chunk. Currently they each reserve a part of the chunk for their own download till they run out of unreserved parts. Each process downloads the parts it reserved and then skips to a next chunk.
But what if there is no next chunk? There may be no chunks left available on a host. In that case the process just finishes when all reservations are downloaded and there is no unreserved part available.

But it happens that 1 process is very fast and the other is extremely slow and unstable. I thought about requesting a “block host” option that can temporarily block a host from upload to clear the way for the faster better process to finish the chunk. But that requires manual action and can only work if I am there and pay attention.
I think a different behaviour of the program would be better.

If a process runs out of parts to download and the only parts still available for download are reserved by another process, then the first process should ignore the reservation and just download it. The data that comes in first can then be written to the file. This will save a successful download in many cases that currently run aground a few bytes before finishing.
I know that this will cause double download of some parts, but on average the amount of that will be tiny. It will only happen in the last (say) 200 kB of a 9 MB download in a limited number of cases. Usually only at the very end of the very last downloaded chunk of a file.

Often the download will resume and finish within a reasonable period after this happened, but in my experience it is one of the most annoying situations I encountered using eMule. So I think it is worth the effort of changing it.

[Link64]

DatHebIkWeer, on 08 July 2012 - 12:21 AM, said:

But that requires manual action and can only work if I am there and pay attention.

If you are not there, how does it matter, if the file be ready few minutes earlier or later? Anyway, the network lacks of upload bandwidth, I don't see any reason, why we should waste it just for to finish a file few minutes earlier.

[DatHebIkWeer]

Link64, on 08 July 2012 - 09:42 AM, said:

Anyway, the network lacks of upload bandwidth, I don't see any reason, why we should waste it just for to finish a file few minutes earlier.

1 Because sometimes it is not just a few minutes.
2 Because troll clients can make life bad for us this way
3 malicious clients can try to discourage use of p2p altogether with this.

If the success of a download can not be blocked by another download that will make the whole system more robust and reliable. There is some extra data transfer but that is tiny and never in problematic places (I will explain why). And even that tiny extra data transfer can still be reduced.

When does the problem occur that I want solved?
First when it does not occur:
if 2 downloads are busy at about the same speed. Say we have download 1 going at 5 kB/s and download 2 at 4.5 kB/s. In that case when the end is reached both will have about the same amount of data reserved. They have about the same speed so when 1 of them runs out of reservations there will only be maybe 10 or 20 kB left to download. Or less. With today’s internet really not an amount to worry about.

In another case however, if 2 processes run with great difference in speed, for instance download 1 at 50 kB/s and download 2 at 250 B/s things are different. In that case both processes will reach the end and will have about 200 kB reserved. Download 1 will have no problem with it’s reservation. It finishes almost immediately and dies. Download 2 will struggle on for another 10 minutes ad die too, leaving us with some 150 kB yet to be downloaded. To get those 150 kB we have to place new upload requests at all available sources. How much data traffic does that generate?
Of course there are 2 processes going here. One of them struggling to get data across. The 200 kB is a heavy burden for that one. The other one having seriously less trouble coming from a place where 200k is really nothing to worry about. In this situation the slow process will never have to take the burden of the fast process. It is always the fast process that gets the extra data.

By taking over the download of that last bit the fast process can actually relief the slow process of it’s burden. Because a check can be built in.
Every time the process cannot reserve new data it checks which of it’s reserved data has already been downloaded. If it turns out some other process already did everything, it can abort. In this way we actually move data traffic from a critical problematic place to a non critical non problematic place.
I think that would benefit us all.

This post has been edited by DatHebIkWeer: 09 July 2012 - 06:56 PM

[Link64]

DatHebIkWeer, on 09 July 2012 - 08:53 PM, said:

By taking over the download of that last bit the fast process can actually relief the slow process of it’s burden. Because a check can be built in.
Every time the process cannot reserve new data it checks which of it’s reserved data has already been downloaded. If it turns out some other process already did everything, it can abort. In this way we actually move data traffic from a critical problematic place to a non critical non problematic place.

Yeah, but that's not "double download" but a more or less clean termination of the transfer from the slower source and finishing the download with the data from the faster source. And that's OK.