I've been using eMule for about 5 years and have carefully observed its operation. I think all the posts in this thread have shown a misunderstanding of the true cause of eMule's major download problem. It's not caused by the existence of the queue, and only in a minor way by the credit system.
It's mainly caused by the developers' poor choice of the upload priority factors that are based on file rarity. According to
eMule's support webpage the upload priority factors based on rarity are as follows:
Release: x1.8 [x5]
High: x
0.9 [x2]
Normal: x0.7 [x1]
Low: x
0.6 [x0.5]
Very Low: x0.2 [x0.2]
(The numbers in square brackets are for old versions of eMule, I think. The emule-project webpage could be clearer about the meaning of the square brackets.)
As you all know, widely sourced files' Auto priority is Low (even if some of the sources are incomplete) and rare files' Auto priority is High.
Note how narrow is the range of factors. In particular:
THE .9 FACTOR FOR "HIGH" IS ONLY 1.5 TIMES THE .6 FACTOR FOR "LOW".
This causes trouble for rare files in the typical case that the source sharing them is also sharing popular files.
Let's assume most eMule users leave their upload priorities at Auto, which ranges from High to Low. (It doesn't use Release or Very Low. However, the point I'm about to make is nearly as valid for the few users who optimally use Release and Very Low too, because the factor for Release is only 9 times the factor for Very Low. Although 9 is much better than 1.5, it's still inadequate, as the argument in the paragraphs that follow will show.)
Consider the effect on an eMule client that's sharing both "rare" files (where "rare" means no one else has it yet, and only one client--or a tiny number--is attempting to download it) and "new popular" files (for example, tonight's episode of Game of Thrones). For the rare files, the Auto upload priority will of course be High (and will stay High for a long time, possibly forever). For the new popular files, the Auto upload priority will begin at High, but will very quickly drop to Low as downloading clients receive parts and thus become additional sources of needed parts. (I've observed that the number of clients downloading a new popular file quickly grows to dozens or hundreds soon after it's released. They soon become sources for the parts they've downloaded, so for each of the downloading clients there will quickly be dozens or hundreds of sources that have needed parts, and its Upload Priority soon becomes Low for everyone... assuming Auto priority.)
For a client sharing rare and new popular files (not necessarily complete yet), the number of upload requests received for his/her parts of new popular files is typically MUCH greater than 1.5 times the number of requests received for his/her parts of rare files, which means eMule's "High = 1.5 x Low" causes a huge bias toward uploading new popular files and against uploading rare files, which greatly slows down the people trying to download rare files.
It would be easy to fix the upload priority formula in the next version of eMule. For instance,
if you wanted to avoid a bias one way or the other, you could simply divide by the number of known sources of parts, instead of multiplying by the priority factor given in the upload priority table.
Files (or parts of files) that have only one source should be considered "endangered species" and eMule should automatically push them. So, rather than eliminating the bias for or against rare files (which I appeared to advocate in the previous paragraph) I actually favor changing eMule so it will be strongly biased toward pushing rare files. (Better yet, push rare parts, if the overhead to calculate this degree of precision wouldn't be significant.)
HERE'S AN EASY WAY TO DO IT: If the eMule client believes it's the only source for at least one part of a requested file, that upload request should be given the best possible queue score so the request will soon be serviced, and one of those sole-sourced parts should be uploaded when the request is serviced.
The two simple changes described above might be enough to fix eMule. (Assuming enough users update to the newer version. Or to a mod that makes those changes, or some other changes that automatically push rare parts.)
Something else to consider are the reasons why many people stop sharing files "soon" after they finish downloading them. I can think of several reasons to stop sharing a file:
1. Fear of being hassled by anti-P2P police.
2. Evil selfishness. ("I've got mine; screw you!")
3. To speed up uploading of rarer files.
4. To speed up downloading.
The reasons are worth considering because eMule design decisions can exacerbate or mitigate #3 and #4. I've already described above how to greatly mitigate #3 by speeding up the uploading of rare files. So next I'll elaborate on #4:
Suppose a user X is downloading some file(s) D, and also sharing some complete files C. Due to eMule's credit system, clients with parts of D needed by X will upload them to X sooner if X has uploaded parts (typically parts of D) to them... the more parts that X sends to them, the quicker they will send other parts to X. To maximize the parts that X sends to them, user X can simply stop sharing C. This is because X's eMule will distribute its upload bandwidth not only to the clients who need X's parts of D, but also to clients who need X's parts of C and have no parts X needs. X will accumulate more credit with those who have needed parts of D if X stops sharing C (especially the popular files in C) than if X continues to share C. In other words, eMule's credit system creates an undesirable incentive to stop sharing files, especially popular files. (The credit system also creates desirable incentives to keep C large--to build up credit in general--but my point is that eMule should not create any undesirable incentives to keep C small.)
In my own practice, I tend to stop sharing a file when the following 3 conditions are all true:
1. The amount of the file I've already uploaded exceeds the size of the file. (This is a "nonleech" criterion.)
2. eMule shows the number of complete sources is still reasonably large (say 6 or more). (This is an "availability" criterion: other people should still be able to download it in a reasonable amount of time.)
3. The file isn't small. (Sharing small files doesn't consume much upload bandwidth.)
I also tend to set upload priorities to Release as soon as possible (which is after I've downloaded a part of the file), and I leave it at Release until it's complete and condition 1 is true. So, for example, a couple of hours after I finish downloading the latest episode of Game of Thrones, all 3 conditions will hold and I probably stop sharing it. This lets my eMule share rare files much better, and also benefits me a little by improving my credit accumulation for incomplete files I'm downloading.
As an alternative to unsharing, I sometimes set those files' upload priority to Very Low. But I presume that sharing a large number of popular files will interfere with uploading of rare files even if set to Very Low, so I don't use this alternative much. I don't share many large well-sourced complete files after condition 1 is met.
I think there's another possible change to eMule that would induce me (and hopefully many other people too) to share many more files. (And certainly it would make unnecessary my tedious manual practice described above.) For any file larger than, say, one part (9.28 MBytes), modify the upload score calculation so it also considers how much of the file the client has already been uploaded to everyone. To be specific:
divide the upload score by the fraction of the file already uploaded (within the last 6 months, or however long eMule remembers such stats). For example, if client X is sharing a 200 MBytes file and clients have received 700 MBytes of that file from X (during the last 6 months), then X's eMule should divide the file's upload score by 3.5 so it will spend longer in the queue and thus consume less upload bandwidth relative to other shared files. Then the file will automatically interfere less with uploading of other files;
this will reduce the incentive to stop sharing it. For another example, suppose client X is currently downloading a 200 MBytes file and has, thus far, uploaded only 9.28 MBytes of it to other clients. In this case, a client who is also downloading the file and is requesting part of it from X will be greatly boosted because X will divide the upload score by 9.28/200 (which is the same as multiplying by approximately 20). One more example, to avoid introducing a "divide by zero" bug: Suppose a client is requesting an upload from X of a file that X hasn't yet uploaded any parts; in this case, rather than dividing by zero, multiply by a large number.
I began this post by saying the upload queue and the credit systems are not serious problems. I'll elaborate here:
Regarding the claim that the upload queue causes problems... The reason I disagree is that my eMule is always uploading at about the maximum rate that I set in its Communications Options. It follows logically that eliminating its upload queue could not possibly increase the rate at which it uploads. Furthermore, in principle, the queue helps keep the upload rate at the maximum by providing eMule info about more requests, thus avoiding moments when there aren't enough known requests to saturate the upload maximum.
Anyone who says s/he is slowed down by having to wait in other eMule clients' upload queues is really saying those clients should serve him/her instead of the people they're currently serving. This is selfish and immature, like jumping any other kind of queue. The purpose of a queue is to equitably manage a scarce resource... in this case the source clients' scarce upload bandwidth. Eliminating the queue won't make the upload bandwidth less scarce.
The claim also seems self-contradictory, because the time that client X spends waiting in Z's queue should be balanced by the times when X is one of those being served by Z while Y is waiting in Z's queue. If the claim were true, then by symmetry Y should think Z should serve Y instead of X. X says Z should be serving X and Y says Z should be serving Y, using the identical argument. They can't both be right (since Z doesn't have enough upload bandwidth to serve both). So their argument must be wrong.
I haven't used a bittorrent client, but I suspect the quicker download speed claimed for bittorrent either has a serious penalty or can be explained by a greater number of sources, not because bittorrent has a technical advantage with no penalty. Some of the posts in this thread have claimed bittorrent is more popular than eMule because bittorrent downloads faster, but I suspect they have that backward... that bittorrent downloads faster because bittorrent is more popular. More sources for a file would produce faster downloading of the file.
The claim that bittorrent's greater popularity proves it's technically better is as ignorant as the claim that the popularity of the intel 8088 (in the IBM PC) proved it was technically better than the Motorola 68000 (in the Apple Macintosh) or that the popularity of VHS videotape recorders proved they were technically better than Sony Betamax.
Regarding the claim that the credit system is a serious problem... I would call it a minor problem, because its "punishment" factor is at most 10, which isn't large enough to be blamed for (large) rare files taking years to download. In my opinion, it's based on mistaken reasoning. I would say that anyone whose overall Cumulative UL:DL Ratio (which is listed in eMule's Statistics tab) is at least 1:1 should not be considered a leech. Of course, if eMule were to query another eMule client asking it to report its Cumulative UL:DL Ratio, there would be no way to be certain the response is honest, since an evil eMule mod could pretend to have a high ratio. Despite this lack of certainty regarding whether the response would be honest, I think eMule should have been designed to trust those responses rather than resort to a misleading "local" credit calculation. (The local calculation can't be faked since the source client X remembers how many bytes each requesting client uploaded from X and downloaded to X during the last 6 months). I'm assuming the number of people who would choose to use an evil mod that falsifies its Cumulative UL:DL Ratio will be relatively small. I think it's a reasonable assumption... most people would avoid using an evil mod due to their common decency, and of the minority who aren't as decent, many of them would be deterred by the risk that an evil mod will also contain malware. (Only a tiny minority would compile their own evil mod, agreed?) So, I hope new versions of eMule will be designed to respond to queries for the cumulative upload/download ratio (or will include the ratio in upload request packets, if that would be a better implementation) and I hope new versions of eMule will be designed not to "credit-punish" clients that report an overall cumulative ratio >= 1.
If you are the only source for some part of a file, then from the perspective of the P2P community it is an "endangered" part (and it endangers the entire file even if the rest of the file's parts are widely sourced) because you might stop sharing it at any moment. The P2P community should want your client to push such parts so those files will no longer be endangered, not punish clients trying to download them from you because you haven't downloaded much from them in the last 6 months.
Here's another (unrelated) request for the next eMule:
When downloading a file, improve the detection of clients sending corrupt parts (or sub-parts). I've noticed the last few weeks that new releases of the latest episodes of Orphan Black are being attacked by one or more clients sending corrupt data, and my eMule 0.50a has been very poor at rejecting just the corruption. It throws out entire parts even though only fraction of the parts are corrupt, and downloads the same parts, including the non-corrupt portions, over and over again. This is "throwing out the baby with the bathwater."
Here's my suggested solution: For each incomplete file, the client should keep track of where each portion came from, and it should store the portions instead of deleting them when a part fails the corruption test. As new portions are received, it should test combinations that collectively might make a complete part to see if any combination passes the corruption test. As soon as a combination passes, eMule can increment a goodness count for each source client that provided only good data, and a badness count for each source client that provided corrupt data. As an optimization, the order in which combinations are tested could depend on the goodness and badness counts, so that data from clients with a nonzero badness count will be tested last (which means it won't need to be tested once a good combination is available). Also, eMule could stop requesting from a client known to have sent corruption in multiple parts. If this scheme would have too much storage or cpu overhead, it could be triggered only for the (few) files for which a corrupt part has been detected. (Better late than never.)
Best wishes,
Direkitten
Sign In
Register
inman, on 26 March 2015 - 05:56 PM, said:
Back to top
Reply
