Official eMule-Board: Side Effects Of Too Many Lowid's - Official eMule-Board

Jump to content


  • (2 Pages)
  • +
  • 1
  • 2

Side Effects Of Too Many Lowid's

#1 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 03 January 2021 - 08:04 AM

I share about a thousand files, mostly old music videos and TV shows from the 60's and 70's
using the MorphXT mod. I typically get about 30 users in my upload queue. The bandwidth max
I can safely allocate is about 200KB p/s; if I go beyond that, Spectrum throttles my
upload and I have to call them to fix it (even if they say they don't throttle, it was just
a temporary network problem)

I tried to implement some sharing techniques such as Hide over share and/or share only the need,
But then I realized that this is never going to work when the percentage of LowID's in my
queue is so high, typically about 90% - since they can't share with each other.

Why there are so many LowID's is pretty much a rhetorical question at this point, I wish it were
not so. Seems like much of the sophistication of the sharing techniques built into the
code is just wasted

It's shame that a way wasn't found in the early days to address it and encourage the little bit
of know-how needed to get the High ID. If I had a LowID myself, my upload queue would be
practically empty for days at a time

I enjoy sharing my files, but I wish the situation were more balanced
0

#2 User is offline   Heliotropo 

  • Member
  • PipPip
  • Group: Members
  • Posts: 42
  • Joined: 13-June 05

Posted 03 January 2021 - 11:06 AM

I completely agree with your approach. Something should be done if eMule is to be attractive and effective to appeal to new users.
0

#3 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5800
  • Joined: 07-April 04

Posted 03 January 2021 - 11:20 AM

Generally speaking, you are correct. LowID to LowID is and will remain an issue until some substantial revisions make it to the mainstream client. However, considering your specific desire: You may want to consider using Share only the Need (SOTN) instead of Hide Overshare (HideOS). In a network with so few clients and especially if you are sharing very rare files it's best if your client decides which parts have sufficient coverage and which don't based on the current availability known to it.
I am an emule-web.de member and fan!

[Imagine there was a sarcasm meter right here!]

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

#4 User is offline   omeringen 

  • l÷l
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 983
  • Joined: 01-January 06

Posted 03 January 2021 - 11:42 AM

View Postdavexnet, on 03 January 2021 - 11:04 AM, said:

Why there are so many LowID's is pretty much a rhetorical question at this point, I wish it were
not so. Seems like much of the sophistication of the sharing techniques built into the
code is just wasted

It's shame that a way wasn't found in the early days to address it and encourage the little bit
of know-how needed to get the High ID. If I had a LowID myself, my upload queue would be
practically empty for days at a time

I enjoy sharing my files, but I wish the situation were more balanced

Because a user needs to spend some time to correctly set eMule (investigate what LowID and HighID is and forward ports if UPnP doesn't work) and you know users are lazy and i understand this.

LowID must be handled automatically. There are some ideas but still no roadmap for official client: https://forum.emule-...howtopic=156977
0

#5 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 03 January 2021 - 10:18 PM

Appreciate the comments. Stulle, thank you for your suggestion


My client is running now, I see two LowID clients downloading the same file from me;
does "share only the need" help? I see the "obtained parts" on the "uploading" screen is colored
purple.

This post has been edited by davexnet: 03 January 2021 - 11:08 PM

0

#6 User is offline   omeringen 

  • l÷l
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 983
  • Joined: 01-January 06

Posted 04 January 2021 - 08:26 AM

Yeah using SOTN always helps healthy part distribution. Purple means that : wiki.emule-web.de/Share_only_the_need
0

#7 User is offline   emulers 

  • Newbie
  • Pip
  • Group: Members
  • Posts: 7
  • Joined: 04-September 20

Posted 07 January 2021 - 07:16 PM

It's not so much about knowledge of users or them being lazy anymore.

CGNAT is pretty common nowadays and it's getting more and more wide every other day because of IPv4 limitations.

And it's not possible to get a High ID with a CGNatted connection.
0

#8 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 13 February 2021 - 10:28 PM

View Postemulers, on 07 January 2021 - 11:16 AM, said:

It's not so much about knowledge of users or them being lazy anymore.

CGNAT is pretty common nowadays and it's getting more and more wide every other day because of IPv4 limitations.

And it's not possible to get a High ID with a CGNatted connection.


Ok, there may be some. I doubt if it's the majority.
The common thing I see is a file I am sharing being uploaded from me to 3 (for example)
Lowid users at the same time.
I have to upload the whole file 3 times.
All of the thought that went into the code to optimize uploads/sharing is wasted because of this.
I wish there was a way to give priority to HighID's in the client; both in terms of priority
and bandwidth.
I have now taken to watching it and giving this priority myself manually. But this is not sustainable.
0

#9 User is offline   Campo 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 62
  • Joined: 18-June 19

Posted 14 February 2021 - 01:06 AM

i think i remember that high id clients get more credits. low ids have to wait much longer until they get the upload slot. of course a low id dont mean, that no uplaod slot is reached as long as a high id is waiting too. is that correct?

in germany with vodafone and unitymedia, most of there customers have only a "real" ipv6. and quite often the normal router are garbage with nearly no options... i have trouble with some games multiplayer, because of that.
0

#10 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 14 February 2021 - 01:31 AM

View PostCampo, on 13 February 2021 - 05:06 PM, said:

i think i remember that high id clients get more credits. low ids have to wait much longer until they get the upload slot. of course a low id dont mean, that no uplaod slot is reached as long as a high id is waiting too. is that correct?

in germany with vodafone and unitymedia, most of there customers have only a "real" ipv6. and quite often the normal router are garbage with nearly no options... i have trouble with some games multiplayer, because of that.


yes I understand what you're saying, trouble is, most of the clients (about 90%)
on my queue are LowID's.
My own equipment was just swapped out by the internet company, now have a very limited router,
have to control it from an app on the phone.
But it still enables me to forward ports.
0

#11 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5800
  • Joined: 07-April 04

Posted 23 February 2021 - 07:53 PM

View PostCampo, on 14 February 2021 - 01:06 AM, said:

i think i remember that high id clients get more credits. low ids have to wait much longer until they get the upload slot. of course a low id dont mean, that no uplaod slot is reached as long as a high id is waiting too. is that correct?

No it's not correct. Credits are, strictly speaking, just a record of sent and received bytes per secure user hash. We calculate (one way or another) score from that which basically acts as a multiplier to the waiting time. So waiting time multiplied with said score will give a result which ranks all clients in our queue. However, High ID clients will have a higher chance of jumping to the upload list because we need to wait for Low ID clients to connect when we can (basically) arbitrarily connect to High ID clients. So even when a Low ID client is on the top of the list we may promote a lower ranked High ID client and not promote that same Low ID client later because at that point we may not need it to fill our bandwidth. It should be noted that different mods handle Low ID clients differently in regards to their promotion to the upload list.
I am an emule-web.de member and fan!

[Imagine there was a sarcasm meter right here!]

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

#12 User is offline   Campo 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 62
  • Joined: 18-June 19

Posted 24 February 2021 - 04:42 PM

Thank you Stulle.

Can you recommend a credit system? iam currently at the lovelace, but i cant say anymore, why i choose that.
0

#13 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5800
  • Joined: 07-April 04

Posted 24 February 2021 - 06:36 PM

I prefer lovelace myself because it gives substantial credits to those who have given to me. There used to be some documentation on its details which you can find here:

ed2k://|file|credit.system.[lovelace].rar|129865|AF2348D4C8C871DF2127DE558D6FF5CC|h=FWSD7FHAEWW24JFIL3VSA7ODOYH3Z7WL|/

lovelace put it this way:

Quote

new credit system (start:1, max:100, min:0.1, ratio:1:1.5, only one formula)
CreditThefts will not get any credits. Only clients using the 'SecureHash' are able to get a multiplier of 100. All others will stick at 10.

In contrast to the original credit system, credits are evaluated more on differences and not on quotients. Using the orginal system you have the best credit values shortly after generating a new userhash. With the new credit system you get good credit values faster if you already have uploaded many MB before (and did not cheat by killing the userhash).

(old system: 5up/ 5down = DLModifier of 2, additional 5up = DLModifier of 4
10up/10down = DLModifier of 2, additional 5up = DLModifier of 3
-> for the same amount of additional upload you get less score (-25%)
new system: 5up/ 5down = DLModifier of 1.16, additional 5up = DLModifier of 2.31
10up/10down = DLModifier of 1.85, additional 5up = DLModifier of 5.09
-> for the same amount of additional upload you get more score (+120%)
because you already uploaded a certain amount before.)

This is only one simple example; new system has even more advantages. So in general generally generous uploaders get a nicer DLModifier than tightwads.


Other credit systems work differently. Would need to check either their code or the respective changelogs...

Anyhow, the importance of credit systems has waned since the days they have been invented. Right now there are so few clients in the network and uploaders have gotten bigger lines. I don't really think there is an actual shortage of upload bandwidth in the network anymore. In my personal experience, I rarely ever see my bandwidth filled. I mean, I used to have like 12 kB/s available some 15 years ago. Now I cap my upload at 700 kB/s and could potentially go above it some more...

This post has been edited by Stulle: 24 February 2021 - 06:37 PM

I am an emule-web.de member and fan!

[Imagine there was a sarcasm meter right here!]

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

#14 User is offline   Campo 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 62
  • Joined: 18-June 19

Posted 24 February 2021 - 08:17 PM

yeah, youre right with the unused upload. funny: i have 3 clients nearly every week or in long sessions, which i know since over 15 years. they are donwloading so slow (5kb/sec), i think they have limited the bandwith. i see them even on new files^^ Emule and BiglyBT are both at autospeed. The few torrents i have active are using the fullspeed nearly instantly, so if i shutdown the pc, i close torrent some minutes earlier.

Lovelace: Thanks, i will read into that. You made the important points already out. I want the whole System as fair as possible. The security settings against leechermods, thiefs ect are as hard as morph allows it. Even if my upload is free,the use of unfair functions/clients should be punished.
0

#15 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 25 February 2021 - 04:36 AM

View PostCampo, on 24 February 2021 - 12:17 PM, said:

yeah, youre right with the unused upload. funny: i have 3 clients nearly every week or in long sessions, which i know since over 15 years. they are donwloading so slow (5kb/sec), i think they have limited the bandwith. i see them even on new files^^ Emule and BiglyBT are both at autospeed. The few torrents i have active are using the fullspeed nearly instantly, so if i shutdown the pc, i close torrent some minutes earlier.

Lovelace: Thanks, i will read into that. You made the important points already out. I want the whole System as fair as possible. The security settings against leechermods, thiefs ect are as hard as morph allows it. Even if my upload is free,the use of unfair functions/clients should be punished.


regarding leeching, there's some guy downloading some of my files, he's downloading two files from me at the same time.
How is this possible? According to the "client details" He's using eMule V0.48a VeryCD 090304
I've checked the user hash and downloaded total, they're the same, both active together
0

#16 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5800
  • Joined: 07-April 04

Posted 26 February 2021 - 08:07 AM

Did you check the client's hash is legit? Some have corrupted hashed (basically 0s with a couple of other digits in 2 places) and they appear like the same when they are not. Some times people also just copy their eMule folder including the config which would create (upon starting emule) two clients with the same hash etc. I can't quite remember how remote clients would handle it if they encounter two clients with the same legit secure hash.
I am an emule-web.de member and fan!

[Imagine there was a sarcasm meter right here!]

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

#17 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 26 February 2021 - 10:12 PM

I tried to post an image of the client details, but I got a forum message saying
"You do not have permission to carry out that function"

This is the hash:
A1D7FF5CC00E13DCC12FB7F3AA9B6F99

This post has been edited by davexnet: 28 February 2021 - 12:34 AM

0

#18 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 28 February 2021 - 11:05 PM

View Postdavexnet, on 26 February 2021 - 02:12 PM, said:

I tried to post an image of the client details, but I got a forum message saying
"You do not have permission to carry out that function"

This is the hash:
A1D7FF5CC00E13DCC12FB7F3AA9B6F99


At one point yesterday that same user was downloading from me two instances of the same file at the same time.

As I mentioned earlier, one thing I would have liked to see in the program is a way to allocate certain
amounts of bandwidth to the clients depending on whether they're High or Low ID's.
Perhaps the current situation (too high a proportion of Low ID's) was not envisaged when the thoughts
about balancing the network were considered at the program design.
For now, I'm using facilities in the Morph mod to do it myself manually. There are so may LowID's,
their activity in the network is a form of leeching - since once they get the files it's gone into a black hole
0

#19 User is offline   Enig123 

  • Golden eMule
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 534
  • Joined: 22-November 04

Posted 01 March 2021 - 07:07 AM

View PostStulle, on 26 February 2021 - 12:07 AM, said:

Did you check the client's hash is legit? Some have corrupted hashed (basically 0s with a couple of other digits in 2 places) and they appear like the same when they are not. Some times people also just copy their eMule folder including the config which would create (upon starting emule) two clients with the same hash etc. I can't quite remember how remote clients would handle it if they encounter two clients with the same legit secure hash.


Even worse, there're some of the user hashes be shared publicly on internet.

I've been working on a solution that's not based on the now false assumption that userhash is unique for each eMule client instance, the AttachToAlreadyKnown is re-written to address that issue, based on the rule that a client should be preferably identified by it's client IP then userhash.

The current codes made the userhash duplication detected more easily and accurately.

For upload side, uploadqueue, waitingqueue only allow unique userhash, which means duplicated userhash clients will have to kick each other out of the queues. Basically, the new rule will assume no interruption for a 'normal' client with a unique userhash.
0

#20 User is offline   davexnet 

  • Advanced Member
  • PipPipPip
  • Group: Members
  • Posts: 72
  • Joined: 21-February 05

Posted 04 March 2021 - 01:52 AM

View PostEnig123, on 28 February 2021 - 11:07 PM, said:

View PostStulle, on 26 February 2021 - 12:07 AM, said:

Did you check the client's hash is legit? Some have corrupted hashed (basically 0s with a couple of other digits in 2 places) and they appear like the same when they are not. Some times people also just copy their eMule folder including the config which would create (upon starting emule) two clients with the same hash etc. I can't quite remember how remote clients would handle it if they encounter two clients with the same legit secure hash.


Even worse, there're some of the user hashes be shared publicly on internet.

I've been working on a solution that's not based on the now false assumption that userhash is unique for each eMule client instance, the AttachToAlreadyKnown is re-written to address that issue, based on the rule that a client should be preferably identified by it's client IP then userhash.

The current codes made the userhash duplication detected more easily and accurately.

For upload side, uploadqueue, waitingqueue only allow unique userhash, which means duplicated userhash clients will have to kick each other out of the queues. Basically, the new rule will assume no interruption for a 'normal' client with a unique userhash.

What ever is/isn't being considered, it's clear that these rogue clients have surpassed the ability to be properly detected,
and I have a situation where a user with the same hash is downloading the same file twice in active clients,
and there's a third, on the queue, same user/hash different file.
0

  • Member Options

  • (2 Pages)
  • +
  • 1
  • 2

2 User(s) are reading this topic
0 members, 2 guests, 0 anonymous users