Official eMule-Board: Ics, Sotn, Hideos Etc... - Official eMule-Board

Jump to content


Page 1 of 1

Ics, Sotn, Hideos Etc...

#1 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 09 June 2011 - 03:31 PM

I didn't analyze any code about SOTN & hideOS, so i base my thoughts only on information gathered with a fast search here:sotn@wiki, here:hos@wiki, and sotn&hos by stulle .

SOTN: share only the need.
hideOS: hide overshare.
ICS: intelligent chunk selection.

Let's begin!
given that a client should use (imo) the ICS feature to download a chunk, how these feature works with ICS? (obviously i can write false statements if the code is different compared to the gathered informations)

In few words, if i understand correctly, ICS should pick the rarest available chunk of a file for download. Thanks to kad in first minutes and source exchange after, a client can reach almost all sources (except ones that didn't publish the file because them share a lot of files, so kad publishing needs time) in the network and can reach also almost all clients that are downloading the file. let's call clients that download the chunk as "downloaders", while the sources as "releaser".

A downloader have, after some time, the best picture of the network regard to chunk availability.

For the same reason, sooner or later (except for small files) a releaser if it publish the file on kad, will have all the downloaders in queue.

Now, a releaser can know with a good approximation the global distribution of file chunks? yes! In fact others releasers should be not counted, because them have, obviously, all parts of the file and moreover a releaser normally don't know other releasers of the file directly.
Then the chunk distribution can change only if a downloader gets a new chunk, but the releaser know all downloaders so it knows all changes.

That said, let's analyze the features.

With hideOS a releaser can nonetheless hide the rarest chunk, if it was recently sent to a downloader, and this is bad and works against ICS design.
For example: a file with 3 chunks A, B, C.
A is the rarest.
Then there is B and after C.
The releaser sent A and B, after it will show only C, even if A is still the rarest chunk.


With SOTN, if SOTN works with the information gathered from clients in queue (only clients in queue is enough) for the file, a scenario such as the previous one can never happen. In fact sotn recognize the rarest chunk thanks to information gathered from the downloaders, that is a good snapshot of chunks availability on the network, and will hide the others, thus allowing to spread faster the rarest chunk without harm to ICS.

Conclusion: SOTN should be enabled (in an hardcoded fashion) for all files.

This post has been edited by pier4r: 09 June 2011 - 03:37 PM

>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#2 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 09 June 2011 - 04:01 PM

Basically, ICS should be used by all clients to optimize bandwidth utilization and file spreading. SOTN and HideOS are just "hacks" (bad ones in case of HOS) that try to enforce what ICS would already do.
0

#3 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 09 June 2011 - 04:43 PM

View PosttHeWiZaRdOfDoS, on 09 June 2011 - 06:01 PM, said:

Basically, ICS should be used by all clients to optimize bandwidth utilization and file spreading. SOTN and HideOS are just "hacks" (bad ones in case of HOS) that try to enforce what ICS would already do.


yes but, if i didn't use incorrect informations to write my argument, SOTN do nothing against ICS but can improved the chunk spreading for clients without ICS. (obviously i start with ICS because it is a good thing).
>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#4 User is offline   omeringen 

  • l÷l
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 973
  • Joined: 01-January 06

Posted 09 June 2011 - 05:17 PM

I see your question is different but at the begining, you know these features is here because of official eMule's chunk choosing method. I still can't understand why official eMule doesn't use ICS. So noone would need these hacks.

See request topic ;
http://forum.emule-p...ndpost&p=971173

This post has been edited by omeringen: 09 June 2011 - 05:24 PM

0

#5 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 09 June 2011 - 05:46 PM

View Postomeringen, on 09 June 2011 - 07:17 PM, said:

I see your question is different but at the begining, you know these features is here because of official eMule's chunk choosing method. I still can't understand why official eMule doesn't use ICS. So noone would need these hacks.


Obviously! (see my signature!)
>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#6 User is offline   taz-me 

  • I'm taz (a modder)
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 587
  • Joined: 07-December 06

Posted 10 June 2011 - 06:07 AM

Just a quick one :

Newest (intelligent) SOTN (by WiZ coded in eMF v >= 1.0 and derived) is the best solution for now : it exposes all chunks to ICS using clients and applies SOTN logic for the rest (... mainly official).
P2P is about sharing, ed2k is my choice !
2

#7 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 10 June 2011 - 11:01 AM

View Posttaz-me, on 10 June 2011 - 08:07 AM, said:

Just a quick one :

Newest (intelligent) SOTN (by WiZ coded in eMF v >= 1.0 and derived) is the best solution for now : it exposes all chunks to ICS using clients and applies SOTN logic for the rest (... mainly official).


Thanks for the note (so i can check the code) but even if there is no information on what chunk chooser is used by the client, the statements on the first post are valid. SOTN is not harmful for ICS.

This post has been edited by pier4r: 10 June 2011 - 11:02 AM

>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#8 User is offline   taz-me 

  • I'm taz (a modder)
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 587
  • Joined: 07-December 06

Posted 10 June 2011 - 12:04 PM

View Postpier4r, on 10 June 2011 - 02:01 PM, said:

SOTN is not harmful for ICS.


For the sake of argument let's look on a scenario where the downloader and the releaser source lists are not the same - SOTN might hide "common" (due to sources not seen by downloader - for example ICS equipped client without KAD / KAD2) chunks that are "rare" for the downloader ...
P2P is about sharing, ed2k is my choice !
0

#9 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 10 June 2011 - 03:31 PM

View Posttaz-me, on 10 June 2011 - 02:04 PM, said:

For the sake of argument let's look on a scenario where the downloader and the releaser source lists are not the same - SOTN might hide "common" (due to sources not seen by downloader - for example ICS equipped client without KAD / KAD2) chunks that are "rare" for the downloader ...


Mumble, i don't recognized this scenario with the assumption that i have done for the proof, for the following reasons:
1. Except the first minutes, after some time (1-2 h? Is not so much time) a downloader can reach all sources that share at least one complete chunk thanks to source exchange. So the downloader will have an almost perfect snapshot of the chunks availability
2. If the downloader can't reach some sources, so the ICS works well for the point of view of the downloader, not for the global chunks availability. Then is not a big problem, imo.
>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#10 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 10 June 2011 - 03:51 PM

SOTN will always show the least spread chunks... this may not necessarily be accurate or the best solution because you may show the same chunk to a LOT of other clients which may then request the same chunk instead of different ones (though IIRC my iSOTN solution takes care of that issue) - long talk, short solution: the way to go is to fix the downloaders side (maybe with the help of the uploaders' oberservations).
0

#11 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 10 June 2011 - 04:00 PM

View PosttHeWiZaRdOfDoS, on 10 June 2011 - 05:51 PM, said:

SOTN will always show the least spread chunks... this may not necessarily be accurate or the best solution because you may show the same chunk to a LOT of other clients which may then request the same chunk instead of different ones (though IIRC my iSOTN solution takes care of that issue) - long talk, short solution: the way to go is to fix the downloaders side (maybe with the help of the uploaders' oberservations).


green: obviously i agree (and it will be also less costly)
red: sure! But is not, at least after a quick thought, difficult to unhide other chunks, e.g: show at least the 3 rarest chunk that are requested by the downloader if there are a lot of people that request the file. Anyway this is more costly instead of ICS.
>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#12 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 10 June 2011 - 04:25 PM

IIRC I solved the issue in my iSOTN by counting which chunks we published to other clients... that way, we'll usually show different chunks to all users which eliminates that problem.
0

#13 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 10 June 2011 - 05:30 PM

View PosttHeWiZaRdOfDoS, on 10 June 2011 - 06:25 PM, said:

IIRC I solved the issue in my iSOTN by counting which chunks we published to other clients... that way, we'll usually show different chunks to all users which eliminates that problem.


is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client show the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

This post has been edited by pier4r: 10 June 2011 - 05:32 PM

>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#14 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 10 June 2011 - 05:34 PM

View Postpier4r, on 10 June 2011 - 07:30 PM, said:

is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client shown the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

Yes, absolutely... that may sound arrogant but it's a fact ;)
Should be available in the latest eMF release...

It's best you take a look at the source code... basically we check the queue for the available AND shown chunks and make our decision with that data.
0

#15 User is offline   pier4r 

  • Ex falso quodlibet ; Kad is the major concept behind emule.
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 571
  • Joined: 31-March 09

Posted 10 June 2011 - 05:47 PM

View PosttHeWiZaRdOfDoS, on 10 June 2011 - 07:34 PM, said:

View Postpier4r, on 10 June 2011 - 07:30 PM, said:

is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client shown the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

Yes, absolutely... that may sound arrogant but it's a fact ;)
Should be available in the latest eMF release...

It's best you take a look at the source code... basically we check the queue for the available AND shown chunks and make our decision with that data.


Ok i will check the code (but maybe this is already included in ZZUL TRA base, pre 2.1), thanks. If i find some action that aren't clear to my eyes (remember, i'm not so skilled), i will fill you pm box or this thread :flowers:
>>> My wiki (ITA) on emule >>>Feature Request (ICS) or SOTN, ClientAnalyzer(fixing fastXs and reask punishment),, EmuleCollectionV2 >>> Emule on old hardware (intel pentium 2 or 3 - via c3 - and so on) with good OS settings and enough ram (256+ mb): great >>>user of: eMule - Xtreme - ZZUL bastard - SharX - SharkX 1.8b5 pierQR - ZZUL-Tra - ZZUL-Tra-TL - kMule - Beba

Extended signature: click.
0

#16 User is offline   Sir_Boagalott 

  • Golden eMule
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 466
  • Joined: 23-September 02

Posted 30 July 2013 - 02:03 AM

View PosttHeWiZaRdOfDoS, on 10 June 2011 - 01:34 PM, said:

It's best you take a look at the source code... basically we check the queue for the available AND shown chunks and make our decision with that data.


Hmm, interesting. I was aware that your iSOTN did this. However I'm now confused because I have asked you numerous times to calculate this and you wont because of your counter argument of you cant trust remote data, clients can hide chunks. Dont get me wrong, you are right about clients can & do hide chunks, which can mess up the "true" amount, but your iSOTN does make use of this non trustable remote data.

Basically iSOTN checking the queue for the available AND shown chunks in theory is in fact calculating the Global DL availability :angelnot:

So what is the difference?

If iSOTN calculates the Global DL availability then why not show that info somewhere, and start using it? Apparently you already are trusting Global DL availability by calculating it in iSOTN.

:respect:
0

#17 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 30 July 2013 - 05:49 AM

You cannot trust any data you receive from remote clients, that's a fact... you have, however, to use some of that data :flowers:
What and where should be shown?
0

#18 User is offline   Sir_Boagalott 

  • Golden eMule
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 466
  • Joined: 23-September 02

Posted 30 July 2013 - 08:53 AM

View PosttHeWiZaRdOfDoS, on 30 July 2013 - 01:49 AM, said:

You cannot trust any data you receive from remote clients, that's a fact... you have, however, to use some of that data :flowers:


I have been telling you that for years too. :angelnot:

I know you have always been against it because yes, it can be faked, but even still whats available, is whats available. i.e. you release a new file with 8 chunks, you UL 4 chunks to Budy Al, and the bugger hides them. Whats the availability of that file? 1.50? No, its still technically just 1. Even though the answer is truly 1.50, those chunks are hidden, so therefore they arent actually physically available = technically not available. It might sound silly to some, but the true physical availability is completely irrelevant if it isnt technically physically available. i.e if a chunk is hidden, it doesnt matter if they have it, its unavailable.

View PosttHeWiZaRdOfDoS, on 30 July 2013 - 01:49 AM, said:

What and where should be shown?


Whan you say "we check the queue for the available AND shown chunks" what does that mean? and called - name? I refer to that as calculating the Global DL Availability. i.e. watch clients in Q and their partfilestatus for which chunks they have, then make a % calculation of it. i.e. 100 clients in q, 100% have 0 comp, Global DL Availability= 0. Or 100 clients in q for a 10 chunk file thats unique, UL 3 chunks Global DL Availability= .300. Oh noes, better UL that file, when would right now be a good time? :sleep: (wink wink, nudge nudge)

If iSOTN calculates this info, then why not make excellent use this info and show it in the shared files win?

You asked for it:
Posted Image

Note: pic isnt finished, its for a "bigger" Req. It should help you "see" what I mean, and potentially where I am going with it. :flowers:
0

  • Member Options

Page 1 of 1

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users