[pier4r]

I didn't analyze any code about SOTN & hideOS, so i base my thoughts only on information gathered with a fast search here:sotn@wiki, here:hos@wiki, and sotn&hos by stulle .

SOTN: share only the need.
hideOS: hide overshare.
ICS: intelligent chunk selection.

Let's begin!
given that a client should use (imo) the ICS feature to download a chunk, how these feature works with ICS? (obviously i can write false statements if the code is different compared to the gathered informations)

In few words, if i understand correctly, ICS should pick the rarest available chunk of a file for download. Thanks to kad in first minutes and source exchange after, a client can reach almost all sources (except ones that didn't publish the file because them share a lot of files, so kad publishing needs time) in the network and can reach also almost all clients that are downloading the file. let's call clients that download the chunk as "downloaders", while the sources as "releaser".

A downloader have, after some time, the best picture of the network regard to chunk availability.

For the same reason, sooner or later (except for small files) a releaser if it publish the file on kad, will have all the downloaders in queue.

Now, a releaser can know with a good approximation the global distribution of file chunks? yes! In fact others releasers should be not counted, because them have, obviously, all parts of the file and moreover a releaser normally don't know other releasers of the file directly.
Then the chunk distribution can change only if a downloader gets a new chunk, but the releaser know all downloaders so it knows all changes.

That said, let's analyze the features.

With hideOS a releaser can nonetheless hide the rarest chunk, if it was recently sent to a downloader, and this is bad and works against ICS design.
For example: a file with 3 chunks A, B, C.
A is the rarest.
Then there is B and after C.
The releaser sent A and B, after it will show only C, even if A is still the rarest chunk.


With SOTN, if SOTN works with the information gathered from clients in queue (only clients in queue is enough) for the file, a scenario such as the previous one can never happen. In fact sotn recognize the rarest chunk thanks to information gathered from the downloaders, that is a good snapshot of chunks availability on the network, and will hide the others, thus allowing to spread faster the rarest chunk without harm to ICS.

Conclusion: SOTN should be enabled (in an hardcoded fashion) for all files.

This post has been edited by pier4r: 09 June 2011 - 03:37 PM

[tHeWiZaRdOfDoS]

Basically, ICS should be used by all clients to optimize bandwidth utilization and file spreading. SOTN and HideOS are just "hacks" (bad ones in case of HOS) that try to enforce what ICS would already do.

[pier4r]

tHeWiZaRdOfDoS, on 09 June 2011 - 06:01 PM, said:

Basically, ICS should be used by all clients to optimize bandwidth utilization and file spreading. SOTN and HideOS are just "hacks" (bad ones in case of HOS) that try to enforce what ICS would already do.

yes but, if i didn't use incorrect informations to write my argument, SOTN do nothing against ICS but can improved the chunk spreading for clients without ICS. (obviously i start with ICS because it is a good thing).

[omeringen]

I see your question is different but at the begining, you know these features is here because of official eMule's chunk choosing method. I still can't understand why official eMule doesn't use ICS. So noone would need these hacks.

See request topic ;
http://forum.emule-p...ndpost&p=971173

This post has been edited by omeringen: 09 June 2011 - 05:24 PM

[pier4r]

omeringen, on 09 June 2011 - 07:17 PM, said:

I see your question is different but at the begining, you know these features is here because of official eMule's chunk choosing method. I still can't understand why official eMule doesn't use ICS. So noone would need these hacks.

Obviously! (see my signature!)

[taz-me]

Just a quick one :

Newest (intelligent) SOTN (by WiZ coded in eMF v >= 1.0 and derived) is the best solution for now : it exposes all chunks to ICS using clients and applies SOTN logic for the rest (... mainly official).

[pier4r]

taz-me, on 10 June 2011 - 08:07 AM, said:

Just a quick one :

Newest (intelligent) SOTN (by WiZ coded in eMF v >= 1.0 and derived) is the best solution for now : it exposes all chunks to ICS using clients and applies SOTN logic for the rest (... mainly official).

Thanks for the note (so i can check the code) but even if there is no information on what chunk chooser is used by the client, the statements on the first post are valid. SOTN is not harmful for ICS.

This post has been edited by pier4r: 10 June 2011 - 11:02 AM

[taz-me]

pier4r, on 10 June 2011 - 02:01 PM, said:

SOTN is not harmful for ICS.

For the sake of argument let's look on a scenario where the downloader and the releaser source lists are not the same - SOTN might hide "common" (due to sources not seen by downloader - for example ICS equipped client without KAD / KAD2) chunks that are "rare" for the downloader ...

[pier4r]

taz-me, on 10 June 2011 - 02:04 PM, said:

For the sake of argument let's look on a scenario where the downloader and the releaser source lists are not the same - SOTN might hide "common" (due to sources not seen by downloader - for example ICS equipped client without KAD / KAD2) chunks that are "rare" for the downloader ...

Mumble, i don't recognized this scenario with the assumption that i have done for the proof, for the following reasons:
1. Except the first minutes, after some time (1-2 h? Is not so much time) a downloader can reach all sources that share at least one complete chunk thanks to source exchange. So the downloader will have an almost perfect snapshot of the chunks availability
2. If the downloader can't reach some sources, so the ICS works well for the point of view of the downloader, not for the global chunks availability. Then is not a big problem, imo.

[tHeWiZaRdOfDoS]

SOTN will always show the least spread chunks... this may not necessarily be accurate or the best solution because you may show the same chunk to a LOT of other clients which may then request the same chunk instead of different ones (though IIRC my iSOTN solution takes care of that issue) - long talk, short solution: the way to go is to fix the downloaders side (maybe with the help of the uploaders' oberservations).

[pier4r]

tHeWiZaRdOfDoS, on 10 June 2011 - 05:51 PM, said:

SOTN will always show the least spread chunks... this may not necessarily be accurate or the best solution because you may show the same chunk to a LOT of other clients which may then request the same chunk instead of different ones (though IIRC my iSOTN solution takes care of that issue) - long talk, short solution: the way to go is to fix the downloaders side (maybe with the help of the uploaders' oberservations).

green: obviously i agree (and it will be also less costly)
red: sure! But is not, at least after a quick thought, difficult to unhide other chunks, e.g: show at least the 3 rarest chunk that are requested by the downloader if there are a lot of people that request the file. Anyway this is more costly instead of ICS.

[tHeWiZaRdOfDoS]

IIRC I solved the issue in my iSOTN by counting which chunks we published to other clients... that way, we'll usually show different chunks to all users which eliminates that problem.

[pier4r]

tHeWiZaRdOfDoS, on 10 June 2011 - 06:25 PM, said:

IIRC I solved the issue in my iSOTN by counting which chunks we published to other clients... that way, we'll usually show different chunks to all users which eliminates that problem.

is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client show the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

This post has been edited by pier4r: 10 June 2011 - 05:32 PM

[tHeWiZaRdOfDoS]

pier4r, on 10 June 2011 - 07:30 PM, said:

is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client shown the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

Yes, absolutely... that may sound arrogant but it's a fact
Should be available in the latest eMF release...

It's best you take a look at the source code... basically we check the queue for the available AND shown chunks and make our decision with that data.

[pier4r]

tHeWiZaRdOfDoS, on 10 June 2011 - 07:34 PM, said:

pier4r, on 10 June 2011 - 07:30 PM, said:

is your iSOTN, as far as you know (xD), the best out there? (At least from your point of view) can i find it in which mod version...emf 1.1 final?
Anyway how does it manage the case that: "it show different chunks to all clients [and only to one client shown the rarest?], but the previous clients that get the chunk disappear from the queue?" ?

Yes, absolutely... that may sound arrogant but it's a fact
Should be available in the latest eMF release...

It's best you take a look at the source code... basically we check the queue for the available AND shown chunks and make our decision with that data.

Ok i will check the code (but maybe this is already included in ZZUL TRA base, pre 2.1), thanks. If i find some action that aren't clear to my eyes (remember, i'm not so skilled), i will fill you pm box or this thread