Official eMule-Board: Emule V0.47a [ese V1.0b] (2006-04-24) - Official eMule-Board

Jump to content


  • (5 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »

Emule V0.47a [ese V1.0b] (2006-04-24) Updated DBR and ReadBlockFromFileThread

#1 User is offline   netfinity 

  • Master of WARP
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1658
  • Joined: 23-April 04

Posted 04 March 2006 - 10:40 AM

eMule v0.47a [ESE v1.0b]

Not an NetF Mod, but a standard eMule 0.47a with Dynamic Block Requests, Safe Hash and SiRoB's File IO threading added.

Dynamic Block Requests in this Mod isn't as fast and reliable as in NetF as there is no dropping of stalled slots and no delayed NNP, but performance is still quite good. NEW in 1.0b is that DBR is now much more likely to complete with most sources downloading to the last few seconds before completion.

Quote

about:
A simple Mod that adds a few features to eMule that probably should already have been there.

Author:
netfinity

Changelog:
*** ESE 1.0b
- Added 'Variable compression' (reduce CPU usage at high upload rates)
- Updated 'Flush Thread' [by SiRoB / MorphXT 8.10]
- Updated 'ReadBlockFromFileThread' [by SiRoB / MorphXT 8.10]
- Raised block count locking thresholds in CUpDownClient::SendBlockRequests (makes DBR to work much better)
- Fixed various cosmetic stuff in the code

*** Missed in ESE 1.0a changelog
- Fixed GetClosestTo() functions in KAD (file searches gives much more results) [by Unknown1]

*** ESE 1.0a
- Based on eMule 0.47a
- Added 'Dynamic Block Requests' (speeds up file completion)
- Added 'Active Ratio'(must have when using 'Dynamic Block Requests' to prevent excessive downloading)
- Added 'Safe Hash' (part hashing is done in separate thread) [by SLUGFILLER]
- Added 'ReadBlockFromFileThread' (prevents GUI to become sluggish when uploading) [by SiRoB / MorphXT]
- Added 'Flush Thread' (may help a little when doing heavy downloading) [by SiRoB / MorphXT]
- Added 'Mod version' (makes it possible to distinguish the client from the official one)
- Added 'Anti Shape' (makes credit shaping less profitable)


Binary:
ed2k://|file|ESE_1.0b_binary.zip|2365785|A715A5551DA47C0EA132E46192721204| |h=IUJ6BDWNKBPBLQXVBOHMH33OGC66GW2B|/
HTTP Link (DDoS Isle of Tortuga)

Sources:
ed2k://|file|ESE_1.0b_source.zip|3933827|5337E5F242883C92AF893B04628EA923| |h=WR37NSTZMEAWGUNFEHK7IFYC4DPZXUHI|/
HTTP File (DDoS Isle of Tortuga)

This post has been edited by netfinity: 24 April 2006 - 06:43 PM

eMule v0.50a [NetF WARP v0.3a]
- Compiled for 32 and 64 bit Windows versions
- Optimized for fast (100Mbit/s) Internet connections
- Faster file completion via Dynamic Block Requests and dropping of stalling sources
- Faster searching via KAD with equal or reduced overhead
- Less GUI lockups through multi-threaded disk IO operations
- VIP "Payback" queue
- Fakealyzer (helps you chosing the right files)
- Quality Of Service to keep eMule from disturbing VoIP and other important applications (Vista/7/8 only!)
0

#2 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5738
  • Joined: 07-April 04

Posted 04 March 2006 - 11:18 AM

damn, you really kinda confused me at first... ;)

thx for the mod! :)
I am an emule-web.de member and fan! Hate me or people will get suspicious about you! Ever wondered if it's all worth the trouble?

Posted Image

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

#3 User is offline   pesti@eMF 

  • Splendid Member
  • PipPipPipPip
  • Group: Members
  • Posts: 171
  • Joined: 28-February 05

Posted 04 March 2006 - 12:14 PM

eMule 0.47a [ESE v1.0a] :+1:


Downloadarea eMule 0.47a [ESE v1.0a] -Binaries & Sources- @ eMuleFuture.de

Medion Akoya MD 96380
Intel Core 2 Duo CPU T5450 // 1,67 GHZ // 2048 MB DDR RAM // 256 MB Nvidia GeForce 8600 M GS
Vista Home Premium // ADSL2+ 16000 / 1024

0

#4 User is offline   moloko+ 

  • ...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1209
  • Joined: 18-August 05

Posted 04 March 2006 - 12:22 PM

it sounds good,

thanks. :)
0

#5 User is offline   Da GuRu 

  • Premium Member
  • PipPipPipPipPip
  • Group: Members
  • Posts: 272
  • Joined: 25-December 02

Posted 04 March 2006 - 12:48 PM

Download Mirrors:
eMule-0.47a-ESE-1.0a-bin.rar
eMule-0.47a-ESE-1.0a-src.rar


Download more versions of eMule eMule ESE.MoD
Rate eMule eMule ESE.MoD @ emule-mods.de !!!

This post has been edited by Da GuRu: 04 March 2006 - 02:10 PM

0

#6 User is offline   netfinity 

  • Master of WARP
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1658
  • Joined: 23-April 04

Posted 04 March 2006 - 05:37 PM

Just realised I might need to explain some of the features a little bit more.

Dynamic Block Requests
This features try to split up download requests in different sizes dependant of the datarate of each source. This will make it more likely that all the download requests will complete simoultaneously and therefore speed up the file completion.

Active Ratio
With active ratio, the ratio limit in the options settings of eMule is no more. Instead the ratio between download and upload for the current session is calculated. As long as upload is greater than the download, download rate can be unlimited.

Download limit is set according to the following;
    Dl:Ul < 1  => DownLimit = UNLIMITED
1 < Dl:Ul < 2  =>  DownLimit = 5 * UploadRate
2 < Dl:Ul < 3  =>  DownLimit = 4 * UploadRate
3 < Dl:Ul < 4  =>  DownLimit = 3 * UploadRate
4 < Dl:Ul        =>  DownLimit = 2 * UploadRate
Where UploadRate is your actual upload (or upload limit if the upload waiting queue is empty) as shown in the transfer window. DownLimit is the maximum allowed download rate allowed for the specific ratio.

Anti Shape
Sets a minimum threshold for credit calculation. If a client has uploaded less than 2MB when 'Try to transfer full chunks to all uploads' is not set, or 4.64MB if it is, the credit ratio will always be 1 independant of if any data has been downloaded.
eMule v0.50a [NetF WARP v0.3a]
- Compiled for 32 and 64 bit Windows versions
- Optimized for fast (100Mbit/s) Internet connections
- Faster file completion via Dynamic Block Requests and dropping of stalling sources
- Faster searching via KAD with equal or reduced overhead
- Less GUI lockups through multi-threaded disk IO operations
- VIP "Payback" queue
- Fakealyzer (helps you chosing the right files)
- Quality Of Service to keep eMule from disturbing VoIP and other important applications (Vista/7/8 only!)
0

#7 User is offline   lacouz 

  • Member
  • PipPip
  • Group: Members
  • Posts: 24
  • Joined: 11-February 05

Posted 04 March 2006 - 06:09 PM

Thanks for explanation
0

#8 User is offline   Xman1 

  • Xtreme Modder
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1955
  • Joined: 21-June 03

Posted 04 March 2006 - 07:10 PM

thanks for this mod! :thumbup:
0

#9 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 04 March 2006 - 09:46 PM

ThX a lot - awesome how small you made the DBR-Code!
However I'd recommend to change the code a bit (proper preprocessor usage+some missing comments):

PartFile.h said:

#ifdef DONT_USE_DBR
bool GetNextEmptyBlockInPart(const uint16& partnumber, Requested_Block_Struct* result) const;
#else
bool GetNextEmptyBlockInPart(const uint16& partnumber, Requested_Block_Struct* result, uint64 bytesToRequest = EMBLOCKSIZE) const; //netfinty: Dynamic Block Requests
#endif


PartFile.cpp said:

#ifdef DONT_USE_DBR
bool CPartFile::GetNextEmptyBlockInPart(const uint16& partNumber, Requested_Block_Struct *result) const
#else
// netfinity: DynamicBlockRequests - Added bytesToRequest
bool CPartFile::GetNextEmptyBlockInPart(const uint16& partNumber, Requested_Block_Struct *result, uint64 bytesToRequest) const
#endif
...
...
  // If this gap has not already been requested, we have found a valid entry 
#ifndef DONT_USE_DBR
// BEGIN netfinity: DynamicBlockRequests - Reduce bytes to request
  bytesToRequest -= bytesToRequest % 10240;
  if (bytesToRequest < 10240) bytesToRequest = 10240;
  if (bytesToRequest > EMBLOCKSIZE) bytesToRequest = EMBLOCKSIZE;
  if((start + bytesToRequest) <= end && (end - start) > (bytesToRequest + 3072)) // Avoid creating small fragments
  end = start + bytesToRequest - 1;
// END netfinity: DynamicBlockRequests - Reduce bytes to request
#endif 
...
...
// Define and create the list of the chunks to download
const uint16 partCount = GetPartCount();
CList<Chunk> chunksList(partCount);

// BEGIN netfinty: Dynamic Block Requests
#ifndef DONT_USE_DBR
uint64 bytesPerRequest = EMBLOCKSIZE;
uint64 bytesLeftToDownload = GetFileSize() - GetCompletedSize();
uint32 fileDatarate = max(GetDatarate(), UPLOAD_CLIENT_DATARATE); // Always assume file is being downloaded at atleast 3 kB/s
uint32 sourceDatarate = max(sender->GetDownloadDatarate(), 10); // Always assume client is uploading at atleast 10 B/s
uint32 timeToFileCompletion = max((uint32) (bytesLeftToDownload / (uint64) fileDatarate) + 1, 10); // Always assume it will take atleast 10 seconds to complete

bytesPerRequest = (sourceDatarate * timeToFileCompletion) / 2;

if (bytesPerRequest > EMBLOCKSIZE)
  bytesPerRequest = EMBLOCKSIZE;
if (bytesPerRequest < 10240)
{
  // Let an other client request this packet if we are close to completion and source is slow
  // Use the true file datarate here, otherwise we might get stuck in NNP state
  if (!requestedblocks_list.IsEmpty() && timeToFileCompletion < 30 && bytesPerRequest < 3400 && 5 * sourceDatarate < GetDatarate())
  {
  DebugLog(_T("No request block given as source is slow and file near completion!"));
  return false;
  }
  bytesPerRequest = 10240;
}
#endif
// BEGIN netfinty: Dynamic Block Requests
...
...
  Requested_Block_Struct* pBlock = new Requested_Block_Struct;
#ifdef DONT_USE_DBR
  if(GetNextEmptyBlockInPart(sender->m_lastPartAsked, pBlock)) 
#else
  if(GetNextEmptyBlockInPart(sender->m_lastPartAsked, pBlock, bytesPerRequest)) //netfinty: Dynamic Block Requests 
#endif

0

#10 User is offline   netfinity 

  • Master of WARP
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1658
  • Joined: 23-April 04

Posted 04 March 2006 - 10:21 PM

tHeWiZaRdOfDoS, on Mar 4 2006, 11:46 PM, said:

ThX a lot - awesome how small you made the DBR-Code!
Yes, the original code was a bit ambigous; althought it had its qualities. The new code only focus on the file completion and not trying to speed up part completion as the old code, this makes it much less complex.

tHeWiZaRdOfDoS, on Mar 4 2006, 11:46 PM, said:

However I'd recommend to change the code a bit (proper preprocessor usage+some missing comments):
I thought about removing those preprocessor statements as I only added them for debugging purposes. But yes, good idea. Thanks! :+1:
eMule v0.50a [NetF WARP v0.3a]
- Compiled for 32 and 64 bit Windows versions
- Optimized for fast (100Mbit/s) Internet connections
- Faster file completion via Dynamic Block Requests and dropping of stalling sources
- Faster searching via KAD with equal or reduced overhead
- Less GUI lockups through multi-threaded disk IO operations
- VIP "Payback" queue
- Fakealyzer (helps you chosing the right files)
- Quality Of Service to keep eMule from disturbing VoIP and other important applications (Vista/7/8 only!)
0

#11 User is offline   leexgx 

  • UK MAD FOR LESS
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 2533
  • Joined: 04-November 02

Posted 05 March 2006 - 12:42 PM

hay :) (time for an sig change)
in and around
0

#12 User is offline   SiRoB 

  • Retired Morph Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1691
  • Joined: 28-June 03

Posted 05 March 2006 - 09:59 PM

Hi netfinity,
you may have to change some of the slugfiller code to make thing better with my Threading stuff, i completly forget that point.

SetFirstHash() said:

  //MORPH - Changed by SiRoB, Need to check buffereddata otherwise we may try to hash wrong part
  /*
  if (pOwner->IsComplete((uint64)i*PARTSIZE,(uint64)(i+1)*PARTSIZE-1, false)){
  */
  if (pOwner->IsComplete((uint64)i*PARTSIZE,(uint64)(i+1)*PARTSIZE-1, true)){

FlushDone() said:

  //MORPH - Changed by SiRoB, As we are using flushed data check asynchronously we need to check if all data have been written into the file buffer
  /*
  if (IsComplete(PARTSIZE * (uint64)partNumber, (PARTSIZE * (uint64)(partNumber + 1)) - 1, false))
  */
  if (IsComplete(PARTSIZE * (uint64)partNumber, (PARTSIZE * (uint64)(partNumber + 1)) - 1, true))

eMule 0.47c MorphXT v9.5 ::binary::source::
0

#13 User is offline   netfinity 

  • Master of WARP
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1658
  • Joined: 23-April 04

Posted 05 March 2006 - 10:51 PM

@SiRoB
Thanks, I'll look into that! And thanks for your threading code, nice job!
eMule v0.50a [NetF WARP v0.3a]
- Compiled for 32 and 64 bit Windows versions
- Optimized for fast (100Mbit/s) Internet connections
- Faster file completion via Dynamic Block Requests and dropping of stalling sources
- Faster searching via KAD with equal or reduced overhead
- Less GUI lockups through multi-threaded disk IO operations
- VIP "Payback" queue
- Fakealyzer (helps you chosing the right files)
- Quality Of Service to keep eMule from disturbing VoIP and other important applications (Vista/7/8 only!)
0

#14 User is offline   SiRoB 

  • Retired Morph Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1691
  • Joined: 28-June 03

Posted 05 March 2006 - 11:11 PM

netfinity, on Mar 6 2006, 12:51 AM, said:

@SiRoB
Thanks, I'll look into that! And thanks for your threading code, nice job!
View Post

Thank's too for your work, essential usefull code will be surly well used.

Anyway you missed an important change in the our 8.5 version arround CReadBlockFromFileThread::Run().
You need to force read into the harddrive and not into the permanent partfile handler, otherwise we may move the seek position. :x

Quote

  /*if (!srcfile->IsPartFile()){*/
  CString fullname;
  if (srcfile->IsPartFile() && ((CPartFile*)srcfile)->GetStatus() != PS_COMPLETE){
    ((CPartFile*)srcfile)->m_FileCompleteMutex.Lock();
    lockFile.m_pObject = &((CPartFile*)srcfile)->m_FileCompleteMutex;
    // If it's a part file which we are uploading the file remains locked until we've read the
    // current block. This way the file completion thread can not (try to) "move" the file into
    // the incoming directory.

    fullname = RemoveFileExtension(((CPartFile*)srcfile)->GetFullName());
  }
  else{
    fullname.Format(_T("%s\\%s"),srcfile->GetPath(),srcfile->GetFileName());
  }
 
  if (!file.Open(fullname,CFile::modeRead|CFile::osSequentialScan|CFile::shareDenyNone))
    throw GetResString(IDS_ERR_OPEN);

  file.Seek(StartOffset,0);
 
  filedata = new byte[togo+500];
  if (uint32 done = file.Read(filedata,togo) != togo){
    file.SeekToBegin();
    file.Read(filedata + done,togo-done);
  }
  file.Close();
  }
  /*else{
  CPartFile* partfile = (CPartFile*)srcfile;

  partfile->m_hpartfile.Seek(StartOffset,0);
 
  filedata = new byte[togo+500];
  if (uint32 done = partfile->m_hpartfile.Read(filedata,togo) != togo){
    partfile->m_hpartfile.SeekToBegin();
    partfile->m_hpartfile.Read(filedata + done,togo-done);
  }
  }*/

eMule 0.47c MorphXT v9.5 ::binary::source::
0

#15 User is offline   netfinity 

  • Master of WARP
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1658
  • Joined: 23-April 04

Posted 06 March 2006 - 06:43 PM

@SiRoB
Hmm, thanks! I thought I took all the code from MorphXT 8.5 but obviously not.

Guess I will have to make a 1.0b release. The idea with this Mod is to make it easy to merge useful features into the official client or Mods based on it. I think the threading stuff is an essential feature to be added to the official client, as I can't use eMule without it (GUI gets incredible sluggish, without it), therefore I made this Mod especialy to help the Devs to incorporate these features.

/netfinity
eMule v0.50a [NetF WARP v0.3a]
- Compiled for 32 and 64 bit Windows versions
- Optimized for fast (100Mbit/s) Internet connections
- Faster file completion via Dynamic Block Requests and dropping of stalling sources
- Faster searching via KAD with equal or reduced overhead
- Less GUI lockups through multi-threaded disk IO operations
- VIP "Payback" queue
- Fakealyzer (helps you chosing the right files)
- Quality Of Service to keep eMule from disturbing VoIP and other important applications (Vista/7/8 only!)
0

#16 User is offline   niRRity 

  • Avid Post Editor
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 1229
  • Joined: 28-January 03

Posted 06 March 2006 - 08:37 PM

@netfinity: Thanks for this mod. I'm a sucker for simple mods. Waiting for bugfix version :+1:
0

#17 User is offline   BlueSonicBoy 

  • Magnificent Member
  • PipPipPipPipPipPip
  • Group: Members
  • Posts: 396
  • Joined: 26-September 04

Posted 07 March 2006 - 04:37 PM

netfinity, on Mar 4 2006, 05:40 AM, said:

emule v0.47a [ESE v1.0a]

Not an NetF Mod, but a standard eMule 0.47a with Dynamic Block Requests, Safe Hash and SiRoB's File IO threading added.

Dynamic Block Requests in this Mod isn't as fast and reliable as in NetF as there is no dropping of stalled slots and no delayed NNP, but performance is still quite good.

View Post


That's great! Thank You!!!! :clap:

I have only just found time to take a look at your code!
I did notice a couple of tiny minor things. :ph34r: :flowers:

Quote


        // BEGIN netfinty: Dynamic Block Requests
        uint64 bytesPerRequest = EMBLOCKSIZE;
#if !defined DONT_USE_DBR
        uint64 bytesLeftToDownload = GetFileSize() - GetCompletedSize();
        uint32 fileDatarate = max(GetDatarate(), UPLOAD_CLIENT_DATARATE); // Always assume file is being downloaded at atleast 3 kB/s
        uint32 sourceDatarate = max(sender->GetDownloadDatarate(), 10); // Always assume client is uploading at atleast 10 B/s
        uint32 timeToFileCompletion = max((uint32) (bytesLeftToDownload / (uint64) fileDatarate) + 1, 10); // Always assume it will take atleast 10 seconds to complete

        bytesPerRequest = (sourceDatarate * timeToFileCompletion) / 2;

        if (bytesPerRequest > EMBLOCKSIZE) bytesPerRequest = EMBLOCKSIZE;
        else
        if (bytesPerRequest < 10240)
        {
              // Let an other client request this packet if we are close to completion and source is slow
              // Use the true file datarate here, otherwise we might get stuck in NNP state

              if (!requestedblocks_list.IsEmpty() && timeToFileCompletion < 30 && bytesPerRequest < 3400 && 5 * sourceDatarate < GetDatarate())
                {
                        DebugLog(_T("No request block given as source is slow and file near completion!"));
                        return false;
                }
                bytesPerRequest = 10240;
        }
#endif
        // END netfinty: Dynamic Block Requests


Thank you again for your great work! :+1:
0

#18 User is offline   Sydney 

  • Splendid Member
  • PipPipPipPip
  • Group: Members
  • Posts: 120
  • Joined: 27-September 02

Posted 07 March 2006 - 07:31 PM

Nice Mod. The GUI thing helps me a lot. :D
0

#19 User is offline   tHeWiZaRdOfDoS 

  • Man, what a bunch of jokers...
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5630
  • Joined: 28-December 02

Posted 07 March 2006 - 08:21 PM

@BlueSonicBoy: else isn't necessary but saves one comparison :flowers:
@Netfinity: just as a proposal: replace UPLOAD_CLIENT_DATARATE by 3072 because a lot of mods modify that variable and then the feature won't work correctly :lol:
0

#20 User is offline   Stulle 

  • [Enter Mod] Dev
  • PipPipPipPipPipPipPip
  • Group: Members
  • Posts: 5738
  • Joined: 07-April 04

Posted 07 March 2006 - 08:36 PM

tHeWiZaRdOfDoS, on Mar 7 2006, 08:21 PM, said:

@Netfinity: just as a proposal: replace UPLOAD_CLIENT_DATARATE by 3072 because a lot of mods modify that variable and then the feature won't work correctly :lol:
View Post

thought you don't like those "C&P mods"¿¡ why do you care, though¿ :confused:
I am an emule-web.de member and fan! Hate me or people will get suspicious about you! Ever wondered if it's all worth the trouble?

Posted Image

No, there will not be a new version of my mods. No, I do not want your PM. No, I am certain, use the board and quit sending PMs. No, I am not kidding, there will not be a new version of my mods just because of YOU asking for it!
0

  • Member Options

  • (5 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users