[SlugFiller]

From now on, all subsequent test clients will be available via these solid mirrors:
binary | source
You can also view these mirrors and the current test version number from the SF-IOM main website.

Anouncements about new test versions available will be made in this thread.

[SlugFiller]

A new test version, #12, is available for download(see first post).

New:
-Fixed crash bug in upload bandwidth throttler(Thanks, Firnus)
-AR fixes. Should no longer crash.

[Meuh6879]

#12 on test for 24h

Crash

-crash report sended...heuuu...VBScript error ? (on the main page to send the .ZIP report)

-on search section, when i open the contextual menu on a file and choose "details", the eMule crash

Feedback 24h



eMule v0.46c BETA SlugFiller-itsonlyme v0.42a Statistics

Upload Sessions: 744
Total successful upload sessions: 649 (87.23%)
Total failed upload sessions: 95 (12.77%)
Average Uploaded Per Session: 5.37 MB
Average upload time: 20:23 Minutes
Total remaining upload on premature disconnections: 2.09 GB
Average remaining upload per session: 3.38 MB

Download Sessions: 2288
Successful Download Sessions: 1755 (76.7%)
Failed Download Sessions: 533 (23.3%)
Average Downloaded Per Session: 5.05 MB
Average Download Time: 25:08 Minutes

General
Reconnects: 1
Active Connections (estimate): 151 (Half:5 | Compl:46 | Other:100)
Average Connections (estimate): 147
Peak Connections (estimate): 501
Max Connection Limit Reached: 89 : 24/10/2005 18:09:00

Clients
Known Clients: 4444
Clientsoftware:
Network
Port
Low ID: 728 (16.4%)
Secure Ident (OK : Failed ): 4262 (99.6%) : 18 (0.4%)
Problematic: 0 (0.0%)
Banned: 11
Filtered: 204

This post has been edited by Meuh6879: 24 October 2005 - 04:09 PM

[leuk_he]

@ slug you might want to take a lok at this (morph=stullemule) bug report:

http://forum.emule-p...ndpost&p=669243

since the code is tagged:

// SLUGFILLER: SafeHash - one is enough

MAybe the possibilty that a part.001.met file can exists in multiple disrs in 0.46c should be taaken into account?

[SlugFiller]

This is already taken care of since the first 46c port. I've modified the code to match the new multi-tempdir. It seems Morph and StulleMule simply forgot to update to the latest version.
First, my function looks like this:

CPartFile* CDownloadQueue::GetFileByMetFileName(const CString& rstrPath, const CString& rstrName) const

Second, the code in question looks like this:

	for (int i=0;i<thePrefs.tempdir.GetCount();i++) {
  // SLUGFILLER: indexPartFiles - create an index for each tempdir
  CPartFileIndex* partfileindex = new CPartFileIndex(thePrefs.GetTempDir(i));
  m_partfileindex.Add(partfileindex);
  // SLUGFILLER: indexPartFiles
  CStringList metsfound;	// SLUGFILLER: SafeHash - ensure each met is loaded once per tempdir

  	// SLUGFILLER: SafeHash - one is enough
  	if (metsfound.Find(CString(ff.GetFileName()).MakeLower()))
    continue;
  	metsfound.AddTail(CString(ff.GetFileName()).MakeLower());
  	// SLUGFILLER: SafeHash

Aside from counting files per temp directory, it prevents the index file recovery being triggered twice - once for a corrupt file, and once for a corrupt backup(If you clicked "No" it means no).
Though, admittedly, it might be nicer to test the backup before resorting to the index, but the index is more reliable anyway.

@Meuh6879:
If you can, please mirror the crash dump manually. The website doesn't exactly give proper notification for new crash reports, and it's hard to sort them out there. It's not exactly the most organized crash report script.

On a side note, I've been messing with the official crash reporter, and may pull the plug on crashRpt in the next version. That said, I might also provide a mirror to dbghelp.dll v6, since it's needed in order to produce unicode crash dumps on 9x.

Meanwhile, I'm trying to pinpoint why the current test version crashs under stress. I'm stress-testing it in the hopes of producing good dumps. So far it's been working so-so.
Once I get the crashs to stop, I'd be able to deliver one hell of a new version.

[leuk_he]

SlugFiller, on Nov 29 2005, 10:16 PM, said:

This is already taken care of since the first 46c port. I've modified the code to match the new multi-tempdir. It seems Morph and StulleMule simply forgot to update to the latest version.

CPartFile* CDownloadQueue::GetFileByMetFileName(const CString& rstrPath, .....

Aside from counting files per temp directory, it prevents the index file recovery being triggered twice - once for a corrupt file, and once for a corrupt backup(If you clicked "No" it means no).
Though, admittedly, it might be nicer to test the backup before resorting to the index, but the index is more reliable anyway.

Misread the scope of metsfound and now i understand that it is in it. Thx for your reply. The Cparfileindex thingy is not in morph, and it not clear to me (yet) what has to be recovered with it.

[SlugFiller]

The CPartFileIndex saves a backup of the main data(hash, size, name, hashset, AICH) of all part.met files in a temp folder in a single file called downloads.met.
When eMule is started, if a met file fails to load, or one is missing from the directory while being mentioned in downloads.met, eMule pops up a dialog asking you if you want to restore it, and if you click "Yes" it recreates the part.met file from the data in downloads.met.

As you can see, with the addition of multiple temp dirs, there is a CPartFileIndex for each temp dir, which means a downloads.met for each temp dir(previously, it was in config).
Note that this means it might detect moving a temp file between temp directories(outside of eMule) as a corruption, and ask to recover the old one(just click "No").

The most important aspect of it is that downloads.met, unlike the other files, isn't saved hundreds of times, but only when it really needs to be updated(and even then, it doesn't exactly cover all cases), so the chances of it being corrupted due to an OS crash or a power failure are very low.

[leuk_he]

SlugFiller, on Nov 30 2005, 10:45 PM, said:

The CPartFileIndex saves a backup of the main data(hash, size, name, hashset, AICH) of all part.met files in a temp folder in a single file called downloads.met.

Thx, i understand now. (and i understand why morph does not have it )

How would you liketo name this recovery feature, so i can document it in the
http://www.fileshari...hp/English:List

(or you are free to edit the wiki as well of coarse.)

[Firnus]

leuk_he, on Dec 1 2005, 09:35 AM, said:

SlugFiller, on Nov 30 2005, 10:45 PM, said:

The CPartFileIndex saves a backup of the main data(hash, size, name, hashset, AICH) of all part.met files in a temp folder in a single file called downloads.met.

Thx, i understand now. (and i understand why morph does not have it )

How would you liketo name this recovery feature, so i can document it in the
http://www.fileshari...hp/English:List

(or you are free to edit the wiki as well of coarse.)

SPAM !!!

Quote

[10] Safe Hash(Tag: "SLUGFILLER: SafeHash"):
* The ultimate fix against sharing corrupt downloads, or missing or having corrupt hashlists. See details below.
* Part file completion is done regardless of currently hashing known files.
* Temp files and their backups can't be shared, even if the temp directory is shared. Only temp files actually used by eMule and their backups are filtered, so other files in the directory can still be shared.
* Config files can't be shared, even if the config directory is shared. Only config files actually used by eMule are filtered, so other files in the directory can still be shared.
* Only complete chunks of part files are rehashed at startup, instead of the entire part file.
* If eMule is closed while a part file is hashing, it will now rehash at startup, instead of assuming it's been hashed.
* All part files hashing, including part finishing and ICH, are now done in another thread.
* Even though multiple files can be hashed simultaniously, only one chunk is hashed at the same time.
* Only hashed and verified chunks are shared. Completed chunks are shared only after they are fully hashed. File previewing also checks that the chunks are hashed, and not just complete.
* When a hashlist is recieved the part file is instantly hashed with it. Each already completed chunk is shared as soon as they are hashed.
* A part file doesn't wait for the initial hashing to complete before looking for sources and starting to download. It's only shared, though, after it's hashed.
* Only blocks which are currently gaps can be written to, data that has already been downloaded is discarded. This prevents finished blocks from being corrupted by overflowing data.
* Better handling of part files with less than one chunk. Prevents alot of bugs, including a bug which could allow such a file to finish without it's hash being checked.
* The shared files are loaded last, so that the part files are already loaded and can be shared when the shared files first load. Also, when the shared files are sent to the server, the part files are sent in the same packet, instead of being sent one by one afterwards.
* All directories can be shared. Unsharing of eMule-used files is done per-file.
* When loading met files, the type of the tags are now checked, rather than just the tag names, to prevent errors caused by tag type mismatch.
* A known file without a filesize tag, or with a filesize tag set to 0 is no longer loaded, and is considered as an invalid entry.
* Fixed a bug which could cause a part file to remain without a hashset. Now the hashset requests are monitored and have timeouts.
* Fixed a bug when loading a backed up part file which would cause several errors during hashing and other operations.
* The download queue will now only load one part file instance per part.met filename. Prevents a backup from colliding with a valid or recovered met file.
* Fixed a bug caused by the first time wizard opening before eMule is fully loaded.
* Fixed a bug where eMule might start working before it finished loading.
* Reloads shared files whenever a requested files found in the shared files is not found on the harddrive.

I'm not sure there is no modification in last release but maybe we can wait SF-IOM 46c final, no ?

[leuk_he]

Firnus, on Dec 1 2005, 10:44 AM, said:

leuk_he, on Dec 1 2005, 09:35 AM, said:

SlugFiller, on Nov 30 2005, 10:45 PM, said:

The CPartFileIndex saves a backup of the main data(hash, size, name, hashset, AICH) of all part.met files in a temp folder in a single file called downloads.met.

Thx, i understand now. (and i understand why morph does not have it )

How would you liketo name this recovery feature, so i can document it in the
http://www.fileshari...hp/English:List

(or you are free to edit the wiki as well of coarse.)

SPAM !!!

Some would call it spam. Some would call it promotion to a documentation project.

But after reading it twice i still miss the description of the download.met mechanism. in the safe hash stuff

[SlugFiller]

It's already in the feature list:

Quote

* eMule now keeps index of all current downloads. If one of the download files becomes corrupt, or is unexpectedly deleted, or doesn't match the index, eMule offers to restore the download from the indexed backup. The index is saved only when it's changed, to reduce chances of it becoming corrupt(Tag: "SLUGFILLER: indexPartFiles").

Still, the feature list descriptions are a bit trimmed down. I've already expanded on a couple of my features, where I thought the information in the wiki was misleading.
You can feel free to include a more elaborate description to the feature, I think it should be obvious to you how it works.
If not, try deleting/renaming one of your part.met files, then run eMule and see for yourself.

Also, here's a little trick for using that feature to recover corrupt downloads from the official or from partial downloads made with another program.
Note the age of that post, and it should give you a clue as to how old this feature is.

Finally, why is SafeHash posted as "NiceHash"? These are two utterly seperate features, with the latter also being the newer one.

I mean this line:

Quote

Run the diskintensive (complete file) hashing task in a other lower priority task.

Doesn't exactly describe SafeHash. In fact, one of SafeHash's features is doing the exact opposite - running it in normal priority instead of low.
Other than that, it's mostly a hashset maintanance feature, and should be described as such. I would think it deserves it's own section, rather than be stuffed as part of a lesser feature.

When it does have it's own section, I would be happy to write a complete description for it, rather than a mere bullet-list.

[leuk_he]

SlugFiller, on Dec 1 2005, 02:53 PM, said:

Quote

Run the diskintensive (complete file) hashing task in a other lower priority task.

Doesn't exactly describe SafeHash. In fact, one of SafeHash's features is doing the exact opposite - running it in normal priority instead of low.
Other than that, it's mostly a hashset maintanance feature, and should be described as such. I would think it deserves it's own section, rather than be stuffed as part of a lesser feature.

When it does have it's own section, I would be happy to write a complete description for it, rather than a mere bullet-list.

Well you are really free to edit the wiki! I made an list item for you. YOu might want to create an account (to hide your ip) but it is not even required.

http://www.fileshari...glish:Safe_Hash

http://www.fileshari...Index_partfiles

[SlugFiller]

This morning I had to ask myself a question I would have otherwise never thought I would ask: "Do I suck at math or something?"
So anyway, after about 30 seconds of coding and several minutes of compiling, I've finally produced a numerically stable rand-queue. Just a simple matter of replacing the exponential ratios with logarithmic random values.

Long story short, #13 is now available. It has refresh shared files re-enabled, RandCS added, and various other fixes and improvements.

With this new-found stability, this version border-lines with release candidate. It's possible that within another version or two, there will actually be a release.

[LeGaulois]

Great news for your french fans.
I test it.

Thx.

[Mithgol]

SlugFiller, on Dec 3 2005, 06:51 PM, said:

Long story short, #13 is now available. It has refresh shared files re-enabled, RandCS added, and various other fixes and improvements.

Will it be possible for user to disable shared files and/or RandCS, if he (or she) needs to?

[SlugFiller]

If you don't like refresh shared files, just set it to 0.

If you don't like RandCS, I'm afraid you'll have to turn off rand-queue. The reason is that rand-queue, on it's own, like any other random process, produces a gaussian curve of probabilities around the desired results. RandCS narrows it down to nearly a pillar. This is necessary for using rand-queue.

[Firnus]

#13 for a while , no crash with the same configuration than before ,

but a lot of memory cusumption.

I use probalistic queue AND the AR at 20% , usage of CPU is between 33% and 40%.

testing without AR to see .

Thank you !!!

[Enig123]

@SlugFiller, I noticed the known part file rehashing problem happened only when quit with downloading. Besides, since the rehashing still happened when starting other version of emule, there's nothing wrong with that part of code in SF-IOM. It clearly shows that the problem is related to the buffer flush when quiting.

Since introduction of rand queue, I wonder why the credit file was not compatible with 0.30 style. Back-compatibility is quite important for letting more people to use it.

Edit: I got a weird error message:

2005-12-22 15:11:47: ERROR while saving part file: 001.part.met (xxxxxx.avi) - File exists

which may have something to do with the rehashing problem.

Corresponding log messages are:

2005-12-22 15:11:47: Failed to remove "D:\Program Files\eMule\Temp\001.part.met" - 敐浲獩楳湯搠湥敩d:\Program Files\eMule\Temp\001.part
2005-12-22 15:11:47: Failed to move temporary part.met file "D:\Program Files\eMule\Temp\001.part.met.backup" to "D:\Program Files\eMule\Temp\001.part.met" - 楆敬攠楸瑳s湥敩d:\Program Files\eMule\Temp\001.part

This post has been edited by Enig123: 22 December 2005 - 07:22 AM

[SlugFiller]

Quote

I noticed the known part file rehashing problem happened only when quit with downloading.

If it's while a chunk is being completed, this is to be expected. It's possible that the chunk hashing thread is running while eMule is quitting, causing the file to become "untrusted". It would then be rehashed on the next start-up.
This is not an issue, since the rehash only checks completed chunks anyway, and won't effect downloading in any way.
A good way to avoid these rehashs altogether is not to close eMule while a download is active.

Quote

Since introduction of rand queue, I wonder why the credit file was not compatible with 0.30 style.

Rand-queue is optional. SUQWT isn't. Besides, the new credits format has more advantages to it than just SUQWT. I'm hoping to get the official devs to use it, rather than the other way around.

Quote

I got a weird error message

Were you previewing the file at the time, or something? Was it open by some program? When was the last time you ran scandisk?

[Enig123]

SlugFiller, with your mod test 13, RandQueue enabled, there's strange thing happened. I was downloading 3 hot files, one file (The first download one) have 2.6 GB upload while the other two have almost nothing, like the effect of reward uploaders gone away. Does it behave as expected?

In my opinion this feature has no good for end users (almost no rewarding for later downloading file?), that's why it was disabled by default, right?

In the feature list of SF-IOM, it read that RandQueue was designed for releasers. Could you please explain in detail about the aim and algorithm behind the idea of RandQueue? Is there some other options work together with RandQueue that could make it fit for end users?

Regards,

This post has been edited by Enig123: 23 December 2005 - 05:25 AM