[ducho]

I updated from 0.60d to 0.70a/b recently. I noticed a huge disk activity, around 1.2 MB/s sustained write in my temporary folder.
Upon investigation, it was determined that writing .part.met and .part.met.bak files was the culprit.

I would say that the bigger issue is update .part.met and .part.met.bak files without any change to .part itself.
I do believe that we could trigger updates to .part.met and .part.met.bak only after some change to .part file.

There is also a lot of refactoring, i.e. CPartFileWriteThread, and more updates that I didn't look at.
I need to find time to update libraries to compile 0.70b to experiment.

This post has been edited by ducho: 10 October 2024 - 03:30 AM

[fox88]

What was download rate while writing at 1.2 MB/s?

[ducho]

Download rate was zero. If something was downloading sporadically I am sure it didn't go higher than 50 KB/s.
I have 1225 files on download list,
then I have 1225 x 2 (.met & met.bak) rewritten in 62 seconds - this value from comparing timestamps.

Total size of 2450 files is 461,916 bytes o.O
I can start emule again and maybe it is possible to measure more accurately how many times per second each file is written.

This post has been edited by ducho: 10 October 2024 - 09:41 PM

[Enig123]

Current community version calls FlushBuffer() regularly even if there's no data in the buffer.

	// If buffer size exceeds limit, or if not written within time limit, flush data
	if (m_nTotalBufferData > thePrefs.GetFileBufferSize() || curTick >= m_nLastBufferFlushTime + thePrefs.GetFileBufferTimeLimit())
		FlushBuffer();

should be like

	if (m_nTotalBufferData == 0)
		m_nLastBufferFlushTime = curTick;
	else
		// If buffer size exceeds limit, or if not written within time limit, flush data
		if (m_nTotalBufferData > thePrefs.GetFileBufferSize() || curTick >= m_nLastBufferFlushTime + thePrefs.GetFileBufferTimeLimit())
			FlushBuffer();

[fox88]

This is questionable for official, and unacceptable for 0.70.

[Enig123]

What's not acceptable for this kind of practical optimizations of avoiding unnecessary disk ios?

I found no obvious bugs regarding to similar logic that has been applied to my mod for quite several years.

I just checked the .part.met files and found no change since previous time eMule started to run for those without download data in current session. Actually I am thinking about even further optimization by not re-writing .part.met files unless the downloading file has actually downloaded something in this session.

This post has been edited by Enig123: 17 October 2024 - 12:38 AM

[ducho]

I did run again for 29 minutes.

Only 2 files, $LogFile and $Mft had together 0.69MB/s writing, measured with Procmon. Total disk writings were double than that.
This makes sense as small files are written directly on MFT.

Is it useful to present additional data?

[ducho]

So I moved back to 0.60d to try to reproduce the issue with 0.70.
All part files have been recreated - it looks like 0.70 has a new format. I didn't see any mention in release notes.

So 0.60d is online and disk activity is negligible while eMule is idling waiting for connections.

[fox88]

First of all, 1250 active downloads is orders of magnitude above recommended numbers (most should be stopped, and enabled option to start next when download completed).
On the other hand, it helped to disclose a problem.

Things should be improved in the next release, and thanks for the report.