[emule_user_downunder]

This whole thread is full of utter misinformation. Data magically appearing from unconnected drives, unshared data being sent elsewhere, unwritten files being shared, and other such garbage.

Verified chunks are shared. You cannot share swap files as by their very nature they are dynamic and locked by the operating system. Data on a drive that is not live is not being shared, even though you can get to it via a sector editor. You cannot get to it via eMule. Unwritten secotrs by the very definition are not accessible and will not be shared.

eMule uses the underlying operating system for file I/O, therefore being bound and limited by its' rules. Unless you are working with corrupted files on a corrupted drive with cross-linked files, you shouldn't be able to access other data. If you are, it is highly likely your system will shortly crash anyway, and you will have more serious problems than 'data privacy', namely 'data loss'!

Windows doesn't overwrite sectors of data when deleting or opening files - only the file lookup tables. That is by design. Should you wanted heightened security to satisfy your paranoia, I suggest you look elsewhere for something more robust.

Can we give this a rest? Please?

[Tritonio]

I see several users coming very close the correct explanation for this. I'll offer what I know is the most precise explanation and a SOLUTION.

Windows offer several ways to create files. The normal one involves the program telling windows how large a file to create, windows finding empty space on the drive to allocate it, overwriting it with zeros, and then passing back the file handle so that the program can actually start writing to the file.

A second way involves the same procedure without overwritting the allocated sectors with zeros. This saves lots of time and drive wear but has a big security disadvantage. The file is created with random data in it. Random data that comes from files that used to be on the same sectors at some point in time (perhaps they got defragged or deleted by now, but the old data is still in those sectors and now belong to your newly allocated file). That data could be passwords and it could even be data belonging to ANOTHER user on your computer. Passwords that belong to your sister for example. The security risk is very big especially on multiuser systems since you have no control over what other users run so this kind of file creation is ONLY available to administrator users and programs ran with administrator rights.

There is a third way involving sparse files on NTFS systems which combines pros and cons from the two ways above. The allocation takes zero time, sectors are not allocated to the file until you actually need to write data to some point in the file when the allocation happens (AFTER doing the overwrite with zeros so no privacy leaks). This causes more fragmentation though. This way is not supported by eMule as far as I know.


Now, do you have to worry about the apparent privacy leak? No. When a remote user asks for a part of a file, eMule checks some sort of database to see if it has that part of the file. If that part is filed by private data, then it has never been downloaded, therefore the database will say it's unavailable. So eMule will never share such parts. To share a part it has first got to be written once with downloaded data and then marked on that database as available for sharing.

Of course this is how eMule is supposed to work. If you don't trust eMule then simply run it under a simple account. No compatibility options, no administrator priviledges. It will then be impossible for eMule to allocate sectors without overwritting them so new files will be a white page of bits and you'll have nothing to worry about.

Now if you run it as an unprivildged account and new files still contain private information, either you did something wrong, or your windows is totally broken or compromised since there is no way for an unpreviledged program to tell windows to do that kind of space allocation. (without the overwriting)

Hope that clears everything out.

This post has been edited by Tritonio: 08 September 2012 - 03:25 PM

[fox88]

Tritonio, on 08 September 2012 - 06:22 PM, said:

Windows offer several ways to create files. The normal one involves the program telling windows how large a file to create, windows finding empty space on the drive to allocate it, overwriting it with zeros, and then passing back the file handle so that the program can actually start writing to the file.

Basically Windows API provides only one way: CreateFile...() functions, which return handle to an empty (zero length) file when file is created. There is no space allocation and no writing.

File space allocation with Seek() function does not imply overwriting. For any specific data (zeros included) you must do explicit writing.

Tritonio, on 08 September 2012 - 06:22 PM, said:

If you don't trust eMule then simply run it under a simple account. No compatibility options, no administrator priviledges. It will then be impossible for eMule to allocate sectors without overwritting them so new files will be a white page of bits and you'll have nothing to worry about.

Since free drive space does not belong to anybody, that space could be allocated by any user/program - and privileges would not be an issue. Hence that SOLUTION would not help.

Tritonio, on 08 September 2012 - 06:22 PM, said:

Hope that clears everything out.

Probably it adds more confusion. I quote from above message:

emule_user_downunder, on 16 August 2012 - 08:39 AM, said:

Can we give this a rest? Please?

[Tritonio]

Fox88,

The seek() operation is something provided by C libraries. Windows don't offer such a fuction directly if I am not mistaken. To set the file size you can use one of the following functions: (along with SetFilePointer)
SetFileValidData
or
SetEndOfFile

The first one requires special privileges on the disk which normal users don't have by default. It allows you to allocate uninitialized space into the file which is immediatelly overwriten by zeros IF YOU WANT. Specifically it sets both the logical and the physical length of the file to the position you have placed the "seek" pointer. And it gives you the option to avoid zeroing the allocated sectors.

The second which is more common can be used by any user and allocates the space without overwriting it with zeros BUT it only increases the logical size of the file and not the physical size. This causes windows to return zeros when you read from the uninitialized sectors even though they contain past data. Now even if you write a single byte at the (logical) end of the file this will cause the physical size to rise up to the logical size and Windows will have to automatically fill the intermediate data with zeros, overwritting past data to avoid privacy leaks. So using that function you can never see past data in the file even though the overwriting isn't done at the moment of the space allocation.

It would be ridiculus even by Microsoft's standards to let all users run a function like SetFileValidData. It would pretty much make file permissions pointless if ANY user on a system can easily grab portions of data from the disk.

So my previous post doesn't actually add any confusion. It's Windows that automatically do the overwrite with zeros, not eMule. And you need to be an admin to tell Windows not to do that.

Refer to these too:
http://stackoverflow...th-setendoffile
http://www.bittorren...#diskio.no_zero

My only inaccuracy was to say that the allocation is done on file creation which is not true but I thought the details wouldn't really matter.

EDIT: And my solution will work. Just try it and if you'll find a way to still read data from past files you'll have found an amazing privacy leak in Windows that many hackers will be happy to hear about. But you won't unless you run an outdated OS.

This post has been edited by Tritonio: 08 September 2012 - 10:50 PM

[fox88]

Tritonio, on 09 September 2012 - 01:45 AM, said:

The seek() operation is something provided by C libraries. Windows don't offer such a fuction directly if I am not mistaken.

I did not write that seek() is Windows API function. The Visual Studio library translates seek() calls to SetFilePointer API functions.

Tritonio, on 09 September 2012 - 01:45 AM, said:

To set the file size you can use one of the following functions: (along with SetFilePointer)
SetFileValidData

MSDN tells SetFileValidData does not change file size; and this function is not called by eMule directly.

Tritonio, on 09 September 2012 - 01:45 AM, said:

SetEndOfFile
...
can be used by any user and allocates the space without overwriting it with zeros BUT it only increases the logical size of the file and not the physical size.

From MSDN: Sets the physical file size for the specified file to the current position of the file pointer.

Tritonio, on 09 September 2012 - 01:45 AM, said:

Now even if you write a single byte at the (logical) end of the file this will cause the physical size to rise up to the logical size and Windows will have to automatically fill the intermediate data with zeros, overwritting past data to avoid privacy leaks.

That's how .part files are allocated in eMule. It is the same for ordinary users and administrators.

Tritonio, on 09 September 2012 - 01:45 AM, said:

It would pretty much make file permissions pointless if ANY user on a system can easily grab portions of data from the disk.

There is a big difference between allocating unused space and accessing disk blocks at will; but that's a sidenote.

Tritonio, on 09 September 2012 - 01:45 AM, said:

And my solution will work.

Will it? The problem was: user found old data in .part files.
I use XP, have admin rights, and always see zero-filled .part files. Probably XP and later OS users do not need any solutions.

Tritonio, on 09 September 2012 - 01:45 AM, said:

But you won't unless you run an outdated OS.

According to MSDN, SetValidFileData exists since Windows XP/2003, while the topic starter used Windows ME and Windows 2000. Which means, your solution probably would not work for the OP too.

This post has been edited by fox88: 09 September 2012 - 07:15 PM