[yonatan]

@Superlexx/all:
The 1GB test file is only for testing the new pre-release v01.00 which sends a proxy-ip with each OP_HTTPCACHEDBLOCK packet and is not compatible with webcachetest 00.06.

For testing 00.06 clients please use this one: ed2k://|file|Randomdata|100390638|FF7EA91FE4F75AE9022C50D12075F096|/

If anyone gets 'Received bad HTTP response' messages (in the verbose log) with 00.06, especially ones which don't contain a server error message, please let me know.

@doerrlamm:
Thanks for the mirror.

@everybody on t-online.de:
T-online's proxy has two names, www-proxy.btx.dtag.de and www-proxy.t-online.de. You will only get cached block info from clients using the same proxy name as you.

This post has been edited by yonatan: 01 July 2004 - 04:28 AM

[duffbeer]

Works well for me on the testfile (www-proxy.btx.dtag.de)

But sometimes I get some errors in the verbose log:

01.07.2004 12:25:42: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 12:25:42: Sent HTTP GET to proxy server
01.07.2004 12:25:42: Received bad HTTP response: HTTP/1.1 504 The requested document is not cached, and you indicated that you wanted the document only if it was in the cache.
01.07.2004 12:25:42: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Close


My Stats Graph

[jpkoester1]

Quote

Works well for me on the testfile (www-proxy.btx.dtag.de)

But sometimes I get some errors in the verbose log:

01.07.2004 12:25:42: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)

This line indicates that ababa downloaded a 180k block of the file you requested and sent you notification that that part of the file should be available at your proxy server

Quote

01.07.2004 12:25:42: Sent HTTP GET to proxy server

this line indicates that you sent a request for the 180k block to your proxy server

Quote

01.07.2004 12:25:42: Received bad HTTP response: HTTP/1.1 504 The requested document is not cached, and you indicated that you wanted the document only if it was in the cache.

well, this line is pretty self explanatory. the file you requested from the proxy server couldn't be found there. Could be many reasons for why this happens.

1. could be that ababa downloaded the file from someone using an emuleport that is not cached by the proxy (think Superlexx can clarify that for me). Solution to that would be to only allow http-connections between clients with ports that usually get cached by proxy servers.

2. T-online has several proxy servers. As we tested today requesting a file from one proxy server also checks all other proxy servers for the file. If all cached files are stored in a centralized index there might not have been enough time for proxyserver A to send that information to the central index. (anyone here that can offer some input on how different proxy servers of one ISP communicate with each other, especially how long it takes for proxyserver B to learn that proxyserver A cached a certain file?).
One solution we tried was to send the IP-adress of the exact proxy server where the file is stored. This did not alleviate the problem. Another solution might be to delay the proxy-download by a certain amount of time so the proxy servers all know about the cached file. Lastly we could just accept that some proxy downloads don't work while others will (off course that's not preferred). Anyways, we are looking into how to fix it

3. other reasons that we can do nothing about

Quote

01.07.2004 12:25:42: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Close

that's just the message that you disconnected from the proxy-server. (as you should because the file is not there)

Cheers
JP

[Superlexx]

Squid, the most popular proxy / web cache software on the net, has only the ports 80, 21, 443, 563, 70, 210 and the range 1025-65535 set to cachable per default. I could confirm this, traffic on port 81 wasn't cached for me, port 80 worked well.

Later, webcache-eMule will deny setting the server port to one of the blocked values, or warn the user that webcache will most likely not work with this setting

[prototyp]

hey i think its a test client with a test file, so why i see peoples with 3000 queues?

and we need more than 4-5 users

This post has been edited by prototyp: 01 July 2004 - 11:51 AM

[duffbeer]

OK, next status report:

I get some Timeouts on Proxy:

01.07.2004 14:53:54: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:53:54: Sent HTTP GET to proxy server
01.07.2004 14:53:59: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Timeout

And there seems to be parts sent I already have:

01.07.2004 14:55:13: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:55:13: OP_HTTPCACHEDBLOCK: Not a pure gap (180K block already downloaded) from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:55:19: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Timeout

But the disconnect is a few seconds later, maybe it has to be fixed with a prompt disconnect.

This post has been edited by duffbeer: 01 July 2004 - 01:06 PM

[jpkoester1]

Quote

OK, next status report:

I get some Timeouts on Proxy:

01.07.2004 14:53:54: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:53:54: Sent HTTP GET to proxy server
01.07.2004 14:53:59: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Timeout

Ok, this is a known bug. Currently the connection to the proxy server is not terminated after a proxy-block is completed. You should see the timeout message about 80 seconds after a 180k block from the proxy server is finished. Solution: Connection to the server needs to be disconnected after proxy-block is finished. (working on it)

Quote

And there seems to be parts sent I already have:

01.07.2004 14:55:13: OP_HTTPCACHEDBLOCK received from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:55:13: OP_HTTPCACHEDBLOCK: Not a pure gap (180K block already downloaded) from 217.229.*.* 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],NoNeededParts/None)
01.07.2004 14:55:19: Disconnected DownloadHTTPSocket, client: 0.0.0.0 '(null)' (,Downloading/None), reason: Timeout


But the disconnect is a few seconds later, maybe it has to be fixed with a prompt disconnect.

Ok, first to the "Not a pure gap" message:
The clients only know the completed chunks (thats the 9.28MB chunks) of each other. So there is ababa and you. lets say ababa is downloading chunk number 10. You already have part of chunk number 10, but you haven't got it completely yet.
ababa cannot know which 180k blocks of that 9.28MB chunk you don't have yet because it only sees that you don't have that chunk completed yet. So it sends you all 180k blocks it receives (of that chunk) as proxy sources (it doesn't send you all that data of course, just the link to the location on the proxy server where the data can be downloaded from). Your client then checks if it already has that 180k block. If you already have it it is not downloaded from the proxy and the "not a pure gap" message is displayed. (note that no connection to the proxy server is established -> see below)
There is currently no way to tell the other client exaclty which 180k block of each 9.28MB chunk you are missing so that message is as designed.

Now to the disconnect message (timeout). As I explained above that message is displayed approximately 80 seconds after a 180k block gets finished. It had nothing to do with the "not a pure gap" message. Actually if you see the "not a pure gap" message there isn't even a connection to the proxy server established.

Hope that cleared up those messages. If you have any more questions ask away and I'll do my best.

Cheers
JP

[Köppchen]

Sometimes the download of webcache stopps suddenly. When this happens there is no queue number, but also nothing is downloaded anymore from the actual user. I see the following lines in the log:

02.07.2004 20:31:00: Sent HTTP GET to x.x.x.x 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],Downloading/None)
02.07.2004 20:34:08: Sent HTTP GET to x.x.x.x 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],Downloading/None)
02.07.2004 20:34:08: Disconnected DownloadHTTPSocket, client: x.x.x.x 'ababa 06' (eMule v0.42g [WebCacheTest00.06y],Downloading/None), reason: Close

In the proxy log i can see that the second HTTP GET never reached the proxy.

What happens is the following:
The first HTTP Block ist transferred completely. Now emule tries to load the second block.
The HTTP GET request ist created and sent to the same socket used for the first request. But in the meantime the proxy disconnects this socket. When the sending thread tries to send the packet (with the second http get) it finds out that the socket is closed. Now the packet is lost. In HTTP 1.1 the server/proxy can use the same socket for many requests, but it can also disconnect.

HTH Markus

[geearf]

What is the order between normal exchange and exchange with the proxy ?

I mean is it something like t a guy with the same isp asks me something => proxy,

different isp => normal emule ?

[Superlexx]

it does not matter if the other client has same ISP. If he has a webcache set up in his webmule (and there is another client that can benefit from the webcached transfer - though I am not sure if it's already implemented) then it's proxy transfer.

client A, using ISP a, will not benefit from webcached download of another client B that is using ISP b. So for highest speed boost you should be customer of a "big" ISP.

[geearf]

i asked that cause we are trying it with a friend right now, on the same isp.


But as we did not empty our shared, we have to wait with the queues, and therefore i did wanted to know if emule protocol would be the one to handle it or the proxy,

is there anywhere to see if i am well connected to it or something ?

[Superlexx]

geearf, on Jul 2 2004, 10:40 PM, said:

is there anywhere to see if i am well connected to it or something ?

you mean the proxy? Set up your internet browser to use it; if you still can load web pages, then you are well connected.

In eMule, use one of the good ports as I said before. eMules TCP port is meant, nothing else.

You and your friend should be able to make and receive webcache sources. In your case it doesn't matter if your shares aren't empty, because it's not the case with the uploaders of the test file (you know, your download is other peoples upload ).

[geearf]

well i've created one file to test it also,
was i wrong ?

also my proxy is on port 3128
what do you call a good port ?

do you mean changing my default tcp ? if so already done long ago,

[Superlexx]

you should use the test file that is posted in this thread. Testing with just 2 clients doesn't make sence, you need at least 3 to see some effect: A uploads, B downloads through webcache, C downloads from the webcache.

It doesn't matter much, what port your webcache uses. Just enter it in the settings correctly. Don't set eMule default port too low (<1024) since only _some_ of the "small" ports are cached by the servers.

This post has been edited by Superlexx: 02 July 2004 - 09:22 PM

[Andu]

Guys eMule 0.43a has been released and it contains something called PeerCache. Now I haven't read into the details yet but it sounds exactly like the feature you are testing right now.

P.S.: After all the hard work I can imagine that you think that life suxx.

[Superlexx]

yes, we demand that you remove this feature until we aren't done with our. Otherwise we will DDoS you until you're on your knees and praying for forgiveness

This post has been edited by Superlexx: 02 July 2004 - 09:48 PM

[Andu]

Yea it's all my fault. I confess. I will right away remove the feature from the source I downloaded. Please don't Ddos me now.

[yonatan]

Superlexx, on Jul 3 2004, 12:48 AM, said:

yes, we demand that you remove this feature until we aren't done with our. Otherwise we will DDoS you until you're on your knees and praying for forgiveness

I agree. Peercache support must be removed IMMEDIATELY.

[geearf]

it seems the isp has to do something for peercache, therefore we won't see many use of it ?

If so we will still be needing webcache a lot

[Nils]

so for this mod the support for peer cache could be removed

Please continue development