Volume 7 - Issue 3
Advanced distributed crawling system for KAD network
Abstract
Many distributed hash tables have been proposed recently, but only very few of them have been applied actually. KAD, a Kademlia based DHT, is only an exception which is widely used in eMule peer-to-peer system. The measurement of the network will be able to guidance for the designing of such systems. In order to make the results of measurement more accurate, it is essential to develop a high-performance crawler. In this paper, we introduce a new distributed KAD crawler and apply a new condition for stopping crawling, so that the crawler can collect information faster and more completely. On the basis of this, a crawler is developed and full KAD ID space is crawled. Then on the basis of statistical analysis, we have found some interesting phenomena.
Paper Details
PaperID: 79953753928
Author's Name: Wu, Q., Chen, X.
Volume: Volume 7
Issues: Issue 3
Keywords: DHT, Distributed crawler, KAD network, Network measurement
Year: 2011
Month: March
Pages: 677 - 684