Gnutella Crawler

A Gnutella crawler is a software program used to gather statistic information on the gnutella file sharing network, such as the number of users, the market share of different clients and the geographical distribution of the userbase.

Early crawlers used the Ping/Pong messages to discover hosts connected to the network. Although this method is still usable, it is too slow to capture enough data for a topological overview of the gnutella network as it requires initiating full gnutella connections; this involves several roundtrips to perform the header processing. An extension has been added to the gnutella protocol to allow crawlers to quickly access ultrapeers. Right now, there is no public accessible crawler online on the gnutella network, since the last one hosted by Lime Wire LLC has been taken down.

Gnutella2 (G2) also supports crawlers for the gain of statistical data such as the network size or the network composition (clients, versions, usernames and usercountries). Right now, there is only one crawler existing on the G2 network, called g2paranha. It is written and maintained by dcat and licensed under the GPL.