Method
The developers of SET found that for if a particular piece of content has several different versions available for download from a P2P network, there may be enough similarity between the files in the different releases that they can all be used as a download source for a single version. In particular they found, (quoted from ):
- MP3 music files with identical sound content but different header bytes (artist and title metadata or headers from encoding programs) were 99% similar.
- Movies and trailers in different languages were often 15% or more similar.
- Media files with apparent transmission or storage errors differed in a single byte or small string of bytes in the middle of the file.
- Identical content packaged for download in different ways (e.g., a torrent with and without a README file) were almost identical.
SET uses a technique called handprinting - which is based on earlier techniques known as "Shingling" that have been used to filter junk e-mails - to seek out the files that contain similar chunks of data to those in the requested file. The SET system computes a handprint for each file, and can take chunks of data from files which are both identical and similar to the one being searched for. The lower similarity ranking that SET searches for, the more sources for that data are likely to be found. The authors claim that the extra overhead of locating these sources does not out-weigh the benefit of using them to help saturate the recipient's available bandwidth and that exploiting similar sources can significantly improve download time.
In tests, SET improved the transfer time of an MP3 music file by 71% and a 55Mb movie trailer went 30% faster using the researchers' techniques to draw from movie trailers that were 47% similar. SET could help most with less popular files, but it is not believed to improve transfer rates much for popular data, where there is already a huge set of people downloading it. Experiments suggest that in the other cases, SET can help a lot.
Note however, that SET can only improve download speed when the downloader's connection is not the bottleneck. This is more often the case for unpopular downloads.
Read more about this topic: Similarity Enhanced Transfer
Famous quotes containing the word method:
“Protestantism has the method of Jesus with His secret too much left out of mind; Catholicism has His secret with His method too much left out of mind; neither has His unerring balance, His intuition, His sweet reasonableness. But both have hold of a great truth, and get from it a great power.”
—Matthew Arnold (1822–1888)
“The method of scientific investigation is nothing but the expression of the necessary mode of working of the human mind. It is simply the mode in which all phenomena are reasoned about, rendered precise and exact.”
—Thomas Henry Huxley (1825–95)
“The country needs and, unless I mistake its temper, the country demands bold, persistent experimentation. It is common sense to take a method and try it. If it fails, admit it frankly and try another. But above all, try something. The millions who are in want will not stand idly by silently forever while the things to satisfy their needs are within easy reach.”
—Franklin D. Roosevelt (1882–1945)