History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“The greatest horrors in the history of mankind are not due to the ambition of the Napoleons or the vengeance of the Agamemnons, but to the doctrinaire philosophers. The theories of the sentimentalist Rousseau inspired the integrity of the passionless Robespierre. The cold-blooded calculations of Karl Marx led to the judicial and business-like operations of the Cheka.”
—Aleister Crowley (1875–1947)
“In the history of the human mind, these glowing and ruddy fables precede the noonday thoughts of men, as Aurora the sun’s rays. The matutine intellect of the poet, keeping in advance of the glare of philosophy, always dwells in this auroral atmosphere.”
—Henry David Thoreau (1817–1862)
“When the coherence of the parts of a stone, or even that composition of parts which renders it extended; when these familiar objects, I say, are so inexplicable, and contain circumstances so repugnant and contradictory; with what assurance can we decide concerning the origin of worlds, or trace their history from eternity to eternity?”
—David Hume (1711–1776)