History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“The only thing worse than a liar is a liar that’s also a hypocrite!
There are only two great currents in the history of mankind: the baseness which makes conservatives and the envy which makes revolutionaries.”
—Edmond De Goncourt (1822–1896)
“History, as an entirety, could only exist in the eyes of an observer outside it and outside the world. History only exists, in the final analysis, for God.”
—Albert Camus (1913–1960)
Related Phrases
Related Words