History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“The thing that struck me forcefully was the feeling of great age about the place. Standing on that old parade ground, which is now a cricket field, I could feel the dead generations crowding me. Here was the oldest settlement of freedmen in the Western world, no doubt. Men who had thrown off the bands of slavery by their own courage and ingenuity. The courage and daring of the Maroons strike like a purple beam across the history of Jamaica.”
—Zora Neale Hurston (1891–1960)
“The second day of July 1776, will be the most memorable epoch in the history of America. I am apt to believe that it will be celebrated by succeeding generations as the great anniversary festival. It ought to be commemorated, as the day of deliverance, by solemn acts of devotion to God Almighty. It ought to be solemnized with pomp and parade, with shows, games, sports, guns, bells, bonfires and illuminations, from one end of this continent to the other, from this time forward forever more”
—John Adams (1735–1826)
“Don’t give your opinions about Art and the Purpose of Life. They are of little interest and, anyway, you can’t express them. Don’t analyse yourself. Give the relevant facts and let your readers make their own judgments. Stick to your story. It is not the most important subject in history but it is one about which you are uniquely qualified to speak.”
—Evelyn Waugh (1903–1966)