History
The invention of "robots.txt" is attributed to Martijn Koster, when working for WebCrawler in 1994. "robots.txt" was then popularized with the advent of AltaVista, and other popular search engines, in the following years.
Read more about this topic: Robots Exclusion Standard
Famous quotes containing the word history:
“Free from public debt, at peace with all the world, and with no complicated interests to consult in our intercourse with foreign powers, the present may be hailed as the epoch in our history the most favorable for the settlement of those principles in our domestic policy which shall be best calculated to give stability to our Republic and secure the blessings of freedom to our citizens.”
—Andrew Jackson (1767–1845)
“The history of mankind interests us only as it exhibits a steady gain of truth and right, in the incessant conflict which it records between the material and the moral nature.”
—Ralph Waldo Emerson (1803–1882)
“It would be naive to think that peace and justice can be achieved easily. No set of rules or study of history will automatically resolve the problems.... However, with faith and perseverance,... complex problems in the past have been resolved in our search for justice and peace. They can be resolved in the future, provided, of course, that we can think of five new ways to measure the height of a tall building by using a barometer.”
—Jimmy Carter (James Earl Carter, Jr.)