History
Labor-intensive manual text mining approaches first surfaced in the mid-1980s,Template:Http://www.ppc.sas.upenn.edu/cave.htm but technological advances have enabled the field to advance during the past decade. Text mining is an interdisciplinary field that draws on information retrieval, data mining, machine learning, statistics, and computational linguistics. As most information (common estimates say over 80%) is currently stored as text, text mining is believed to have a high commercial potential value. Increasing interest is being paid to multilingual data mining: the ability to gain information across languages and cluster similar items from different linguistic sources according to their meaning.
The challenge of exploiting the large proportion of enterprise information that originates in "unstructured" form has been recognized for decades. It is recognized in the earliest definition of business intelligence (BI), in an October 1958 IBM Journal article by H.P. Luhn, A Business Intelligence System, which describes a system that will:
"...utilize data-processing machines for auto-abstracting and auto-encoding of documents and for creating interest profiles for each of the 'action points' in an organization. Both incoming and internally generated documents are automatically abstracted, characterized by a word pattern, and sent automatically to appropriate action points."
Yet as management information systems developed starting in the 1960s, and as BI emerged in the '80s and '90s as a software category and field of practice, the emphasis was on numerical data stored in relational databases. This is not surprising: text in "unstructured" documents is hard to process. The emergence of text analytics in its current form stems from a refocusing of research in the late 1990s from algorithm development to application, as described by Prof. Marti A. Hearst in the paper Untangling Text Data Mining:
For almost a decade the computational linguistics community has viewed large text collections as a resource to be tapped in order to produce better text analysis algorithms. In this paper, I have attempted to suggest a new emphasis: the use of large online text collections to discover new facts and trends about the world itself. I suggest that to make progress we do not need fully artificial intelligent text analysis; rather, a mixture of computationally-driven and user-guided analysis may open the door to exciting new results.
Hearst's 1999 statement of need fairly well describes the state of text analytics technology and practice a decade later.
Read more about this topic: Text Mining
Famous quotes containing the word history:
“The whole history of civilisation is strewn with creeds and institutions which were invaluable at first, and deadly afterwards.”
—Walter Bagehot (1826–1877)
“All history attests that man has subjected woman to his will, used her as a means to promote his selfish gratification, to minister to his sensual pleasures, to be instrumental in promoting his comfort; but never has he desired to elevate her to that rank she was created to fill. He has done all he could to debase and enslave her mind; and now he looks triumphantly on the ruin he has wrought, and say, the being he has thus deeply injured is his inferior.”
—Sarah M. Grimke (1792–1873)
“It may be well to remember that the highest level of moral aspiration recorded in history was reached by a few ancient Jews—Micah, Isaiah, and the rest—who took no count whatever of what might not happen to them after death. It is not obvious to me why the same point should not by and by be reached by the Gentiles.”
—Thomas Henry Huxley (1825–95)