Accuracy
Making a meaningful comparison of the accuracy of different speech analytics systems can be difficult. The output of LVCSR systems can be scored against reference word-level transcriptions to produce a value for the word error rate (WER), but because phonetic systems use phones as the basic recognition unit, rather than words, comparisons using this measure cannot be made.
When speech analytics systems are used to search for spoken words or phrases, what matters to the user is the accuracy of the search results that are returned. Because the impact of individual recognition errors on these search results can vary greatly, measures such as word error rate are not always helpful in determining overall search accuracy from the user perspective.
Measures such as precision and recall, commonly used in the field of information retrieval, are typical ways of quantifying the response of a speech analytics search system. Precision measures the proportion of search results that are relevant to the query. Recall measures the proportion of the total number of relevant items that were returned by the search results. Where a standardised test set has been used, measures such as precision and recall can be used to directly compare the search performance of different speech analytics systems.
These measures of accuracy can be illustrated by the following example. Imagine a user searches a set of audio files for a specific phrase, and the search returns 10 files. If 9 of the 10 search results do in fact contain the search phrase, the precision is 90% (9 out of 10). If the total number of files that actually contain the phrase is 18 then the recall is 50% (9 out of 18).
Read more about this topic: Speech Analytics
Famous quotes containing the word accuracy:
“U.S. international and security policy ... has as its primary goal the preservation of what we might call “the Fifth Freedom,” understood crudely but with a fair degree of accuracy as the freedom to rob, to exploit and to dominate, to undertake any course of action to ensure that existing privilege is protected and advanced.”
—Noam Chomsky (b. 1928)
“In everything from athletic ability to popularity to looks, brains, and clothes, children rank themselves against others. At this age [7 and 8], children can tell you with amazing accuracy who has the coolest clothes, who tells the biggest lies, who is the best reader, who runs the fastest, and who is the most popular boy in the third grade.”
—Stanley I. Greenspan (20th century)
“Such is the never-failing beauty and accuracy of language, the most perfect art in the world; the chisel of a thousand years retouches it.”
—Henry David Thoreau (1817–1862)