Nielsen//Net Ratings - Measuring Ratings - Criticism of Ratings Systems

Criticism of Ratings Systems

There is some public critique regarding accuracy and potential bias within Nielsen's rating system, including some concerns that the Nielsen ratings system is rapidly becoming outdated due to new technology like smartphones, DVRs, tablet computers, and Internet viewing. In June 2006, however, Nielsen announced a plan to revamp its entire methodology to include all types of media viewing in its sample.

Since viewers are aware of being part of the Nielsen sample, it can lead to response bias in recording and viewing habits. Audience counts gathered by the self-reporting diary methodology are sometimes higher than those gathered by the electronic meters which eliminate any response bias. This trend seems to be more common for news programming and popular prime time programming. Also, daytime viewing and late night viewing tend to be under-reported by the diary.

Another criticism of the measuring system itself is that it fails the most important criterion of a sample: it is not random in the statistical sense of the word. A small fraction of the population is selected and only those that actually accept are used as the sample size. In many local areas of the 1990s, the difference between a rating that kept a show on the air and one that would cancel it was so small as to be statistically insignificant, and yet the show that just happened to get the higher rating would survive. And yet in 2009 of the 114,500,000 U.S. television households only 25,000 total American households (0.02183% of the total) participated in the Nielsen daily metered system In addition, the Nielsen ratings encouraged a strong push for demographic measurements. This caused problems with multiple TV households or households where viewers would enter the simpler codes (usually their child's) raising serious questions to the quality of the demographic data. The situation further deteriorated as the popularity of cable TV expanded the number of viewable networks to the point that the margin of error has increased due to the sampling sizes being too small. Compounding matters is the fact that of the sample data that is collected, advertisers will not pay for time shifted (recorded for replay at a different time) programs, rendering the 'raw' numbers useless.

A related criticism of the Nielsen ratings system is its lack of a system for measuring television audiences in environments outside the home, such as college dormitories, transport terminals, bars, jails, and other public places where television is frequently viewed, often by large numbers of people in a common setting. In 2005, Nielsen announced plans to incorporate viewing by away-from-home college students into its sample. Internet TV viewing is another rapidly growing market for which Nielsen Ratings fail to account for viewers. Apple iTunes, Hulu, YouTube, and some of the networks' own websites (e.g., ABC.com, CBS.com) provide full-length web-based programming, either subscription-based or ad-supported. Though web sites can already track popularity of a site and the referring page, they can't track viewer demographics. To both track this and expand their market research offerings, Nielsen purchased NetRatings in 2007. However, noted in a February 2012 New York Times article the computer and mobile streams of a show are counted separately from the standard TV versions, further degrading the overall quality of the sampling data. As a result there was no way for NBC to tell if there was any overlap between the roughly 111.3 million standard TV viewers and 2.1 million live stream viewers the Super Bowl.

After Nielsen took over the contract for producing data on Irish advertising in 2009, agencies said that they were "disastrous" and claimed that the information produced by them is too inaccurate to be trusted by them or their clients.

In 2004, News Corporation retained the services of public relations firm Glover Park to launch a campaign aimed at delaying Nielsen's plan to replace its aging household electronic data collection methodology in larger local markets with its newer electronic People Meter system. The advocates in the public relations campaign charged that data derived from the newer People Meter system represented a bias toward underreporting minority viewing, which could lead to a de facto discrimination in employment against minority actors and writers. However, Nielsen countered the campaign by revealing its sample composition counts. According to Nielsen Media Research's sample composition counts, as of November 2004, nationwide, African American Households using People Meters represented 6.7% of the Nielsen sample, compared to 6.0% in the general population. Latino Households represent 5.7% of the Nielsen sample, compared to 5.0% in the general population. By October 2006, News Corp. and Nielsen settled, with Nielsen agreeing to spend an additional $50 million to ensure that minority viewing was not being underreported by the new electronic people meter system.

In 2011, CBS and Nielsen proposed a model consisting of six viewer segments which according to their empirical research are more relevant for advertisers than older models based on sex and age. The segments are based on user behavior, motivations, and psychographics. It is argued that the model can increase reaching the desired audience as well as message recall and advertisement likeability.