>When the television ratings come in each morning, there is often a lot of attention paid to a rating that might be up only one tenth of a rating point from the prior week or the prior year. (More recently, most ratings in most time periods have been trending down instead of up, which makes an uptick all the more noteworthy.) But TV ratings are now at such low levels generally, a one-tenth of a rating point change can translate to a very large percentage movement. A network show struggling under a 1 rating can boast a “14% increase!” if it moves from a 0.7 to a 0.8 rating, for example.
Maybe you’ve wondered how many people in the Nielsen sample represent a tenth of a rating point. In the all-important Adult 18-49 demographic, the answer is 22 people. Not even enough individuals to fill the roster of one Major League Baseball team or occupy all the seats on a city bus. On a recent day (April 19, 2012), the Nielsen People Meter sample had 22,015 Adults 18-49 “in tab” or in the sample and properly reporting data. The number of “installed” sample members is slightly higher, and the “in tab” number fluctuates from day to day due to equipment malfunctions or compliance issues. (If a sample member stops pressing buttons routinely as they watch television, a sample member is taken “out of tab” and contacted by the ratings service and coached to press their assigned button on the people meter remote control when they watch TV.)
So with 22,015 Adults 18-49 in the national sample, a 10.0 rating would mean 2,202 sample members were watching a particular program, while 220 sample members would yield a 1.0 rating. When we get down to 22 people moving a rating by one-tenth of a rating point, it is not hard to imagine a few people getting sick, some deciding to go out on a night when they usually watch TV, a couple working an extra shift, or a few others God forbid doing something else than watching a favorite show.
The numbers become even more volatile with smaller but still important demographic groups. Six (6) Men 18-34 in the Nielsen sample can move a rating in that demographic group by a tenth of a point, and only four (4) Teens 12-17 can do the same thing. It becomes even more treacherous if you look at cable networks with low coverage levels (like HBO). For example, 1,850 Men 18-34 in the national sample subscribe to HBO. So if you were tempted to compare ratings with young men for a couple of boxing matches, one or two young men could impact an HBO rating by a tenth of a point.
This does not mean you should distrust ratings or even other polls. It is merely a caution to not push the data too far. A one-tenth of a rating point change in one isolated week for one program really should be taken with a grain of salt. Look at a program’s ratings over the course of several weeks. If a show is falling a tenth of a rating point a week for five weeks in a row, something real is going on rather than a transitory blip that could very well be reversed the following week. Look at multi-week network averages. Larger trends are confirmed by repeated measurements, something we attempted to do with our recent review of American Idol and The Voice ratings over the course of a few seasons.
It might surprise you to hear that the Nielsen sample has never been larger. The 22,015 Adults 18-49 in the national sample live in 19,906 households (as of April 19), roughly four times the 5,000 household sample size usually associated with the People Meter panel for years. The current sample has been expanded rapidly so new households can be hooked up to measure traditional television viewing, DVR playback and online video viewing on computers and other devices. Also, in very large markets such as New York, Los Angeles and Chicago, households in the local people meter samples also deliver data to the national sample.
Is the ratings system perfect? Of course not. There is immediate bias introduced because people who would never consider allowing someone into their home to wire their televisions, cable/satellite boxes, and computers are excluded from the sample. There is panelist fatigue that sets in as a household stays in the sample for up to two years. People simply get tired of pressing a button on a remote to “check in” as they watch a show live or on their DVR. And some family members, especially young men and teenagers, have notoriously low willingness to even start pressing their buttons from the moment the household enters the sample. But the initial sample design, a random selection of every address in the United States (where everyone theoretically has an equal chance of being invited into the sample) is far superior to most online research, which is based on woefully incomplete sample frames. Further, any research methodology with a sample size around 20,000* is miles ahead of most public opinion polls (with sample sizes around 1,000) or most network programming research (a typical pilot is tested with fewer than 400 people total, and decisions are often based on sub-samples of under 100 people in those tests).
* Note: the sample size in the Nielsen People Meter panel is actually 50,368 when you look at all household members, Viewers 2+.
