How accurate are Nielsen ratings?

No one really knows.

There are statistical models to support their validity. They’ve been doing it a long time and have revised and improved methods over the last fifty years. But still – one schlep in Clovis, New Mexico who hates Amy Poehler because she reminds him of a girl who once dumped him could represent 2,000,000 viewers.

With more and more people getting cable boxes or satellite boxes it’s easier to track just what is being watched. But those still don’t tell you WHO is watching. How many are in the room? What are their ages? How many left the room after the 15th vagina joke on 2 BROKE GIRLS (so five minutes in)?

And then there is the issue with shows on the DVR. How long do you give someone to watch a program before it no longer counts? Nielsen has a formula but is it valid or just arbitrary?   Last week I watched a sitcom episode that I had DVR’d and noticed there were Christmas commercials in it. I’m sure the sponsors could care less that I finally got around to it. And just because someone records a show doesn’t mean he’ll ever watch it.

And if you record a show and don’t watch it for weeks, is that your pattern or you were just on vacation?

How many viewers like a show but won’t watch it on the air, preferring to wait for the DVD to come out so they can watch the whole season at once? So you’re big fans of the show but Nielsen doesn’t know that.

But wait! There’s more!

Watching content on-line. That’s become even more popular because there’s that much more television being streamed and with inexpensive devices like AppleTV you can watch these shows on your big screen TV.  And don't forget about people now watching shows on their iAnythings.  The A.C. of A.C. Nielsen stands for “Aw crap!”

And yet, all programming and advertising decisions are based on the Nielsen numbers.  Yes, it's a crooked card game but it's the only game in town. 

Radio had a similar problem. Ratings were taken via diaries issued to listeners. On good faith, you were expected to keep a detailed log of what station you listened to, when you turned it on and when you turned it off. So essentially it charted the listening habits of diligent people.

And then a few years ago the People Meter was introduced. This is a little device that looks like a beeper. You just take it with you. It picks up and documents any radio signal you hear. How accurate is this an indicator? You could be in your car, stopped at a light for two minutes and the clown next to you has Eminem blasting. The People Meter hears it and registers that you -- a 78 year-old white lady from Beverly Hills --  listens to a hip hop station.

Still, it’s a far more accurate system than the diaries. And interestingly, the results when they started using People Meters were very different from the diary input. Big case in point: the oldies format.

Before People Meters, the oldies format was out of favor. Lots of station that had been playing oldies for decades saw their numbers dwindle and bailed on the format. Even WCBS-FM in New York, maybe the country’s number one oldies station, abandoned the format for the ill-advised “Jack” format.

Then People Meters came along and what do you know? Not only did listeners still listen to oldies stations, there were way more of them and listened way longer than anyone had ever thought. WCBS-FM is back to playing the Beatles. In some markets, the oldies station -- once thought passe -- is the number one station in town.  That's a big discrepancy between diaries and People Meters. 


Might the same be true of television? If an accurate measurement could be made, might we find that shows we thought no one was watching actually had decent numbers? How many series got cancelled that should have been given another chance? Is it possible that even fewer viewers watch WHITNEY than NBC thinks and they renewed a certified bomb?

This always reminds me of the chain gang in football. These are the guys who stand off on the sidelines with the yardage markers. When a ball is close to a first down the chain gang is summoned out to the field. Two guys run out, they place the markers down and rule. Is the nose of the ball over the line? We’re talking inches here. Entire games sometimes depend upon these calls. And two nudniks run out to the field, not even in a straight line, and their word is final. Huh? In this day and age of high technology they can’t install lasers that accurately tell you whether the football has crossed the first down plane?

And the NFL could install this hardware. At least TV has an excuse – they don’t have such sophisticated methods of calibration. Damn Steve Jobs for dying before he could solve this problem too!

In the meantime, the livelihood of thousands of people in the television and advertising industry still depend on the two nudniks zig-zagging their way to mid-field.