Thursday, January 22, 2009

How I really feel about testing

Aloha. Winging to Hawaii. But assuming they have the internet over there my blog posts will continue uninterrupted. I just won’t hate everything as much.
Friday questions comin’ at ya:

Richard Y asks:

A 3 Episode ARC. Is 'arc' an abbreviation or an acronym? If either, what does it spell out to be?

Writers use it describe a storyline that stretches over two or more episodes. One season on CHEERS our yearlong story arc was that Sam was trying to get into Rebecca’s pants. Throughout the course of the episodes he tried every sleazy, lying, despicable tactic he (us) could think of. NBC did research testing on that season and Sam tested way higher than anyone else. Why? Because he “cared” about everybody in the bar and was the father figure everyone could trust. Huh???? What shows were THEY watching???

From Damian in California:

I have a question regarding those preview shows such as “CBS Television City” at the MGM Grand in Las Vegas. Do they help you at all?

We saw about 3 of them.
1 was great – Big Bang Theory
1 was terrible – never made it to TV
1 was so bad we left half way through (they claimed it was kid appropriate, so our kids were watching it with us, but actually they had changed the TV show that day from a Discovery family friendly show to some trashy adult show)

CBS decided that they get a much better cross-section of Americans in Vegas than LA. And they’re probably right. Just walk through the MGM Grand casino, check out the clientele, and you’ll know why THE WIRE is not a big hit.

Here’s the bottom line with research testing: If it is used as a tool, another source of input then yes, it is valuable. All feedback is valuable. But if it becomes the final determination on whether your show is picked up or not, or if it dictates the direction your show must go, then it is very destructive. How do you measure creativity and ideas? Executives rely on research because they’re covering their asses. But when ALL the shows that get picked up test well and 90% of them fail anyway then doesn’t it stand to reason that this is a faulty system?

And finally, from Erich Eilenberger:

I remember that you and your partner wrote one of my favorite episodes of CHEERS, "Rat Girl," which had a really great argument scene between Frasier and Lilith. Then I noticed that you also wrote "Room Service," which was one of my favorite episodes of FRASIER and also featured Lilith. Looking at IMDb, it seems like you wrote a lot of the episodes of FRASIER featuring Lilith. I have to assume that this was intentional, but it seems unusual, especially given how collaborative writing for a half-hour comedy is. Were you considered to be the Lilith writers? And did that happen for a particular reason?

For whatever reason, David and I seemed to have a flair for that character. I always worry what it says about us. I was a Psych major at UCLA and pretty good at slinging around that psycho-babble so that helped too. In fact, the only use I ever got out of my four years of studying psychology was the ability to better write Lilith’s bullshit.

Got a question? I'm here to help (unless I'm snorkeling)


Anonymous said...

Since you're flying off for (I assume) some time for relaxation, here's a kind-of related topic -- What's your opinion/outcome of trying to be creative in comedy writing while either having imbibed a little, or under the influence of any other intoxicants?

(My limited trial runs have been it can open up the mind to some incredible new angles of comedy you might not think of while straight/sober, or it can inspire you to dash of two hours of naval-gazing tripe that seems hilarious at the time, but is -- let's say -- sparse in the humor aspects come the light of dawn, and you're looking at the thing the way other people will.)

Anonymous said...

So did you write the recent Frasier/Lilith Dr. Pepper commercial? ;)

geewits said...

Why do writers sometimes rewrite "history" on a TV show? In "Frasier" Marty clearly laments that he has trouble understanding his sons' odd relationship because he never had a brother and then a few years later he has a brother and has to go to a Greek wedding. Couldn't it have been a cousin? And also early on, his chair is a plug-in massage chair and then later when Frasier buys Marty a fancy leather massage chair, Marty freaks out and says it is obscene. Do the writers think people forgot and do not reckon that the show will reach syndication? Or do they just not care to be consistent? I've always wondered about stuff like that. Have fun in Hawaii. I had a great time in Maui a few years ago.

Anonymous said...

And it's no factor at all that Bebe Neuwirth is a great actress, not to mention HOT!

Anonymous said...

Richard Y's question remains unanswered, so:

"Arc" is not an abbreviation or acronym. It's an actual word, used originally in geometry, where it means "segment of a circle." Out of that context, it has been used to describe such things as the shape of a ball's path through the air when it has been thrown. So metaphorically, it came to be used (originally in the 1980s, I think, because that's when this type of plotting came to primetime TV) for the "curve" of a story line that began in (say) episode 3 and ended in episode 5.

Anonymous said...

CBS has used basically the same test system since the 1930's when Frank Stanton was in the network's audience research department (he later became president of CBS).

According to Wikipedia and other sources, he and Dr. Paul Lazarsfeld of Columbia University developed the system called "Little Annie." All the sources claim an accuracy rate of 85 percent. This might be because they all quote the same source.

Some reasons why the 85% figure might be too high:

Katie Couric on the "CBS Evening News"

Anonymous said...

MCP--I think I know the original source of that 85% success figure for "Little Annie". In a book from the 1970's about CBS, "Reflections in a Bloodshot Eye". author Robert Metz interviewed a man named Robert Goldfarb. He had recently been the network monitor for the testing sessions, writing the summary of the results for the executives. Goldfarb admitted the process was unnatural and shouldn't work..but the batting average was 85%.

(Verification.."hotte"--the way the French describe an attractive and desirable young woman, rather than the plebian American "hottie")

LouOCNY said...

I was in NYC in the mid 80s, and got approached by the people with CBS pins, to do one of those test things, so being bored, I went up to Black Rock and did it. A fairly memorable experience for couple of reasons:

Waiting by the elevators, then mayor Ed Koch swept by, so of course I said something snarky - probably 'how ya doin?'and he nodded his head and went 'ooooook'

The show turned out to be the very short lived Whiz Kids, which was about mid 80s teenage computer geeks. It 'starred' Matthew Laboteaux ( I had to look up how to spell it)- but I was thrilled because Max (Barney Miller)Gail was in it, so of course, I held the little green button down whenever he was on... it also was a 'special cross over' show, as the cute half of Simon and Simon was a guest star! BAD sign...
Even I knew the show was in trouble, as this was October, during the season, so I ended up, during the writing your comments section, writing all sorts of junk - in fact I was the last one thee, obviously annoying all the interns there. I like to think I got someone a job, though, as I praised the girl who hung out with the nerd gang quite a bit, and she ended up as the older daughter on ALF,a year or so later!

belub - how you would say 'beloved' with a nasty head cold!

Anonymous said...

Ref's right.

WVW bilidica the next big venereal disease

Anonymous said...

I'm just not buyin' it. In the 60s and early 70s, all of us psychs were behaviorists. If you wanted nothing but Freud, you had to walk over to the English dept. :) :( :)

Anonymous said...


I think that I speak for all of us when I say that every bit of pain you suffered while studying psychology was well worth it - at least for us.

Anonymous said...

it's probably called an arc because it's like an arc. it starts on the bottom, has a high point, and then winds down. Like an arc on a show (if it ended at the high point, it'd be a cliffhanger).

Anonymous said...

Word verification:
Retant - When you take back calling someone retarded.

Mister Charlie said...

I did one of those tests once up here in San Fran. It was obvious how they were gaming it...first they showed a fairly decent pilot, nothing great but ok.
Then they showed an excrable old pilot that had been around forever (I had actually seen it once on tv beforehand) that was the best example of how NOT to make a tv show.

THEN they asked which show we viewers liked best. I mean, really! That was a no brainer. And based on your questioners breakdown of the 3 shows it sounds like S.O.P.


Anonymous said...

Focus group testing is all bullshit. The testers want the results to be a hair over good or a hair over bad, but they do not want a wild, 99% say it's great... because when that happens and it fails, it proves that focus group testing does not mean anything. And because of that, if a group is starting to stampede in the direction of "this show is great", you will actually see the person who is leading the questioning start to dampen enthusiasm with questions like, "Yes, but what was the one thing about Character A that you did NOT like"... or what parts did you find unbelievable"... even if no one had said a word about Character A or about anything being unbelievable. Ken hit the nail on the head when he said that focus group testing is strictly there for studio and network execs to cover their jobs. None of them have the balls to just get behind something and say it's great, let's put it on the air and then take their lumps if they're wrong. Trust me, I've been through this for years on countless pilots... focus group testing and studio and network notes are ALL worthless. 100% worthless. Do you think Springsteen took notes from a studio exec when he was writing "The Risisng"? Did Mark Twain, John Steinbeck, and a zillion other great authors need notes from some one who was probably hired from the sales department and probably based on their ethnicity? No. Writers wrote, and wrote things that were true, creative and had a point of view. Television will always be crap as long as the execs are a part of the creative process. My opinion? Their job is to say a simple "yes" or "no" to a project and then shut the hell up and collect their paycheck that is only a gift of charity anyway.

Anonymous said...

Well, Springsteen wrote at least 100 songs that are better than "The Rising"... notes from network execs are hardly the only reason for mediocre results.

[Given that his band wasn't part of his lead-off performance at Sunday's (HBO/NPR) Lincoln Memorial variety show, I would have loved for him to sing, instead of "The Rising," something from Nebraska - maybe "Reason to Believe."]

Anonymous said...

For shows like "Two and a Half Men" it seems like the creators write half of the episodes. For shows like "Cheers" or "M*A*S*H", was it common for the creators to offer input to the writers?

Anonymous said...

Ken said:
"Huh???? What shows were THEY watching???"

Could be Ted Danson deserves some credit here, since apparently doing dispicalbe things and being likable at the same time is difficult. Or it could just be that in a cast so big he was the central character, which meant he the only one to have a well-developed relationship
with ALL the characters.

Anonymous said...

Speaking of getting one's info from only one source, I was very surprised to read in a book called Desperate Networks that CSI (original), Survivor, Desperate Housewives, Lost, Grey's Anatomy, and American Idol were all on the verge of being scrapped before they ever went on the air. In each of these cases, it was the work of an executive (usually fairly high in the chain) who fought for the show that caused the network to go with it. In the case of Lost, I believe the executive lost his job for it - and was never reinstated when the show became a hit.

Perhaps the whole thing's fiction and I'm totally off-base, but it suggests to me maybe a good barometer of a show's eventual success is whether it can inspire even a bottom-line business mind to champion it through the system, against strident opposition if necessary, because he/she believes in it so much.

It might also explain why some really good shows finding an audience never get the chance to do so... it takes balls (testicular or... ovacular?) to go up against a system, whether it be financial or artistic in nature, and I imagine such people are rare in both.

WV: latio: Relation between the time you're supposed to be somewhere and the time you actually show up.

Anonymous said...

"One season on CHEERS our yearlong story arc was that Sam was trying to get into Rebecca’s pants. Throughout the course of the episodes he tried every sleazy, lying, despicable tactic he (us) could think of. NBC did research testing on that season and Sam tested way higher than anyone else. Why? Because he “cared” about everybody in the bar and was the father figure everyone could trust. Huh???? What shows were THEY watching???"

And what kind of father's did they have?!?! [/shudder]

Anonymous said...

Aloha! I don't know if this kind of question is allowed but here goes: I'm buying a new car for the first time ever. It's very overwhelming. I've researched, consumer reported and all that but what tips do you and your diverse and astute readers have?

Velocity DeWitt said...

I once went to a focus group that was allegedly going to be for upcoming television shows. But the pilots were all clearly at least ten years old (and in once case starred an actress who had died!) and the whole thing was designed to test the efficiency of the commercials they showed between the pilots.

Anonymous said...

Take it from me, over half the people in LA doing focus groups are movie extras and other down-and-outers on the fringe of the biz.