June 1, 2011

Word of Mouth: Measuring What You Think

More articles by »
Written by: Mitch Metcalf
Tags: , ,

>Word of Mouth is the most powerful and trusted form of marketing in many industries, not least film and television.  Good Word of Mouth can save a movie or make it a surprise hit, whether it’s The 40-Year Old Virgin or Black Swan, while bad Word of Mouth can sink even a “sure thing” movie with a strong premise and a slick advertising campaign (like Watchman).  But Word of Mouth is notoriously hard to measure and quantify.  But here at ShowBuzzDaily we have found a way to include Word of Mouth in predicting ultimate box-office, and the solution might surprise you.  

Most movies decline about 50% a week throughout their run in theaters. A difference of only a few percentage points can make a movie a hit or a bomb. A 40-45% decline per week indicates good Word of Mouth and a much healthier and longer play in theaters. A 55-60% decline each week means the buzz is awful and the film will limp to homevideo.
A couple of examples:  40 Year Old Virgin started with $22 million and in its first 6 weeks, never declined by more than 42%–the result was that it was still pulling in $6 million in Weekend 6; conversely, Watchmen began with $55M in its opening weekend, 2.5x as large as Virgin, but tumbled around 60% each week after that, down to $550,000 in Weekend 6.  In the end, despite its much faster start Watchmen had a total of $108M, behind Virgin‘s $109M.
It is natural to assume that the difference between the film that declines 40% and the film that plummets 60% is the Word of Mouth from audience opinion leaders who see film early in their run. But how do we accurately measure Word of Mouth?
Over the years, we have tried a number of variables to represent word of mouth in the ShowBuzzDaily model that predicts Final Domestic grosses, including:
  • CinemaScore grades, the polled reaction to films in their opening nights in three to four markets across the country, reflecting the views of several hundred paying customers.  Each film receives an A-F grade and demographic breakdowns include women, men, under age 25, and 25 or older. (The average movie, by the way, is 53% female and 57% 25+, and audiences under 25 are the easiest graders by far.)
  • RottenTomatoes score, the percentage of reviews judged by RT to be positive on the movie. This number ranges between 0 and 100 (yes, movies have received 0% positive reviews — remember Transylmania and Witless Protection?).  At the other extreme, Toy Story received a 100% positive score. The average score is 46% positive by an average 100 reviewers (print, electronic and bloggers). 
  • Elite Film Critic Opinion. The average film review score among a much smaller set of critics (think New York Times and Los Angeles Times film reviews).  There are two ways to measure this: the Metacritic score and the Top Critic RottenTomatoes score. In both measures, each film review is rated on a scale of positivity (from 0-100 on Metacritic and 0-10 on RT), usually based on around 30 reviews.  
Read more by clicking below.

    When the three measurements are fed into the model (using stepwise multi-variate regression — that’s the last fancy statistical term, I promise), the RT % positive score consistently is found to be significantly related to ultimate box office, while the CinemaScore grades and Elite Opinion variables are rejected.
    It makes some sense that Elite Opinion does not correlate with box office success. It’s a smaller sample of critics, probably very homogeneous in terms of opinion, and, one can argue, more removed from “regular” moviegoers. But why don’t CinemaScore grades correlate with box office?  Perhaps it could be sampling error — is there a different breed of person that is willing to talk to an interviewer when exiting a theater?  Are the interviewers in the right markets?  By interviewing only on the opening Friday, rather than throughout the weekend, are the grades skewed to the most enthusiastic filmgoers? 
    It is interesting to note that the broad RT positive score includes not just traditional film critics but also the newer breed of citizen bloggers, who might be a better representation of average filmgoers. Whatever the reason, the numbers show that a big increase in broad critical sentiment will translate to higher box office potential after the opening weekend, all things being equal. Roughly speaking, an extra 10 percentage points of positive reviews results in $2 million extra box office.  That doesn’t sound like much, but think about a panned film with 10% positive reviews and a widely-praised film with 90% positive reviews.  That means the better film can make an extra $16-20 million over its run, enough to make a very big difference in mid-level pictures.  Until we have a more accurate measurement of elusive “buzz” and Word of Mouth, we will stick with the broad RottenTomatoes positive score.  

    About the Author

    Mitch Metcalf
    MITCH METCALF has been tracking every US film release of over 500 screens (over 2300 movies and counting) since the storied weekend of May 20, 1994, when Maverick and Beverly Hills Cop 3 inspired countless aficionados to devote their lives to the art of cinema. Prior to that, he studied Politics and Economics at Princeton in order to prepare for his dream of working in television. He has been Head of West Coast Research at ABC, then moved to NBC in 2000 and became Head of Scheduling for 11 years.