Putting P-O-P To The Test

Jim Litwin is on a mission. After 20 years of working in P-O-P advertising, Litwin, senior vice president of strategic initiatives at Wunderman Cato Johnson, Chicago, is attempting what few have tried -- to accurately measure the effectiveness of P-O-P. Still in its infancy, Litwin's research program, called P-O-P Metrics, incorporates real world sales measurements and observations, as well as new laboratory methods. The goal of the research is to establish a set of benchmarks so that the sales impact of future P-O-P campaigns can be assessed before they're placed in stores."No corner of the marketing mix has been more under-researched, and yet I think few areas are as researchable. And it's fairly easy to do some of this," says Litwin.WCJ, with its roots in direct marketing and sales promotions, boasts a long list of P-O-P clients, such as Blockbuster Entertainment, Citibank and Taco Bell. As the company has expanded its P-O-P operations, research has become vitally important. "Given the fact that P-O-P is an integral part of what we deliver to our clients, we've achieved what we call 'critical mass,' which enables us to really invest heavily in people, time, energy and in money, to really get under the issue of how to measure the effectiveness and the payback of P-O-P," says Litwin."We can all talk about our opinions, but I want to put on the table that this is what the customer said and this is why we're doing this," says Patricia Millo, vice president of merchandising strategy for Citibank, New York. "A lot of the things we were doing in the past were very subjective. It was really a matter of opinion. We're trying to get away from the subjective and develop a more disciplined process for merchandising."What Is P-O-P, Really?To begin the process of developing new tools to measure P-O-P effectiveness, Litwin started by defining what P-O-P really is and how it functions. The basis of Litwin's research assumes that the consumer experience with P-O-P is a low-involvement communication mode that fills very specific "need states." Litwin further defines four separate need states and the way P-O-P addresses these needs.When customers need to find something in a store, P-O-P provides direction.When consumers need to learn about a product, P-O-P offers information.When they need value, P-O-P communicates promotions and bargains.When patrons need a pleasant shopping experience, P-O-P offers brand reinforcement and decor.After taking into account the way consumers actually use P-O-P, Litwin and his staff created a series of tests and measurement tools designed around the usage of P-O-P. Both laboratory and real world methods were adopted to put different P-O-P campaigns to the test and gauge their effectiveness."Traditional forms of persuasion advertising research don't work very well with P-O-P because shoppers use P-O-P when they need it and then forget it," says Litwin. "The most effective P-O-P recall is next to zero after just a second of coming into contact with it."The cornerstone of Litwin's research is a series of laboratory tests designed to evaluate ad copy and quantitatively measure customer behavior at the moment of contact with P-O-P materials. It's called Promotional Appeal & Readability (P.A.R.). "P.A.R. gives you very definitive results. Some interpretation is needed but not a lot," says Litwin. "It's like, this campaign is definitely better than that one."The P.A.R. system measures the three stages of customer interaction with P-O-P materials -- catching the eye, main message comprehension and persuasion. To measure each of these components that lead to a purchase, WCJ developed three different gauging methods.The Eye-catching Stopping Power (E.S.P.) test is used to measure a P-O-P sign's ability to attract customer attention. The test flashes two alternative designs simultaneously on two screens placed in the participants' line of peripheral vision and measures which sign they look at first. "Of course, we reversed the designs from one side to the other because we know that people have a tendency to look left. We wanted to eliminate that as a factor. We mixed them up and made sure that they had to physically move their heads so we could see which design they looked at first," says Litwin.Secondly, a tachistiscope (T-scope), a computer-controlled shutter that attaches to a slide projector and opens and closes the shutter for a precise amount of time, is used to measure the readability and main message comprehension of P-O-P materials. Images are flashed on a screen for a brief interval in front of a room full of participants who comment on what they see into a microphone. "Each person is recording something like, 'I see orange' or 'I see the word free'," says Litwin. The image is flashed in three separate intervals to see how additional exposure to P-O-P impacts perceptions. The multiple exposures not only measure how quickly test subjects respond and receive the main message, but also capture the order in which different elements of a design are noticed.The third element of P.A.R. attempts to measure the appeal of the P-O-P piece, the most subjective and personal element of the purchasing process. Participants individually rate each sign on appeal, clarity and relevance. Afterward, a focus group discussion attempts to explain why consumers responded the way they did.Several steps are taken during the testing process to ensure the validity of the test scores and create a real world atmosphere. For example, P-O-P elements are grouped with similar types of signs based on store location, complexity, size and purpose. "It wasn't fair to judge a sign that said 'comedy,' identifying the comedy section of Blockbuster, against a more complicated promotional offer," says Litwin.The P.A.R. study was initially developed through a partnership with WCJ and its client, Blockbuster Entertainment. "A large share of Blockbuster's consumers make decisions on what to rent in-store, so it is essential that our P-O-P communicate effectively," says Alan Klose, director of marketing research for the Dallas-based video rental company. "P-O-P Metrics allowed us to evaluate different types of P-O-P based on the strategy for each piece. It is critical that P-O-P be evaluated based on its specific design criteria -- environment, directional, informational and promotional."Ultimately, the goal of the program is to build a database of benchmarks, organized by category and type of P-O-P element. WCJ hopes that eventually its clients will be able to evaluate each new element of a campaign or a complete campaign against these benchmarks to see if the new designs are above or below category standards.Study FindingsMany of the findings of the initial P.A.R. study confirmed what good designers have instinctively known for a long time. Bright and colorful images catch the eye; simple layouts have the highest main message readability; and promotional cues, such as price areas, tend to catch the eye.The study also revealed some unexpected results. For example, simple text isn't always the most readable. The study suggested that plain or unattractive signs are not as thoroughly read as more attractive signs. In fact, when simple, unappealing signs were shown during the T-Scope test, participants began commenting on the background and store setting, rather than the sign."People were getting bored with the sign," says Litwin. "They weren't being engaged because it wasn't interesting. It's uplifting to know that the aesthetics had something to do with the readability of the sign. In fact, we found that an increase of the main message recall from the first to the second exposure did correlate pretty well with the appeal scores. There were a bunch of signs that had really high main message recall and really low appeal," he says. "Those were more of the directional/navigational signs that didn't have to be aesthetically appealing. Simplicity of the design gets their attention. We know that through the E.S.P. tests. The appeal of the design keeps their attention."We're hoping that through P.A.R. we'll get the results we need to be able to speculate just from the P.A.R. research what the sales impact would be with a certain sign," says Litwin.M.I.M.S. The WordDesigned to test P-O-P in the real world, the Merchandising Impact Measurement Study (M.I.M.S.) measures the actual impact on sales with P-O-P in matched store settings. Two similar stores, with similar sales volumes, customer patterns, and locales are paired, each using different P-O-P. The sales results are then compared to see which P-O-P designs had the greatest impact. Also, this system can be used to determine the most effective combinations of elements or the impact of a single element."The way these two relate is that you pre-test creative designs with P.A.R. and you post-test the actual work in the field with M.I.M.S. and hopefully you start building a relationship between these two," says Litwin. "Once you have a strategy in place, you do things like P.A.R. and M.I.M.S. to make sure you're executing it optimally. The copy is the best it can be. The colors are the most readable, the most eye-catching."In addition to P.A.R. and M.I.M.S., Litwin is experimenting with a number of other devices that he hopes will eventually aid in the measurement process. For example, a Quicktime VR project called Virtual In-store Environment Window (V.I.E.W.), uses virtual reality technology to allow consumers to take a self-guided tour through a virtual store while their actions are recorded. Different P-O-P signs can be placed within the virtual store environment to see how alternative designs affect consumer response. V.I.E.W. can also be used to coordinate multiple campaigns within the same store setting."V.I.E.W. and other techniques are what I call in-store media planning, where we can simulate what the store is going to look like in a very realistic way before we actually print the materials. This is especially useful when you get into multiple campaigns, where you have a kids' meal promotion, an adult program and a charity thing all going on and you get this conflict area. When you put it all together, you can see what the store is going to look like and realize that maybe you should adjust something or maybe you have too much of something in this part of the store and not enough in that part. It's sort of a media planning tool," says Litwin."We're using V.I.E.W. as a tool for internal purposes to present campaigns to senior management, which is helpful to us because a lot of people aren't very visual. For us, it's more of a tool to show the creative and sell it in," says Citibank's Millo.Also, Litwin is testing what he calls "video snakes," small video cameras hidden in a hat, worn by a consumer to record exactly what the consumer sees in the retail environment. Similarly, a "visor-cam" system incorporates video snakes and eye-tracking. The device is clipped on the visor of a hat and tracks what the consumer sees, as well as precisely what the consumer is looking at."The ultimate goal for us is to have guidelines for placement, guidelines for design, and guidelines for types of messaging. Then we'll apply those guidelines and test again," says Bruce Dorr, vice president of merchandising at Citibank. "I don't think we're done researching. It's not just a one-time deal. It's an ongoing and dynamic process."In the end, P-O-P still has a long way to go before it can be considered a measured media. In a sense, much of the research done to date is really science designed to improve the art. However, with P-O-P playing an increasingly important role in the advertising mix, the demand for research data is sure to increase. Will marketers ever be able to correlate P-O-P dollars directly to a sales volume increase? Litwin says yes. "Traditionally, P-O-P has been very easy to brush off by senior marketing staff as a second-tier medium," he says. "Now I'm starting to see more recognition of the importance of P-O-P. In many cases, it is the most influential medium."

Putting P-O-P To The Test

P2PI MEMBERS

JOIN P2PI

Related Images

Business Topic