How to Creatively Experiment
Alex Tajirian
May 26, 2010
The essay first identifies conversations as a necessary component of a bottom-up approach to creative experimentation, then lays down a roadmap for creative experimentation, and finally lists and explains lessons from past secondary-label Web site experiments.
Conversations
As a marketer, before starting to experiment, you have to set your strategic goals. Thus, you must (1) understand your customers and what matters to them, (2) understand their preferred information- delivery medium, (3) determine which of their problems you must solve, and (4) analyze alternative approaches to solving the problems. After setting your course, you need to regularly monitor the performance of your experiment and make adjustments accordingly.
Setting your strategic goals can best be achieved through conversations with customers. A conversation is a two-way communication based on a common language. Conversation is different from the traditional, top-down marketing approach, whereby marketers feel that they know best and determine what the customer should buy . Successful conversations should be designed to go beyond engagement (through games and other devices) but favorably influenced and ready to buy.
Conversations work as a marketing tool because they are instinctive to your customers. Moreover, recent technological developments, namely Web 2.0 tools, have made such exchanges easy and cheap to participate in. Thus, we are experiencing an end to mass markets, mass media, and limited, arm’s-length relationships—limited because, for certain types of conversations, arm’s length crowdsourcing (a concept popularized by James Surowiecki in his book The Wisdom of Crowds) is the best medium, as when the outcome is the average of responses by a large, unbiased crowd. Examples include logo and t-shirt designs.
For centuries humans have gravitated toward community membership for protection, socialization, and economic gain. However, before opening up to strangers, they first instinctively try to ascertain that the strangers in question are not their enemies , an assessment that is sometimes made in a blink (to use the term made famous by Malcolm Gladwell in his book of that name).
Web 2.0 tools go beyond empowering Internet users to engage in communications within and across communities through direct online read and write capabilities. The tools have also made possible cocreation of products and services through a process that is beyond arm’s-length crowdsourcing.
To hold successful conversations, you must show that you care. Listen to community members, encourage them to open up and participate, and guide their participation toward mutually beneficial goals. You also need to provide them with the tools to facilitate the ongoing conversations. Through the process you aim to understand what they really want, not just what they say they want.
Creative Experimentation
Corporate experimentation presents a number of obstacles and constraints that you need to overcome. The work is like trying to complete a jigsaw puzzle without a picture to guide you. The key success factors are information gathering, creativity, skepticism about assumptions, accepting failure as a learning tool, and willingness to adjust your course.
The first obstacle is that performance information is limited or non existent. T here is no clear roadmap as to where to start, how much money to allocate, whether to start small or with a bang, h ow t o decide your next move (expand, shrink, or change direction?), or how to decide on when and how to make path adjustments. You cannot rely on crowdsourcing because doing so might give away important information to your competitors. Moreover, it’s difficult to estimate the expected profits from the venture, as there are no historical revenue estimates to rely on.
There is only one solution to this predicament: creative experimentation. The approach requires choosing an intelligent starting point, being ready to make mid course adjustments, and setting an exit strategy. An intelligent starting point reduces expensive mistakes and increases the speed of reaching your target.
The following are steps to guide you through the process:
-
You should have a good answer to “W hat is the problem to be solved?” and whether your solutions are viable.
-
Analyze experiments within and across industries, when available. Why did the experiments succeed? Why did they fail?
-
Select solutions. Due to potential unresolved uncertainties, use science and art to narrow down the solutions to a minimal number.
-
Analyze whether you have the right organizational structure to efficiently implement the venture.
-
Determine whether you have the necessary resources and competencies for the project— i.e., whether to do the experiment by yourself or find a partner.
-
Creatively develop initial performance measures and be prepared to monitor them and tweak them regularly.
-
Unlike a hockey game with its three 20-minute periods, experiments don’t have a preset time frame . You need experience to help you decide the time necessary, and you have to fix the time allocated to various milestones.
- You must analyze your exit options. With no exit strategy, there should be no entry. You cannot afford to wait for a crisis to decide what action to take.
Once you have started experimenting, the analysis does not end; it is an ongoing process. At this stage, to make any necessary directional adjustments, you need to tweak your performance measures. Performance measure should indicate whether you are satisfied with progress. If not, can you improve it? But satisfaction is personal. One way to deal with the subjectivity is to ask yourself if you would be satisfied if a consultant presented the results to you. Alternatively, if you were a consultant, would you consider relaying the information to your client as good news or bad?
You need a compass for moving in the right direction, a guide that must include the following elements:
-
Measurements. At a minimum, you need to measure profit and time. Measure total profit, not only direct profit but also the experiment’s positive and negative impact on other projects, including effects you might not have even thought of. Also keep track of time. An unexpected setback may require a revision. However, you should stick to your set time allocation (from Step 7) unless special circumstances warrant a finite overtime.
-
Use feedback from the lead user when available.
-
Find ways to compromise on different opinions within the team.
-
Have a healthy dose of skepticism about assumptions. Regularly validate and modify them when necessary.
Flop Example: Bud.TV
I use Anheuser-Busch’s (A-B) launch in February 2007 of Bud.TV, its secondary- label Web site.
Bud.TV had a grand vision: creating an entertainment network/portal that would show original videos and sell ads on the network to an estimated 3 million annual unique visitors. The initial cost estimate was $30 million.
The grand vision was to a large extent the product of the prelaunch environment. First, A-B was in a state of panic as its sales and market share deteriorated . Second, t here was ubiquitous euphoria over video- sharing Web sites, and A-B wanted to be part of it. Third, customer engagement was the buzzword for effective marketing—i .e., “I f we engage customers, they will stay on the Web site and come back.” Finally, online marketing budgets were increasing across various industries.
It did not take long for management to have a rude awakening. Monthly visitors came in below estimates and were falling sharply. A-B decided to shut down the site in early 2009.
The failed experiment demonstrates incompetence in experimentation analytics:
-
Management adopted a top-down approach instead of having conversations with their customers.
-
They had the wrong starting point. In the planning phase, they did not ask themselves what problem a secondary label would solve , let alone whether such a label was the best solution.
-
They did not have the necessary competencies. The management team members, despite their solid track record, were not video users and had no expertise in video development or budgeting. How do you budget for a video? What are the risks? They didn’t know. Because they didn’t understand the industry or the relevant technologies, they overestimated the first-mover advantage. Members of the team felt an urgency to get on the video bandwagon, believing that otherwise they would not be able to catch up. Furthermore, their vision called for a closed destination, one with limited offerings compared to those found on YouTube. Management underestimated how customers would gravitate toward YouTube, a centralized service, and did not provide cross platform portability to blogs, YouTube, etc. The closed destination and the age- verification process resulted in a cumbersome user-registration process.
-
The experiment’s measures of success were flawed. Engagement is not enough to generate profit. Thus, traditional performance measures such as traffic volume and time spent on site were not useful.
-
There was no change in organizational structure for an area that required new competencies and a new mindset.
- The team ignored Coca-Cola ’s failed attempt at promoting a YouTube– style destination.
With all these incompetencies and oversights, what were the reasons for A-B to approve such a bad idea? For one thing, management had a solid track record, albeit in an area that required different competencies. Moreover, there was no serious skepticism about the assumptions.
Nevertheless, one needs to place the flop in perspective. Although cost was 30% over budget, the figure still represented a small percent of the company’s total marketing budget of over $1 billion, with more than $600 million of that total being allocated to traditional advertising venues.
Success Case Study: LEGO® (LegoFactory.com)
Success, in general, increases if consumers’ emotional attachment to a product is high. In LEGO’s case, kids and parents love the product. Parents are nostalgic for the days they enjoyed playing with LEGO and now fantasize about their children becoming inventors and engineers. Thus, parents are enthusiastically paying for their kids’ purchases of LEGO products.
LEGO takes a bottom-up approach. They provide tools to facilitate two-way conversations on technical and design through a friendly user-generated content (UGC) environment. They also provide standardized tools for cocreation. The conversations demonstrate that they care. They handle logistics (manufacturing custom users’ designs and delivery) when a kid submits a design for desired LEGO components through standardized tools. A simple “Submit your idea here” page isn’t going to generate much trust or demonstrate a company’s caring and seriousness, not about meeting the needs of their customers or about cocreation in particular.
Lessons
-
“Build it and they will come” is not enough , as the Bud.TV experiment demonstrates. Think beyond the benefits to your customers. If a potential customer is entertained and happily engaged, what is in it for you? You should ask if engagement and entertainment convert to purchases.
-
Find useful measures to evaluate ongoing performance. Return on Investment (ROI) is not applicable for newly launched experimental projects and need not be meaningful in measuring returns on social media marketing.
-
Communicate in the customer’s preferred language. Top-down decisions fail in a conversation-based marketing environment.
-
Experimentation must be disciplined: You must choose an intelligent starting point, be willing and ready to modify course, and have an exit strategy.
-
You must incorporate UGC and genuinely care about customer participation. A successful secondary-label hub must be directed at a harmonious group and/or potential customers who are not on a centralized hub. Otherwise take advantage of centralized hubs such as Facebook and YouTube, where a customer mass is already there.
-
Have the right organizational structure and competencies.
- When designing your hub, and your Web site in general, visitor experience must take priority. Bud.TV’s registration interface is a straightforward proof.
Happy experimentation!
Connect & Share
   
|
