Archive for March, 2010

Actionable Attribution Analysis: A Three-Phased Approach

March 11, 2010

“Last-click attribution is dead!”

“Media-mix modeling is the key to marketing success!”

“Without an accurate attribution model you’re throwing away your marketing dollars!!”

With SMX West in town last week the halls were echoing with passionate cries about attribution analysis. It seemed as if all topics (other than the Yahoo-Microsoft search deal) had taken a back seat for a moment, and suddenly the most important thing to consider was attribution analysis, specifically whether or not you are giving too much credit to SEM and not enough to other media.

Believe me, I share people’s enthusiasm on the topic. It’s clearly one of the next big online marketing problems to solve. And despite the fact that moving away from last-click attribution toward a more elegant and accurate attribution model can really only serve to divert budget away from SEM and toward other channels, I do think it’s the right thing to do. But after talking to as many people as I could, gathering my own data and soliciting opinions, I’m convinced that we are a still a long way from being any good at this at all.

Currently there appear to be two basic types of approaches, both of which seem to me to be fatally flawed.

One approach I see in the market, offered by various otherwise credible services, has the advertiser entering percentages into boxes on a screen, assigning portions of the conversion value to different marketing channels – 25% for SEM, 35% for display, 15% for email, and so on. As a large advertiser myself, I can safely say that this approach gives someone like me entirely too much credit as a sophisticated marketer. I don’t know anyone who has a good enough grasp on their business and the implications of attribution analysis to make an intelligent decision in this type of situation. No knock on my fellow advertisers, but seriously, this is way out of our league. Even so, a Google representative stated during a panel I was moderating, that they intend only on providing attribution-related data, placing the burden of analysis on the advertiser.

The other approach I see emerging is a black-box math-based approach. This is more likely to be done in-house by large advertisers, using statistical and predictive modeling to simulate different attribution models, and mapping their outcomes to business metrics like profit, revenue or ROI. While I do think there is significant value in doing the hard math and understanding these problems from a statistical point of view, this methodology tends to be short-sighted. I don’t believe there is a one-size-fits-all approach to attribution analysis where you simply dump your marketing data in, and out magically pops an attribution model that maximizes profit, for example. It’s just not that generic of a problem.

It’s easy for me to sit back and criticize the status quo – so why not offer some solutions, you say? Well, here goes: I envision a three-phased approach that takes some elements of the existing practices, then combines and expands upon them to provide a more complete, appropriate solution for each advertiser.

The first phase involves smart people talking to each other. Revolutionary, no? We need an attribution specialist to lead off this effort by conducting a fairly exhaustive analysis of the advertisers’ business and online marketing programs. Starting with business goals and product adoption cycle, to conversion window analysis, on to a channel-by-channel audit of on- and off-line marketing. The purpose of this consulting and analysis is to provide the proper inputs into phase two.

Phase two is the super-math modeling I describe above. With the proper inputs as they relate to an advertiser’s business and its metrics, statistical modeling is needed to predict all possible outcomes and understand which model will best support the advertiser’s business goals.

Finally, phase three makes all of this actionable. We need a way to pluck the wisdom out of phase two and apply it directly to actual media channels the advertiser is running. Ideally we’ll find a way to automate this or at least automate the recommendations, which can then by easily implemented into the media buys themselves.

But before any of us sprint into the world of attribution analysis and media mix modeling, let’s step back and take a long look in the mirror: I don’t know of a way to realistically pull any of this off if an advertiser doesn’t have a common tracking/analytics system for all marketing channels. So before we start hiring expensive analysts, consultants and statisticians, let’s be sure to clean our own houses and get our own data in order. Standardize your analytics and measurement on a single platform so you can compare ‘apples to apples’. Then you can start to focus on the fun stuff.

Bidding Strategies for a Complex SEM Landscape

March 11, 2010

If you’re managing multiple SEM programs across a broad landscape of web assets, as more and more of us are, it’s important to realize that when it comes to managing keywords and their bids, it’s not one-size-fits-all.  In this reality of diverse business goals and revenue models, marketers looking for a singular approach to SEM program management will be sorely disappointed.Bidding Strategies

In the past I’ve talked about the fact that at Yahoo!, we have many different properties (think Yahoo! Personals, Yahoo! Sports, Yahoo! Shopping, etc.), and each of these properties has a business that’s unique in it’s own way. Some properties are supported primarily by display media (Sports, Finance, News, etc.). In others, users can subscribe to services (web hosting, merchant solutions). Yet other properties list products sold by other vendors e.g. Shopping and Travel. Since each of these businesses has a different way of making money, for each property we need a unique way of defining the value that a user can bring to the company. On this topic of customer valuation, I went into much deeper detail a while back.

But what about SEM campaign management? How do you manage different SEM programs for a diverse set of businesses? It turns out that each type of business requires a unique approach to SEM program management, particularly as it relates to keywords, targets, and bidding. Let’s look at the following 3 examples I pulled straight out of the trenches:

Keeping it Simple

For our branding campaign, as I wrote a few months ago, our goal was to deepen engagement with the Yahoo! Brand and Products. We were able to track and measure engagement through a proxy of web events to which we assigned points to define relative value. In executing against this goal, we (with the help of our Agency) managed to a cost per point or cost per value model on a keyword-by-keyword basis. We did this on a fairly manual basis, as the keyword set was finite and manageable, and performance was fairly stable. This is the simplest of examples I can provide from purely a bid management perspective.

The Tail Wagging the Head

In our orders-based businesses, where we’re driving users toward a common conversion point such as a subscription or sale, we (again, with Agency assistance) use a somewhat more complex management strategy that is now gaining traction in the industry, though we’ve been employing it for some time. We try to break up our keyword portfolio into the smallest possible discrete datapoints that we can, while maintaining a sufficient quantity performance data for bidding purposes. For example, a high-volume keyword such as ‘dating’ gets broken down into bits that look like ‘dating on exact match in Chicago during nighttime hours with a value-focused call-to-action ad’, and so on. Bids can then be managed on these micro-units so the campaign can be optimized to a level of efficiency not possible when managing solely at the keyword level.

Kicking It Up a Notch

For our listings properties, where users come in, research products and services then leave, the model is completely different. For one thing, our revenue is mostly earned in-session (as opposed to a 30-day conversion window, for example. To make matters more complex, we have keywords of every type that number in the millions. High volume, seasonal, long tail, you name it. In order to get the most out of these programs, we must deploy multiple management strategies within a single program. For head or high-volume keywords, we can employ automated bid management algorithms to optimize to a desired business metric – revenue, profit, ROI, etc. For other keywords we need to take a more rules-based approach. For example, we may want to take all the keywords that fit a certain profile – ranks on the 2nd page, has above 100% ROI, etc. and perform calculated bid management techniques – bid a percentage of revenue or profit, increase bids by 20%, bid to higher position, etc. As if that weren’t enough to worry about, for seasonal and other reasons, at any given time we have a large number of keywords that aren’t getting any impressions at all. In cases like this we periodically ‘re-activate’ keywords and try to bid them back to profitability once again (it’s assumed that we bid them down to inactivity at some point because they weren’t profitable). This takes a measured, rule-based approach to qualify and bid systematically to ensure the best chance at regaining profitability.

As you can see, as the landscape across which you’re managing SEM campaigns becomes increasingly varied, the more complex and custom your approach to search marketing will need to be. It’s only when you can match your businesses one-to-one with uniquely suitable approaches that you’ll be able to bring your ad spend to a truly optimized level.

Paid and Organic Search: Lift, Cannibalism, or Both?

March 9, 2010

‘Why are we buying our brand keyword when we already rank #1 in the organic results?’ ‘Why are we paying for traffic if we’re already getting it for free??’ It turns out that the question isn’t whether or not you should be buying your brand keywords. The question is how much should you be willing to pay for that ad, and what should it say

For search marketers like me (and probably you), the question of the paid/organic dynamic has been around for years. So why is there such an amazing dearth of good information on this topic? Why isn’t there any kind of industry-accepted framework with which to address the age-old question?

I believe that the reason for this is that the conversation around the interaction between paid and organic search has historically been sorely lacking any good data. As a result, we get stuck talking about opinions and assumptions, and we typically don’t come to any meaningful conclusions. I am grateful that at this point in my career, I am surrounded by savvy marketers who understand how search results pages (SRPs) work. They understand that the SRP is a complex landscape, that each link has its own clickthrough rate (CTR), and that any link’s CTR is affected by the other links with which it shares the SRP. This is the path to meaningful dialogue on the subject, so I encourage everyone to get intimately familiar with the data around the paid/organic dynamic.

So how do we look at the data in a way that can help us understand this phenomenon? First let’s get a few ground rules straight:

  1. What keywords are we really talking about? Those that match exactly with your brand name or branded product name, where there is generally no competition. So if you are the Acme Widget Company we would be talking about keywords like ‘acme’ and potentially ‘acme widget’.
  2. What are we actually trying to compare? Ultimately we want to compare two different conditions: a) a SRP where the organic link for brand kw is ranked #1 with no PPC ad (and no competitors’ ads) present and b) a SRP where the organic link for a brand kw is ranked #1 with a #1 rank PPC ad (and no competitors’ ads)
  3. What phenomena are we trying to measure? In the above cases there are two things that normally happen. I call them cannibalization and lift. ‘Lift’ is the net amount of traffic that is added to the mix by virtue of the PPC ad. Cannibalization is the portion of PPC ad traffic that comes at the expense of the organic link. If you can quantify cannibalization and lift in any situation, you can then begin to think intelligently about what to do.

One thing we need to also acknowledge is the fact that the many variables affecting paid and organic search traffic – search volume, page layout, keyword bids and rankings – prevent us from doing any rigorous scientific testing around the paid/organic dynamic. It’s simply impossible to isolate all the variables necessary to completely understand what’s going on. However, there are some terrific ways that you can at least gather some meaningful data that can be interpreted and analyzed, and from which we can actually draw very useful and actionable conclusions.

Next, let’s agree on a few basic principles:

  1. Internet (and search) traffic patterns move in weekly cycles
  2. Search volume is affected by seasonality, media, and other factors
  3. You’ll want to ‘test’ in a period of minimum volatility (avoid holidays and seasonal peaks and dips if possible)

Now, consider the following approaches to gather the data required to quantify cannibalization and lift:

  1. On/off weekly: Pause your paid ads for one week and then resume. This is the simplest approach and takes the least amount of time. If you have more time, try alternating weeks as long as you need.
  2. On/off daily: For a two week period, alternate pausing and activating your paid ads on consecutive days. Why two weeks? This is the minimum duration required to get both ‘on’ and ‘off’ data for each day of the week.

These are just examples. Use your imagination to design something more elegant if you have more time or budget.

Now What?

Now, you need to gather your data and estimate your lift and cannibalization. The incredibly tricky (and potentially inaccurate) part of this is trying to establish a baseline for organic traffic. Naturally, you will want to use the organic traffic during ‘off’ periods as a baseline, but what about the ‘on’ periods? What would the organic traffic have been without the paid ads present? For this you will need a third data point. Either use averages of organic traffic during ‘off’ periods that bookend an ‘on’ period, or if you have access to data like search volume for a given keyword, you can use this trend to estimate what your organic baseline should be.

The key here is to come up with an approximation for cannibalization and lift. It doesn’t have to be perfect, because you’re going to use this data to determine, based on your business goals, what you should be willing to pay for a click on your PPC ad. Here’s an example:

Let’s use a day’s worth of data, and suppose we determine that our organic baseline traffic is 100 clicks. When we add a PPC ad, that ad provides us with 100 clicks, but when we do so, our total organic+paid total is only 180 clicks. That means that of the 100 PPC clicks we bought, 80 were ‘lift’, and the other 20 were ‘cannibalization’. Then, all other things being equal, you should discount your maximum allowable CPC on your brand keyword by 20% to account for the cannibalization, and adjust your bids accordingly. Make sense?

Now let’s look at the extremes. If your PPC traffic is 100% lift, then you can confidently say that buying your brand terms is absolutely justified, and you have the data to prove it. What, then, if all your PPC traffic is cannibalized organic traffic? If that’s the case, then you had better have an incredibly good reason for paying for the PPC ads. One reason might be that you want to put a differentiated message in front of people, a message that’s not reflected in your organic link. Possible reasons for this might be a brand re-launch or a strategic event like an important product launch or corporate milestone.

This may sound complicated, but I can assure you it’s both do-able and worthwhile. I just completed a study for one of our keywords and I can tell you that I am ecstatic about the results. I can’t wait to share them around the company! Good Luck!


Follow

Get every new post delivered to your Inbox.