Divorcing yourself from opinion, or qualitative analysis, or wishy-washy suggestions can be a tough task.  Some stakeholders are statistically savvy and can look at numbers objectively, opting to do their own analysis – perhaps by using separate business intelligence. However, most people want to read interesting insights, even if it’s just to look smart at the water cooler.  For everyone to get on board, the trick is to stop saying, “I think” and start saying “the data shows.”

Finding valuable insight in segments of your traffic is a great first step.  Focus on one product, one traffic segment, and one conversion pathway through a site.  Slice and dice the traffic different ways until you notice a trend, or even something interesting.  You might find that first time visitors coming through organic search terms convert five times better than paid search visitors.  A perfect starting point for deeper dives across alternate product lines.

Oh oh, what if your data is sketchy?

What if your segments look a lot alike, and you think that 5% difference could mean something but aren’t sure?  That’s where significance testing and maybe even multivariate testing can help:

• If you DON’T have access to web development teams, test theories using a statistical significance test, such as PRC’s statistical significance calculator.
• If you DO have access to a friendly web dev, consider working together to run an A/B or multivariate test using Google Website Optimizer.  Using it is free, significance testing built-in.

All of a sudden, the data starts showing room for improvement.  Creativity shifts to testing to find a winning combination, instead of intuition- or experience-based decision-making.  Perhaps for the first time ever, you can start attributing measurable difference in revenue and start prioritizing projects by potential shifts in “ROI” (using the term liberally here).

In addition, the data analyst can save measurable results to their war chest.  A bit different than intuition or experience, the war chest becomes a repository of educated guesses.  As you proceed to make improvements in ROI, you can start to reliably predict lost revenue for projects that are put on the back-burner.  Data (not the Star Trek android) becomes your friend; an ally in the fight (ok, tone that down to “challenge”) in development resources.

A new ally in data, new friends in web development, new supporters across business units, new weapons in your war chest.  Becoming a data diplomat is win-win-win.

Return on investment is a sexy KPI right now; everything and everyone claims to provide easy calculations of ROI, value added services that show ROI, and other far-fetched promises.  Probably one of the hardest things a web analyst can do is measure ROI — true ROI.  We’ve become accustomed to using the term “return on investment” to include subjective value propositions; soft and cuddly marketing statements.  True ROI is just not available through analytics tools and web analysts do not have the luxury of full financial disclosure.

Unfortunately, as data-driven decision-making has become more popular, so too has the search for more meaningful business-oriented metrics.  You know, stuff that actually means something to say, your CFO — money.  Somewhere along the way, we lost sight of the true definition of ROI; people have been bastardizing the term ever since.  Let’s look to Wikipedia for a refresher:

Rate of return, or return on investment is the ratio of money gained or lost (whether realized or unrealized) on an investment relative to the amount of money invested.

What’s missing in this definition?  Oh right, all those wonderful value-add statements marketers (myself included) use to inspire confidence in products and services.  Statements describing potential positive ROI should never include subjective benefits, because that’s not ROI.  With respect to web analytics, the same is true.

Regardless of how much data you collect using your web analytics weapon of choice, you’re not seeing the whole financial picture.  If I had to wager my salary towards the truth of that statement, I would probably win 99% of the time.  Web analysts simply aren’t plugged into each and every profit and loss source within an organization, and frankly we shouldn’t be.

The typical web analyst is a highly detail-oriented individual, that includes figures in context of what is being observed online.  So we have to ditch the use of ROI as a KPI and look to more specific calculated metrics that make sense, such as return on ad spend (ROAS), average revenue per order/user (ARPO/ARPU), and maybe just aggregate revenue figures.

Does that mean we should live a sheltered life?  Hell no.

There are wonderful departments known as finance, with highly skilled numbers people just like us that have access to a heck of a lot more business information than we do.  I encourage you to reach out to these financial gurus and ask a ton of questions.  Don’t expect each and every profit and loss center in the company, but if you’re lucky, they might just give you an aggregate ratio to aim for to determine a winning versus losing product, campaign, or change initiative.

Happy New Year and welcome back to what will be a make or break quarter for many of us. It’s time to hunker down and turn what was a pretty terrible economic run last year, into the building blocks of recovery.  Many of us will have to focus our efforts on hitting hard in early 2010, gaining an important early foothold amongst strong competition.  That makes measuring continued success so much more difficult, especially if you haven’t yet established clear KPI’s on which efforts will be judged.  So with all the vague “just measure ROI” discussion aside, I’m predicting the following to be the most important key performance indicators for early 2010.

Why the hate geared towards “ROI” as a metric?  Simple, it’s pretty obvious that everyone brandishing the term about does not use it appropriately.  When most people say “ROI” they don’t mean to take the conversation up to the next level, and don’t have research to back it up.  Measuring true ROI can be pretty difficult, especially without tag-teaming efforts between a web analyst and financial controller/comptroller.  So let’s all put the term “ROI” aside for now, and refocus on a term someone smarter than me used once upon a time, “business impact”.

Response or Conversion Rate

The perennial favorite!

We’ve all been measuring conversion rate for years, and that’s not likely to change anytime soon.  However, what will change is the attribution any one department or campaign or strategy is awarded when a response or conversion is achieved.  Departments do not live in a bubble, be prepared to share and share alike as divisional lines begin to blur.  Marketing folks will have to talk to sales folks, support folks will have to talk to IT folks, and everyone will want a piece of the organization’s success metrics, regardless of who reports on those KPI’s.

Social Influence

You can’t ignore social media anymore.  To web analysts, social media is just another platform that can be measured independently using external sources, or analyzed with software by segmentation.  Avinash Kaushik goes into greater detail on this subject, and while I agree his equation of influence on Twitter is useful, “influence = RT’s per 1000 followers”, a greater holistic metric is needed that can be applied to all other social media platforms.

Why not blend external and internal metrics?  Do they blend?  They sure do!

Consider segmenting social media sources as a group, and apply ball-park cost figures to the time you invest on social platforms versus lead or order volume to your site.  In this manner, you can measure social media cost per acquisition (CPA) the same way you would banner ads, paid/organic search traffic, and email campaign success.

Voice of Consumer

As we delve deeper into advanced competitive intelligence reports from sources such as Compete.com, Hitwise, and even Google Trends, we can start measuring the impact of external traffic patterns, get additional context to put other metrics in perspective, and establish checks and balances for branding efforts.  While this is great for marketing departments, it doesn’t do much to help other folks in the company.  Enter your new favorite tool for 2010: the survey.

You may be aware of vendors that offer survey solutions for abandoned shopping carts, or as a last-ditch effort to people leaving a site, but what about using surveys elsewhere.  Partnering with research houses can give you the best measure of voice of consumer because they go beyond the scope of your site, include all your competitors, and normalize the data.

Not the surveying type?  Depending on how sophisticated your analytics solution is, you may be able to relate segments of authenticated users to your CRM or billing software.  By applying external competitive intelligence metrics to internal web analytics, and then web analytics to real customer information, you can provide ample insight to customer services, sales, and support teams.