Measuring ROI – Lesson 2: Becoming a data diplomat

Divorcing yourself from opinion, or qualitative analysis, or wishy-washy suggestions can be a tough task.  Some stakeholders are statistically savvy and can look at numbers objectively, opting to do their own analysis – perhaps by using separate business intelligence. However, most people want to read interesting insights, even if it’s just to look smart at the water cooler.  For everyone to get on board, the trick is to stop saying, “I think” and start saying “the data shows.”

Finding valuable insight in segments of your traffic is a great first step.  Focus on one product, one traffic segment, and one conversion pathway through a site.  Slice and dice the traffic different ways until you notice a trend, or even something interesting.  You might find that first time visitors coming through organic search terms convert five times better than paid search visitors.  A perfect starting point for deeper dives across alternate product lines.

Oh oh, what if your data is sketchy?

What if your segments look a lot alike, and you think that 5% difference could mean something but aren’t sure?  That’s where significance testing and maybe even multivariate testing can help:

  • If you DON’T have access to web development teams, test theories using a statistical significance test, such as PRC’s statistical significance calculator.
  • If you DO have access to a friendly web dev, consider working together to run an A/B or multivariate test using Google Website Optimizer.  Using it is free, significance testing built-in.

All of a sudden, the data starts showing room for improvement.  Creativity shifts to testing to find a winning combination, instead of intuition- or experience-based decision-making.  Perhaps for the first time ever, you can start attributing measurable difference in revenue and start prioritizing projects by potential shifts in “ROI” (using the term liberally here).

In addition, the data analyst can save measurable results to their war chest.  A bit different than intuition or experience, the war chest becomes a repository of educated guesses.  As you proceed to make improvements in ROI, you can start to reliably predict lost revenue for projects that are put on the back-burner.  Data (not the Star Trek android) becomes your friend; an ally in the fight (ok, tone that down to “challenge”) in development resources.

A new ally in data, new friends in web development, new supporters across business units, new weapons in your war chest.  Becoming a data diplomat is win-win-win.

3 thoughts on “Measuring ROI – Lesson 2: Becoming a data diplomat”

  1. Hi Gary – Great post (will RT!).

    I especially like this comment: “However, most people want to read interesting insights, even if it’s just to look smart at the water cooler.”. There’s too much of that going around versus what is really important: “Focus on one product, one traffic segment, and one conversion pathway through a site.” is much smarter.” Baby steps first.

    BTW: I partially address the issue of too much data and not knowing what to do with it in this Blog post: http://www.webfuel.ca/dfc

  2. Hi Gary – Great post (will RT!).

    I especially like this comment: “However, most people want to read interesting insights, even if it’s just to look smart at the water cooler.”. There's too much of that going around versus what is really important: “Focus on one product, one traffic segment, and one conversion pathway through a site.” is much smarter.” Baby steps first.

    BTW: I partially address the issue of too much data and not knowing what to do with it in this Blog post: http://www.webfuel.ca/dfc

Leave a Reply

Your email address will not be published. Required fields are marked *