There comes a time in every analyst’s life when an organization decides to change up the game and move from one web analytics solution to another.  Transitioning the reporting data isn’t usually that painful, if done correctly; it’s decommissioning the old solution that is most challenging.  Having survived a number of these transitions already, I came up with a brief list of best practices when decommissioning most paid web analytics solutions.

1. Audit URLs to be migrated.
   The one thing any analytics solution will be good at will be to output a list of URLs that are tracked.  Use this as your dashboard every day, to ensure these URLs go dark.
2. Audit JavaScript file locations.
   Think your JS file is centrally hosted?  Think again.  In my experience, very few organizations with multiple web properties keep a centrally-hosted instance of their analytics JS file.
3. Audit custom event tagging.
   Keep in mind that server hits (and I do meant hits) can be generated asynchronously (onClick, onLoad, embedded in redirects, in frames, etc), that may account for a significant portion of important business goals.  If your only goal is to turn off analytics to a particular vendor, feel free to remove these and forget they ever happened.  If you want some continuity in reporting from one solution to another, you will want a record custom events and prioritize their inclusion in the new solution so that you don’t lose sight of important metrics.
4. Decommission the JavaScript library at least 30 days before end of contract.
   Browser caching is a royal PITA, but most busy sites will see a drop-off of cached JavaScript tags within a week.  Users however, do all kinds of crazy nonsense with webpages; some will actually save your homepage to their desktop and use it as a bookmark.  Leaving at least thirty days to sort out your odd user behavior and factors beyond your control leaves enough time to take affirmative action to kill server hits.
5. Audit remaining URLs sending hits to the analytics log server.
   Using the list you generated in the first step, hone in on wayward pages that may still contain JavaScript, server-side code, or web services that are still logging hits.  In one implementation, I discovered a web service – long forgotten – uploading classification data to a vendor on a daily basis.  In another implementation, “no script” tags were still being triggered on redirect pages.
6. Decommission any DNS entries supporting the old solution. (optional)
   Most self-respecting web analytics solutions will offer the ability to customize an implementation to support first-party cookies.  This is most commonly achieved through the use of a CNAME (DNS) entry on your primary domain, and may involve maintenance of a SSL certificate for secure areas of your site.  If you have the luxury of a CNAME, decommissioning is easy as pie; you kill the CNAME, you pretty much kill all server calls to your vendor’s analytics log server.

Know of any better solutions?  Feel free to share tactics you may have used on previous decommissioning efforts below.  I would love to hear from you.

Implementation of Omniture SiteCatalyst isn’t that difficult by any means, however validating that scripts are in place to populate variables in your solution design can be somewhat challenging for those outside of the web analytics community.  The challenge is not that it’s particularly hard to debug Omniture implementations, but it can be hard to explain why, for instance, Omniture uses separate traffic and conversion variables.  The second challenge is finding tools that support cross-platform testing and validation efforts; the subject of my post today.

Firefox is probably the easiest platform to test because most developers will use Firefox in their development environments when coding.  Firefox has a myriad of tools to choose from, but the most popular tools for validating Omniture implementations that I have come across include the following:

• Firebug, and optionally Omnibug plugin for Firebug:
  If using Firebug, activate the Firebug interface by clicking the bug icon located in the lower right hand corner of Firefox and activate the “Net” panel.  Look for image requests starting with “s” and incorporating a long string of numbers, for example, “s1230923w84…” or an image call originating from a subdomain of 112.2o7.net (for third-party cookies) or metrics.[root domain].com.  If you opt to choose Omnibug, Omniture image requests are automatically filtered and shown using friendly variable names, which makes life much easier.
• Fiddler2
  Fiddler is not usually required with Firefox, because Firebug and Omnibug are much easier to use. However if you’re testing different versions of Firefox, some with varying levels of compatibility with Firebug and Omnibug, Fiddler comes in as a strong second opinion.
• Omniture Digital Pulse Debugger
  It can’t get much easier than this… the new Omniture Digital Pulse Debugger makes validating implementations so much easier, but it comes with consequences.  First, it’s not that fast when trying to debug variables while navigating page to page, you have to give it a second to update.  Second, I’ve had issues where I had to reload it when switching from one domain to another.  Otherwise, it’s a very simple tool to use and share amongst your QA teams.

Gosh, how I loathe Internet Explorer.  It’s just so bad, at so many things, that most development teams would prefer to ignore the segment of users entirely than test in old versions of IE.  However, versions of IE such as 6, and 7 still represent significant portions of our internet audiences, so we have no choice, we have to live with testing for these *shudder* browsers.  It comes as no surprise then, that IE doesn’t have a lot of reliable plugins to test with, so unfortunately your options are to test externally of the browser itself using packet sniffers or proxies.

• Fiddler2
  Once again, Fiddler is an application that lives outside of a browser, and can make your life testing different versions of IE much easier.
• Charles Proxy
  A nice little tool that does pretty much the same thing as Fiddler, if you’re only interested in debugging.
• Omniture Digital Pulse Debugger
  Again, throw the debugging JS into a “Favorite” and testing can be as easy as clicking that favorite in a toolbar.

Chrome is relatively easy to debug with.  Check out the built-in debugger that Google programmed into Chrome to look for image requests hitting Omniture servers, or use the same options as IE above.

Mobile platforms are interesting, because many of them still don’t support javascript, but even if they did, debugging them on the device itself is never intuitive.  For platforms that supply emulators, debugging Omniture implementations is a little bit easier, but still far from straightforward for most.  Luckily, Omniture provided an excellent write-up on how to debug mobile implementations.

There is a lot to be said about implementing web analytics on a site, and using that data to improve customer experience and a company’s bottom line.  Guys like Avinash Kaushik, Eric Peterson and Jim Sterne all advocate that baseline implementations provide a boat-load of traffic data that will never be looked at in a million years, with segments out the ying-yang.  However, I’m going to argue that although free tools do offer a whack of great reports out of the box, the real power of web analytics solutions comes from custom variables.  Your web analytics tags are the cake, and custom variables are the oh-so-delicious frosting on top.  Regardless of your tool of choice, here are some of the best custom variables to implement on your site for segmentation bliss.

My top custom variables you should implement:

• Registered versus non-Registered
  This answers the question about what content motivates two entirely different sets of visitors.  Most websites these days will have various mechanisms that enable visitors to register in some way.  On a retail website this would be registering to purchase or formulate a gift registry, on service sites this might include a free user account, and on blogs the registration mechanism might be used for commenting or subscribing to posts via email.
• Logged-in versus not-logged in
  Used in conjunction with registered/non-registered, this can be used to filter out much of the mundane repetitive pathing that occurs with repeat customers/local visitors.  Subtle user behavior can often be lost in a flurry of routine page views and visits resulting from people that know your site, brand, products, and services inside and out.
• Custom site sections and sub sections
  Sometimes the layout of your site doesn’t jive with directory structure or page naming conventions, by coming up with your own silos, you’re saying “yeah, our site organization kinda sucks, but what if it were organized this way.”
• Content ID
  Particularly useful on sites with a lot of content that’s organized using a content management system.  Rather than referring to each page by it’s page name, or by it’s permalink or URL, it might be easier to categorize pages by the ID it’s assigned in your CMS.  Different strokes for different folks.  This is why discovery questions with internal stakeholders can be so important, what do your people prefer?
• Content type
  Answers the age old questions of which pages contain specialty content get the most love, for example pages with articles, photos, audio, video, flash, etc.
• User key
  Sometimes you need an identifier that resolves in two separate systems: your web analytics solution and your business system.  If you’re going to key your users, I’d recommend a somewhat  stronger privacy policy, and only tagging visitors after they are logged into secure areas of your site.
• Demographics
  Gender and age group can really help you form valuable insights into visitor patterns, but collecting representative samples can be tough on low traffic sites.  Again, use this with a bit of caution and respect for visitor privacy.  Ensure that the mechanism whereby you collect this information is documented and people know they are opting-in some tidbits of personal information.

Of course, implementing custom variables will vary by web analytics tool and website, so chasing the holy grail of data overload might not be feasible.  But if the sky’s the limit, try to brainstorm as many different custom variables you can that answer your valuable business questions.

Based on just the title of this post alone, can you guess what the #1 sticking point is between web analysts and the I.T. department?  You got it, tagging up pages across your organization’s website usually falls to the coders, webmasters, outside consultants, or I.T. folk within your company, and not the research, marketing, or business development department.  Although many software solutions are turning to single smart tags that can be defined later from within the tool, lots of enterprise class vendors still require on-page variable calls or customized javascript functions to assist in segmenting visitors.  So is perfect tagging mission impossible or the holy grail?  A rare discussion plus a Dwight Schrute bobble-head after the break.

I don’t think I’m speaking out of school here when I say, page tagging can be tough!

Step 1 – Identify the critical few, but don’t ditch the laundry list

In a world of changing business requirements, dynamic content, interactivity, and obscure “engagement” metrics, it’s essential to minimize your tagging requirements down to a critical few metrics or key performance indicators (KPI).  It’s usually easier to win over your implementation resources with a short list of “must-haves” and “nice-to-haves” versus a laundry list including the kitchen sink.

Step 2 – Champion a Web Analytics Dev Lead

This part isn’t easy because development personnel are notoriously overburdened with more important things to do (obviously!).  However, as one ally in development once told me,

“Garry, you probably taught me more than I ever cared to know about Omniture, but it helped!”

That statement implies two things.  Development gurus won’t necessarily care to know all there is to know about any given web analytics platform or tagging methodology, but with that knowledge under their belt, they can go far in increasing their own personal net worth in an organization, or expand their skill set for future opportunities.

Step 3 – Evangelize and Train Followers

Turn into a huge proponent of comprehensive data collection while evangelizing user testing, statistical significance, and challenging intuition.  Consider a strategy posed by Avinash Kaushik: talk to anyone that will listen on the business side of an organization that might support the move to data-driven decision-making, and make them profitable.  Great idea, but not that easy.  What I would suggest is this: evangelize and train enough people within an organization, and funny things start to happen.

When you evangelize and train a critical mass of “followers,” the collective philosophy of an organization becomes fragmented and ripe for change.  I’m not talking about full-scale revolution here; what I’m talking about is evolution.  All of a sudden, the demand for additional metrics and specific KPIs starts to drive the development of better tagging.

Step 4 – Impress with the Impossible

There are ways to get what you need out of a vanilla web analytics implementation.  Many tools offer the ability to slice and dice segments after the fact, regardless of how whacky your requirements are.  The limiting factor being time, of course.  Given enough time, a hundred monkey’s could calculate the value of pi to eleven decimals, and so can you.

One of my favorite asks of all time was a geosegmentation question spanning a distinct site section of a popular media site.  Of course, this question was made all the more difficult by combining conversion metrics with unique visitor calculations.  Regardless of how sexy your implementation is, you wouldn’t be able to answer this question – guaranteed.  But I’m not here to show off, I’m here to prove a point.  If you demonstrate the impossible, you will get closer to your holy grail.  It may have taken a week to process that request, but it posed the question, “was the report that I provided insightful enough to introduce more complex tagging to cut reporting time to an hour?”

Step 5 – Make due with what you got

We’re not conceding defeat, but recognizing that even with the most complex implementation and site tagging you can get, you’ll still have funkified data.  A poorly coded banner ad could mess up your variables, browsers might not accept your cookie, visitors might disable javascript, page encoding might garble your image call, etc.  It’s not the cleanliness of your data that will make you successful, it’s what you do with it.