Traffic Metrics Require Context

Oct 2 2012

Last Friday, Chris posed a question to the team regarding the traffic numbers that various analytics tools provide: why do we trust the numbers so much, regardless of their source? This got me thinking… why do we put so much faith into the numbers we’re looking at? Are we applying the correct context to these metrics?

I did some further research into two different tools we leverage at RD2, Inc. (AWstats and Google Analytics) to provide a reason as to why the numbers don’t line up.

AWstats (Awwww, stats!)

AWstats is a product of the late 90s as a solution to address the concerns of “how many people are visiting my website?”; I can only assume web hosting during that time wasn’t nearly as cheap as it is today and ROI was an even bigger focus than it is now. Getting people to the website then was a big deal; today, getting traffic could be viewed as a relatively easier task (given the social media technologies at our disposal.) AWstats has been updated several times over the years, but its base function has remained the same: present the data in access log files in a way that makes sense to the layman (read: graphs and tables.) This is where the question, “So it’s like Google Analytics?” comes up and my response is along the lines of, “Ehh… in a way, but not really.” AWstats provides a more easily read representation of the server access logs… so what does that mean when talking “traffic” and “visits” and “page views”? Well to be quite honest, it means nothing without an understanding of how the tool works and appropriate context is applied to what is being viewed (we’ll get to this in a bit.)

Google Analytics (or, cloud-computing crunching the numbers and giving you insane amounts of detail on your visitors behavior)

Positioned neatly among the many tools found in the Google webmaster’s “tool belt”, Google Analytics is a platform offering many different methods of quantifying, displaying, processing, comparing, and filtering data on user behavior while visiting a web property. Google doesn’t charge for their service (which is interesting, seeing as it’s one of the more powerful tools they offer) so pretty much anyone with a Google ID can use it. When discussing the “how” tracking occurs, Google’s method is completely different: JavaScript.

So why would I use both?

Scenario: Company X provides a variety of consumer services online and in-person. After launching a new version of their social home base, Company X wants to keep a diligent eye on visitor traffic patterns as a means of quantifying whether or not the new platform is successful. Eager Analyst 1 working for Company X goes to Google Analytics and prepares a detailed report based on this information; Eager Analyst 2 prepares a similar report to Eager Analyst 1′s Google Analytics report, but opts to include data from AWstats. During the presentation, all of the numbers line up between the two reports until AWstats is discussed. For the first quarter of the year, Google Analytics reports 613,718 total pages viewed, whereas AWstats reports 3,740,453 total pages viewed. Logic tells us that these 2 numbers don’t really match up; AWstats is reporting approximately 6.1 times higher than Google Analytics. So which number is correct? As it turns out, both are. Stay with me for a minute and I’ll explain why that is… If you recall, I mentioned a tool is only valuable when the correct context is applied; in this case, it’s the definitions of the terms being used. This is key when reviewing information from any source, e.g., store foot traffic metrics, the number of customers served, and the number of visitors. So we know that Google Analytics tracks visitors using JavaScript upon page load and AWstats reads log files, great; is it possible Google has a margin of error this large? Highly unlikely… Digging into the definitions used for “page” with both services, I was surprised to find that the definitions are widely different. Google Analytics defines a page view as a single page (www.rd2inc.com and www.rd2inc.com/team are 2 different pages to Google Analytics, loading both pages twice would result in 4 page views); AWstats definition is very different. When calculating “page” views, AWstats calculates the page itself, any additional PHP functions run to display dynamic content, and AJAX queries to the server that occur after the page has loaded (so for each 1 page, there could be 6+ page “views” when a content management system is used for page generation.)

Aside from pages viewed, another variable metric differing between the two is how user sessions are tracked. By default, AWstats tracks user sessions in what is roughly 60 min segments and Google Analytics tracks in roughly 30 minute segments. Since Google Analytics is JavaScript based, multiple users from the same public IP address are tracked separately; AWstats would view all visitors from the same public IP address as one unique visitor, and if multiple people from the same public IP address visit Company X’s website within approximately 60 minutes of the first visit from said IP address, the number of unique visitors will be greatly skewed (this is a limitation stemming from how AWstats captures its data.)

The take away from this shouldn’t be that AWstats is antiquated and shouldn’t be used; it should be understood that each tool has its strengths and opportunities (AWstats can track which pages are being crawled by search engines, Google Analytics can’t.)

So go forth, review the traffic metrics and make sure the data being presented is presented with the correct context applied


Reference URLs
AWstats

One Comment to “Traffic Metrics Require Context”

  1. On planet earth we must look to multiple sources, gather the evidence, and ask the hard questions. Sure, some of us may be showing our age by bringing up AWStats. But there’s always the hair that stands up in the back of my neck when we covet one tool as the authoritative tools. Competition is good. Parity makes better products and better people.

    By Chris Ronan on October 2nd, 2012 at 11:54 am

Leave a Reply