I noticed on the IRM posted their April website stats on their blog. I found the numbers very interesting. But made me wonder how many of the hits were spiders vs hits by real people. I also wonder how many organizations are tracking this sort of data and more importantly how it is being put to good use. Any comments or thoughts?

http://www2.irm.org/blogs/archives/917- ... -2011.html

The website tools for the host I use allow you to view hits by spiders seperately from actual visitors. You can also see whether they were directed to your website by a link or a message in an online mail system.

They also offer detailed stats into what pages on your site are being accessed, and how long they spend looking at your site. Both of these in conjunction with your visitor counts would be useful in determining where to spend your limited IT resources in expanding or improving your website.

You can also see what kind of browser and operating system they use; which may be useful if your content or product or geared towards a particular browser and operating system.

Why the IRM made these states publically available; my guess is that they have a wealth of stats behind the scene that are not available to the public.

We have used the web stats to track impact of media coverage, as well as the relationship between web hits and attendance.

Wesley

Like many things, what starts out seeming to be simple can get complicated in a hurry. Webstats can be useful information or it can be a can of worms that wastes your time.

Spiders are both good and bad. They shouldn't count as site visitors, but they're what bring many of the visitors in the first place.

The next question is how to track visitors. The old method of simply using "hits" has given way to more sophisticated menthods.

Pageviews is the simplest method, counting how many times each page is viewed.
Tracking unique visitors is a bit more complex, but it's also a better measurement.

Stat programs can be fooled by a lot of things, and some services, notably AOL, cache your pages off site, so you may not see of the views you're gettng.

My suggestion? Use the info, but don't obsess about it! Look at the big picture, trends, and that sort of thing. Did you run an ad and get twice as many hits as usual? That's useful information.

Did you look at the IPs to see where your visitors are from? Not so useful, as they may show as being in a totally different location, for example I think AOL often shows up as "Fairfax Virginia", even if your guest is actually in Seattle. I just checked mine, and it shows me a good 40 miles from my actual location. That could fool you if you were trying to see what locations a specific ad was reaching. You might think your Seattle Times ad was doing great while I actually saw it in the Tacoma News Tribune.

A couple of general comments about your website, based on years of observation.

First of all, visit the main page of your website. Pretend you have no idea what your operation is, or where it's located. How quickly can you find that info? You'd be amazed at how many websites say something like "West Winfield and East Bugbite RR" but give no indication what state or country they're in? "Located just 15 minutes west of Bugbite on Highway 21" doesn't help either. This is the worldwide web, make that info obvious and easy to find.

Make sure the info is up to date. It's May, does your site still advertise the Christmas Trains?

Sell tickets online. Yeah, it's a PITA. Yeah, the ticket vendors take a big cut. But you'll sell more tickets, lots more.

Make the ordering process simple and easy too. Once you have it all set up, bring in somebody's grandmother, say "Go buy tickets for you and the grandkids on this train ride", then sit on your hands and watch her. If she can't do it in less than 5 minutes, without ANY coaching from you, you have a problem.,

I hate the term "hits", it is not an accurate measure of visitors. Our old site had a million hits a month, that worked out a couple hundred visitors a month.

We recently reworked our site. The old site was old, and we really did not have anyone who understood the modern web, so the site suffered. Our new site was developed in house, along with the ticketing system, and is working really well. Selling tickets online, a must. As much as I hate Paypal, that has worked out really well too. Last time I talked to the web guy, we were running around a 100 unique visits a day.

Here is the site, it still needs some info filed in, but is doing very well.

http://www.hoosiervalley.org/

Jason Annen
HVRM

I should also note that we geared our site towards the general public, as a destination for something to do. The general public provides over 90% of our income. Mom and her kids are the number 1 customers.

Jason Annen
HVRM

What makes it difficult is the various technologies at hand. When you browse a website, that is a "stateless" transaction: you load the webpage, you're done; when you go to load another webpage, it's a whole different interaction, and the server doesn't "remember" you.

Needless to say, they started working on schemes to "remember" you. The most effective scheme is "cookies". Imagine you want to measure usage at an aquarium. At each tank you position a volunteer with a stack of ID tags. They give an ID tag to any visitor who is not wearing one. They log the ID number of that tag, and also any others they see. Someone is given ID 24601 at the shark tank, and they will then be recorded at the octopus and salmon tank. That's the idea. However, nobody makes people actually wear them, and some people toss them. Very few people do, but most who do are robots, and they visit every tank about once a week. All who don't wear ID tags are given a new ID tag at each tank with a different number. The ID tags are magical; only your staff can see them. Now you have the basics of how cookies work and why they are tricky. What if no-one tells the volunteers to ignore robots? They burn a LOT of ID numbers, and they can skew your numbers to where you think most visitors visit only one tank, and never return.

I will leave out the "Third party cookie" issue, where you allow DoubleClick into your aquarium to hand out their own magical ID tags, which they also hand out at the zoo, symphony, Borders, teabagger rally and whorehouse, so they can track people's movement among all.

Your browser handles the cookies for you. Its default settings are highly promiscuous; take every cookie, even third party's, and wear them forever. By the way, the "Browser" is the thing on your computer that makes the internet work. Many people do not realize it actually is an application you run, like Microsoft Word or Solitaire. Common browsers are Internet Exploder, Firefox or Safari.

Anyway, back to IRM and their cookies.

I checked twice on IRM. The site "www.irm.org" gave me a cookie called "BANID=1". That's like giving everyone an ID tag with the number "1". Not useful for tracking, more useful for "I've seen this guy before, so he's not new". This cookie is set to expire when I close my browser program. (cookies can be set for longer, 20 years in the case of the doubleclick cookies, but this one is not.) That means after the person quits for the night, they're a new visitor.

The IRM blog site "www2.irm.org" gave me a cookie "PHPSESSID=l0luuekoblahblahrandomcodes". When I visited it with a different browser, I got a different cookie. This is also set to expire when I close my browser. This cookie is unique; it distinguishes me from another visitor. However, this cookie is on www2.irm.org. http://www.irm.org cannot see it. (it's possible to set a cookie with a domain of ".irm.org", note leading dot, which both sites could see; but they didn't.)

So it must be monitoring unique users by noticing when a user arrives who DOES NOT have the "BANID=1" cookie. As for repeat visits, I can only guess it's looking at IP address, two visits in the same month from the same IP is a repeat visitor. About that:



An IP address is the routing address of your PC. It contains passive geo-location information. Take 12.3.221.44. The 12. says this is an AT&T customer; the 12.3. says the Valley, 12.3.(216-223) is a pool of IP addresses allocated to AT&T customers in Burbank. There are people who have figured all that stuff out for most IP addresses, and that data is readily available. That is how you get ads that say "Find sexy singles in Burbank" and how IRM can know its percentage of USA visitors.

An IP address is a 32 bit number, for 4 billion possible IP addresses. There are 6.9 billion souls on our big blue marble, and half of them have iPhones. PROBLEM. Yes, we are "feeling the crunch" on IP addresses. AT&T can no longer afford to dedicate a whole pool of IP addresses to possible customers in Burbank. The pools are being merged, which means the quality of geo-location is declining dramatically. In a year, we expect to be lucky to tell which coast a customer is on. If you own a website and you want to get specific "down to the town" geolocation data for your customers... DO IT NOW!

A new IP numbering scheme, called IPv6, is almost ready. None of the major websites want to enable it, for fear they'll look broken if someone's ISP or personal PC is misconfigured. So all the majors agreed to turn it on, on June 8, to see what happens. See if you are ready, here: http://test-ipv6.com/

The Internet just keeps getting more interesting.

The IPs in a pool like that one are dynamic, i.e. they change. AT&T doesn't have enough IPs for everyone who has an account, just like they don't have enough phone lines for everyone to make a phone call at the same time.

When you log in, you have an IP assigned to you, and you get a "lease" on that IP. It expires after a set period of time. You then renew your IP, which may, or may not, assign you the same number.

Some pools are assigned to a specific area while others are nationwide. (Well, they're all assigne to a region when it comes right down to it, but some ISP don't provide that info publically)