One of F5’s best resources is its DevCentral community. On DevCentral, users can find tutorials, code samples, podcasts, forums, and many additional resources to help them leverage their investment in F5’s technologies. As an active contributor and reader of DevCentral, I was very pleased to see a tutorial on combining F5’s new built-in Geolocation database with Google’s charting API to make heatmaps to illustrate traffic patterns.
One of F5’s DevCentral employees, Colin Walker, first wrote a tutorial for using iRules to show domestic traffic patterns and then added the ability to illustrate world-wide patterns. By using these iRules, users are able to see a visual representation of how often their site is accessed from different areas of the country.
First, there’s a US view:
Then, there’s a world-view.
In both cases, the logic is relatively straight-forward. A user hits your site which triggers an iRule that increments a table on your F5 LTM. Based on the source address of the client, the F5 can determine from which state they originated and using the Google Charts API, can overlay that data onto a map, using different colors to represent different hit counts.
While this is great data, we still have to find a tangible use for it. Here are some thoughts I’ve had so far:
1. For companies using Akamai, the client_ip this iRule uses to determine the source of traffic will actually be Akamai’s server. If you want the true source, you need to change [IP::client_addr] to [HTTP::header “True-Client-IP”]. What’s even cooler is doing 1 heatmap with client_addr and 1 heatmap with True-Client-IP. The maps should actually look the same since Akamai has such a distributed computing model. Far more often than not, a user will hit an Akamai resource in their own state. If the maps aren’t the same, you have a problem.
2. Rather than simply using colors to illustrate access, keep a table of HTTP requests per state, look at the amount every 60 seconds, and divide by 60 to get HTTP Reqs/Sec for each state.
3. For E-Commerce sites that use promotions to target certain areas of the country, look at the heatmap before and after a promotion to see whether or not access from that area increased and if so, by how much.
4. If you don’t have any legitimate international customers, using the world view map can help you determine with which frequency your site is being accessed from outside the US. If often enough, it might be worthwhile using the built-in Geolocation services to block access for users outside the US.
5. Rather than looking at every single HTTP request, have the rule only look at certain ones – for instance a checkout page so you can compare conversion rate between states.
6. Same concept as number 5, but if you release a new product page, have your rule look at that page so you can determine where it’s most popular.
7. Watch the heatmap throughout the day to see during which hours different locations most frequently hit your site. In an elastic computing situation, this might allow you to borrow resources from systems that might not get hit until later in the day.
8. If you release a new mobile site, look at mobile browser user-agents as well as client ip address to see if mobile users in certain areas of the country are hitting your site more often than others. If you have bandwidth intensive applications, this might help determine where you’d derive the most benefit with another DC, or using a CDN.
These are just a few thoughts. I’m sure there are many many more opportunities to leverage these great technologies. It’s nice to see that F5 recognizes the value of including a Geolocation database with it’s product, but it’s even more impressive that they’re giving tangible examples of how to use this information to make a site better.
Another challenge is demonstrating these capabilities to the folks who make decisions based on them. In the past, IT has been criticized for finding solutions to problems that didn’t exist yet. New capabilities are being added so frequently that architects really need to look at very solution, determine whether there’s an opportunity, and then send such opportunities to decision-makers.
3 comments
Comments feed for this article
July 20, 2010 at 7:49 pm
Andrew Hoyos
Interesting read….
While F5 seems to partnered up with Quova, which is one of the more accurate databases, there is always alot of room for error (since they are based on things like RIR data, DNS, etc). Always hard to connect an IP address to a location, as there are so many different factors that could cause it to “appear” elsewhere than it really is.
One use for this for folks that host their own content, might be to find out into what areas to build a network presence and peer with providers in that area, or place their hosted content closer to the end destination.
July 20, 2010 at 8:02 pm
Andrew Hoyos
Also, was going to mention this CAIDA project:
http://www.caida.org/projects/cybersecurity/geolocation/
which is looking at the array of geolocation services out there today.
July 21, 2010 at 6:58 am
millerch
Andrew – I actually chatted with our Akamai PM about how companies are using Geolocation today. I was wondering whether more companies were starting to use it to automatically tailor a site for folks – like how certain weather sites automatically give you the zip code for your current area.
The most interesting thing he said was that some companies have found that giving incorrect information is even better as it forces a user to put in a different zip-code and through them being interactive with the site, they are more likely to purchase.
We also chatted about the accuracy. It obviously degrades as you get more precise. Country accuracy is like 99%, region is like 95%, state is like 90%, and city gets a lot worse since as you said, it’s all about who owns the IP space.
I had actually thought about your idea for using the information to determine where to locate DCs and it’s an awesome idea. Since most Global Traffic Management products can respond with a DNS address of a system closest to the user, the accuracy of Geolocation is vital.
Thanks for the comments…and also the link to CAIDA!