You are currently browsing the category archive for the ‘Work – Non-Technical’ category.

As I discussed in a previous post, simply redirecting a user to a “friendly” 404 page isn’t the best option. First, the user might not remember what they clicked/typed to get them to the page and also, simply clicking the back button might not be an option, especially if they submitted form data. Fortunately, as F5 LTMs are “Strategic Points of Control,” we can use them to better handle the situation.

 

First off, let’s determine the desired behavior when a user request induced an error code. In our case, let’s choose 404 as the error code to watch for. If we detect this error code being sent to the user, let’s redirect them to our Home Page (www.sample.com) rather than simply keeping them at an error page. To make their experience better, let’s also hold the user at an custom page for a few seconds while explaining their issue as well as the request that caused the problem.

 

Since we’re looking for error codes sent by the servers, we’ll need to run our commands from within the “HTTP_RESPONSE” event. As you’ll see from the DevCentral wiki page for HTTP_RESPONSE, there are examples for using the “HTTP::status” command to detect error codes and redirect users. For some, the rule below is perfectly acceptable.

 

when HTTP_RESPONSE {

if { [HTTP::status] eq “404” } {

HTTP::redirect “http://www.sample.com” }

}

 

Unfortunately, that rule would result in the user being sent to the redirect page without any explanation as to what they did wrong. So, we’re going to beef the rule up a bit. As you’ll recall from this post, we can set variables from the HTTP_REQUEST event and then reference them from our HTTP_RESPONSE event in order to show the user the link that caused the error.

Here’s a nice sample rule I just whipped up. We’re using “HTTP::respond” so the response comes directly from LTM. Also, I’m setting a variable “delay” to the amount of seconds to keep the user at the hold page.

 

when HTTP_REQUEST {

set hostvar [HTTP::host]

set urivar [HTTP::uri]

set delay 4

}

when HTTP_RESPONSE {

if { [HTTP::status] eq “404 } {

HTTP::respond 200 content \ “<html><head><title>Custom Error Page</title></head><body><meta http-equiv=’REFRESH” content=$delay;url=http://www.sample.com/></head>\<p><h2>Unfortunately, your request for $hostvar$urivar casued a 404 error. After 4 seconds, you’ll automatically be redirected to our home page. If you feel you’ve tried a valid link, please contact webmaster@sample.com. Sorry for this inconvenience.</h2></p></body></html>” “Content-Type” “text/html”

}

}

 

So, with that rule, the user requests a page that causes a 404 error. LTM will detect the 404 error, and instead of sending it to the user, it will respond with a 200 status code and HTML showing the user the link the requested as well as apologizing and telling them to contact your webmaster if there’s an issue. I was too lazy to use the HTML to make the e-mail address clickable, maybe next time. Also, by using “Meta Refresh,” we’re holding the user at the page for 4 seconds and then sending them to our error page. As you can see, HTTP::respond is a very powerful command. It’s pretty cool being able to use LTM to send HTML to a user.

 

 

As everyone knows, retail is an extremely seasonal industry. Retail E-Commerce is no different so when building an environment to support a retail site, architects and engineers have to plan for the highest demand. Let’s pretend cloud computing doesn’t exist or isn’t feasible in this case.

 

You’ve got a site that has an average daily peak of 50Mbps but on Black Friday, the peak is 1.2Gbps. Besides Black Friday, no other day of the year exceeds 200Mbps. Naturally ISPs can provide burstable ethernet so you’re only paying for what you use, but switches, load balancers, etc might not provide the same capability. So, you might have to build (and buy) an infrastructure that supports 10 Gbps to provide for your “peak” growth as that 1.2Gbps number might grow at 40% a year or more.

 

Before building out this environment though, it might be beneficial to learn more about your “peak” demand. For instance, let’s say the peak happens at midnight on Black Friday and that it’s sustained from 12:00 – 12:50 AM. High demand continues the rest of the day, but never exceeds 500Mbps. Why are so many people hitting your site from 12:00 – 12:50 AM? Let’s assume the marketing people tell us that they release some sort of promotion allowing shoppers huge discounts starting at 12:00 AM and going throughout the day. Unfortunately, there’s only enough inventory for 100 of each discounted item, so shoppers hit the site as soon as they’re available.

 

Before this conversation, we were planning on building an infrastructure to support that 1.2Gbps (and beyond) number that’s only hit once per year, and for only an hour. Now that we know more about why that time period is so popular, it’s time to determine whether it’s “cost-effective.” Let’s say we’re spending $1M extra to support demand that exceeds 1Gbps. If we want to avoid that spend, what options do we have to keep our traffic spikes under 1Gbps? What if the promotions are released the night before Thanksgiving? What if different promotions were released each hour during the day? What if there was enough inventory to assure all customers the items they want? What if promotions were e-mailed to different customers at different times? Obviously a marketing group would be better able to answer these questions than I, but there’s a decent chance that such methods could eliminate the short (duration), large (size) spike. Perhaps rather than a 1.2Gbps spike from 12:00 – 12:50 AM, we see a 500Mbps spike from 11:00 PM – 3:00 AM. Assuming profitability isn’t tied to when folks are buying goods, such a change in traffic spikes would allow us to delay a large expense for at least another year.

 

Naturally, retail is a great arena for public cloud. What happens, though, when all retailers are on public cloud? Wouldn’t the cloud provider have to have a huge hardware footprint to support Black Friday for all of its retail customers? At any rate, supporting seasonal demand is definitely a challenge, but it poses some interesting opportunities.

One of F5’s best resources is its DevCentral community. On DevCentral, users can find tutorials, code samples, podcasts, forums, and many additional resources to help them leverage their investment in F5’s technologies. As an active contributor and reader of DevCentral, I was very pleased to see a tutorial on combining F5’s new built-in Geolocation database with Google’s charting API to make heatmaps to illustrate traffic patterns.

One of F5’s DevCentral employees, Colin Walker, first wrote a tutorial for using iRules to show domestic traffic patterns and then added the ability to illustrate world-wide patterns. By using these iRules, users are able to see a visual representation of how often their site is accessed from different areas of the country.

First, there’s a US view:

Then, there’s a world-view.

image

In both cases, the logic is relatively straight-forward. A user hits your site which triggers an iRule that increments a table on your F5 LTM. Based on the source address of the client, the F5 can determine from which state they originated and using the Google Charts API, can overlay that data onto a map, using different colors to represent different hit counts.

While this is great data, we still have to find a tangible use for it. Here are some thoughts I’ve had so far:

1. For companies using Akamai, the client_ip this iRule uses to determine the source of traffic will actually be Akamai’s server. If you want the true source, you need to change [IP::client_addr] to [HTTP::header “True-Client-IP”]. What’s even cooler is doing 1 heatmap with client_addr and 1 heatmap with True-Client-IP. The maps should actually look the same since Akamai has such a distributed computing model. Far more often than not, a user will hit an Akamai resource in their own state. If the maps aren’t the same, you have a problem.

2. Rather than simply using colors to illustrate access, keep a table of HTTP requests per state, look at the amount every 60 seconds, and divide by 60 to get HTTP Reqs/Sec for each state.

3. For E-Commerce sites that use promotions to target certain areas of the country, look at the heatmap before and after a promotion to see whether or not access from that area increased and if so, by how much.

4. If you don’t have any legitimate international customers, using the world view map can help you determine with which frequency your site is being accessed from outside the US. If often enough, it might be worthwhile using the built-in Geolocation services to block access for users outside the US.

5. Rather than looking at every single HTTP request, have the rule only look at certain ones – for instance a checkout page so you can compare conversion rate between states.

6. Same concept as number 5, but if you release a new product page, have your rule look at that page so you can determine where it’s most popular.

7. Watch the heatmap throughout the day to see during which hours different locations most frequently hit your site. In an elastic computing situation, this might allow you to borrow resources from systems that might not get hit until later in the day.

8. If you release a new mobile site, look at mobile browser user-agents as well as client ip address to see if mobile users in certain areas of the country are hitting your site more often than others. If you have bandwidth intensive applications, this might help determine where you’d derive the most benefit with another DC, or using a CDN.

These are just a few thoughts. I’m sure there are many many more opportunities to leverage these great technologies. It’s nice to see that F5 recognizes the value of including a Geolocation database with it’s product, but it’s even more impressive that they’re giving tangible examples of how to use this information to make a site better.

Another challenge is demonstrating these capabilities to the folks who make decisions based on them. In the past, IT has been criticized for finding solutions to problems that didn’t exist yet. New capabilities are being added so frequently that architects really need to look at very solution, determine whether there’s an opportunity, and then send such opportunities to decision-makers.

Since you’re reading this blog entry, you more than likely know a bit about me and in turn, know that I’ve recently accepted a new position with Kohl’s Department Stores as a Network Analyst focused on their E-Commerce infrastructure. I plan to write an entry about why I decided to leave Direct Supply, but this won’t be that entry.

It’s now been 2 weeks since I began working at Kohl’s and it’s gone ok. I really miss the people at Direct Supply and I also miss being the “go-to guy” for so many different systems. It took me awhile to become the subject matter expert (SME) for Direct Supply’s network infrastructure and while my goal is to fill that SME role for Kohl’s E-Comm infrastructure, it’s going to take some time. Starting over is never fun!

I really like my new leader so far. He has a great handle on the technologies Kohl’s requires to be successful. That really helps him get us (his team) the resources we need as far as training, mentoring, etc. During my first couple days, he was kind enough to introduce me to at least 50 people. Each time he introduced me, he said, “this is Chris” and then some sort of derivative of  “he’s the F5 guy.” The thing about which I was most surprised is that everyone to which he said that immediately understood why I was there, what I was doing, and how critical it was that I do it. I was being introduced to VPs who already knew how Kohl’s depended on the ability of the devices I supported. What an amazing concept!

When I left Direct Supply, I spent the last couple weeks training my coworker and another person on how easy it was to use F5’s systems to accomplish tasks. I already knew F5s were easy to use…gotta love that GUI. Seriously though, I caught myself thinking that it was my familiarity with F5 equipment that led Kohl’s to take an interest in me. Since I was able to teach someone 95% of the tasks they’ll need to know in just 2 weeks, obviously familiarity with the systems weren’t my value proposition.

Through being constantly referred to as “the F5 guy,” I started to realize what “F5” meant to other people. While to most of us, it simply means an Application Delivery vendor, to others, it means Application Delivery on its own. It’s almost like when I ask someone for a “kleenex” instead of a tissue. Wow, imagine your name being interchangeable with the product you sell. I’m not “valuable” because I know how to get around in an F5 device. Sure, I can create pools, virtual servers, set BIG-IPs up in a high-availability pair, but all of those things can be done but just about anyone. It’s understanding why you’re doing it that’s so powerful! My backfill position suggests it would be advantageous for an applicant to be “knowledgeable” in load balancing technologies and that “F5 would be a plus.” While I have my strong feelings about that whole sentence, it will be addressed in the next entry I right. Back to the point – it’s all about understanding why you’re using a load balancer, and then why a load balancer isn’t enough, and then why going with F5 is key.

My current career goal is to get into more infrastructure architecture and design. Being at Kohl’s has been great because we’re essentially doing a re-design of our E-Commerce infrastructure. It’s always an amazing challenge when you’re given an anticipated amount of Page Views Per Second, told that demand can fluctuate 100-fold depending on the time, and that downtime is absolutely not an option. Oh, and response time is critical! Someone who is “knowledgeable in load balancing” will likely understand that using a load balancer in this scenario would help spread load across servers and even ensure that healthy servers are the only ones receiving traffic. Someone who understands Application Delivery, though, will understand that your F5 device won’t just load balance, but it will also compress data, cache it, and get it to your users faster. That person also understands how to use contextual-awareness to accomplish those goals. So, you want to compress data but only when it’s beneficial? Well, someone who is “knowledgeable in load balancing” is going have a tough time with that. On the other hand, someone with a good handle on how applications, servers, and networks interact should be able to say, “hmmm, let’s figure out which browsers have problems with compression and also, since compression can actually add latency when content is accessed over a LAN, let’s only compress if the RTT is longer than 1 second.  So now, only someone with Internet Explorer 6+ or Firefox 2.5+ that’s accessing our pages over a “slower” link is going to get compressed data. That, to me, is what it’s all about. If being introduced as the “F5 guy” can convey that I’m an application delivery guy, then by all means!

I’ve only recently begun understanding how critical contextual-awareness is to the future of technology. Now that we access sites from phones, laptops, desktops, iPads, video game consoles, and TVs, the industry needs people who can understand not just how those devices are different, but what that means to delivering content. Today, there are people who understand things well enough to say, “if you’re a mobile user, you should enter mobile.***.com so you can access our specially designed mobile site. That’s not going to be good enough guys! Your systems had damn well better be smart enough to view that user’s agent and automatically direct them to your mobile site. Oh, and if they’re not using 3G, you’d better send them to your “light-weight” mobile site.

The above paragraph is a perfect example of why I love application delivery. Did Direct Supply “let” me tackle problems like this? No…because we didn’t necessarily need them solved. If we did, it wasn’t a “Network Engineer’s” job to solve them. Kohl’s definitely understands and needs its network folks to be creative and understand their customers, but whether or not I’m the guy who’s able to accomplish things like this, we’ll have to wait and see. The first part is to find the problem, the second part is to make sure I’m the person to whom they come when they realize the problem needs to be fixed! A big reason I’m pursuing my MBA soon is so I’m the guy who can find the problem, understand why it needs to be fixed, and then understand how to fix it. Whether all those functions exist in my role is another story, but at least I’ll have the tools.

If you chose working in IT as a profession, you probably didn’t do so in order to be recognized. If you work on the Infrastructure side of IT, you likely encounter plenty of people who don’t know what you do. My team, which encompasses Data Center, Network, and Telephony, actually has the mission statement “We Work so You Can.” It’s through our laying a technical foundation that our customers are able to use their applications to generate revenue. While it’s unlikely that a customer realizes how my team fits into their experience, they will certainly learn about and notice us if we have an outage. The way I’ve always explained it to people is that my goal isn’t to make money, it’s to prevent our organization from losing it. When I choose to replace an End-of-Life switch, it’s not going to make us any more money. Certainly there might be enhancements that speed up content, but tying that to increased revenue is hard. Instead, I replace the switch because the other one has a higher risk of failure which can cause our procurement engines to be offline.

Take QoS for instance. Since our campus consists of fiber connected buildings and all of our applications are run over our LAN, bandwidth has never really been a concern. Of course, before we could do a full-campus VoIP conversion, we had to do a network readiness assessment. Since I couldn’t find a good wiki link and don’t want to force folks to click on a sponsored link, especially one from which I don’t make money; I’ll explain a bit about the assessment. Since VoIP uses UDP and not TCP, it is easily effected by network latency. This often manifests in dropped words, echoes, static, etc. In order to ensure a network is properly capable of carrying voice calls, a vendor will formulate a test plan and measure the network’s readiness. Since we have about 850 phone users on campus, we had to come up with a maximum number of concurrent calls to simulate. While the number we chose is of no consequence, it’s the exercise that I found very interesting.

We knew going into the assessment that we might peak at around 100 concurrent calls. So, if we’d used 100 calls, we could have certified our readiness for typical load conditions. Of course, we also knew that we were growing at 20% a year and adding roughly 50 employees during the same time period. If we assume our network and phone system need to last 5 years, we can extrapolate that number to about 200 concurrent calls. Since our system had to be designed around and tested with a certain number of maximum calls, someone had to pick the number. Since whoever picked a maximum number would probably be accountable if our real-use ever exceeded that number and a failure happened, I imagine the number would be pretty high. Having QoS enabled was also part of the check. Since our LAN was designed to never be the bottleneck, we didn’t really need it on. That’s the thing about technologies like QoS. In most cases, QoS doesn’t fix a problem. It prevents a problem from happening in the first place! If I have bandwidth issues, I should implement QoS to prioritize traffic but I should also investigate and determine whether I need additional bandwidth. QoS is really just there to minimize the impact of bandwidth constraints to our critical systems. Interestingly enough, we had some spirited debates because we didn’t feel like our not having QoS enabled on our LAN should be a success criteria for our assessment. When our vendor told us that a single user could fully saturate our LAN by FTPing something, I knew he couldn’t prove it. He’d have to find an FTP site that could read/write at 100Mbps just to saturate a single user…good luck having a client machine that can even handle a transfer like that!

That’s when I believe “What if You’re Wrong?” thinking comes into play. Many of us refer to it as being “What If’d” to death. Having a solution challenged is extremely valuable because it forces the designer to consider possible risks. There should be a limit to the validity of those challenges though. Since there’s always a “What If” scenario around which you can’t design, (nuclear war comes to mind) the risk discussion can’t go on forever. The problem system designers often face is knowing most of the built-in redundancies might not ever be used. Imagine only having your work noticed if something goes wrong? I suppose that’s often IT in a nutshell. It’s also during failures that blame begins to be assigned. If your system has an adequate level of redundancy, the outage will be transparent to most users and perhaps others in IT will commend you for your good planning. On the other hand, if you didn’t build enough redundancy or didn’t account for a certain risk, you’ll be noticed by everyone and not in a good way. While we might not believe that QoS is required or that it’s a good use of our time, we’d still be prone to implementing it because there’s so little upside and so much downside. If having QoS enabled prevents a problem 1 in 100,000 times, I’d almost be tempted to implement it because I don’t experience any benefit by not implementing it the other 99,999 times we don’t experience a problem.

That brings us to two options.

1. Design a solution with every risk in mind that might never be used and even if it is, might not get you any commendation outside of “way to do your job.”

2. Design a solution that is most cost-effective but doesn’t necessarily plan for every risk.

Naturally, most of us are prone to option 1. While people are constantly spending time defining the probability of certain risks and assigning a dollar value to them (lost revenue x probability of risk), we always play it safe. One reason might be that system designers don’t experience any of the benefits of having a cost-effective solution. They certainly experience the negatives of not having an extremely resilient one though!

Outside the technical world, we all face the “what if you’re wrong” question all the time. Consider the amount of death bed religious conversions that happen and why. From my perspective, there’s no downside for someone about to die to become religious. From their point of view, if there is a heaven and they’re about to experience death, they should obviously increase their chances by whatever means they can. The reason most of us get insurance isn’t because we experience positive value from it, it’s because our lives aren’t long enough for variance to kick in. If 10 of us pool money together to buy insurance and only 1 of us ever uses it, 9 of us have essentially thrown away money but since none of us could have afforded to have an injury, it was still the right thing to do.

Since I’ve jumped around quite a bit, I should probably re-state my theme. Asking “What if” or “What if you’re wrong” is an extremely valid question in IT. It should, however, be exercised in moderation. A systems designer should have clear requirements about uptime and from there, exercise their own discretion given a fixed budget. If you have $10,000 and want someone to create a hosting solution, you should be prepared to have a dialogue about your requirements. You can probably get 99.99% for that, but if you want that extra 9, you obviously have to change your budget as well. If given a clear budget and a clear uptime requirement, an architect or engineer should analyze risk and build-in proper redundancy. It’s not fair for an IT person to assume what level of uptime someone wants because we’ll always assume 100% uptime and design accordingly. We’re not usually recognized for uptime…we’re noticed for downtime. That naturally leads us to “over-build” solutions.

Part of our “Performance Management” program entails us identifying and being measured against Quarterly Objectives. This applies to pretty much everyone in the company. My leader decided to make an objective for himself to build a “High-Performance Team.” What I thought was a pretty clearly positive idea drew a bit of discussion once he blogged about it.

The central theme of dissent was that every team should strive to be high-performing and implying that our team declared itself high-performing, we would by default be contending that others weren’t. While I agree that every team should and hopefully does aspire to be high-performing, I see the value in identifying and attempting to measure such a goal. It’s only through measuring that we can find out whether we’re successfully reaching a high-performance level. I’m not sure if it’s just my personality type, but I can’t stand ambiguity as it relates to goals. That’s probably the reason I love working on systems that have an ability to impact performance. By measuring our progress toward the goal of being a high-performance team, we can see what we’re doing wrong, what we’re doing right, and work to identify our gaps. If executed properly, this can only lead to improvement and perhaps a bit of inspiration for others. After discussing more, I was made to understand that the issue wasn’t with the objective but with how its success criteria would be communicated. A good challenge indeed!

I’ve seen plenty of goals, missions, visions, etc that use phrases like best-of-breed, world-class, and top-of-the-line. Without quantifying what those terms mean, you’re unable to determine whether you’re successful. Sadly enough, that is sometimes the goal of such statements. If I don’t tell you what having a world-class infrastructure means, you can’t tell me that I’ve failed. Even better, I can come up with 10 initiatives to reach such a goal, ignore the failures, and say that the positive ones should be attributed to my great planning. That’s where accountability comes in. I think it’s through seeing so many ill-measured initiatives that I’ve become obsessed with identifying success criteria up front. It gets a bit more difficult though when it’s someone higher up the ladder coming up with the ideas. It’s not so easy to challenge whether they’ve in fact been successful.

I’m re-reading “The World is Flat” and really enjoying it.  I’d encourage everyone who works in IT or business to read it as quickly as possible. I didn’t have as much “real-world experience” my first time through so it’s nice to read it again and actually be able to apply things. I just finished my 40th page or so and had to post something. Friedman is in China speaking with a city leader who tells him about how the US currently architects a building and has China build the bricks. This Chinese man wants to see China start doing the Architecture too. I was really captivated by this point because of how often it happens. I used to work for a company called APAC who basically handles inbound and outbound calling for companies. Instead of outsourcing calling to China/India, a company could send it to APAC. Let’s say APAC someday decides to re-outsource their own campaigns to China. Consider what they’ll teach the Chinese in order for them to become capable to take calls. They’ll take them through company education, maybe some tech support training, and likely some accent-neutralization classes. So, the product is cheaper labor capable of doing the same job. It isn’t too long after that the Chinese company realizes how easily they can do what APAC does. Now, APAC has essentially created and educated a competitor that companies can hire for 1/5th the price.

The same applies to IT. Dell can hire companies in India for a fraction of the price but it still has to teach them a bit about how things work in Dell. Again though, how easy is it for that company to start selling “tech support for Dell computers” to customers or other countries?

For those who have not yet heard, I was fortunate enough to have been selected to the University of Wisconsin’s Evening MBA program beginning in the Fall of 2010.  Since I’ll basically be giving up every Monday and Thursday evening for 32 consecutive months, I felt it necessary to provide my readers with some background information.

As I’ve become infatuated by Application Delivery, understanding my customers and the impact of my decisions has become a high priority. Since my goal is to be an effective business leader who happens to specialize in technology, I want to make sure I have the best possible skills at my disposal. As I’ve executed projects recently, I’ve found myself asking questions of people who don’t necessarily have good answers. I want to know whether putting an extra power supply in a device is worth the cost. I want to know whether shaving 200 milliseconds off page-load time makes us more money. I want to know how to articulate the value proposition for a organizational changes. Most of all though, I want to know how to be as effective as possible!

UW-Wisconsin describes their program as follows – “The goal of the Evening MBA program is to create a challenging learning experience that integrates students’ in-class activities with their daily practice on the job. Applicants are screened and selected to produce a diverse blend of academic backgrounds, professional disciplines, industries, and personal perspectives. In many of their classes, students will work on team and small group projects, reflecting the need for cross-functional approaches to problem solving found in today’s work world.” Since the groups with which I want to work aren’t strictly composed of IT folks, being able to interact with groups like Finance, HR, and Operations is a must. The course content aside, I’m very excited that UW-Madison brings together people with each of those backgrounds and lets them work together in groups.

My first 3 classes will be “Financial Accounting,” “Values-Based Leadership,” and “Managerial Communication.” I’m certainly more excited for the latter two courses but definitely understand the value in honing my accounting skills as well. All in the interest of being more valuable to my employer!

Given the huge time and monetary investment, I want to be as specific as possible about the motivation behind my decision. It was earlier in this year that I began asking others and myself questions for which there were no good answers. I was faced with opportunities to spend 100s of hours of my time to research and implement features on our application delivery controllers that would speed up the delivery of content to our customers while also making our hosting environment more resilient. To many, implementing such features would be an obvious decision. Unfortunately, since I’m usually spread across so many projects, I need to ensure I’m able to properly prioritize my tasks. For these reasons, I wanted to understand the “value” the business would derive from my efforts. A more stable environment provides obvious value to customers, but how much? If our environment is down, will a customer simply wait for us to come back up or will they order elsewhere? Does our reputation take a hit such that the company will do less business with us? I’m not naive enough to think that getting an MBA will make me a good manager or help me answer questions like this. I simply expect the courses to give me the necessary “tools” to find the answers on my own. As I said in my admissions essay, I believe now is the right time for me to begin an MBA program because I’ve encountered opportunities in which I can apply what I learn. I’m very excited to learn skills on a Monday night and be able to use them the next day.

I’ve created a new Blog Category for my MBA entries and will place them as such. While the program doesn’t begin until next August, I’ll continue discussing my goals around the program.

I was a bit bored this weekend and decided to watch some documentaries available on Netflix Instant-View. The second one I watched was “Wal-Mart: The High Cost of Low Prices.” It bashes Wal-Mart for driving out hometown stores, ruining towns, paying poor wages, encouraging underpaid employees to go on social welfare programs, etc. As we were watching the movie, my wife and I had a long discussion about whether Wal-Mart was doing “the right” thing to each group of people to whom it was accountable. Part of the movie discusses how Wal-Mart uses labor in India and China. The movie shows quite a few employees who work 84 hours a week for about $3 a day or less. It’s cheap labor like that which allows Wal-Mart to sell items more cheaply than other stores. The question then becomes, should Wal-Mart be concerned with the welfare of its customers, its domestic employees/shareholders, or the people to whom it outsources employment? In a perfect world, the answer would be all of the above. If Wal-Mart pays Chinese workers wages more comparable with those of domestic workers, Wal-Mart won’t be able to sell its items as cheaply. This will likely cause customers to look elsewhere which in turn will cause Wal-Mart’s stock to go down which in turn will cause its domestic employees to lose value in their options/retirement plans. Most agree that they’d have more respect for Wal-Mart if it used all domestic labor and paid better wages/benefits but in a world where respect doesn’t always equal revenue, what are they to do? Can you imagine a single mother who struggles to give her children everything they need trying to explain to her son that he can’t eat dinner today because Wal-Mart had to raise prices…but it’s ok, because we “respect” them more?

Then, there’s the discussion about Wal-Mart not paying its domestic associates enough. There were some accusations that Wal-Mart wouldn’t pay overtime when employees worked it. That’s something with which I absolutely disagree. However, the discussion about Wal-Mart not providing good enough insurance or pay is one with which I struggle to take a position. Typically, if you’re unhappy with the benefits/pay Wal-Mart provides yet still work there, it’s because there isn’t an alternative. Can we be upset that Wal-Mart has more employees on Welfare in certain states than any other company? It would be tough to feel that way knowing that Wal-Mart employs more people than any other company in the state. We should also consider how much more utilized the programs would be if Wal-Mart didn’t exist at all.

The general theme I derived from the movie and outsourcing in general is this – Someone has to be happy and someone else has to be unhappy. When a company chooses to outsource jobs, it typically does so at the expense of some employees but because it’s able to operate more efficiently, customers often benefit and “higher-level” positions within the company do as well. I’m sure it doesn’t help Wal-Mart’s case that the 3 primary shareholders (the remaining Waltons) have $90 billion between them.

I’m sure those of us in the IT industry will be asking ourselves these questions more and more as support and even hosting become outsourced more often. If it takes 10 people to support a company hosting its website while Amazon can do it with just 2, there go 8 jobs. The company has saved money and might be able to offer its product more cheaply. The folks still employed by the company might even get raises or see their stock appreciate. Of course, none of that helps the 8 people who no longer have jobs.