Are you sane, rational, and of sound mind? And your colleagues too? Are you sure? You are! I’m so glad! OK then, given all that, how come you spend so much of your time making daft decisions, and endlessly back-tracking and re-opening issues you thought had been resolved? How come you have projects that are not aligned to orgainzation’s strategic goals? Do you even know what those goals are? Which ones are the most important? Honestly? You fibber!
Well, last fall I was introduced to some eye-opening decision science tools and techniques that made me realize how “decision illiterate” I was. I’d like to share those experiences with you here. And if you’re in a telecom-related industry, this should give you some ideas on how to effect change in your organization towards and end-to-end compatible business model.
Some groundwork first. We’ve all seen an athletic race, if only on TV. The competitors may have been wearing a number. Those numbers are merely labels. Being assigned #2 doesn’t make you any better that the person with #5 on their back. The race is then run, and a big scoreboard shows who won. Whoever came first got a gold. The margin of victory was immaterial. Coming second by a slim margin doesn’t get you a silver medal with gold trimmings. The board will show how much the winning margin was. Was the race a close one? And it will also show the absolute times. A win of a second on a 100m race is a very different thing to winning by a second in a marathon. The total time puts the winning interval into context. Was competitor #2 10% faster than #5, or .1% faster?
There are four types of number here: nominal numbers, which act as labels; ordinals, which give position; intervals, which show the differences; and ratio numbers, which give relative performance. You can compare ordinals; add and subtract intervals; and divide and multiply ratios. You can’t do arithmetic on nominal numbers.
So that’s some basic terminology sorted out. Now, what about the confusing world of business?
I bet you’ve seen the following happen. There was an annual budget round, or some other big resource allocation decision. A guru from finance or strategic planning was tasked with producing the World’s Most Complex Spreadsheet. Filled with tabs and links, it lists the options and the measurement criteria. Some criteria are hard numbers, some are softer issues that reasonable people could differ about. Each option is given a score for each criterion. That score could be its rank (inverted, to make the best one score highest). Or could just be a simple scale like 1-10. Or it could be some formulaic derivative of something like expected revenue. At the end, they all get combined by the formula from Hell (or just SUM), and a summary splurged into a Powerpoint deck.
Then there is a Big Meeting™. A number of exceedingly well paid executives are called in to bless the result. But they don’t like it. There is a big argument, and some “adjustments” are made. Success in getting up the priority list is largely guided by force of personality and imaginative over-statement of expected project benefits. A few weeks later the senior VP of finance comes back from vacation, decides she doesn’t like the outcome, and strikes out a project or two. In the meantime, a product development team keeps working on a project that didn’t make the cut, because the company has already invested so much in it, and you can’t kill the thing now. It’s just work in progress, you know. The organization flounders in meeting it’s mission.
So, what happened? Well, several things. Firstly, the values for the “soft” issues in that spreadsheet were probably bogus. A small clique of people got to guess at them. There was little agreement on what the criteria should be. The weightings were pretty aribtrary, at the judgement of the “strategic” folks at middle management level. The criteria were also probably both incomplete and overlapping.
But that’s not the worst bit. The biggest problem is that the results of such a spreadsheet are meaningless. You’ve made a classic Computing 101 error: you’ve combined things of incompatible types. Abort, Retry, Ignore. Blue Screen of Death. The spreadsheet was riddled with ordinal, interval and ratio numbers. Just giving ranked items labels ‘1’, ‘2’, ‘3’, … doesn’t mean you can then add up ranks for different criteria. After all, you could have called them ‘A’, ‘B’, ‘C’, … and A+C isn’t D — it’s a total wibble. You can’t multiply “$10m revenue boost” (an interval) by a weighting and assume the resulting score is twice as good as a $5m revenue boost somewhere else in your model.
I’ll say that again. Such a spreadsheet has no meaning. None. Zero. Throw it away. Stop reading now — come back once you’ve emptied the trash can and deleted the contents of your PC’s recycle folder.
Now, an alternative method is to throw away the spreadsheet before you start. Present the raw data. Get the decision makers together. Hold a Bigger Meeting™ (start at 7am, donuts and strong coffee included). This process has a name. Quite a cute name, too. BOGSAT. Bunch Of Guys Sat Around Talking. Next time you’re in a tedious meeting trying to make a group decision around some directionless agenda, that’s the methodology you’ve adopted. No structure, no collaborative decision, no buy-in. Biggest ego wins. World’s #1 choice today.
OK, so what to do about this mess. Well, these problems were identified in the 1970s. Two business school professors came up with a solution. It’s called Analytic Hierarchy Process (AHP). Not the obvious name to choose if you want to market a methodology to a general business audience. Metaphyscial Acronym Confusion. It puts people off. Just imagine it’s called Collective Justifiable Decisions — trust me, it’s good, despite the academic naming goof.
So, what are the key concepts? A lot is familiar: sets of alternatives, scoring criteria, and numeric scores. But there are some special twists.
Firstly, the criteria are put into a hierarchy (the “H” of “AHP”). Your strategic objective of “increase revenue” can be broken into “enter new geographic markets”, “launch new products”, “increase prices”, etc. Want a quick-and-dirty decision to avoid all those icky costs of feasibility analysis? Just stop at the 1st or 2nd level of the hierarchy when evaluating alternative options.
Secondly, everything is a ratio metric. The inputs might be soft opinions, or hard facts. But there are mechanisms to turn these into a number that can be maningfully combined with other such numbers. And the weightings of the criteria, at every level of the hierarchy, are themselves ratio metrics. The end result of multiplying and adding the criteria weights and alernative scores is something mathematically meaningful (i.e. Analytic — the “A”).
Thirdly, it’s deeply collaborative. You can blend many people’s opinions into every level of the decision. Be that weighing the strategic goals (where the CEO might have a disproportionate say) to evaluating the minutest criterion (where some subject matter expert may have a greater say). It’s a Process.
Finally, it can use pairwise comparisons between alternatives to recover your “true” evaluations (as ratios), using a simple scale of extreme/strong/medium/weak to express preference either way. You aren’t asked to score each alternative in isolation. So, say you’re using the tool to choose a new headquarters location. You wouldn’t be asked whether New York has great transport facilities. Instead you would say how New York compared to San Diego. And San Diego with Chicago. And Chicago with New York. At the end they get assigned scores that tell you just how much you preferred New York to San Diego. Ratio scores.
This is a pretty subtle aspect of the methodology. If I just say “eigenvalues”, will you take my word for it that there’s more going on behind the scenes than your traditional Excel spreadsheet? And, in the style of QVC, not only all of that, but (for no extra charge) it tells you when you’ve been inconsistent, and where that inconsistency arises. There are “redundant” questions which do this for you. So if you said New York was much better than Chicago, which in turn was a bit better than San Diego, which itself was about the same as New York, then you’ve got a problem.
This pairwise evaluation is first done to the evaluation criteria (to get their weightings at each level of the hierarchy). What’s the relative importance of increasing revenue compared to complying with regulatory and legal mandates? And so on. It can then be applied to the alternatives. How well does project A contribute to improving customer satisfaction compared to project B?
This whole thing clearly needs a tool. It’s not you average problem to be solved with an afternoon of Excel macro hacking. And such a tool, thankfully, exists. It’s called Expert Choice. No, I’m not being paid by them to promote it (although they did buy me dinner once). It just works. I like stuff that works. It does a ton of things I’ve not mentioned above. Sensitivity analysis, efficient frontier resource allocation, reports, wireless group voting using remote controls (lots of fun!), blah and blah. You can read, call a salesman if you want.
But even the salesman won’t tell you what’s really important to know. Because it isn’t about eliminating the World’s Most Complex Spreadsheet and worshipping at the ratio metric alter. It’s about people. Making people involved in the process. Getting buy-in. There even being a clearly identified decision for people to discuss and a forum to discuss it — and not a hundred invisible water cooler conversations and e-mail chains. The existence of a decision at the end that everyone can see. The ability to challenge the decision, and extract new objective criteria that might have created an illogical decision. Forcing people to re-think their prejudices about the outcome when their own opinions on the criteria don’t produce the result they expected.
Decisions are about people, not mathematics, and the AHP methodology and Expert Choice tools are very good at dealing with people. Even the simple pairwise comparison is extremely user-friendly. No arbitrary weighting scales, no inconclusive mixing of objective and subjective criteria.
Now for the telco spin. As the Innovator’s Dilemma pointed out, the Internet can be sustaining or disruptive to your business. For Dell it was a sustaining way of cutting out middle-men and talking directly to the customer for build-to-order. For Compaq it was the opposite, creating channel conflict and challenging the batch-built mentality. So a decision tool alone won’t re-structure your business to deal with a disruptive technology innovation, should that be needed. But used appropriately, you can reasonably argue that “end-to-end compatibility” should be a corporate goal. Without a decision tool, you had no focus for influencing the corporate direction. But when the criteria for resource allocation become explicit, you can campaign to have yours added. At first they might be weighted near 0%. But repeated matching of proposed projects is likely to get questions asked. And over time, the value of that strategic objective becomes clearer and accepted. (For quite what constitutes the right criteria to become “end-to-end compatible”, watch this space…)
Let’s move down twenty pegs from the overall strategic problems of the telecom industry to your own career. If you don’t want to have your knowledge work sent offshore, you need to be doing something beyond the ordinary. As the WSJ recently put it, you need to “move up the job chain” to survive.
Beyond basic technical knowledge lies design, innovation and business ingenuity. What better tool could there be to create personal success in these spheres than to enable greater co-operative decision making and improved strategic focus? To be the catalyst people to turn to when they need to make things happen?
Now AHP may not be the best decision process — I’m not an expert in decision science. You’ll need to work out how AHP fits into other quality methodologies like QFD. But if you’re not aware of it, you’re a business illiterate. And that’s not where you want to be with billions of quality-focused Indians and Chinese breathing down your neck. You need to learn to read, before it’s too late.
Posted by Martin Geddes at 06:17 PMTrackBack URL for this entry:
http://www.telepocalypse.net/cgi-sys/cgiwrap/mgeddes/MT/mt-tb.cgi/133.