In my previous post, I mentioned that there are methods that a layman, untrained in formal decision analysis, can use to make multi-criteria decisions. A multi-criteria decision is any decision where there are multiple objectives to be met; in the last post, that decision was buying a car, where the objectives include things like low cost, nice style, good performance, high reliability, and so on.

Here’s one such approach that, in general, can be applied to any multi-criteria decision problem:

1. Determine the specific objectives of importance to you. In the case of a car buying decision, this would be things like cost, exterior style, interior style, performance, fuel economy (mpg), reliability, longevity, and so on. These become the scoring criteria for the decision. Alternatively, you could just use the same criteria that U.S. News & World Report applied (see the previous post): exterior style, interior style, safety, reliability, and performance. Personally, I do not think these criteria have sufficient granularity, and I wonder if they used more detailed criteria before bubbling everything up to a nice easy-to-publish summary level of detail.
2. Take a hundred points of “importance” and divide them up among the scoring criteria to reflect their relative importance to you. The number of points you assign to a given criterion becomes the weighting factor for that criterion. So, for example, if you assign 20 points to reliability and 10 points to performance, you are indicating that reliability is twice as important to you as performance.
3. For each candidate decision choice — in this example, each car you are considering — determine its score for each scoring criterion. If you are happy with the U.S. News & World Report scoring criteria, you could use their scores, but you will have to navigate to each specific car’s review page to find them, and some cars you are interested in may not have been scored.
4. For each candidate decision choice (e.g., car), multiply its score in each criterion by the weighting factor you assigned to that criterion. This gives you the weighted scores for that car for each of the criterion.
5. Add up the weighted scores for each car to obtain the overall weighted score.
6. You can now compare the overall weighted scores to find the right car for you rather than for some mythical average consumer.

Here is an example. Suppose I am trying to decide between purchasing a Honda CR-V and a Toyota FJ Cruiser, both of which I like. Here’s my analysis:

Note that I chose to split out “stereo quality” separate rather than just lump it in with “interior styling”, because this particular feature is something important to me that I wanted to score separately. I also chose to split out “drives in snow” separate from “performance”. I also lumped in fuel efficiency with cost, since that’s really why we care about fuel efficiency. The scores shown for total cost are actually the sum of purchase price plus estimated fuel costs over a 10 year ownership period. To convert this to a 1-10 scale, I gave the top car a score of 10 and the other car scaled down from there.

You can see that the sum of the weighted scores indicates that the Honda CR-V (with a score of 822.5) would be a better car for me than the Toyota FJ Cruiser (with a score of 811.71). But without the weighting for my personal criteria preferences, I would probably have chosen the Toyota, which scored (unweighted) a 75.28 to the Honda’s 73. This just goes to illustrate the importance of the weighting. If you look at the U.S. News & World Report rankings, you are seeing the rankings that result from criteria weightings chosen by a consumer survey; thus, the rankings reflect the opinion of a mythical “average” consumer.

There is one last step you might choose to do before accepting the results of this process, and that is to perform a sensitivity analysis. There is a degree of uncertainty with respect to the weighting factors, and to the scores that I picked for each car, due to their subjective nature. On a different day I might have chosen slightly differently. So it’s a good idea to go tweak things a little bit and see what happens. If I change the weighting factors just a tiny amount, do the results come out differently? If I drop the Honda’s interior styling score from an 8.5 (love that leather interior) to a 7.5 (but not in the summer!), does it change the final result? If a small change to the weighting factors and/or scores would cause a different outcome, then for all intensive purposes it’s a tie. At that point I might choose to look at additional criteria, or do some more research to reduce the uncertainties in my scoring.