How to Make Multi Criteria Choices?
Fleeting- see,
I believe that making a choice is generally considered quite easy and often tackled with complacency. I think this subject needs more attention.
multi criteria decision making is not a mathematical topic. Its result heavily depends on the preferences of the person making the choice. The slight mathematical background only gives a framework to support this decision making.
Do pros and cons, but don’t consider they are the objective of the method. They only serve as note taking to help applying a decision method afterwards (pros and cons fallacy).
Use pareto efficiency. When there are several criteria, there is no objectively unique solution. You have to use a tool that won’t give you the answer, but that will provide a mean for you to clearly see your preferences1.
Also, be intellectually honest. If you have a solution that you want to adopt and you want the method to help you prove it to others, acknowledge this instead of fouling you in believing you make an objective choice. To me, it is ok to make the tool show your preferred answer, as long as you are aware of this and that you trust the tool for setting up limits that will prevent you from lying to yourself. In other terms, you don’t need a tool to spot the correct solution, you rather need a tool to eliminate the choices that you would intuitively make but that don’t align with your objectives. This is why pros and cons may not be the only method as people generally need to discuss and chose a solution that pros and cons initially discouraged.
I believe that using the following method won’t allow you at the same time write objectives you believe in and make a choice that you don’t believe is compatible with them.
The method of making a choice is (to me) iterative, this is not a sequence of action but stuff that you need to iterate on until they all are satisfying to you.
- find the objectives that you want to achieve. Those are the axis of the space of your choice, (see consider each dimension separately)
- write a pros/cons table for each objective. That helps the following step,
- write down the hypothesis/constraints that impact the choice of dimensions
- write down the non hypothesis/non constraint that you might consider but decided they won’t impact your decision
- draw a line for each objective and put the choices on it, in order and with
distance to show how far stuff are from each other with regards to this
objective.
- don’t put an explicit beginning and end of the line, so as not to undergo anchoring. Instead, put the borders after putting the choices,
- don’t use graduations, for the same reason. Instead, use your visual intuition,
- then, measure the distances and convert them to values, normalized so that the size between beg and end is 1,
- then, use those values to find your pareto frontier, eliminating stuff not in it,
- then, watch the result and how each solution provides a difference set of compromise,
Actually, even though I think this is the more sensible way to present the steps of the method, I firmly believe that you’ll actually start by looking at solutions, then you’ll have an idea of your objectives. I believe that the repetition of watching candidate solution, inferring objectives is very beneficial.
If after all you don’t like the result, feel free to rationalize and change variables so that the result suits your intuitive judgment. After all, multi criteria decision making is as subjective as the choice of your objectives.
Listen to what the system tells you. If you don’t manage to rationalize your preferred answer using such a method, chances are you have no sensible reason to prefer it. Either decide to choose your preferred solution anyway, it try to make this solution fit the objectives and measures. Either way, it helps you being more intellectually honest and have good reasons to believe.
Also, don’t fear adding plenty of dimensions, chances are that after all you will compact them afterwards. To me, it is important that the method supports the discussion. Thus if at some point a dimension is split in the conversation, then it should be split in the tool as well. If it is lumped again later, it will be lumped in the tool as well.
In the end, the tool becomes an accurate visualization on the state of the discussion/decision taking process. As so, it should be easy, attractive and timely. If you feel some cognitive resistance feeding it, it has lost its purpose.
using this method in a team
For having tried the method a few times with colleagues, I realised that thinking in term of objectives is not natural at all and that people tend to focus on solutions and provide reasons that implicitly contain their objectives but that they don’t manage to make them explicit. In that case, I recommend that you elect an objective keeper whose role is to dig out and make explicit everything that looks like an objective.
Also, I’ve realised that at first, there are so many arguments that there are numerous objectives. They tend to naturally diminish when you finally take the time to consider them.
Therefore, this method also has the advantage of focusing the debate around whatever consensual objective gets out of the conversation.
Yet, people tend to have very long oral discussion and forget to capture them. It seems important to continuously remind them that what is told is forgotten, but what is captured will be persisted and is more likely to have a positive impact on our decision.
In the end, it is felt as unattractive and people tended to let it go.
what does it look like
From the left to the right:
- non hypotheses
- the stuff that won’t impact your decisions
- hypothesis
- stuff that you consider true that impact your decisions
- dimensions
- each line contains a dimension and solutions are evaluated from the left (best) to the right (worst)
- solutions
- associated with colored papers to easily place them in the dimensions frame
In this example, the decision taking process was at its beginning, people added many hypothesis and dimensions, but in the end it will most likely compact again (breathing model).
how to evaluate them?
Oftentime, you won’t know exactly how to put the solutions in the dimensions. To me, this is normal. I think we always take decisions based on incomplete knowledge (bayesian thinking), that does not mean we don’t have to make decisions (analysis paralysis).
If it is too difficult to evaluate them, you can consider
- thought experiments,
- role games,
- POCs,
In the end, if you cannot evaluate, just remember to always do your best. This method, to me, help you make the best decision you can with the knowledge you have.
I’m actually starting to realize that those role games are crucial and should be put forward at the beginning of the method. People need to focus on examples rather that general rules when making decisions.