Konubinix' opinionated web of thoughts

Beyond Bias With Olivier Sibony

Fleeting

Some examples of decisions hygiene:

don’t use scales when measuring something

Every judgement implies a scale. Two people with the same opinion will have different absolute scales, like one will need perfection to rate with higher scale while another one would rate with higher scale unless it is a terrible stuff.

Instead of having 1, 2, 3 etc, anchor them to have relative comparisons, like Stuff 1, Stuff 2, Stuff 3. Those are called anchor cases1.

And them, instead of assigning a number, ask yourself, is the measured stuff better than Stuff 1, better than Stuff 2 etc.

It looks like it is less subject to cognitive bias noise.

Putting anchors in relative scales also allow having an ubiquitous language. What we choose as reference points will tell a lot about ourselves.

mediating assessment protocol

Write down the dimensions that are useful to you and consider the ideas on each of them2.

Use anchoring to be in the appropriate state of mind when taking the decision.

First of all, specify what makes sense, what is important (it may be 1, 2, 7 or 20 dimensions) and assess your choices on each dimension separately (this is the hard part).

Unless dimensions are highly correlated, assessing dimensions together makes harder decision taking as the information is diluted. For instance, if people are to assess a method without separating how they liked it and what they learned, the result will have a lot of noise.

Then, take a holistic judgement based on seeing all those dimensions (pareto front).

It is hard, but not that hard after all.

It helps mitigate the binary thinking. Also, it helps you make clearer what actually matters to you and incentivizes being intellectually honest. For instance, a company that does not take into account the employee comfort in a decision taking tells a lot about the culture of the company.

This takes into account the fact that wysiati. You tend to provide an intuitive judgment based on what you previously had in mind (anchoring). This protocol tries to ensure what is in your mind before you make your assessment is as nuanced and holistic as possible. Like nudges, instead of fighting against your cognitive bias, it empowers them to allow reach a greater goal. This is similar to the idea that gtd is about regularly refreshing your mind about the tasks that matter. (don’t fight the way your brain works, work with it)

Therefore, its important not to discuss the final assessment before you have discussed each dimension separately, to avoid possible biases that would come with it.

People, inspired by several decision making processes, want to add weights to the dimensions so that after assessing on each dimension, a single formula will tell what it the winner. This approach is iterative and depending on what the winner is, you adjust the weights. Here, Olivier Sibony claims that people tend not to trust such system and unconsciously bias it towards the decision they like. This has the counter productive effect of incentivizing them to think about the final decision they want beforehand, which is exactly what this protocol wants to avoid.

Plus, even thought there are several decisions for which formulas tend to work better than intuitive judgements (because of their determinism and their tendency to be bias proof) here, it is not the case. We don’t know yet how to reduce a multi criteria decision making process to a formula. Plus, having a formula incentivizes the precision fallacy

Therefore, this is more about putting the best conditions for a fruitful discussion and a gut feeling decision than an automatic process to provide an answer. It provides a framework to structure the conversation and keep a holistic view of what is wanted. Having the dimensions on a MAP really helps the discussion. People can say that some solution might be good but after seeing the dimensions it might not be that good after all. You don’t do things like a mean of the assessments. You actually need people to believe in the solution (otherwise they will retro engineer the protocol to make it provide their prior opinion), hence it must comes from their guts.

This protocol will show you how well your decision fits with your objectives and incentivizes intellectual honesty. There will be cases when a last minute you will realize some dimension is actually important to you. This will be captured in the system easily. Unlike the pros and cons fallacy where people tend to write + and - and finally ignore the system to provide some gut feeling decision3

To me, this might mitigate but not necessarily remove the biais de bonne conscience. People might well end taking a decision that theoretically works but that they know (or should know, if they were intellectually honest) did not work in the past and then will likely fail again (induction).

use EAT in the mediating assessment protocol

I suggest that people use at least the EAT dimensions in their assessment. Decisions that are small on those scales might well be theoretically sexy, they are likely to remain on paper only.

people tend to expect behavior science to provide answers that will prevent the discussion, when it actually only provides appropriate fundation to help people have a fruitful discussion

What matters the most is the discussion and reaching a consensus together that is as biased/noise proof as possible.

Like in the tragedy of bdd: 10 years of Doing Behaviour-Driven Development All Wrong where once the implementation of BDD was done, people stopped discussing with each other, doing the exact opposite of what the method was about.

use the mean of several independent assessments

Even a person can provide several assessments if a long running project (one assessment every 3 months for example).

make people write down their assessment before knowing the one of their colleagues

Knowing the assessments of others tends to allow groupthink.

Writing it down before hearing about the assessment of other people and reading it before telling yours makes it more difficult to have groupthink. You will experience cognitive dissonance if you tell something different than what is different from what is written down.

Nevertheless, the fact no one know what you wrote makes it possible and easy to genuinely change your opinion hearing the arguments of others.

Notes linking here

  1. this makes me think about the way we evaluate story points ↩︎

  2. makes me think about how to make multi criteria choices?

     ↩︎

  3. to me, people that do so fall into the biais de bonne conscience and a shot of intellectual honesty should make them realize that their method does not fit their goals ↩︎