Konubinix' opinionated web of thoughts

Failure to Disagree

Fleeting

Simon (1992) offered a concise definition of skilled intuition that we both endorse: “The situation has provided a cue: This cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition” (p. 155).

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

accuracy of a judgment (our else’s) is by considering the validity of the environment in which the judgment was made as well as the judge’s history of learning the rules of that environment.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

people perform significantly more poorly than algorithms in low-validity environments.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

see prefer algorithms

determination of whether intuitive judgments can be trusted requires an examination of the environment in which the judgment is made and of the opportunity that the judge has had to learn the regularities of that environment.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

There is no subjective marker that distinguishes correct intuitions from intuitions that are produced by highly imperfect heuristics. An important characteristic of intuitive judgments, which they share with perceptual impressions, is that a single response initially comes to mind. Most of the time we have to trust this first impulse, and most of the time we are right or are able to make the necessary corrections if we turn out to be wrong, but high subjective confidence is not a good indication of validity (Einhorn & Hogarth, 1978)

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Checking one’s intuition is an effortful operation of System 2, which people do not always perform - sometimes because it is difficult to do so and sometimes because they do not bother.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Subjective experience is not a reliable indicator of judgment accuracy.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Simon defined intuition as the recognition of patterns stored in memory

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

An environment of high validity is a necessary condition for the development of skilled intuitions. Other necessary conditions include adequate opportunities for learning the environment (prolonged practice and feedback that is both rapid and unequivocal). If an environment provides valid cues and good feedback, skill and expert intuition will eventually develop in individuals of sufficient talent

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

A central goal of NDM is to demystify intuition by identifying the cues that experts use to make their judgments, even if those cues involve tacit knowledge and are difficult for the expert to articulate. In

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

experts are expected to successfully attain vaguely defined goals in the face of uncertainty, time pressure, high stakes, team and organizational constraints, shifting conditions, and action feedback loops that enable people to manage disturbances while trying to diagnose them

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Meehl (1954) believed that the inferiority of clinical judgment was due in part to systematic errors, such as the consistent neglect of the base rates of outcomes in discussion of individual cases. In

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

When presented with the same case information on separate occasions, human judges often reach different conclusions.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

subjective conviction of understanding each case in isolation was not diminished by the statistical feedback from officer training school, which indicated that the validity of the assessments was negligible.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Kahneman coined the term illusion of validity for the unjustified sense of confidence that often comes with clinical judgment.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

conclusion of the study was that sophisticated scientists reached incorrect conclusions and made inferior choices when they followed their intuitions, failing to apply rules with which they were certainly familiar.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Tversky and Kahneman (1971) in their belief (originally based on introspection) that faulty statistical intuitions survive both formal training and actual experience. Many

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

We found that the sharpest differences between the two of us were emotional rather than intellectual. Although DK is thrilled by the remarkable intuitive skills of experts that GK and others have described, he also takes considerable pleasure in demonstrations of human folly and in the comeuppance of overconfident pseudo-experts.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Nevertheless, as this article shows, we agree on most of the issues that matter.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

most common method for defining expertise in NDM research is to rely on peer judgments.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

conditions for defining expertise are the existence of a consensus and evidence that the consensus reflects aspects of successful performance that are objective even if they are not quantified explicitly.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Shanteau (1992) suggested, “Experts are operationally defined as those who have been recognized within their profession as having the necessary skills and abilities to perform at the highest level” (p. 255).

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

When colleagues say, “If Person X had been there instead of Person Y, the fire would not have spread as far,” then Person X counts as an expert within that organization.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

judgments and decisions that we are most likely to call intuitive come to mind on their own, without explicit awareness of the evoking cues and of course without an explicit evaluation of the validity of these cues.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Anyone can recognize tension or fatigue in a familiar voice on the phone.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

intuitive judgments are produced by “System 1 operations,” which are automatic, involuntary, and almost effortless.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

deliberate activities of System 2 are controlled, voluntary, and effortful—they impose demands on limited attentional resources.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

When there are cues that an intuitive judgment could be wrong, System 2 can impose a different strategy, replacing intuition by careful reasoning.2

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

NDM and HB approaches share the assumption that intuitive judgments and preferences have the characteristics of System 1 activity: They are automatic, arise effortlessly, and often come to mind without immediate justification.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

two necessary conditions for the development of skill: high-validity environments and an adequate opportunity to learn them.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

How can skilled intuitions be distinguished from heuristic-based intuitions?

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Simon (1992) offered a concise definition of skilled intuition that we both endorse: “The situation has provided a cue: This cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition” (p. 155).

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

model of intuition as recognition is helpful in several ways. First, it demystifies intuition. Many experts who have intuitions (and some authors who study them) endow intuition with an almost magic aura—knowledge that is not acquired by a rational process.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

In Simon’s definition, the process by which the pediatric nurse recognizes that an infant may be gravely ill is not different in principle from the process by which she would notice that a friend looks tired or angry or from the way in which a small child recognizes that an animal is a dog, not a cat.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

recognition model implies two conditions that must be satisfied for an intuitive judgment (recognition) to be genuinely skilled: First, the environment must provide adequately valid cues to the nature of the situation. Second, people must have an opportunity to learn the relevant cues.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

For the first condition, valid cues must be specifiable, at least in principle—even if the individual does not know what they are.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The child relies on valid cues to identify a dog, without any ability to state what the cues are. Similarly, the nurse and the firefighter are also guided by valid cues they find in the environment. No magic is involved.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

crucial conclusion emerges: Skilled intuitions will only develop in an environment of sufficient regularity, which provides valid cues to the situation.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Validity, as we use the term, describes the causal and statistical structure of the relevant environment.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

For example, it is very likely that there are early indications that a building is about to collapse in a fire or that an infant will soon show obvious symptoms of infection.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

On the other hand, it is unlikely that there is publicly available information that could be used to predict how well a particular stock will do—if such valid information existed, the price of the stock would already reflect

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Thus, we have more reason to trust the intuition of an experienced fireground commander about the stability of a building, or the intuitions of a nurse about an infant, than to trust the intuitions of a trader about a stock.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Consider what the history of the 20th century might have been if the three fertilized eggs that became Hitler, Stalin, and Mao had been female. The century would surely have been very different, but can one know how? In

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

In other environments, the regularities that can be observed are misleading. Hogarth (2001) introduced the useful notion of wicked environments, in which wrong intuitions are likely to develop.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

early 20thcentury physician who frequently had intuitions about patients in the ward who were about to develop typhoid. He confirmed his intuitions by palpating these patients’ tongues, but because he did not wash his hands the intuitions were disastrously self-fulfilling.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Two conditions must be satisfied for skilled intuition to develop: an environment of sufficiently high validity and adequate opportunity to practice the skill.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Charness, Hoffman, and Feltovich (2006) have described a range of factors that influence the rate of skill development. These include the type of practice people employ, their level of engagement and motivation, and the selfregulatory processes they use.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Intuitions that are available only to a few exceptional individuals are often called creative. Like other intuitions, however, creative intuitions are based on finding valid patterns in memory, a task that some people perform much better than others.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

conditions often remain unmet in professional contexts, either because the environment is insufficiently predictable (as in the long-term forecasting of political events) or because of the absence of opportunities to learn its rules (as in the case of firefigh-

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

People, including experienced professionals, sometimes have subjectively compelling intuitions even when they lack true skill, either because the environment is insufficiently regular or because they have not mastered it

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

incorrect intuitions, like valid ones, also arise from the operations of memory.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

“A ball and a bat together cost $1.10. The bat costs a dollar more than the ball. How much does the ball cost?”

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The question invariably evokes an immediate tentative solution: 10 cents.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

many intelligent people adopt the intuitively compelling response without checking it. The incidence of intuitive errors in this question ranges from approximately 50% in top undergraduate schools (MIT, Princeton, Harvard) to 90% in somewhat less selective schools.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

common genre of business literature celebrates successful leaders who made strategic decisions on the basis of gut feelings and intuitions that they did not adequately check,

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The mechanism of anchoring is well understood (Mussweiler & Strack, 2000). The original question with the high anchor brings expensive cars to the respondents’ mind: Mercedes, BMWs, Audis. The lower anchor is more likely to evoke the image of a beetle and the name Volkswagen. The initial question therefore biases the sample of cars that come to mind when people next attempt to estimate the average price of German cars.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

process of estimating the average is a deliberate, System 2 operation, but the bias occurs in the automatic phase in which instances are retrieved from memory. The resulting anchoring effect is large and robust. The answers that come to mind are typically held with substantial confidence, and the victims of anchoring manipulations confidently deny any effect of the anchor.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Attribute substitution has been described as an automatic process. It produces intuitive judgments in which a difficult question is answered by substituting an easier one—the essence of heuristic thinking

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

If people have a skilled response to the task with which they are charged, they will apply their skill.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

even in the absence of skill an intuitive response may come to their minds. The difficulty is that people have no way to know where their intuitions came from. There is no subjective marker that distinguishes correct intuitions from intuitions that are produced by highly imperfect heuristics. An important characteristic of intuitive judgments, which they share with perceptual impressions, is that a single response initially comes to mind. Most of the time we have to trust this first impulse, and most of the time we are right or are able to make the necessary corrections if we turn out to be wrong, but high subjective confidence is not a good indication of validity

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Checking one’s intuition is an effortful operation of System 2, which people do not always perform— sometimes because it is difficult to do so and sometimes because they do not bother.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

“In general these heuristics are quite useful, but sometimes they lead to severe and systematic errors”

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

not that intuitions that arise in heuristics are always incorrect, only that they are less trustworthy than intuitions that are rooted in specific experiences.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

people are not normally aware of the origins of the thoughts that come to their minds, and the correlation between the accuracy of their judgments and the confidence they experience is not consistently high

[…]

Subjective confidence is often determined by the internal consistency of the information on which a judgment is based, rather than by the quality of that information

[…]

evidence that is both redundant and flimsy tends to produce judgments that are held with too much confidence. These judgments will be presented too assertively to others and are likely to be believed more than they deserve to be.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

safe way to evaluate the probable accuracy of a judgment (our own or someone else’s) is by considering the validity of the environment in which the judgment was made as well as the judge’s history of learning the rules of that environment.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

attitude, motivation, talent, and deliberate practice as crucial to skill development

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Shanteau searched for task characteristics that distinguished the domains in which experts did well from those in which experts did poorly. The factors that we identified—the predictability of outcomes, the amount of experience, and the availability of good feedback—were included in his list.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

professionals exhibited genuine expertise in some of their activities but not in others. We refer to such mixed grades for professionals as “fractionated expertise,” and we believe that the fractionation of expertise is the rule, not an exception.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

DK is particularly interested in cases in which professionals who know how to use their knowledge for some purposes attempt to use the same knowledge for other purposes. He views the fractionation of expertise as one element in the explanation of the illusion of validity: the overconfidence that professionals sometimes experience in dealing with problems in which they have little or no skill.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

they may have received ample feedback supporting their confidence in the performance of some tasks— typically those that deal with the short term—but the feedback they receive from their failures in long-term judgments is delayed, sparse, and ambiguous.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

These professionals may have strong subjective confidence in their judgments, but we do not believe that subjective confidence reliably indicates whether intuitive judgments or decisions are valid.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

people do not have a strong ability to distinguish correct intuitions from faulty ones. People, even experts, do not appear to be skilled in detecting patterns in the internal situation in order to identify the basis for their judgments.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

reliance on subjective confidence may contribute to overconfidence.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The idea of algorithms that outdo human judges is a source of pride and joy for members of the HB tribe, but algorithms are usually distrusted by the NDM community.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

people perform significantly more poorly than algorithms in low-validity environments.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

human learning is normally quite efficient. Where simple and valid cues exist, humans will find them if they are given sufficient experience and enough rapid feedback to do so—except in the environments that Hogarth (2001) labeled “wicked,” in which the feedback is misleading.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

statistical approach has two crucial advantages over human judgment when available cues are weak and uncertain: Statistical analysis is more likely to identify weakly valid cues, and a prediction algorithm will maintain above-chance accuracy by using such cues consistently. The meta-analysis per-

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Our analysis suggests that algorithms significantly outperform humans under two quite different conditions: (a) when validity is so low that human difficulties in detecting weak regularities and in maintaining consistency of judgment are critical and (b) when validity is very high, in highly predictable environments, where ceiling effects are encountered and occasional lapses of attention can cause humans to fail.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

conditions necessary for the construction and use of an algorithm are stringent. These conditions include (a) confidence in the adequacy of the list of variables that will be used, (b) a reliable and measurable criterion, (c) a body of similar cases, (d) a cost/benefit ratio that warrants the investment in the algorithmic approach, and (e) a low likelihood that changing conditions will render the algorithm obsolete. We

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Few people enjoy being replaced by mechanical devices or by mathematical algorithms, and many devices and algorithms function less well in the real world than on the planning board

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Despite our different attitudes toward formal methods, we agree on the potential of semi-formal strategies.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

premortem method (Klein, 2007) for reducing overconfidence and improving decisions. Project teams using this method start by describing their plan. Next they imagine that their plan has failed and the project has been a disaster. Their task is to write down, in two minutes, all the reasons why the project failed. The facilitator goes around the table, getting reasons from each of the team members, starting with the leader. The rationale for the method is the concept of prospective hindsight (Mitchell, Russo, & Pennington, 1989)—that people can generate more criticisms when they are told that an outcome is certain.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

It also offers a solution to one of the major problems of decision making within organizations: the gradual suppression of dissenting opinions, doubts, and objections, which is typically observed as an organization commits itself to a major plan. The

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

we attempted to answer one basic question: Under what conditions are the intuitions of professionals worthy of trust?

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

starting point is that intuitive judgments can arise from genuine skill— the focus of the NDM approach—but that they can also arise from inappropriate application of the heuristic processes on which students of the HB tradition have focused.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Skilled judges are often unaware of the cues that guide them, and individuals whose intuitions are not skilled are even less likely to know where their judgments come from.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

True experts, it is said, know when they don’t know. However, nonexperts (whether or not they think they are) certainly do not know when they don’t know. Subjective confidence is therefore an unreliable indication of the validity of intuitive judgments and decisions.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Validity and uncertainty are not incompatible. Some environments are both highly valid and substantially uncertain. Poker and warfare are examples. The best moves in such situations reliably increase the potential for success.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The determination of whether intuitive judgments can be trusted requires an examination of the environment in which the judgment is made and of the opportunity that the judge has had to learn the regularities of that environment.

[…]

We describe task environments as “high-validity” if there are stable relationships between objectively identifiable cues and subsequent events or between cues and the outcomes of possible actions.

[…]

environment of high validity is a necessary condition for the development of skilled intuitions. Other necessary conditions include adequate opportunities for learning the environment (prolonged practice and feedback that is both rapid and unequivocal). If an environment provides valid cues and good feedback, skill and expert intuition will eventually develop in individuals of sufficient talent.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

Although true skill cannot develop in irregular or unpredictable environments, individuals will some524 September 2009 ● American Psychologist This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. times make judgments and decisions that are successful by chance. These “lucky” individuals will be susceptible to an illusion of skill and to overconfidence

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

The situation that we have labeled fractionation of skill is another source of overconfidence. Professionals who have expertise in some tasks are sometimes called upon to make judgments in areas in which they have no real skill.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

difficult both for the professionals and for those who observe them to determine the boundaries of their true expertise.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

We agree that the weak regularities available in low-validity situations can sometimes support the development of algorithms that do better than chance. These algorithms only achieve limited accuracy, but they outperform humans because of their advantage of consistency. However, the introduction of algorithms to replace human judgment is likely to evoke substantial resistance and sometimes has undesirable side effects.

— https://emcrit.org/wp-content/uploads/2016/08/Failure-to-Disagree.pdf ()

see we should prefer algorithms, but people don’t like them

correlation between the accuracy of their judgments and the confidence they experience is not consistently high

necessary conditions for the development of skilled intuitions

  1. environment of high validity,
  2. adequate opportunities for learning the environment,

adequate opportunities for learning the environment

validity of the environment in which the judgment was made

environment of high validity

Notes linking here