Quantitative Estimation from Multiple Cues:  Test and Application of a New Cognitive Model Bettina von Helversen Introduction1 The Traditional Approach to Estimations: Social Judgment Theory2The Exemplar-Based Approach to Estimation 3Heuristic Approach to EstimationsA New Cognitive Theory for Quantitative Estimations from Multiple Cues: The Mapping Model 4Dissertation Outline56Chapter 1:The Mapping Model: A Heuristic for Quantitative Estimation6 Abstract78 The Mapping Model 971#47140#2521#2510Alternative Theories of Estimation1121#25123#471213124#8821#25100#48141516Simulation study 1718Study 119 Method20212223Results242526480#74227Discussion of Study 1 Study 228 Method29Results30313233Discussion of Study 234Study 3 Method3536124#28Results3738603#401Discussion of Study 3Study 439 Method40Results41Discussion of Study 442General Discussion The Success of the Mapping ModelRule-based Estimation43Exemplar-based Estimations4445Simple Heuristics for EstimationComplexity of the Models46Limitations of the Mapping Model47Final Conclusion Appendices484950515253545556256#255713#1358107#28100#2859Chapter 2:Models of Quantitative Estimations: Rule-Based and Exemplar-Based Processes Compared Abstract6061 Models of Estimation62Competing Theories636465124#9921#25100#48666768Methods of Model Selection and Qualitative Tests of Models69Study 170 Method71297#267273747576603#445Results777879332#63780Discussion of Study 181Study 282 Method83Results848586318#62487661#586Discussion of Study 288General Discussion Exemplar Memory: Number of Training Trials and Number of Objects89Knowledge Abstraction 90Conclusion Appendices919293949596Chapter 3:Predicting Sentencing for Low-Level Crimes:A Cognitive Modeling Approach97 Abstract98 Heuristics in Legal Decision Making99Sentencing Decisions by the Prosecution100Models of Sentence Magnitude101141#46The Mapping Model: A Cognitive Theory of Quantitative Estimation 102103485#510Fines versus Incarceration 104Study: Analysis of Trial Records105 Method106107108109110197#2521#27Results111381#361112113604#432114115116560#395Discussion117 Predictors of Sentencing Decisions118Model Comparison119Bayesian ApproachLimitations of the Study120Conclusion and Outlook121Appendices122123124203#2737#2437#2437#2437#24125312#3161#27126173#40233#41251#2919#2337#2519#23448#4244#2419#23127128General Discussion129 Mapping Model Regression Model 130 Exemplar Model 131 QuickEst 132Implications for the Process of Estimation Assumptions of the Mapping Model 133Adaptive Behavior in Quantitative Estimation134Model Selection Methods Generalization Method: Out of Sample Prediction Qualitative Tests135Bayesian Model AveragingLimitations and Extension of the Mapping Model Cue Selection 136Cue Weighting137ExtrapolationContinuous Cue Information138Generalizability and Applications of the Mapping ModelConclusion139ReferencesAcknowledgementsErklärungdeInhaltsverzeichnisHilfe

Quantitative Estimation from Multiple Cues: <br/>Test and Application of a New Cognitive Model Dissertationzur Erlangung des akademischen Grades
Dr. rer. nat. im Fach Psychologieeingereicht an der
Mathematisch-Naturwissenschaftlichen Fakultät II
der Humboldt-Universität zu Berlinvon
Dipl. Psych. Bettina von Helversen,
geboren am 27.12.1977 in Freiburg im Breisgau

Präsident der Humboldt-Universität zu Berlin
Prof. Dr. Christoph Markschies

Dekan der Mathematisch-Naturwissenschaftlichen Fakultät II
Prof. Dr. Wolfgang Coy Prof. Gerd Gigerenzer Prof. Peter Frensch Prof. Peter Juslin Tag der Verteidigung: 18.01.2008 English Summary

How do people make quantitative estimations, such as estimating a car’s selling price? Often people rely on cues, information that is probabilistically related to the quantity they are estimating. For instance, to estimate the selling price of a car they could use information, such as the car’s manufacturer, age, mileage, or general condition. Traditionally, linear regression type models have been employed to capture the estimation process. These models assume that people weight and integrate all information available to estimate a criterion. In my dissertation, I propose an alternative cognitive theory for quantitative estimation: The mapping model, inspired by the work of Brown and Siegler (1993) on metrics and mappings, offers a heuristic approach to decision making. In the first part of my dissertation, I laid the theoretical foundation for the mapping model, and tested this against established alternative models of estimation, namely, linear regression, an exemplar model, and a simple estimation heuristic. The mapping model provided a valid account of people’s estimates outperforming the other models in a variety of conditions. Consistent with the “adaptive toolbox” approach on decision making (Gigerenzer and Todd, 1999), which model was best in predicting participants’ estimations was a function of the task environment. In the second part of my dissertation, I further investigated how task characteristics influence the models’ ability to predict participants’ estimations by focusing on the assumptions the models make about the estimation process: While the exemplar model relies on the establishment of an exemplar memory base, the mapping model requires the abstraction of knowledge. I examined how different task features affect these assumptions and thus explain shifts in processing contingent on the task structure. My results indicate that explicit knowledge about the cues is decisive. When knowledge about the cues was available, the mapping model was the best model; however, if knowledge about the task was difficult to abstract, participants’ estimations were best described by the exemplar model. In the third part of my dissertation, I applied the mapping model in the field of legal decision making. In an analysis of fining and incarceration decisions, I showed that the prosecutions’ sentence recommendations were better captured by the mapping model than by legal policy modeled with a linear regression. These results indicated that the mapping model is a valid model which can be applied to model actual estimation processes outside of the laboratory. Furthermore, they suggest that deviations from legal policy can be explained by considering the cognitive processes of the decision maker

 Deutsche Zusammenfassung

Wie schätzen Menschen quantitative Größen wie zum Beispiel den Verkaufspreis eines Autos? Oft benutzen Menschen zur Lösung von Schätzproblemen sogenannte Cues, Informationen, die probabilistisch mit dem zu schätzenden Kriterium verknüpft sind. Um den Verkaufspreis eines Autos zu schätzen, könnte man zum Beispiel Informationen über das Baujahr, die Automarke, oder den Kilometerstand des Autos verwenden. Um menschliche Schätzprozesse zu beschreiben, werden häufig linear additive Modelle herangezogen. Diese Modelle nehmen an, dass Menschen alle Informationen, die sie zur Verfügung haben, gewichten und dann zu einer Schätzung integrieren, indem sie die gewichteten Informationen addieren. In meiner Dissertation schlage ich ein alternatives Modell zur Schätzung quantitativer Größen vor. Das Mapping-Modell präsentiert einen heuristischen Ansatz auf der theoretischen Grundlage von Brown und Sieglers (1993) Arbeit zu metrics und mappings. Im ersten Kapitel meiner Dissertation lege ich die theoretische Basis des Mapping-Modells dar und teste es gegen weitere, in der Literatur etablierte, Schätzmodelle wie zum Beispiel eine lineare Regression, ein Exemplar-Modell und eine Schätzheuristik. Es zeigte sich, dass das Mapping-Modell unter unterschiedlichen Bedingungen in der Lage war, die Schätzungen der Untersuchungsteilnehmer akkurat vorherzusagen. Allerdings bestimmte die Struktur der Aufgabe ― im Einklang mit dem Ansatz der „adaptiven Werkzeugkiste“(Gigerenzer and Todd, 1999) ― im großen Maße, welches Modell am besten geeignet war, die Schätzungen zu erfassen. Im zweiten Kapitel meiner Dissertation greife ich diesen Ansatz auf und untersuche, in wie weit das Zusammenspiel von Aufgabenstruktur und den Annahmen, die die Modelle zum Schätzprozess machen, bestimmt, welches Modell die Schätzprozesse am Besten beschreibt. Das Exemplar-Modell setzt die Speicherung von Exemplaren im Gedächtnis voraus, während das Mapping-Modell die Abstraktion von explizitem Wissen über die Aufgabe postuliert. Meine Ergebnisse zeigten, dass die Struktur der Aufgabe beeinflusste, welches Modell die kognitiven Prozesse am Besten beschrieb. Das Mapping-Modell war am Besten dazu geeignet die Schätzungen der Versuchsteilnehmer zu beschreiben, wenn explizites Wissen über die Aufgabe vorhanden war, während das Exemplar-Modell den Schätzprozess erfasste, wenn die Abstraktion von Wissen schwierig war. Im dritten Kapitel meiner Dissertation, wende ich das Mapping-Modell auf juristische Entscheidungen an. Eine Analyse von Strafakten ergab, dass das Mapping-Modell Strafzumessungsvorschläge von Staatsanwälten besser vorhersagte als eine lineare Regression. Dies zeigt, dass das Mapping-Modell auch außerhalb von Forschungslaboratorien dazu geeignet ist menschliche Schätzprozesse zu beschreiben. Weiter weisen die Ergebnisse darauf hin, dass Abweichungen von gesetzlichen Regelungen auf die kognitiven Prozesse der Entscheidungsträger zurückgeführt werden können.

Inhaltsverzeichnis

  • Introduction

    • The Traditional Approach to Estimations: Social Judgment Theory

    • The Exemplar-Based Approach to Estimation 

    • Heuristic Approach to Estimations

    • A New Cognitive Theory for Quantitative Estimations from Multiple Cues:  The Mapping Model 

    • Dissertation Outline

  • Chapter 1: The Mapping Model: A Heuristic for Quantitative Estimation

    • Abstract

      • The Mapping Model 

      • Alternative Theories of Estimation

      • Simulation study 

    • Study 1

      • Method

      • Results

      • Discussion of Study 1 

    • Study 2

      • Method

      • Results

      • Discussion of Study 2

    • Study 3

      • Method

      • Results

      • Discussion of Study 3

    • Study 4

      • Method

      • Results

      • Discussion of Study 4

    • General Discussion

      • The Success of the Mapping Model

      • Rule-based Estimation

      • Exemplar-based Estimations

      • Simple Heuristics for Estimation

      • Complexity of the Models

      • Limitations of the Mapping Model

      • Final Conclusion

    • Appendices

  • Chapter 2: Models of Quantitative Estimations: Rule-Based and Exemplar-Based Processes Compared

    • Abstract

      • Models of Estimation

      • Competing Theories

      • Methods of Model Selection and Qualitative Tests of Models

    • Study 1

      • Method

      • Results

      • Discussion of Study 1

    • Study 2

      • Method

      • Results

      • Discussion of Study 2

    • General Discussion

      • Exemplar Memory: Number of Training Trials and Number of Objects

      • Knowledge Abstraction 

      • Conclusion

    • Appendices

  • Chapter 3: Predicting Sentencing for Low-Level Crimes: A Cognitive Modeling Approach

    • Abstract

      • Heuristics in Legal Decision Making

      • Sentencing Decisions by the Prosecution

      • Models of Sentence Magnitude

      • The Mapping Model: A Cognitive Theory of Quantitative Estimation 

      • Fines versus Incarceration 

    • Study: Analysis of Trial Records

      • Method

      • Results

    • Discussion

      • Predictors of Sentencing Decisions

      • Model Comparison

      • Bayesian Approach

      • Limitations of the Study

      • Conclusion and Outlook

    • Appendices

  • General Discussion

    • Mapping Model

    • Regression Model

    • Exemplar Model

    • QuickEst

    • Implications for the Process of Estimation

      • Assumptions of the Mapping Model 

      • Adaptive Behavior in Quantitative Estimation

    • Model Selection Methods

      • Generalization Method: Out of Sample Prediction 

      • Qualitative Tests

      • Bayesian Model Averaging

    • Limitations and Extension of the Mapping Model

      • Cue Selection 

      • Cue Weighting

      • Extrapolation

      • Continuous Cue Information

    • Generalizability and Applications of the Mapping Model

    • Conclusion

  • References

  • Acknowledgements

  • Erklärung

Tabellen

  • Table 1: Mobile Phone Example for Illustrating the Predictions of the Models 

  • Table 2: Models’ Average Accuracies (Root Mean Square Error) in the Simulation Study for the Two Environments

  • Table 3: Task Structure of Study 1

  • Table 4: Correlations Between Cues and Criteria in Study 1

  • Table 5: Models’ Average Accuracies in Predicting Participants’ Estimations in Study 1 

  • Table 6: Mean Consistency of the Participants in the Test Set of Study 2

  • Table 7: Models’ Average Accuracies in Predicting Participants’ Estimations in the Test Phase of Study 2 (Test Set)

  • Table C1: Test Set in the J-shaped Environment in Study 1

  • Table C2: Test Set in the Linear Environment in Study 1 

  • Table D1 Average Predictive Accuracy of the Models in the Test Set of Study 1 

  • Table E1 Model Accuracies in the Training Set of Study 2

  • Table 8: New test objects in the condition with a large number of training objects

  • Table 9: Model accuracies in Study 1

  • Table 10: Cue–criterion correlations in Study 2

  • Table 11: Model accuracies in Study 2

  • Table A1: Sets of objects for the training phases of Study 1

  • Table A2: Sets of objects for the training and test phases of Study 1 for the condition with a small number of training objects and of Study 2 for the condition with six predictive cues

  • Table A3: Sets of objects for the training and test phases of Study 2 for the condition with three predictive cues

  • Table B1: Accuracies of the regression model and the standard exemplar model in predicting participants’ estimations

  • Table 12: Overview of the categorization system

  • Table 13: Results of correlation analysis and model comparison for fines

  • Table 14: Results of correlation analysis and model comparisons for incarceration

Bilder

  • Figure 1: Models’ predictions and participants’ estimations in the test phase for (A) the linear environment and (B) the J-shaped environment of Study 1. The profiles in the test set are rank ordered according to the participants’ average estimations. In the linear environment, profiles 1, 2, 3, 5, 7, 12, 13, and 21 were included in the test and training set. In the J-shaped environment, profiles 1, 2, 3, 4, 5, 7, 8, and 21 were included in the test set and the training set.

  • Figure 2: Models’ predictive accuracies for the new profiles of the test phase of Study 3. The average root mean square deviation (RMSD) between the models’ predictions and the participants’ estimations for the linear and the multiplicative condition is depicted. The error bars represent the 95% confidence intervals.

  • Figure 3: Qualitative model predictions. The models’ predictions for the two qualitative tests, when varying the values of the exemplar model’s attention parameter s. The “4 vs. 2” denotes the predicted average differences in estimations for the criterion values of test objects with a cue sum of 4 and test objects with a cue sum of 2. The “3” refers to the predicted average differences in estimations for the criterion values of the pair of test objects with a cue sum of 3, with maximally different cue profiles (e.g. 111000 and 000111). (A) The predictions for the condition with a small number of training objects. (B) The predictions for the condition with a large number of training objects

  • Figure 4: Qualitative test in Study 1. (A) Qualitative predictions of the models and the participants’ estimations in the condition with a large number of training objects (N = 20). (B) Qualitative predictions of the models and the participants’ estimations in the condition with a small number of training objects (N = 19). Sum of cue values 3 gives the average difference in estimations for the criterion values of the pair of test objects with a cue sum of 3 with maximally different cue profiles. Sum of cue values 4 vs. 2 gives the average difference in estimations for the criterion values of test objects with a cue sum of 4 and test objects with a cue sum of 2; error bars denote ±1 SD.

  • Figure 5: Models’ accuracy in predicting the participants’ estimations for the new objects in the test phase of Study 2. (A) Models’ accuracy when the cues’ directions were known (N = 40; 20 for each condition). (B) Models’ accuracy when the cues’ directions were not known (N = 40; 20 in each condition).

  • Figure 6: Qualitative test in Study 2. (A) Qualitative tests for the condition with known cue directions but only three predictive cues are shown. (B) Qualitative tests for the conditions with known cue directions and six predictive cues. (C) Qualitative tests for the condition with unknown cue directions and three predictive cues. (D) Qualitative tests for the condition with unknown cue directions and six predictive cues. Sum of cue values 3 gives the average difference in estimations for the criterion values of the pairs of test objects with a cue sum of 3 with maximally different cue profiles. Sum of cue values 4 vs. 2 gives the average difference in estimations for the criterion values of test objects with a cue sum of 4 and test objects with a cue sum of 2. Error bars denote ±1 SD; N = 20 in each panel.

  • Figure 7: The processing steps of the mapping model. In the first step, the relevant cues are evaluated and rated according to their severity. In the second step the cues are integrated by establishing the average severity score. Then, the case is categorized according to its average score and the typical criterion value, that is the sentence for this category is retrieved. In the last step the retrieved criterion value is used as an estimate

  • Figure 8: Scatter plot of the sentence recommendation for fines by the prosecution and the corresponding verdict by the judge. The magnitude of the fines is given in number of days a payment has to be made.

  • Figure 9: The posterior model probability of the best 1,500 of all 4,096 models to describe the fining process, differentiated by model class. Of the 1,500 best models, 99% belong to the class of mapping models and 1% to the class of regression models.

  • Figure 10: The posterior model probability of the best 100 models describing the incarceration decisions, differentiated by model class. Of the 100 best models, 65% belong to the class of mapping models and 35% to the class of regression models