I will now present a discussion on the interactions which were found in the expression discrimination task of Part I, and the familiarity discrimination task of Part II. However, it is not proposed to repeat again the discussion of the two different discrimination tasks separately. The reader is refered to the summaries and discussions of the two parts. Thus for comprehensiveness, results will be repeated in short.
It was the main question of the present dissertation as to whether there is an interaction between the discrimination of facial familiarity and facial expressions. The general hypothesis was to find a facilitative interaction between facial familiarity and the discrimination of facial expressions. Additionally, the opposite was also expected, namely that facial expressions might influence the discrimination of facial familiarity. By means of performance data and electrophysiological recordings I attempted to find the functional locus of interaction. Event related potentials were used as markers for the temporal properties of the functional processes that are involved. Although the functional model of face recognition by Bruce and Young (1986) presumes these processes t be independent, recent results suggest an interaction in one or the other direction (Baudouin et al., 2000, 2000a; Schweinberger & Soukup, 1998; Endo et al., 1992). In addition, the distributed neural system for face perception proposed by Haxby et al. (2000) allows for the possibility for such an interaction. This is based on interlinked brain regions which subserve the perception of invariant aspects like identity, and changeable aspects like expression. In order to elucidate the issue of the main question, a chronometric paradigm was applied. Throughout all experiments a two-choice RT-task was used demanding either an expression discrimination task or a familiarity discrimination. The task-irrelevant stimulus dimension was always varied independently of the task-relevant dimension. For the expression discrimination task half of the presented faces were familiar, and the other half was unfamiliar. However, all faces displayed both expressions which had to be discriminated by the participants. Accordingly, facial expression was varied within the familiarity discrimination task. By using three stimulus sets throughout the experiments, different degrees of familiarity were included. They contained either personally familiar, famous, or experimentally familiarized and unfamiliar faces. RT and error rates were recorded for all experiments. Event-related potentials were recorded only in some experiments. They served as time markers between the stimulus-response interval to separate the duration of proposed functional processing stages.
Results revealed the expected facilitative interaction between facial familiarity and facial expressions for the expression discrimination task. This only holds true for personally familiar happy faces, but not for faces displaying disgust. Late perceptual processing stages might be the possible functional locus of interaction as was suggested by event-related components. A trend was found for the P300 peak latency peaking earlier for personally familiar faces displaying happiness when compared unfamiliar faces. This facilitation propagated to the following response selection stage and was indicated by a significant difference for the S-LRP onset. Because of the somewhat unclear results based on the stimulus set with personally familiar faces, two further experiments were conducted using experimentally familiarized and unfamiliar faces. No interaction emerged between facial familiarity and the discrimination of facial expressions in both experiments. Perceptual familiarity per se is not relevant in explaining the interaction which was observed for personally familiar faces. Semantic knowledge about a person is also not the basis of an interaction between both processes. This was suggested by Experiment 6 which used famous and unfamiliar faces. Analysis of amplitude and topographical distribution revealed early differences starting at 200 ms poststimulus between personally familiar and unfamiliar faces as well as between happiness and disgust. No such topographical differences based on vector scaled data emerged for famous and unfamiliar faces although amplitude differences were evident.
In Part II the question was raised whether there is an interaction in the opposite direction. It was hyopthesized, that facial expression affects the discrimination of facial familiarity. Again, different stimulus sets were used comprising personally familiar or experimentally familiarized and unfamiliar faces. For the familiarity discrimination task it was expected that a happy expression would facilitate the decision for familiarity, because it is seen on familiar faces more often. In everyday life a happy expression is probably the most common social gesture when someone familiar approaches us and hence the decision for familiarity may be facilitated. Results revealed that the decision for a face as being familiar is faster when it displays a happy or neutral expression. Again, this only holds true for personally familiar faces but not for experimentally familiarized or unfamiliar faces. Event related potentials clearly pointed to a facilitated response selection stage – as was indexed by an earlier onset of the S-LRP.
From the results of the expression and the familiarity discrimination tasks it is obvious, that an interaction between facial expressions and familiarity was present in both directions. This contradicts the functional model of face recognition by Bruce and Young (1986) and a number of other findings (Young et al., 1986; Schweinberger, Klos, & Sommer et al., 1995a; Bobes et al., 2000). In addition, it contradicts previous findings which only found an asymmetrical interaction between facial identity / familiarity and the discrimination of facial expressions (Schweinberger & Soukup, 1998; Baudouin et al., 2002). However, previous results have also suggested an interaction between facial expressions and the discrimination of facial familiarity (Endo et al., 1992). These findings were extend in two important ways by the results of the present dissertation. Firstly, they show that a symmetrical interaction is observable under some circumstances, and secondly, with event-related potentials it was possible to observe, in which processing stage, in the information processing chain, the locus of confluence for the interaction of both processes might be represented. All previous studies which have examined the question of an interaction between facial familiarity and facial expressions have only used performance data (Young et al., 1986; Endo et al., 1992; Schweinberger & Soukup, 1998; Baudouin et al., 2000, 2000a) and, were therefore not able to answer such questions.
The interaction between facial familiarity and facial expressions was exclusively observed for personally familiar faces. No interaction emerged for famous or experimentally familiarized faces. A special characteristic for familiar faces is that they are familiarized through personal encounter. They often bear high personal importance. Therefore, personally familiar faces might induce a higher arousal level which may in return mediate an interaction between facial familiarity and facial expressions. This is supported by a slightly increased SCR amplitude to personally familiar faces when compared to unfamiliar ones in Experiment 3 (see also Herzmann et al, in press.). In previous studies it has been shown, that brain regions that are involved in face recognition can be modulated by affective and socially relevant information (Pizzagalli et al., 2002; Hariri et al., 2000; Morris et al., 1998). One might argue that no difference in affective information was present in the presented experiments, since both familiar and unfamiliar faces were always displayed with the same facial expressions. However, it might make a difference by whom an emotion is expressed. The social relevance might be higher when personally familiar faces display happiness. This, in return, may increase the level of arousal. Hence it is not surprising that an interaction between facial [page 118↓]familiarity and facial expressions was only observed for personally familiar faces displaying happiness.
Neural structures subserving arousal processes are widely distributed throughout the brain (Robbins, 2000). Therefore, the effects of arousal on cognitive processing may depend on the task and temporal properties of the processes involved. Thus it can be expected that the faster to be discriminated stimulus dimension shows an earlier functional locus of interaction when the slower dimension is called as task-relevant. In this case, the relevant information is available earlier. Accordingly, the slower to be discriminated stimulus dimension should show a later functional locus of interaction when the faster to be discriminated dimension is task-relevant. Experiments 2 and 7 can be helpful to explore this prediction. Because the stimulus material also has a strong influence on temporal properties, only tasks should be compared which used the same stimulus material. This was the case for Experiments 2 and 7. A comparison of both experiments revealed faster mean RTs for the familiarity discrimination task of Experiment 7 (M = 583) when compared to the expression discrimination task (M = 693; t(34) = 4.2, p < .01). According to the expectation, an interaction between facial familiarity and the discrimination of facial expressions should emerge for earlier processing stages because facial familiarity is processed faster when compared to facial expressions. Hence an effect on earlier processing stages might be possible. On the other hand, facial expression should affect a familiarity discrimination task on later processing stages because this information is processed slower. Indeed, the results of both experiments are in line with this prediction which is illustrated in Figure 37. For the expression discrimination task of Experiment 2 an interaction emerged between facial familiarity. ERP results suggested late perceptual processing stages as the possible functional locus of interaction. By using the same stimulus material Experiment 7 employed a familiarity discrimination task. Here, ERP results clearly pointed to the response selection stage as the functional locus. Although a different functional locus of interaction emerged between facial familiarity and facial expression in both experiments, it might be modulated by the same mechanism. Depending on the temporal properties of the available expression or familiarity information, the functional locus of interaction is found either for earlier or later processing stages.
An alternative explanation of why there was only an interaction between facial familiarity and facial expression for personally familiar faces, might concern the neural processing of faces with a different degree of familiarity. The neural processing might differ between personally familiar, famous, experimentally familiarized, and unfamiliar faces. Intracranial recordings (Seeck et al., 2001) revealed differences in neural processing between [page 119↓]personally familiar and unfamiliar faces within the right amygdala and the mesial and lateral temporal lobe. No differences were found for famous and unfamiliar faces at the same brain regions. This might be a reason why the interaction between the recognition of facial expression and facial familiarity only emerged for personally familiar faces but not for famous or experimentally familiarized faces. However, areas which subserve the representation of perceptual, and semantic facial information were not examined by Seeck et al. (2001). Differences between famous and unfamiliar faces would certainly be expected in these brain regions (Pfütze, Sommer, & Schweinberger, 2002; Schweinberger, Pfütze, & Sommer, 1995).
|Figure 37. Depending on the temporal properties of the task relevant processes, the functional locus of interaction between facial familiarity and facial expression can be expected on different processing stages; (S = stimulus, SE = structural encoding, ED = expression discrimination, FD = familiarity discrimination, RS = response selection, MP = motor preparation, R = response).|
It is worth noting, that selective attention processes play an important role in the emergence of an interaction between facial familiarity and facial expressions. The importance of selective attention is underlined by studies of Schweinberger and Soukup (1998) as well as Baudouin et al. (2002). Both studies used the Garner paradigm (see 1.2.3.), a paradigm to study selective attention for different stimulus dimensions. Results of both studies point to an asymmetric interaction between facial identity and the discrimination of facial expressions. [page 120↓]Hence, the ability to selectively attend to one or the other stimulus dimension or not may determine the emergence of an interaction. Using only unfamiliar faces can explain the observed asymmetrical interaction in the studies of Schweinberger and Soukup (1998) and Baudouin et al. (2002). As mentioned, external features like hairline or facial shape are more important for the discrimination of unfamiliar faces. The more familiar a faces becomes the more importance is gained by internal facial features. In both studies, it may have been easier for participants to selectively attend to facial identity in the identity discrimination task because decisions could exclusively rely on external facial features. Internal features, that are relevant for facial expression recognition, could have been ignored. In this case, for the identity discrimination task an interaction between both dimensions in the orthogonal condition is not expected. On the other hand, the discrimination of facial expressions has to rely on internal features. In this task it may be a lot more difficult to selectively attend to facial expression since there is more overlap with internal features that are also relevant for identity. Accordingly, it is much more likely to find an interaction between facial identity and the discrimination of facial expressions in the critical orthogonal condition. Results of Goshen-Gottstein and Ganel (2000) show that the differentiation between internal and external facial features is also relevant for an interaction between facial identity and the discrimination of gender. Both processes are assumed to be independent according to Bruce and Young’s model (1986). Their results strongly suggest that the reliance on internal features is important for an interaction to emerge between both processes. The authors employed a priming procedure within a gender discrimination task. Priming was only observed for faces with removed external features but not for whole faces including hair and hairline. Hence, in the previously mentioned condition, the reliance on internal features was implied and an interaction between both processes emerged. In the latter mentioned condition, the discrimination could have been easily relied upon external features and an interaction was absent.
Thus, if selective attention processes could exclusively explain an interaction, the lack of effects for the expression discrimination task in Experiment 6 using famous faces remains unresolved. It can be assumed, that internal facial features are important for the recognition of identity of famous faces. Hence an interaction of familiarity should have emerged for famous faces within the expression discrimination task. This was not the case in Experiment 6. Therefore, additional processes like arousal, due to increased personal importance, might subserve the interaction. The observed symmetrical interaction for personally familiar faces might have emerged, because it might be more difficult especially for personally familiar [page 121↓]faces to maintain selective attention to facial familiarity or facial expression. In the familiarity discrimination task of Experiment 7, internal facial features for personally familiar faces may have captured attention more efficiantly because of their importance. Selective attention might have been harder to keep, and an interaction between facial expression and the discrimination of familiar faces should emerge. This does not neccessarily hold true for unfamiliar faces, because discrimination of unfamiliar faces could rely on external features. In this case, an interaction is not expected. This notion is underlined by the data of Experiment 7.
To sum up, increased arousal due to personal familiarity could be the basis for an interaction between facial expression and facial familiarity. This is underlined by the interaction exclusively found for personally familiar faces, but not for famous or experimentally familiarized faces. Due to temporal properties of the processes involved, the locus of facilitation can differ between tasks. This was the case for the expression discrimination of Experiment 2 and the familiarity discrimination of Experiment 7. However, also selective attention processes and the gained importance of internal facial features for highly familiar faces can partly explain the observed interaction of both processes. Arousal due to personal importance in concert with increased reliance on internal facial features for highly familiar faces might explain the symmetrical interaction between facial familiarity and facial expression.
The present data helped to shed more light on the controversy of whether and under which circumstances an interaction between facial familiarity and facial expression can emerge. Previous studies (Schweinberger & Soukup, 1998; Baudouin et al., 2000, 2000a), which contradicted the functional model of Bruce and Young (1986), were extended by finding a sym-metrical facilitative interaction and by localizing this facilitation within the information processing system.
From the observed interaction, implications can be drawn for further research in this field. The observation, that an interaction only emerged for personally familiar faces but not for famous or experimentally familiarized ones underlines the importance for models of face recognition to account for the degree of familiarity. It is also evident from functional imaging studies, that familiar and unfamiliar faces act differently on the neuronal system. Differences have been found within fusiform and ventral occipital regions (Gobbini et al., 2000), within the right middle temporal gyrus and the prefrontal gyrus (Phillips et al., 1998), and the right amygdala (Sugiura et al., 2001; Seeck et al., 2001). The brain regions involved may be more [page 122↓]or less distributed depending on the degree of familiarity. Accordingly, differential assumptions can be drawn for different degrees of familiarity of faces which are presented. Especially for faces with a high degree of familiarity, additional processes like arousal or increased reliance on internal facial features might be relevant for hypotheses and results. The functional model of face recognition by Bruce and Young (1986) allows differential assumptions for familiar and unfamiliar faces, because FRUs are postulated for familiar faces only. However, the degree of familiarity is not taken into account. In addition, it postulates independence for facial expression recognition and facial familiarity. It was suggested, that the affective response and personal importance for faces might be the basis for an interaction between facial familiarity and facial expression. Taking this into account, the emergence of an interaction between both processes could depend on the degree of familiarity and hence on the strength of the linked functional processes (Figure 38).
|Figure 38. A proposed modification based on the functional model of face recognition by Bruce and Young (1986), including the modification that was suggested by Breen et al. (2000).|
As discussed above, an important prerequisite of an interaction might also be the temporal properties of the processes involved. Initially, it was expected to find an interaction between facial familiarity and facial expressions only in a hard condition, because expression discrimination was thought to be fast, and familiarity might only have a chance to interact with facial expression, when the expression discrimination is slow (Baudouin et al., 2000). Results showed that this manipulation was not necessary, since RT in Experiments 1 to 3 were fairly slow in general. However, in section 4.2. it was argued, that the temporal [page 123↓]properties of both processes may explain the pattern of results for both discrimination tasks of Experiments 2 and 7. Faster RTs were observed in the familiarity discrimination task. Accordingly, the effect of familiarity on the expression discrimination task emerged for an earlier functional processing stage when compared to the effect of facial expression for the discrimination of familiarity. Thus, temporal properties of the processes involved should be considered in experiments which try to elucidate an interaction between different processes.
In order to draw clear assumptions for experiments concerning the processing of facial familiarity and facial expression, the kind of emotional expression also has to be considered. It has been shown in many studies, that a unique system can not subserve facial expression recognition. Different brain regions are involved depending on the expression. The amygdala plays an important role for the recognition of fear (Adolphs, 2002), but is also activated for happy expressions (Whalen et al., 1998). Basal ganglia and the insula are important for the recogntion of disgust (Sprengelmeyer et al., 2003). Hence, an interaction between facial expression and facial familiarity might only emerge for some expressions but not for others. Finding only a facilitation for personally familiar faces displaying happiness within the expression discrimination task might be due to the different neurocognitive processing of the other expression of disgust.
Another important point which emerges from using different expressions might be the distortion of facial features. It has been argued, that happiness is the expression easiest to recognize (Leppänen et al., 2003; Ekman et al., 1972). Other expressions might bear more distortion for the configuration of facial features. Hence differential effects can be expected, depending on the kind of expression. This was the case for the familiarity discrimination task of Experiment 7 as was discussed in section 2.7.
In addition to the suggestions for further research and face recognition models which were drawn from the present results, there are still many open questions and implications for further experiments. The observed interaction between facial familiarity and expression for personally familiar faces was not always clearly cut, therefore, further experiments should be conducted with a better control of the stimulus material. By using different stimulus material, reduced variability in RT for the expression discrimination task might also increase the reliability of the P300 peak picking. Hence a significant difference for this component might emerge instead of a trend as was the case in Experiment 2. This would more strongly [page 124↓]underline the suggested interpretation of facilitated late perceptual processing stages within the expression discrimination task.
It would also be interesting to see whether an interaction between facial expression and familiarity might emerge for famous and unfamiliar faces within a familiarity discrimination task. Such an experiment was not conducted in the present dissertation, because of liminations in time, but would have been a useful completion to the experiments presented. It might be, that the interaction between facial expression and the discrimination of familiarity is not just due to arousal and personal importance. Possibly, the advantage of internal features for familiar faces, might acount for an interaction between facial expression and the discrimination of familiarity. Therefore an interaction might be possible for famous faces since the importance of internal facial features is high.
An interesting variation of the stimulus material might be the removal of external facial features. In this case, the task-relevant information has to be extracted from internal facial features and an interaction might emerge due to overlapping features that are also important for the irrelevant dimension (see Goshen-Gottstein & Ganel, 2000), thus, not allowing participants to rely on external feature information.
Another important next step might be to use different facial expressions e.g. for personally familiar faces. It might be, that a facilitation of the discrimination of facial expressions for personally familiar faces could also emerge for neutral, fearful, or angry faces. At least for fearful faces the amygdala is activated because it is important for affective face processing and arousal (Rolls, 1999). Hence an interaction of both processes could be observed for personally familiar faces displaying fear.
Contrary to the influencial functional model of face recognition by Bruce and Young (1986), results of the present dissertation showed that an interaction between facial familiarity and facial expression is observable under some circumstances. Most important for such an interaction is the degree of familiarity because it only emerged for personally familiar faces. Event-related potentials pointed to late perceptual processing stages which are facilitated by familiarity within an expression discrimination task. On the other hand, facial expression affected the response selection stage for a familiarity discrimination task. It was argued that the affective resonse to personally familiar faces could be the basis of an interaction between facial familiarity and facial expression. In addition, the kind of facial expression in concert with the increased importance of internal facial features for familiar faces might also be relevant for an interaction to emerge. Implications for models of face recognition and for [page 125↓]further research have been discussed. In summary, the present dissertation has elucidated upon important points concerning the functional interaction between facial expressions and facial familiarity. However, it also raised new questions and can be used as a basis for further research in this field.
|© Die inhaltliche Zusammenstellung und Aufmachung dieser Publikation sowie die elektronische Verarbeitung sind urheberrechtlich geschützt. Jede Verwertung, die nicht ausdrücklich vom Urheberrechtsgesetz zugelassen ist, bedarf der vorherigen Zustimmung. Das gilt insbesondere für die Vervielfältigung, die Bearbeitung und Einspeicherung und Verarbeitung in elektronische Systeme.|
|DiML DTD Version 3.0||Zertifizierter Dokumentenserver|
der Humboldt-Universität zu Berlin