[page 36↓]

2.  Part I: Does facial familiarity affect the discrimination of facial expression?

There is reason to believe that facial familiarity and facial expressions can interact under some circumstances. Although not assumed by the model of Bruce and Young (1986), an interaction is suggested by the model of the distributed neutral system for face perception (Haxby et al., 2000) via the interlinking of brain regions which subserve both functional processes. Throughout the research concerning this question, empirical evidence is given for the interaction of identity on the recognition of facial expressions. Schweinberger and Soukup (1998) used the Garner paradigm (Garner, 1976; see section 1.2.3.) to study the relationship between the perception of facial identity, facial expressions, and facial speech. They found the processing of facial identity to be uninfluenced by facial expressions and facial speech. In contrast, the discrimination of facial expressions was affected by identity. This hints to an asymmetric interaction between facial identity and the discrimination of facial expressions. By using the same paradigm, similar results were obtained in a study with schizophrenic patients by Baudouin, Martin, Tiberghien et al. (2002). Again, variations in the identity dimension influenced the discrimination of facial expressions in patients and in the control group. In addition, for schizophrenic patients the interference of identity on the expression discrimination task covaried with the severity of their negative symptoms. This indicates that their malfunction in processing facial affect is based on a deficit in selective attention towards facial expressions. In a behavioral study of Baudouin et al. (2000) participants had to decide if a presented unfamiliar or famous face showed either a happy or neutral expression. Hard and easy recognition of the expression was obtained by an uncovered or covered mouth in Experiment 1 or by varying the presentation times being either 15 ms or 400 ms in Experiment 2. In the covered-mouth condition, participants could more accurately discriminate expressions when the face belonged to a famous person when compared to an unfamiliar one. No difference between famous and unfamiliar faces was found for the easy, uncovered mouth condition. In the second Experiment faster RTs were found for famous faces when compared to unfamiliar ones. In addition, there was an interaction of familiarity and presentation time in such a way that differences between both types of face were only found for the short presentation time. The authors concluded that even if the processing of facial expressions is faster when compared to identity, familiarity improves the recognition of expression because it increases the “fluency” of the processing of a face, including the processing of its expression. Therefore, an effect of familiarity would only be found under [page 37↓]conditions where the processing of facial expressions is slowed down. Hence, facial familiarity can act facilitatively on the expression discrimination task.

Given that the above mentioned studies suffer from methodological problems (see section 1.2.3.) it was reasoned that more research is needed to clarify the controversy of whether there is an interaction between the perception of facial expressions and facial familiarity. Nonetheless, the results lead to the main hypothesis of the present experimental Part I. It is expected that facial familiarity facilitates the discrimination of facial expressions. A simple experimental design is used asking participants to discriminate between two different facial expressions whereas facial familiarity is varied independently. For the expression discrimination task it is expected that the discrimination of facial expressions is faster and more accurate for familiar when compared to unfamiliar faces. The discrimination of expression is a relatively fast process, therefore an effect of familiarity on this task might only emerge when the recognition of facial expressions is slow (e.g. slowed down by introducing a condition with low expressive intensity of the displayed expression). Lower error rates are expected for familiar when compared to unfamiliar faces. Error rates should also point to a facilitation of the task which is due to facial familiarity. In addition, parallel recorded ERPs should reflect the facilitation of familiarity within the expression discrimination task. By measuring peak- and onset-latencies it should be possible to localize the functional processing stage which is facilitated for familiar faces when compared to unfamiliar ones.

The following six experiments employ the expression discrimination task. Different stimulus sets were used in order to allow a differential degree of control of facial familiarity. Personally familiar and unfamiliar faces were used as stimuli in experiments 1 to 3. In the experiments 4 and 5 participants had to undergo a learning procedure in order to become familiarized with a set of unfamilir faces. In the subsequent test phase the same number of unfamiliar faces was added to the experimentally familiarized faces. In the final Experiment 6 only celebrities were used within the same experimental design as in experiments 1 to 3. To enable the comparison between familiar and unfamiliar faces one part of the stimulus set – british celebrities – were completely unfamiliar to the participants. The experimental Part I is closed by a final discussion of the results from the six experiments.


[page 38↓]

2.1.  Experiment 1

2.1.1. Rationale

In Experiment 1 participants had to perform an expression discrimination task. They were asked to discriminate the facial expressions happiness and disgust on personally familiar and unfamiliar faces as fast and accurately as possible. In addition, the displayed expressions were either weak or strong in intensity in order to vary the duration of expression processing. Thus, the expression discrimination was either performed under an easy (strong expressive intensity) or a hard condition (weak expressive intensity). Two groups participated in Experiment 1. Participants in the experimental group were personally familiar with half of the portrayed people. For the control group all portraits were unfamiliar. The main goal of this experiment was to show an interaction between personal familiarity and facial expression discrimination in the experimental group. In the same way it was intended to test this particular created stimulus set consisting of personally familiar and unfamiliar faces which were matched in age and gender, respectively. If an interaction between the two processes in question is found in the experimental group, it has to be proven that this effect is not based on the stimulus set per se because of mere increased expressiveness of the personally familiar faces. In order to safely ascribe a facilitation in the experimental group to personal familiarity a facilitation should be absent in the control group if the same task is used and stimuli are divided into ‘familiar’ and unfamiliar according to the experimental group.

It is expected that the variation of expressive intensity has a strong effect on RT in both experimental groups. Increased RTs should emerge for faces with weak expressiveness. For the experimental group it is hypothesized that personal familiarity facilitates the discrimination of facial expressions. If familiarity only has an effect on this discrimination when expression analysis is slowed down (Baudouin et al., 2000), faster RTs for personally familiar faces versus unfamiliar faces are only expected for faces with weak expressive intensitiy. Error percentage should follow the same pattern, being decreased for personally familiar faces. In the control group no effect of ‘familiarity’ nor an interaction with expressive intensity is expected when discriminating these faces into ‘familiar’ and unfamiliar ones according to the experimental group. Accordingly, no effect of ‘familiarity’ nor an interaction should emerge for error rates in this group as well.

2.1.2. Method

Participants. The experimental group consisted of twelve participants (8 women and 4 men, mean age = 26,7 years, aged between 20 and 35). Half of the presented portraits [page 39↓]displayed personally familiar faces to them. Another twelve participants (7 women and 5 men, mean age = 25,6 years, aged between 19 and 33 years) served as a control group. All presented portraits were unfamiliar for them. Participant in the experimental group received either course credit or payment. In the control group all participants received payment. Participants of both groups had normal, or corrected to normal, vision.

Stimuli and Apparatus. Colour pictures were taken from 16 staff members of the psychology department as personally familiar facial stimuli. Sixteen unfamiliar people were matched in age and gender to the personally familiar people. Each person (Figure 4) was photographed with three different expressions (happiness, disgust, and neutral) in three slightly different positions (frontal view and 10 degrees to the left and right, respectively) with two expressive intensities (weak and strong facial expressions). The eyes looked always into the camera. The mouth was always closed. All pictures were edited in Adobe Photoshop ® to 8-bit pictures with 256 colours and a horizontal and vertical resolution of 125 x 166 pixels. They were presented on a 17-inch screen with a size of 5.0 x 6.6 cm which equals a visual angle of 2.9o horizontal and 3.8o vertical at a viewing distance of 1 m. ERTS® served as experimental software for stimulus presentation and response recording.

Figure 4. Examples of portraits displaying one person with the expressions disgust (top row) and happiness (bottom row), three different head positions, and two different expressive intensities.

Design and Procedure. In a two-choice reaction time-task participants had to discriminate the two facial expressions happiness and disgust. In four consecutive blocks different portraits of 32 people were presented in a randomized order. As mentioned before, each person was presented on different pictures displaying happiness or disgust with weak or strong expressive intensity, as well as in three slightly different head positions. After each quarter of trials, the possibility of a participant-determined break was given. Each experimental trial (Figure 5) started with a blank screen followed by a fixation cross for 500 ms. A facial stimulus appeared for a maximum of 2000 ms on the screen. It was aborted by [page 40↓]the reaction made upon choice by the participant. Only in case of early (under 100 ms; “Zu früh”) or late responses (over 2000 ms; “Zu langsam”) was feedback provided immediately. A feedback balance about mean RT, error count and error rate was provided after each block of trials.

The key-to-expression assignment was changed after one half of trials for each participant. The start assignment was counterbalanced across participants. At the beginning and before changing the key assignment participants had to perform 40 practice trials in order to learn the correct assignment. They had to press the correct key according to the verbal description of the expressions (“Freude” or happiness vs. “Ekel” or disgust).

Figure 5. Trial scheme of Experiment 1 and all following experiments.

Data analysis. Statistical analyses of RT and error percentage was performed by means of Huyhn-Feldt corrected repeated measures ANOVA (as in all following experiments). In order to compare the results of the experimental and the control group a between-subject factor group was applied. The within-subject factors familiarity (familiar vs. unfamiliar), facial expression (happiness vs. disgust), and expressive intensity (weak vs. strong) were used. Hence, four different conditions arise for the weak expressive intensity : happiness and disgust for familiar and unfamiliar portraits, and for the strong expressive intensity respectively. For post-hoc comparisons of conditions t-tests were calculated. If more than one comparision was calculated Bonferroni-corrected significance levels were applied (Bortz, 1993).

2.1.3. Results

Reaction time. Figure 6 summarizes the mean RTs and error rate for Experiment 1. In both groups RT was considerably faster for strong when compared to weak expressions (F(1,22) = 220.4; p < .01; M = 832 vs. 936 ms). Most important, the expression discrimination was faster for familiar when compared to unfamiliar faces (F(1,22) = 17.9, p < .01). This effect of familiarity depended on the between-subject factor group (F(1,22) = 4.7; p < .05). It was only present in the experimental group (t(11) = -4.3, p = .001) but absent in the [page 41↓]control group (t(11) = -1.3, p > .05). In both groups a three-way interaction with the factors familiarity, expression and intensity was found. In the experimental group, post-hoc tests revealed an advantage for familiar faces when compared to unfamiliar faces only for portraits with strong happiness (754 vs. 841; t(11)=-5.6; p = .000). In the control group the discrimination for weak happiness was faster for ‘familiar’ than for unfamiliar faces (959 vs. 1013; t(11)= -2.8; p = 0.018).

Figure 6. Reaction time and error rates for the experimental and the control group of Experiment 1.

Error rates. Figure 6 displays the error rates of the experimental and the control group separated for familiarity and facial expression. According to RT results, both groups made less errors for strong when compared to weak expressions (F(1,22) = 139.0; p < .01). In both groups the discrimination of facial expressions was more accurate for familiar when compared to unfamiliar faces (F(1,22) = 24.3; p < .01). It showed a trend towards this effect being dependent upon the between-subject factor group (F(1,22) = 3.9, p < .06). A small but significant effect of ‘familiarity’ was also found for the controls (14.9% vs. 17.0%; F(1,11) = 5.6; p = .04). In both groups the expression influenced the effect of familiarity (Fs (1,11) > 21.1, ps < .01). Less errors were made expecially for happy familiar faces (ts(11) = -4.5, p = .001). In addition, the control group showed more accurate decisions for disgust when displayed on unfamiliar faces (t(11)2.8; p = .016).


[page 42↓]

2.1.4.  Discussion

As expected the discrimination of facial expressions was faster and more accurate for personally familiar faces in the present stimulus set. This main effect of familiarity is evident in the experimental group but not in the control group. Contrary to the expectation, this effect is most pronounced for personally familiar faces conveying strong happiness. Originally, an effect of familiarity was only expected for portraits with weak expressive intensity. If the processing of expression is faster than the processing of identity, an interaction between facial expressions and facial familiarity should only emerge when the expression discrimination is slow. Indeed, mean RT in the experimental group was rather slow (847 ms) and RTs for portraits with strong expressive intensity were not much faster (800 ms). Generally the task was a difficult one most probably due to the small size of the pictures and the closed mouth in all portraits. The relatively slow RT in the condition with strong expressive intensity might be the reason why the expected interaction of familiarity and expression emerged for portraits within this easier condition. On the other hand, increased RT variability due to slower response times in the condition with weak expressive intensity may have blurred the significance of a familiarity effect.

Unfortunately, the control group also showed a three-way interaction of familiarity, facial expression and expressive intensity for RTs. However, it only emerges for portraits with weak happiness, whereas, in the experimental group, an effect of familiarity was found for portraits with strong happiness. For strong happiness this effect is even absent in the control group when compared to the experimental group (t(22) = -4.8; p < .001). One might argue that the depiction of strong or weak happiness leaves the possibility of an incresed variety in expressive intensity, and therefore personally familiar people might have smiled stronger. Accordingly, the same familiarity effect which is present in the experimental group should also be found in the control group. This can safely be excluded. No main effect of ‘familiarity’ on RT is present in the control group. The three way interaction of facial familiarity, expression and expressive intensity shows a different pattern in both groups, respectively. In the experimental group there is a sizable familiarity effect for happy faces with strong expressive intensity, whereas, in the control group a smaller effect of ‘familiarity’ is found for happy faces with weak expressive intensity. Therefore, the facilitative effect of familiarity on expression discrimination cannot be attributed to differences in expressive intensity between familiar and unfamiliar faces per se but on the personal familiarity of a face.

In line with RT results the decision about facial expressions is more accurate for personally familiar faces in the experimental group. Regrettably, this was also the case for the [page 43↓]control group. However, the effect was much smaller. Alongside this problematic ‘familiarity’ effect in the control group there is also an inverse pattern for faces displaying disgust. The accuracy for ‘familiar’ faces is decreased in the control group but not in the experimental group. If the effect of familiarity is based on differences in expressive intensity between familiar and unfamiliar faces, the same effect should be present in the experimental group as well. In contrast, in this group no difference in error percentage within faces displaying disgust is found.

By showing a main RT-effect of personal familiarity in the experimental group, which is absent in the control group, it can be concluded that personal familiarity facilitates the discrimination of facial expressions. This counts especially for personally familiar faces displaying happiness. Although familiarity also affected RT and error percentage in the control group, the effect of familiarity is much more pronounced in the experimental group, and thus, not explainable by the stimulus set per se. In addition, the small effects in the control group emerged on different factor levels when compared to the experimental group. For this and the above outlined reasoning the stimulus set is proposed for use in the following experiments of the present part. As is always the case, precaution is needed for the interpretation of further results. However, it is challenging to use personally familiar faces as a stimulus set for the sake of high ecological validity but at the price of experimental control.

2.2. Experiment 2

2.2.1. Rationale

In Experiment 2 the same expression discrimination task was used as in Experiment 1. Participants had to discriminate between the two facial expressions happiness and disgust. The same stimulus set as approved in Experiment 1 was used. Again, the presented portraits displayed either a personally familiar or unfamiliar face with a weak or strong facial expression.

The main purpose of the present experiment was to replicate the facilitating effect of personal familiarity on the discrimination of facial expressions. This effect should be manifested in RT and error rates. In addition, the functional architecture of the underlying processes is assessed by means of ERPs. In order to pinpoint the functional locus of an interaction between facial familiarity and facial expressions different ERP components will be used as time and latency markers for the processes that are involved. Amplitude and [page 44↓]topographical distributions of the potentials can give a hint to the availlability of extracted and processed stimulus information.

Concerning RT und error rates, the same facilitating effect of personal familiarity on the discrimination of facial expressions is expected as in Experiment 1. This may not necessarily be the case for personally familiar faces with weak expressive intensity. Based on the results of the previous experiment, a facilitative effect of personal familiarity regarding the discrimination of facial expressions is expected, especially for faces displaying happiness.

Different possibilities arise for the functional locus of the interaction between personal familiarity and the discrimination of facial expressions. If personal familiarity facilitates the structural encoding of a face, the temporal properties for early perceptual processes - as indexed by the N170 peak latency (Rossion et al., 1999a) - should be reduced for familiar faces. Although worth mentioning, this possibility is rather unlikely because of a strong body of results showing that the peak latency of the N170 is not influenced by facial familiarity nor facial expressions (Eimer, 2000; Rossion et al., 1999a). If late perceptual processes and accordingly the perception of facial expressions are influenced, an effect should be found on the peak latency of the P300 component peaking earlier for personally familiar faces. In addition, no effect of familiarity should be present for the peak latency of the N170 component. In the case of a later functional locus, namely the response selection stage before hand selection, the S-LRP interval is expected to be shorter for familiar faces when compared to unfamiliar ones. In this case, the N170 and P300 latency should not be affected by familiarity. Otherwise, a functional locus on late perceptual processing stages cannot be excluded because of propagation of the advantage for personally familiar faces to the following stages. Effects might also be found on the LRP-R interval. If familiarity shortens the time needs for motor preparation beyond hand selection, a reduced LRP-R interval for personally familiar faces should be present in the data.

2.2.2. Method

Participants. 16 participants (11 women and 5 men; mean age = 24,6 years; aged between 20 and 32) took part in the experiment. They were personally familiar with half of the portrayed persons presented in the experiment. The participants fullfilled either course requirement or received a payment of 15 €. The mean handedness score (Oldfiled, 1971) was 78 (ranging from –82 to +100).

Design and Procedure. As in Experiment 1, participants had to discriminate the two facial expressions happiness and disgust. The same stimulus set was used as in the previous experiment. It was repeated three times with the exception that portraits of only 28 people [page 45↓]were presented, half of them being personally familiar to the participant. For each participant 14 most familiar persons out of 16 potentially familiar ones were selected, respectively. The trial sequence was the same as in Experiment 1. The response side-to-expression key assignment was changed three times. This was done to calculate an LRP for the whole experiment and for every repetition of the stimulus set, respectively. Participants performed 40 practice trials by pressing the correct button according to the term refering to the crucial expression (“Freude”, happiness or “Ekel”, disgust) presented on the screen at the beginning of the experiment and before changing the key assignment.

Electrophysiological recording. The EEG was recorded from 31 electrode sites (Figure 7) including IO1, IO2, LO1, LO2, Fp1, Fp2, Fz, F3, F4, F7, F8, FT9, FT10, Cz, C’3 and C’4 (4 cm to the left and right of Cz, respectively), T7, T8, Pz, P3, P4, P7, P8, P9, P10, PO9, PO10, O1, O2, Iz and the right mastoid (M2) according to the modified 10-20 International System (Pivik, Broughton, Coppola et al., 1993). Tin electodes were used with ECI Electro-GelTM electrolyt paste and placed with an electrode cap (Electro-Cap International Inc. ). Electrode impedance was kept below 5 kΩ. The electrodes above and below the left eye (Fp1 and IO1) and next to the outer canthus of each eye (LO1 and LO2) served for controling EOG artefacts. All electrodes were referenced to the left mastoid (M1). A low pass filter was set at 30 Hz. The electrophysiological signal was digitized with 250 Hz and recorded continously together with triggers that marked stimulus onset and reaction.

Figure 7. Recording positions of the scalp electrodes for recording the EEG.

Offline the recording was segmented into epochs of 2300 ms for response synchronized onsets starting 1800 ms before the response. The EEG-data were bandpass filtered with high and low cutoff frequencies set to 0.01 and 8 Hz, respectively. All trials with incorrect responses, with signal drifts of more than 120 µV within the recording epoch or with other EEG artefacts were discarded. Blink trials were corrected by the method described by [page 46↓]Elbert, Lutzenberger, Rockstroh, and Birbaumer (1985); if this was not possible, the trial was discarded. Stimulus synchronized epochs of 1200 ms length were generated by averaging the response synchronized epochs around a variable point representing the stimulus onset. It was calculated by subtracting the single trial RT from the response trigger for each epoch, respectively. An average reference was calculated disregarding the electrodes IO1, IO2, LO1 and, LO2. In order to calculate the ERPs, epochs were averaged according to the experimental conditions.

The LRP was derived by calculating the difference between the potentials contra- and ipsilateral to the responding hand at electrode sites C’3 and C’4 and averaged across hands (Coles, 1989). To assess possible influences of horizontal eye movements on the LRP, the lateralized horizontal EOG (LhEOG) was calculated for the electrode sites LO1 and LO2 in the same way as the LRP. This calculation was applied on response synchronized epochs as well as on stimulus synchronized ones.

Data analysis. Statistical analysis of RT and error rates were performed by means of Huyhn-Feldt corrected repeated measures ANOVAs, including the within-subject variables familiarity (personally familiar vs. unfamiliar), expressive intensity (weak vs. strong), and facial expression (happiness vs. disgust). As in the previous experiment Bonferroni-corrected significance levels were applied in the case of post-hoc comparisons by means of multiple t-Tests. In Experiment 1 it was hypothesized that personal familiarity can only act facilitatively on the discrimination of facial expressions when this process is slowed down, e.g. by weak expressive intensity. Surprisingly, a facilitation for personally familiar faces had been found within the strong expressive intensity condition. Therefore, an additional analysis of the RT data was applied based upon a median split. All trials were divided into trials with fast and slow RT according to the median of each condition and participant. A Huyhn-Feldt corrected repeated measures ANOVA was calculated including the factors RT- bin (fast RT trials vs. slow RT trials), familiarity, intensity, and facial expression.

For the event-related potentials a jackknifing based method (Miller, Patterson, & Ulrich, 1996) was applied to measure the onset-difference for any two conditions of the LRP and of the P300 peak latency. For the latter component the condition values at the electrode site Pz were determined. Because of the low signal-to-noise ratio of the LRP and sometimes indistinguishable P300-peaks within a single subject, signal quality was improved by averaging n times n-1 participants. The values of each new jackknifing-participant served as an estimate of variance for the participant which was left out. The t-values of the subsequent one-tailed t-Tests were adjusted by the value described by Miller et al. (1996). In order to [page 47↓]assess possible influences of lateralized eye movements on the LRP, mean amplitudes over an interval of 100 ms around the point where the S-LRP and LRP-R onset occurred were derived from the LhEOG. Mean amplitudes were analyzed by means of Huyhn-Feldt corrected repeated measures ANOVA including the experimental factors familiarity and expression. For analysing the peak amplitude of the N170 component, measures were derived from averaged event related potentials at the electrode site P10. An Huyhn-Feldt corrected repeated measures ANOVA was performed with the factors familiarity, expressive intensity , and expression.

Analysis of mean amplitudes was conducted with average referenced data on a subset of 28 electrodes (omitting electrodes LO1, LO2, IO1 and, IO2). ANOVAs were performed within intervals of 50 ms starting from 200 ms until 600 ms after stimulus onset. The within-subject factors electrode site, familiarity , and facial expression were used. In addition, Huyhn-Feldt corrected repeated measures ANOVAs for all intervalls were performed on vector-scaled data (McCarthy & Wood, 1985) in order to assure that found differences are ascribable to differences in the topographical distribution.

2.2.3. Results

Reaction time and error rate. Mean RTs and error rates for the different conditions of the expression discrimination task are displayed in Figure 8. Like in the previous experiment there was a strong effect of expressive intensity (F(1,15) = 159.8, p < .01) yielding faster RTs for portraits with strong when compared to weak intensity. However, there was no interaction of intensity with other factors. Participants showed also faster RTs on faces displaying disgust when compared to happiness (F(1,15) = 4.7, p < .05). Most important, RTs to familiar faces were faster than to unfamiliar faces (687 ms vs. 698 ms; F(1,15) = 9.9, p < .01). Again, this effect of familiarity is modulated by facial expression (F(1,15) = 5.2, p < .05) and only present in portraits with a happy expression (694 ms vs. 713 ms; t(15) = - 3.77, p < 0.01) but not for disgust.

An additional analysis was performed by splitting RTs according to the median per condition and participant. The 2-level factor RT- bin was included. The main effects of the factors familiarity, intensity, and expression as well as the interaction of familiarity and expression are not mentioned again, as they correspond to the afore presented analysis. In case of significant interactions with the factor RT- bin two post hoc ANOVAs were calculated for the separate bins, respectively. Most important, the speed of RT (factor RT- bin) significantly influenced the effect of familiarity in the expression discrimination task (F(1,15) = 12.5, p = .003). A facilitation for personally familiar over unfamiliar faces was only present for trials with slow RT (801 ms vs. 817 ms; F(1,15) = 11.8, p = .004), but not for trials with [page 48↓]fast RT (773 ms vs. 777 ms; F(1,15) = 2.2, p = .16). The advantage of disgust over happy expressions was affected by response speed (F(1,15) = 5.2, p = .038) and is only observable for slow RTs (797 ms vs. 821 ms; F(1,15) = 5.8, p = .03), but not for fast ones (p = .16).

Figure 8. Reaction time and error rates of the expression discrimination task of Experiment 2 separated for familiarity and facial expressions.

Participants made less errors on portraits showing personally familiar faces when compared to unfamiliar faces (9,4 % vs. 11,6 %; F(1,15) = 16.1, p < .01) as well as on faces with strong expressive intensity when compared to weak intensity (5,8 % vs. 15,3 %; F(1,15) = 210.0, p < .01). This effect depends strongly on facial expression (F(1,15) = 19.7, p < .01) and in addition on intensity also (F(1,15) = 15.5, p < .01). Whereas for happy faces there are more correct reactions for familiar portraits independent of intensity (7,4 % vs. 13,9 %; ts(15) = -4.1, ps < .01); only with faces displaying weak disgust did participants make slightly less errors for unfamiliar faces when compared to familiar ones (t(15) = 3.4, p < .05).

Event related potentials. Figure 9 displays the N170 component at the electrode site P10 for the different conditions peaking exactly at 170 ms after stimulus onset. It is obvious that there are no peak differences and only minor amplitude differences between the different conditions. For this reason I abandoned any statistical calculation of the peak latency of the N170 component. Concering the peak amplitude an ANOVA confirmed the supposition of the lack of any differences between conditions (F < 1).


[page 49↓]

Figure 9. The N170 component for the expression discrimination task of Experiment 2 at the electrode site P10 separated for familiarity and disgust.

Figure 10. The P300 component for the expression discrimination task of Experiment 2 at the electrode site Pz separated for familiarity and expression.

The P300 is most pronounced at the electrode site Pz (Figure 10). Based on the significant interaction of the factors familiarity and expression in RTs, I expected an earlier peak of the P300 component for familiar portraits when compared to unfamiliar ones only for the happy expression. The jackknifing based comparison of the conditions familiar happiness versus unfamiliar happiness at the electrode site Pz revealed a trend (tJ(15) = 1.6, p < .10) with a slightly earlier peak for happy familiar faces.

The onset of the S-LRP (Figure 11) within the expression discrimination task started earlier for personally familiar portraits when compared to unfamiliar ones (tJ(15) = 1.98, p < .05). This difference was most prominent for portraits with a happy expression (tJ(15) = 1.78, p < .05) whereas it was absent for faces showing disgust (tJ < 1).

No difference between conditions was found for the response-locked LRP (Figure 12).

An influence of the LhEOG on the S-LRP, and the LRP-R could be denied (ps > .10).


[page 50↓]

Figure 11. The stimulus-locked LRP for the expression discrimination task of Experiment 2 separated for familiarity and expression.

Figure 12. The response-locked LRP for the expression discrimination task of Experiment 2 separated for familiarity and expression.

Analysis of the mean amplitude distribution revealed significant interactions of electrode position by expression in all time intervals starting from 200 ms until 600 ms after stimulus onset (Fs(27,405) > 4.0, ps < .005; for all results see Appendix 6.1.). As can be seen in Figure 13, faces displaying disgut yielded higher amplitudes at parietal sites as well as lower amplitudes at temporal and occipital sites. In addition, in the same time intervals (200 [page 51↓]ms until 600 ms post-stimulus), the interactions of electrode position by familiarity was also significant (Fs(27,405) > 5.2, p < .001). Higher amplitudes were observed for personally familiar faces at centroparietal sites as well as lower amplitudes at parietal and occipital sites.

Vector scaled data revealed topographical differences between faces displaying happiness and disgust in the intervals from 200 ms to 350 ms after stimulus onset (Fs(27,405) > 3.7, ps < .008). In addition, topographical differences between personally familiar and unfamiliar faces were evident in the time intervals starting from 200 ms until 450 ms after stimulus onset and again in the time interval from 550 ms until 600 ms after stimulus onset (Fs(27,405) > 4.3, ps< .003).

Figure 13. Differences of the mean amplitude distribution between unfamiliar (UF) and familiar (F) faces (top row) and between happiness (H) and disgust (D; bottom row) for the expression discrimination task of Experiment 2 in all tested time intervalls; a grey shading equals a negative difference.

2.2.4. Discussion

Although numerically smaller than in Experiment 1, a facilitation was observed for familiar faces when participants discriminated facial expressions. It was especially pronounced when the face displayed a happy expression. A speed-accuracy tradeoff cannot explain the pattern of results because error rate was decreased for familiar faces and especially for happy familiar faces. As was already evident in Experiment 1 and replicated here, a facilitation of familiarity was only found for happy faces. Hence, happiness might have a special role when recognizing familiar facial expressions. Next to the neutral expression it is probably the most frequently encountered facial expression on familiar faces. This could be one reason why there is an advantage recognizing especially this expression on personally familiar faces. Results from Endo et al. (1992) point in a similar direction, although their results are not exactly compatible with these results. In an identity discrimination task participants were faster in classifying personally familiar faces, as familiar, with a neutral expression and famous faces, as familiar, with a happy expression. [page 52↓]The expression with which a person encounters the most might influence the representation of this persons face. In return, the access to the stored representation might be faster, the more similar the currently encountered face is to this representation. Hence, if personal familiarity was accessed faster for happy faces it could act facilitatively in the expression discrimination task. This might not hold true for faces displaying disgust.

Compared to the previous experiment the faciliative effect of personal familiarity on the discrimination of facial expressions was diminished. Possibly, the whole context of an electrophysiological experiment increased the vigilance and attention of the participants. Results of Baudouin et al. (2002) suggest, that selective attention is important for the independence of both processes. Increased selective attention might have diminished an interaction between facial familiarity and the discrimination of facial expression. Admittedly, in this experiment mean RT decreased by over 150 ms when compared to Experiment 1. An important pre-requisite of the main hypothesis was defused because of overall faster RTs. Hence, the impact of familiarity on the discrimination of facial expressions decreased. The additional analysis of RT by splitting all trails according to the median shed some light on the diminished effect. It was shown that the facilitative effect of personal familiarity on the discrimination of facial expression was most pronounced for trials with slow response speed. This was predicted by the initial hypothesis. Within the slower RT bin the expression discrimination lasted long enough that the processed information about personally familiar faces could act act as facilitative on the expression discrimination task.

An unexpected result was the decreased error percentage for unfamiliar faces when displaying disgust. Possibly, positive expressions (like a happy face) and negative ones (with a disgusted expression) act differently on the neurocognitive system. The detection of negative expressions, in order to decide quickly upon the approach and avoidance of a given situation, might be important for the survival of an organism. Indeed, the perception of disgust in the present experiment was faster when compared to happiness. Perceiving disgust involves structures that are also implicated in the evaluation of offensive stimuli (Phillips, Young, Senior et al., 1997). In this sense, unfamiliar faces may be perceived as more offensive than familiar faces and hence the recognition of disgust is more accurate for unfamiliar face.

Most importantly, but only based on a small effect, the facilitated perception of expression for personally familiar happy faces is reflected in the electrophysiological data. There was no effect of familiarity, nor an interaction with expression, on the peak latency of the N170 component leaving an early perceptual locus of the found interaction apart from [page 53↓]consideration. The same holds true for the LRP-R interval and concurrently for the preparation of the motor system beyond hand selection. In contrast, a shorter S-LRP interval was found for personally familiar faces when compared to unfamiliar ones. Corroborated by RT results, a significant difference within the S-LRP data was also found when comparing only personally familiar and unfamiliar faces displaying happiness. In addition, a trend was present for the P300 peak latency which was decreased for happy familiar faces when compared to happy unfamiliar faces. This observed trend for happy faces might have been propagated to the following response selection stage, because there a significant difference between happy familiar and unfamiliar faces, as reflected by the S-LRP, was found. Therefore, an effect of familiarity on the response selection stage is not safe to conclude anymore. Concerning the trend of the P300 peak latency, the relatively high RT variability may have broadened the peak of the P300 component making the peak detection less reliable. Thus, the effect of familiarity within happy faces was blurred to only a trend.

The analysis of the topographical distribution revealed both an early significant difference for facial familiarity and facial expression. There was no interaction found between those factors. The discriminative information on facial familiarity as well as on facial expressions seems to be available as early as 200 ms after stimulus onset. It may be possible that the information about facial familiarity can influence later processing stages that are relevant for the participant’s task, and this may cause the facilitative interaction between facial familiarity and the discrimination of happy facial expressions which was found for personally familiar faces in the present data. Had an interaction of facial familiarity and facial expression been present for the topographical distribution it could be expected that the facilitative effect for personally familiar happy faces might be subserved by slightly different brain regions as when faces display disgust. However, no such interaction was present. The validity of this supposition remains unclear because brain regions might be involved which do not affect the topographical distribution.

Overall, the results point towards an interaction between the perception of facial familiarity and the discrimination of facial expressions for personally familiar faces. As intended, the event related potentials elucidated the functional locus of interaction within the information processing stream more clearly. They point to late perceptual or response selection stages as the possible loci of interaction between personal familiarity and the discrimination of facial expressions. Based on the presented ERP data no safe decision can be made favouring one or the other functional locus of interaction. Thus, further data are necessary in order to clarify this unclear functional locus.


[page 54↓]

2.3. Experiment 3

2.3.1. Rationale

It was the intention of the present experiment to replicate the results of Experiment 2 with a slightly changed design in order to increase the rather small facilitative effect of familiarity on the discrimination of facial expressions. As discussed before, the numerically small effect of familiarity and the relatively high variability in RT in the previous experiment may be one reason for the somewhat unclear effect in the P300 peak latency. In addition, the present experiment intends to get better control of the degree of personal familiarity by recording the skin conductance response (SCR) and applying a questionaire about people being personally familiar.

Prior to the experimental blocks the SCR was recorded for all personally familiar and unfamiliar faces. Within face recognition the SCR is highly linked to emotional arousal processes, orientating response, and attention. Hence, it can give a clue to the individual significance and emotional value of a face (Tranel et al., 1985) or the familiarity (Breen et al., 2000). In healthy participants but also in a clinical context, it is often found that the SCR is more enhanced for famous faces when compared to unfamiliar ones. Accordingly, the expectation is to find this pattern for personally familiar and unfamiliar faces in the present experiment.

The previous experiments gave the impression that the personal contact to some members of the teaching staff who served as personally familiar faces was either too little or too long ago for some participants. Therefore, after recording the SCR but before the start of the experimental blocks, an additional block was added presenting all personally familiar people with a neutral facial expression and some personal background information about each (e.g. age, children, biographical data, chair, lectures or seminars). It was intended to refresh the memory for personally familiar people. Subsequently after this block the experimental blocks were started.

Again, it is expected that personal familiarity facilitates the discrimination of facial expressions expecially for happy portraits. Because no interaction had been found with the factor expressive intensity in the previous experiment this factor was omitted and only portraits with strong expressive intensity were used. Thus, the hypotheses do not consider expressive intensity anymore. For ERP data more clear cut results are expected than in the previous experiment because of the omission of portraits with weak expressiveness and thus [page 55↓]decreased variability in RT. As already implied by the trend concerning the P300 peak latency in the previous experiment it is expected that this latency should be reduced for personally familiar happy faces when compared to unfamiliar happy faces. Due to propagation, this facilitation may also be present in the onset latency of the S-LRP. No effects should be present for the N170 component as well as for the LRP-R. Hence, the expectation for the present experiment is to pinpoint late task relevant perceptual processing stages as indexed by the P300 as the functional locus of interaction between personal familiarity and the discrimination of facial expressions.

2.3.2. Method

Participants. In the third experiment 12 participants (10 women and 2 men; mean age = 25,8 years; aged between 21 and 54) took part. As in the previous experiments, they were personally familiar with half of the portrayed persons presented in the experiment. For each participant 14 of the most familiar people out of 16 potentially familiar ones were selected. Participants fullfilled either course requirement or received a payment of 15 €. The mean handedness score (Oldfiled, 1971) was 74,8 (ranging from –40 to +100).

Design and Procedure. The dedign and procedure differs from the previous experiments in that two blocks were added prior to the experimental blocks. At the beginning of the session, the SCR was recorded within one block. In order to keep the participants attention as well as to enhance the SCR, a familiarity discrimination task was required from the participants. They were instructed to view all presented portraits and discriminate personally familiar from unfamiliar faces by pressing one of two response keys on a computer keyboard with their index or middle finger of the dominant hand. Participants were told not to move the non-dominant hand where the SCR electrodes were affixed and which rested comfortably on a soft pad. After 3 warm-up trials at the beginning of the block showing unfamiliar people all 14 personally familiar faces as well as all 14 unfamiliar matches were presented successively and in random order with a neutral expression. The trial started with a fixation cross in the middle of the screen presented for a random duration between 11 and 12 seconds. In the last 5 seconds of the interval the fixation cross turned bold in order to capture the attention of the participant. It was replaced by a portrait of a familiar or unfamiliar person. All portraits remained on the screen for 2000 ms. Participants were supposed to make their familiarity decision within this time window. No feedback was provided.

After the SCR recording, the electrodes affixed to the hand were removed. Subsequently, participants had to perform a block to refresh their memory on the personally familiar people. All 14 familiar portraits were presented with a neutral expression together [page 56↓]with information concerning age, children, position at institute, given lectures and seminars, hobbies, and biographical information (e.g. ‘was a competitive ice skater in his youth’). Participants were told that after the experiment there would be a short questionaire in order to test the information. Participants could look at the information at leisure.

Following the block to refresh memory the experimental blocks for the EEG recording started. As in the previous experiments, participants had to discriminate the two facial expressions happiness and disgust. The same stimulus set was used as in Experiment 2 with the exception that all portraits with weak expressive intensity were omitted because of the previous lack of interaction of this factor with any other factor. The trial sequence was the same as in the previous experiments. The whole stimulus set was repeated twice. The response side-to-expression key assignment was changed twice in order to calculate an LRP. At the beginning of the experiment and before changing the key assignment, participants performed 40 practice trials by pressing the correct button according to the expression (“Freude”, happiness or “Ekel”, disgust) written on the screen.

Electrophysiological recording. The SCR was recorded with Ag/AgCl electrodes (1 cm in diameter) and isotonic electrolyte gel (K-Y Jelly, Johnson & JohnsonTM). They were affixed to the thenar and hypothenar of the non-dominant hand. The ground electrode was placed on the forearm of the respective hand. A CoulbournTM Isolated Skin Conductance Coupler (Model V71-23) was used. The sampling rate was 200 Hz. Offline the signal was cut in 8200 ms epoch starting from 200 ms before until 8000 ms after the stimulus and converted in micro-Siemens (µS).

The EEG was recorded from the same 31 electrode sites as in Experiment 2 (see section 2.2.2.). For recording, the same settings and materials were used as in the previous experiment. Offline the recording was segmented into epochs of 2300 ms for response synchronized onsets starting 1800 ms before the response. The EEG-data were bandpass filtered with high and low cutoff frequencies set to 0.01 and 8 Hz, respectively. The criteria and procedures for artefact removal and blink trial correction was kept the same as in the previous experiment. Stimulus synchronized epochs of 1200 ms length were generated by averaging the response synchronized epochs around a variable point as in Experiment 2. An average reference was calculated disregarding the electrodes IO1, IO2, LO1 and, LO2.

The LRP at electrode sites C’3 and C’4 and the LhEOG at electrode sites LO1 and LO2 were calculated with the same procedure as described above (cf. Coles, 1989).

Data analysis. For calculation of the SCR data only artefact free trials with correct familiarity decisions were included. The prestimulus baseline was set at 200 ms. A valid SCR [page 57↓]trial was counted if the maximum peak 1 to 7 sec after stimulus onset exceeded the criterion of 0.01 µS. If this was not the case the trial was counted as invalid and the value of the peak amplitude was set to zero. Different measures were calculated for familiar and unfamiliar faces. The magnitude was defined as the mean peak amplitude of all valid and invalid trials. For the peak amplitude and the peak latency only valid trials were included in the condition means. The frequency defined the percentage of valid responses. For each measure an ANOVA was performed with the two-level factor familiarity.

Statistical analyses of RT and error rate were performed by means of Huyhn-Feldt corrected repeated measures ANOVAs including the within-subject variables familiarity (personally familiar vs. unfamiliar) and facial expression (happiness vs. disgust). If post hoc comparisons were necessary, t-Test were calculated with Bonferroni corrected significance levels. Like in Experiment 2 all trials were split in two bins acording to the median for each condition and participant. Again, an ANOVA was performed with the additional two-level factor RT- bin (fast RT bin vs. slow RT bin).

As in the previous experiment a jackknifing based method (Miller et al., 1996) was used to measure the onset-difference for any two conditions of the LRP and of the P300 peak latency measured at the Pz electrode. As before, mean amplitudes were derived from the LhEOG for a 100 ms intervall in order to assess a possible influence on the LRP. For analysising the peak amplitudes of the N170 component measures were derived from averaged event related potentials at the electrode site P10. An ANOVA was performed with the factors familiarity and facial expression. Analysis of the mean amplitude distribution and the topography was performed the same way as in Experiment 2.

Figure 14. Skin conductance response as the mean of all valid trials for personally familiar and unfamiliar faces during a familiarity discrimination task.


[page 58↓]

2.3.3.  Results

Skin conductance response. Facial familiarity did not affect the peak latency (F < 1) and the frequency (p = .17) of the SCR (Figure 14). In contrast, a trend was found for peak amplitude (F(1,11) = 3.9, p = .07) which was increased for personally familiar faces when compared to unfamiliar ones (1.2 µS vs. 0.8 µS). This statistically weak effect was even reduced for the magnitude (1.5 µS vs. 0.6 µS; F(1,11) = 2.9, p = .12).

Reaction time and error rate. Figure 15 displays the RT and error rate of Experiment 3. There was a strong effect of familiarity (F(1,11) = 11.8, p < .01) for the expression discrimination task yielding faster RT for personally familiar faces when compared to unfamiliar faces (651 ms vs. 678 ms). Statistical analysis revealed a trend for the interaction of facial expression and facial familiarity (F(1,11) = 4.4, p = .061). Personal familiarity facilitated the discrimination of facial expression only within portraits displaying happiness (645 ms vs. 687 ms; t(11) = -7.1, p < .01). This does not hold true for portraits which expressed disgust (658 ms vs. 668 ms; t < 1).

Figure 15. Reaction time and error rates for the expression discrimination task of Experiment 3 separated for familiarity and expression.

Analysing the data after the median split revealed a differential effect of the factor familiarity for the bin which includes trials with slow RT when compared to the bin with fast RTs (F(1,11) = 9.6, p = .01). However, post-hoc ANOVAs, carried out for the two partitions seperately, showed facilitations for the expression discrimination task due to personal [page 59↓]familiarity not just for the slow trials (773 ms vs. 817 ms; F(1,11) = 13.6, p = .004), but also for the fast RT bin (529 ms vs. 538 ms; F(1,11) = 10.5, p = .008).

Error rates were lower for personally familiar faces when compared to unfamiliar faces (3.3% vs. 7.3%; F(1,11) = 31.6, p < 0.1). Corresponding to the RT results error rates are especially low for personally familiar portraits displaying a happy facial expression when compared to unfamiliar happy faces (3.1 % vs. 9.5 %; t(11) = -4.2, p < .01). Although showing the same direction of effect this does not hold true for personally familiar and unfamiliar faces displaying disgust (3.5 % vs. 5.1 %; t < 2).

Figure 16. The N170 component for the expression discrimination task of Experiment 3 at the electrode site P10 separated for familiarity and expression.

Event related potentials. Figure 16 displays the ERP amplitudes of the N170 component. It is most pronounced at the electrode site P10 peaking exactly at 170 ms. Statistical testing of the N170 at this electrode site confirmed, to all appearances, that there is no difference in peak latency nor in peak amplitude between the experimental conditions.

The P300 component (Figure 17) shows its maximum amplitude at the Pz electrode. According to the interaction of familiarity and facial expressions found in the present experiment, the difference between personally familiar and unfamiliar faces within the P300 peak latency was only tested for faces displaying happiness. Although numerically present (527 ms for personally familiar and 571 ms for unfamiliar happy faces) a t-Test based on jackknifing averages showed no significant difference in the P300 peak latency between personally familiar and unfamiliar happy faces. The mean amplitude of the P300 was measured at the Pz electrode. Statistical testing revealed higher amplitudes for personally familiar when compared to unfamiliar faces (9.7 µV vs. 9.8 µV; F(1,11) = 14.0, p < .01). In addition, higher amplitudes are present for faces displaying disgust when compared to the happy facial expression (9.8 µV vs. 8.9 µV; F (1,11) = 11.2, p < .01). No interaction was found between the factors familiarity and facial expression.


[page 60↓]

Figure 17. The P300 component for the expression discrimination task of Experiment 3 at the electrode site Pz separated for familiarity and expression.

According to the RT results of Experiment 3, onset differences in the S-LRP (Figure 18) between personally familiar and unfamiliar faces were only tested for faces displaying happiness. In contrast to the previous experiment no difference was found between these two types of faces within the expression discrimination task. As is evident in Figure 19, the latency between LRP-onset and overt response was of equal length for all experimental conditions. Based on statistical analysis, an influence of the LhEOG on the S-LRP, and the LRP-R could be denied (ps > .10).

Figure 18. The stimulus-locked LRP and LhEOG for the expression discrimination task of Experiment 3 separated for familiarity and expression.


[page 61↓]

Figure 19. The response-locked LRP and LhEOG for the expression discrimination task of Experiment 3 separated for familiarity and expression.

Statistical analysis of the mean amplitudes for the 8 different time intervals (for all results see Appendix 6.2.) revealed a significant effect of facial expression starting at 250 ms until 600 ms (Fs(27,297) > 4.1, ps < .002). As can be seen in Figure 20 more positivity for disgust is present at fronto-central and pareto-occipital sites. In addition, potentials for disgust are more negative at fronto-temporal sites. Amplitudes differ also significantly between personally familiar and unfamiliar faces starting at 250 until 600 ms (Fs(27,297) > 5.0, ps < .002). Additional calculation of the vector scaled data revealed topographical differences between facial expressions from 250 and 350 ms after stimulus onset (Fs(27,297) > 3.9, p < .003). Different topographies are also evident between familiar and unfamiliar faces in all intervalls from 250 ms until 600 ms poststimulus (Fs(27,297) > 3.5, ps < .01).

Figure 20. Differences of the mean amplitude distribution between unfamiliar (UF) and familiar faces (F; top row) and between happiness (H) and disgust (D; bottom row) for the expression discrimination task of Experiment 2.


[page 62↓]

2.3.4.  Discussion

Although only revealing a trend, SCR data showed an increased peak amplitude for personally familiar faces when compared to unfamiliar faces. The statistical power of the calculated SCR measures was very low because they relied only on a maximum of 14 trials per condition and on n=12 participants in this experiment. Therefore, the trend for the mean peak amplitude can be taken as highly suggestive. It indicates a larger autonomic response for personally familiar faces and hence increased affective information processing. A possible alternative explanation of an increased signal value only for personally familiar faces is hard to justify, because all faces in this block were defined as targets. The SCR results differentiated between personally familiar and unfamiliar faces. Therefore, the results served as a good control for the degree of familiarity. It can safely be concluded, that the degree was fairly high for the personally familiar stimulus set when compared to unfamiliar faces.

Replicating results of the RT and error rates from the previous two experiments, personal familiarity facilitated the discrimination of facial expressions in general and especially when the face displayed a happy expression. As was intended by the initial block for refreshing the memory, the facilitative effect of familiarity in RT increased when compared to the results of Experiment 2. This was confirmed by a repeated measures ANOVA including the two experiments as a between-subject factor (F(1,26) = 5.3, p < .05). In the present experiment the difference in RT between personally familiar and unfamiliar faces was 27 ms; in the previous experiment the facilitation for personally familiar faces was, although significant, only 11 ms. This enhancement of the familiarity effect between this and the previous experiment is even more pronounced for happy faces (42 ms difference in the present experiment and, 19 ms in Experiment 2). The increased effect in RT cannot be explained by the pre-exposure of the faces preceeding the experimental blocks. For recording the SCR both personally familiar and unfamiliar faces were presented with a neutral facial expression. Although personally familiar faces were presented again within the block to refresh the memory, only portraits with neutral facial expression were displayed which were not used in the expression discrimination task of the experimental blocks. In addition, it may be possible and can never be excluded for personally familiar faces that participants have met some of them in the last days or even hours.

As was already seen in the previous experiment, the facilitative effect of personal familiarity was expecially pronounced in trials with slow RT, but also present in fast trials. This is an almost expected outcome based on the initial hypothesis to find an interaction of facial familiarity on the discrimination of facial expressions especially when processing is [page 63↓]slow (Baudouin et al., 2000). In addition, finding a significant effect after the median split is more difficult because of the reduced statistical power based an a lower trial number. The robust significance underlines even more the genuine effect within the RT data. The facilitative effect of familiarity on the expression discrimination task also for fast trials might come along with the overall increased effect of familiarity in RT when compared to Experiment 2.

Surprisingly, the RT results were not corroborated by the electrophysiological data. Concerning the onset and peak latencies of the different event related components no differences were found between personally familiar and unfamiliar faces when collapsed over the two expressions, nor when considering only trials with a happy expression. Although the effect of personal familiarity on the discrimination of facial expressions was evident in RT and error rates and even enhanced when compared to Experiment 2 it was completely absent within all tested ERP components of the present experiment. This holds true to all appearances for the N170 component, the S-LRP and, the LRP-R interval. The only numerical difference was present for the P300 peak latency. Here the difference for personally familiar and unfamiliar faces was 44 ms which corresponds to the size of effect in the RT data. It can only be speculated why there was a lack of significance nor a trend. One reason could be the lower statistical power based on less participants (12 in the present Experiment vs. 16 in Experiment 2) and a reduced number of trials, because only portraits with strong expressive intensity were included. Omitting the trials with weak expressive intensity was in fact done to decrease the variability of RT and increase the quality of determining the peak and onset latency measures. In fact the standard deviation of the mean RT decreased only little from Experiment 2 (SD = 197.9) to the present experiment (SD = 187.7), despite using only trials with strong expressive intensity. Increasing the number of repetitions of the stimulus set has not been a useful alternative because of the growing perceptual familiarity with every repetition for unfamiliar faces. This, in fact, could decrease the already weak facilitative effect of familiarity on the discrimination of facial expressions.

Another difference between the present and the previous experiment is the later onset of topographical differences for facial expressions as well as independently for facial familiarity. In Experiment 2 different topographies for both factors started as early as 200 ms after stimulus onset. In the present experiment differential topographies were only evident from 250 ms poststimulus. The 50 ms displacement of the availability of information about facial expressions and familiarity may have disturbed the interaction of both processes.


[page 64↓]

In summary, the facilitative effect of personal familiarity on the discrimination of facial expressions was replicated for RT and error rates. By omitting the portraits with weak expressive intensity and adding a block to refresh the memory for personally familiar faces the numerically small effect in RT of the previous Experiment 2 was increased for the present experiment. Again, the facilitative effect of personal familiarity was especially pronounced for faces displaying a happy expression. It underlines again the special role a happy expression might have when recognizing the facial expression of personally familiar faces. Unexpected was the lack of effects within the ERP data. The slightly changed design, in order to increase the facilitative effect of personal familiarity on the discrimination of facial expressions and to decrease the variability in RT, only significantly affected the overt performance data. However, it failed to improve the ERP results and to clarify the functional locus of interaction between facial familiarity and the discrimination of facial expressions. One expanation for the unclear data may be the used stimulus set. It is always challenging to use personally familiar faces as stimuli because of high ecological validity. However, it is more difficult to control the influence of the stimulus set. A fully balanced design is hard to accomplish. In order to exclude a possible influence of the stimulus set, the following experiments use unfamiliar faces whereas particpants were familiarized with half of the presented persons.

2.4. Experiment 4

2.4.1. Rationale

In the previous experiments it has been shown that personal familiarity facilitates the discrimination of facial expressions especially for faces displaying happiness. ERP results suggested late perceptual processing stages – as indexed by the P300 peak latency – as the functional locus of interaction between both processes. However, due to high variability in RT, results have been unclear. The used stimulus set consisting of personally familiar and unfamiliar faces may have been one source of variance. It is the purpose of the present experiment to improve experimental control over the stimulus set by using only unfamiliar faces, whereas half of the stimulus set is experimentally familiarized within a learning block.

Although the degree of familiarity of experimentally familiarized faces is low when compared to personally familiar faces, an interaction could still emerge. This is conceivable when we consider that ERP results from Paller and coworkers (Paller, Bozic, Ranganath et al., 1999; Paller, Gonsalves, Grabowecky et al., 2000) found parietal ERP differences for visually familiarized faces when compared to newly presented faces. The authors ascribe this [page 65↓]difference to the retrieval of stored visual face information. Striking evidence comes from Rossion et al. (2003) who found lower PET activation for unfamiliar versus learned familiarized faces within the right lateral fusiform gyrus and the right inferior occipital gyrus. Interestingly, the differences are found in visual extrastriate brain regions that subserve both, the categorization of faces on an object level as well as the discrimination based on previous encounters. An overlapping set of brain regions seems to be involved in face detection, individual discrimination, and presemantic recognition (Rossion et al., 2003). The involvement may just occur over different time scales, which ERP data suggests (e.g. Bentin et al., 1996; Eimer, 2000; Schweinberger, Pickering, Burton, & Kaufmann 2002a). Considering the mentioned results, neuronal differences between familiarized and unfamiliar faces are evident and reflected in ERP data. In addition, it is possible that the access to familiar face representations is faster when triggered by typical familiar stimuli such as an often encountered facial expression on a familiar face. Evidence comes from findings of Jemel, Pisani, Calabria et al. (in press), and Schweinberger, Pickering, Jentzsch et al. (2002) who found increased priming effects for the same facial stimuli when compared to different primes of the same face. Therefore, the expression with which a person is familiarized within a learning block is varied in the present experiment.

Only behavioural measures are recorded in Experiment 4 since it is the first time that I use this stimulus material. The expectation of the present experiment is to find an interaction between facial familiarity and the discrimination of facial expressions for experimentally familiarized faces. Thus, one half of the unfamiliar people will be familiarized in an initial learning block. As mentioned before, the visual experience with different facial expressions of the persons to be familiarized is varied. Within the learning block one half of the persons will be presented with a neutral and a happy expression. The other half will be displayed with a neutral and an angry expression. For the expression discrimination task this means that each familiarized person was seen before with only one expression, but not the other. The condition with the expression that was not encountered for a familiarized person is of particular interest. Previous results of Baudouin et al. (2000; Experiment 3) suggest that the facial expression of a person can be discriminated more accurately when it was familiarized with a happy when compared to a neutral expression. In contrast, it might also be possible that the previous encounter of a specific emotional expression can facilitate the discrimination of facial expressions. In this case it is hypothesized that, depending on the expression that was presented during the learning block, the discrimination is facilitated for a familiarized person displaying this already encountered facial expression. It could also be, that general visual [page 66↓]familiarity of a face may facilitate the discrimination of facial expressions. In this case a facilitation should be present for all familiarized persons independent of the learned emotional expression.

2.4.2. Rating

In advance of the experiment a rating was conducted in order to select a stimulus set of 40 male and 40 female people out of 104 different unfamiliar people. For each person 3 pictures were available with either a neutral facial expression, happiness, or anger. Five participants (all female, mean age = 27.2 years, aged between 24 and 33) rated every portrait according to the six categories strong or weak happiness (“starke Freude”; “leichte Freude”), neutral (“Neutral”), strong or weak anger (“starker Ärger”; “leichter Ärger”), or none at all (“Keine von den angegebenen Kategorien”). Participants were instructed to use the last option mentioned sparingly. It did not belong to the rating scale but conveyed important information about the stimuli. Participants had as much time as they needed for rating the faces. Afterwards, the ratings were recoded on a scale ranging from -3 to + 3 and the percentage of correctly classified ratings was analyzed. People, whose portraits were less often correctly categorized were excluded, and 80 people (40 male) remained. They were divided into 4 subsets with 10 men and 10 women, respectively.

Table 1: Mean expression ratings of 4 subsets of unfamiliar faces.

 

anger

happiness

neutral

Subset 1

- 2.34

2.71

1.05

Subset 2

- 2.35

2.73

1.11

Subset 3

- 2.35

2.69

1.11

Subset 4

- 2.34

2.75

1.13

The decisive value was the mean rating of anger, because the variation was larger when compared to happy and neutral expressions. The latter two expressions were correctly categorized most often. The mean ratings for the three facial expressions happiness, anger, and neutral were comparable in each group (Table 1). That is, each group contained about an equal amount of people with high and low mean ratings of happiness or anger.

2.4.3. Method

Participants. Twelve participants (7 women and 5 men, mean age = 25.5 years, aged between 19 and 36) took part in the present experiment. All participants had normal or [page 67↓]corrected to normal vision. They received either course credit or 10,00 € for participation. Handedness was not determined because electrophysiological data were not recorded.

Stimuli and Apparatus. All portraits of the stimulus set consisting of 80 persons were edited in Adobe Photoshop to 256-colour pictures with a bluish grey background. For the initial learning phase, the size of the pictures was edited to 269 by 350 pixels. For the following experimental blocks they were resized to 190 by 247 pixels. This was done to present the pictures in about the same size as the portraits in Experiments 1 to 3. All stimuli were presented on a 17-inch screen with a size of 10.8 x 14.0 cm for the learning phase which equals a visual angle of 6.1o horizontal and 8.0o vertical at a viewing distance of 100 cm. For the experimental blocks the size of the stimuli (5.9 cm by 7.7 cm) yielded a visual angle of 3.4o by 4.4o at the same viewing distance. ERTS® served as experimental software for stimulus presentation and response recording.

Learning-procedure. All participants had to undergo a 1 hour training session in order to become familiar with 40 people (two subsets of 20 people). One half of the presented faces displayed male faces. The assignment for the participants, to respond to two out of four subsets of faces, was counterbalanced. That is, half of the participants were familiarized with set one and two, whereas the other half was familiarized with sets three and four. For each participant, and counterbalanced between the participants, one set of people were presented only with a neutral and happy expression, the other set with a neutral and angry expression. In an initial learning block the participants viewed 80 portraits of 40 people with a neutral as well as a happy expression for the first subset or an angry expression for the second subset. All portraits were presented for 5000 ms. After a blank screen of 500 ms, the next portrait appeared in randomized order. Participants were instructed to view all presented portraits thoroughly in order to recognize them in the following blocks. No response was required. After a short break, 3 testblocks with a matching-to-sample task were added. On each trial, participants viewed 2 faces presented next to each other for 2000 ms. Within this time they had to decide, which of the two faces belonged to the previously learned ones by pressing the corresponsing right or left button on a computer keyboard. Subsequently, only the correct face remained in its position for an additional 1500 ms either with a green frame, when the previous decision was correct, or with a red frame in case of an error or a missed response. Reaction times over 2000 ms were also counted as errors. Forty different people served as distractor faces and were used only in the learning phase. In each block every portrait was presented twice with a distractor face of the same and of different gender. The two faces always had the same facial expression. At the end of each block, participants received [page 68↓]feedback about mean RT, error count, and error percentage. After three test blocks, an error criterion of 5 % or lower had to be reached. If error percentage was above this criterion participants had to go on practising one, but at most three additional test blocks in order to meet this criterion. Overall, each person was viewed at least 14 times (52 s), whereas each single picture was viewed 7 times (26 s).

Design and data analysis. Following the learning phase, and a short break of about 10 minutes, the participants continued with the experimental blocks, that is, an expression discrimination task. The presented portraits comprised the 40 previously learned people as well as 40 completely unfamiliar people. Each person was represented with two portraits displaying either a happy or an angry expression. A fixation cross appeared in the middle of the screen for 500 ms. It was displaced by a target face displaying either a happy or angry expression. Participants had to discriminate the two expressions as fast and correct as possible by pressing the right or left key according to the key assignment. After pressing the button the screen went blank. After 1000 ms the next trial continued, while the order was randomized. If participants failed to respond within the maximum RT of 2000 ms, visual feedback for late response was provided (“Zu langsam!”, too slow). No feedback for wrong responses was provided because the stimulus set was presented twice and stimulus-to-reaction learning should be avoided. After one presentation of the whole set of portraits (divided into two blocks of trials) the key assignment changed. In order to prevent mistakes, it was practiced in a separate block at the beginning and before changing the key assignment. Participants responded to the stimulus words “Freude” (happiness) or “Ärger” (anger) by pressing the corresponding key. After each block, mean RT, error count and error percentage were provided as feedback.

Statistical analyses of RT and error rate were performed by means of Huyhn-Feldt corrected repeated measures ANOVA. The within-factors familiarity-type (familiar person with learned expression; familiar person with unlearned expression; unfamiliar person) and facial expression (happiness vs. anger) were used. If necessary, t-tests were calculated for post-hoc comparisons of conditions. If more than one comparision was necessary Bonferroni-corrected significance levels were applied.

2.4.4. Results and Discussion

Familiarization. Error rates for the three consecutive test-blocks of the learning procedure decreased from 6.6% over 3.4% for the second block to 2.7% for the last block. The decrease from the first to the second block was significant (t(11) = 5.5, p < .001).


[page 69↓]

Reaction time and error rates. Figure 21 summarizes RT and error rates for the expression discrimination task of Experiment 4. Faster RTs were expected for learned faces when compared to unfamiliar ones, especially when they were familiarized with a happy expression (Baudouin et al., 2000). As is obvious, portraits displaying happiness (506 ms) yielded faster RT (F(1,11)=11.9; p < 0.01) when compared to angry expressions (552 ms). Although not important for the main question of this dissertation, it has been found in previous studies that participants show faster RT to positive expressions of happiness when compared to negative expressions (Leppänen, Tenhunen, & Hietanen, in press; Crew, & Harrison, 1994; Kirita, & Endo, 1995; Hugdahl, Iversen, & Johnsen, 1993). It might be easier to recognize happiness, because less visual features may be necessary. Indeed, in the present stimulus set many happy faces display a salient smiling mouth. In contrast, the discrimination of angry faces may be more difficult. Already the ratings of the stimulus material conveyed more incorrect classifications for angry faces when compared to happy faces (t(4) = -5.6, p < .05). The main effect of facial expression in Experiment 2 with faster RTs for disgust when compared to happy faces does not stand against this interpretation. The closed mouth in the former stimulus set may have diminished the advantage for happy faces.

Figure 21. Reaction time and error rates for the expression discrimination task of Experiment 4 separated for familiarity and expression.

Unfortunately, the hypothesized effect of familiarity-type was not evident in a main effect nor an interaction (F < 1) with facial expression. For error percentage the data revealed [page 70↓]only a trend for facial expression (F(1,11)=3.4; p=0.09) with slightly less errors for the angry (4,6%) when compared to happy expressions (6,6%).

As can be summarized from the present results no interaction was found between familiarity and the discrimination of facial expressions when using learned familiarized and unfamiliar faces. This independency is even more underlined by the fact that participants were not faster in discriminating the expression when they had seen the very same expressive portrait of a familiarized person during the learning block. Perceptual familiarity per se does not affect the perception or discrimination of facial expressions at all. Hence, other processes might have been involved which subserved the interaction between facial familiarity and facial expression discrimination for personally familiar faces in the previous experiments. An obvious difference is the missing personal importance and semantic information of the present stimulus set when compared to personally familiar faces. The latter mentioned faces may be more implemented in the neural system when compared to experimentally familiarized faces. Thus, in the present experiment the chance of finding an interaction between facial familiarity and the discrimination of facial expressions may have been smaller.

In general the mean RT of the present experiment was fairly fast when compared to the experiments which used personally familiar faces (Experiments 1-3). This may be due to the stimulus material used where the happy expression is clearly shown with an open mouth on all happy portraits. Faster RT for the happy facial expression when compared to anger underlines this supposition. Hence, because of the easy and fast expression discrimination task a necessary prerequisite of a possible interaction between facial familiarity and the discrimination of facial expressions is missing (c.f. Baudouin et al., 2000). On the other hand, the salient feature of an open mouth for happy portraits may have led the participants to a strategy where very little information is extracted from the internal facial features that are necessary to perform the task. There is evidence that the discrimination of facial expressions and identity share little featural information (Calder et al., 2001). Still, there is also overlapping feature information which is relevant for the discrimination of both processes. When participants have to use all featural information which is available and in part also relevant for identity recognition during an expression discrimination task, it can be assumed that an interaction might emerge. Possibly, the already mentioned lower degree of familiarity, a less distributed neural representation of familiarized faces, and also the lack of semantic knowledge and emotional importance may be more relevant for the unconfirmed hypothesis. To test whether the fast RT and the relatively easy discrimination of this experiment may have [page 71↓]crossed a possible emerging of the expected interaction, a further experiment is conducted with a harder task in order to increase the mean RT for the expression discrimination.

2.5. Experiment 5

2.5.1. Rationale

As mentioned before, the mean RT of the previous experiment was very fast, presumably due to the stimulus material, which showed an open mouth as the most salient feature on portraits displaying happiness. In return, this may have led the participants to a strategy where most of the featural information of the internal face is ignored since it was irrelevant for the easy expression discrimination. In order to apply a harder condition and to increase the overall RT level different facial expressions were used within the expression discrimination tast of Experiment 5. Participants were asked to discriminate between neutral and angry facial expressions. If the lack of an interaction between familiarity type and discrimination of facial expressions is ascribable to the easy condition of the previous experiment, a facilitative effect of familiarity type for the discrimination of angry and neutral faces is expected in the present experiment.

2.5.2. Method

Participants. Twelve participants (9 women and 3 men, mean age = 24,8 years, aged between 17 and 37) took part in this experiment. They all had normal or corrected to normal vision. Participants received either course credit or 10,00 € for participation. As in the previous experiment handedness was not assessed, because electrophysiological data were not recorded.

Stimulus and apparatus. For Experiment 5 the same stimulus set was used as in the previous experiment. The size of the stimuli was kept for the initial learning phase and for the following experimental blocks. Again, all stimuli were presented on a 17-inch screen with the size of stimuli corresponding to the same visual angles as in Experiment 4. The viewing distance was kept constant at 100 cm. ERTS® served as experimental software for stimulus presentation and response recording.

Learning-procedure. Like in Experiment 4, participants had to undergo a 1 hour training session in order to become familiar with two sets of 20 people (half of them being male). In an initial learning block the participants viewed 80 portraits of 40 people with a happy expression and with a neutral or angry expression, respectively. All portraits were presented for 5000 ms in randomized order. The learning procedure including the test blocks [page 72↓]was the same as in the previous experiment with the exception that all people were familiarized with a happy expression. In addition, one set was familarized with a neutral facial expression. The other set was familiarized with anger. As before, the distractor faces in the matching-to-sample task displayed always the same facial expression.

Design and data analysis. Participants continued with the experimental blocks after a short break of 10 minutes. In the subsequent experimental blocks it was asked for an expression discrimination where neutral and angry faces were now displayed. Except for the expression of happiness which was replaced by a neutral expression, the experimental procedure and design were the same as in Experiment 4.

Statistical analyses of RT and error rate were calculated with the same constraints as in the previous experiment.

2.5.3. Results and Discussion

Familiarization. Error rates for the consecutive test blocks within the learning procedure decreased from 7.8% over 2.9% (t(11) = 3.9, p = .002) to 1.5% (t(11) = 4.2, p = .002).

Reaction time and error rates. The present experiment was conducted in order to exclude the possibility that the fairly fast RT, due to the easy discrimination in the previous experiment, might have been the reason for the lack of an interaction between facial familiarity and the discrimination of facial expression. As expected, mean RT was considerably slower in the present experiment when compared to the previous one (692 ms vs. 531 ms; t(11)=-4.3; p < 0.01). In Figure 22 it is evident, that portraits displaying an angry expression (639 ms) yielded faster RT (F(1,11)=34.4; p < 0.01) when compared to the neutral expression (729 ms). This result is in line with previous results showing that RT to expressive faces is faster than to neutral faces (Eastwood et al., 2001, Phillips et al., 1998). From an evolutionary perspective it is conceivable that it is important to recognize and react especially upon expressive faces since they may contain relevant information about an upcomming threat or a dangerous situation. There is a lot of evidence showing a more distributed neural network which is involved in the processing of expressive faces (Kesler/West et al., 2001; Sato et al., 2001). In addition, ERP results revealed that expressive faces are processed more rapidly within the brain (Eimer & Holmes, 2002).


[page 73↓]

Figure 22. Reaction time and error rates for the expression discrimination task of Experiment 5 separated for familiarity and expression.

Unfortunately, the expected effect of familiarity-type was not evident as a main effect nor an interaction (F < 1). No effect at all was present within error rates. Although the task of the present experiment was a lot harder when compared to the previous one, and response speed was fairly slow, no interaction emerged between the discrimination of facial expressions and facial familiarity. It can be supposed, that during the learning block, an extensive perceptual familiaritzation took place, which in return could have subserved an interaction. In contrast, the result of independency between the two processes are more underlined, because it is likely that participants had to use more featural information derived from the internal features in the fairly hard expression discrimination task of the present experiment. This information might even overlap with featural information which is relevant for perceiving identity.

From the present experiment it can be concluded that facial familiarity and the discrimination of facial expressions are independent processes for experimentally familiarized and unfamiliar faces. This also holds true in case of a harder task, which caused increased RT and possibly the increased use of information derived from internal facial features when compared to the easier task in the previous experiment. If an interaction between facial familiarity and the discrimination of facial expressions is not affected through perceptual familiarity, other features might be relevant for this effect. Possibly, semantic knowledge about a person may be sufficint to enhance the speed of expression discrimination. This can not be decided from these and previous results. Although a facilitative effect of familiarity [page 74↓]was evident in Experiments 1 to 3, personally familiar faces comprise both semantic knowledge and personal importance. Both stimulus properties may be sufficient to subserve an interaction. Hence, the following experiment tries to solve this question by using famous faces, since they convey a lot of semantic information but are not necessarily of personal importance.

2.6. Experiment 6

2.6.1. Rationale

It has been shown in experiments 1 to 3 that facial familiarity can interact with the discrimination of facial expressions for personally familiar and unfamiliar faces. Personal familiarity facilitated the discrimination of facial expressions, especially for happy faces. The previous Experiments 4 and 5 contrasted these reuslts by showing no facilitation of expression discrimination for familiarized faces. It has been argued that the degree of familiarity, including the lack of semantic knowledge and personal importance, may have been the most obvious reason for the hypothesis to fail. Therefore, the present experiment will use famous faces since they already convey a lot of semantic knowledge. If pure perceptual familiarization is not sufficient to evoke an interaction of facial familiarity, and the discrimination of facial expressions, the question is raised whether this also holds for famous faces. Because of their higher degree of familiarity, the representation within the brain might be more distributed. Hence, they are processed differently to some degree. The basis for this assumption come from results by Paller et al. (1999; 2000). Participants had to view 40 faces. Half of them were presented with a name, and semantic information was given by a spoken voice. Completely new faces were added in a subsequent recognition test. ERP data revealed posterior differences between unnamed and new faces, which was interpreted as the retrieval of visual information. Most important, named faces elicted an additional anterior positivity when compared to new faces, which could be linked to the retrieval of semantic information. Thus, a small amount of semantic knowledge can alter the processing of previously learned faces when compared to completely unfamiliar ones.

Although the general neural processing of personally familiar and famous faces might be the same, it is conceivable that there are differences between them, because of a lack of direct encounter as well as the associated emotional importance which is not given for famous faces. Intracranial neurophysiological recordings show differences between these two facial types in the mesial and lateral temporal lobe and in the right amygdala (Seek, Mainwaring, [page 75↓]Ives et al., 1993). These areas are linked to memory function as well as to emotional labeling (LeDoux, 1992), and might account for the differential processing of personally familiar faces on the one hand and famous faces on the other hand. Thus, similarities as well as differences in the processing of these face types are found. Therefore, the question is justifiable whether the observed interaction between facial familiarity and facial expressions for personally familiar faces also holds true when unfamiliar faces are contrasted with famous faces.

In addition, there are also differences in the processing of famous and unfamiliar faces. Already visual extrastriate areas are differently involved in the processing of both types of faces. Higher PET activation was found for unfamiliar versus familiar faces in the right fusiform gyrus and the right occipital inferior gyrus (George, Dolan, Fink et al., 1999; Gobbini, Leibenluft, Santiago et al., 2000). Evidence about differential processing of famous and unfamiliar faces within the brain also comes from ERP studies (Bentin, & Deouell, 2000; Eimer, 2000). Priming studies with parallel recorded ERPs suggest that these differences are already evident between 180 to 290 ms poststimulus (Begleiter, Porjesz, & Wang, 1995; Schweinberger, Pfütze, & Sommer, 1995). Schweinberger et al. (1995) proposed a face specific event-related component around 250 ms which emerges for repeated faces within a priming paradigm. For immediate repetitions this potential is higher in amplitude for famous faces when compared to unfamiliar ones. For unfamiliar faces the N250r component even disappeared with two to four intermediate items (Pfütze, Sommer, & Schweinberger, 2002). Converging results suggest that this potential “reflects the stimulus triggered access to stores facial representations in inferior temporal cortex as influenced by very recent visual experience” (Schweinberger et al., 2002, p 407). There are manifest processing differences between famous and unfamiliar faces which presumably lie in the availlability of a stored visual representation and of semantic knowledge. Referring to the purpose of this experiment, it is expected that these differences might also evoke a facilitation of the discrimination of facial expressions for famous faces.

For the present experiment a special stimulus set was created with german, international, and british celebrities. This experiment was planned in cooperation with J.Kaufmann at the University of Glasgow since the same experiment was to be conducted in Germany and Great Britain. The group of british celebrities was completely unfamiliar to the german participants. In a future experiment, to be conducted in Glasgow the german celebrities are thought to be completely unfamiliar to british participants. Hence, if an interaction between facial familiarity and the discrimination of facial expressions is found, this effect is expected to be reversed for the german and british celebrities in their respective [page 76↓]country’s. For the present dissertation only the german experiment, conducted by myself, is described here.

Because of practical constraints and the accessibility of pictures, neutral and happy expressions were used. In contrast to personally familiar faces famous faces are emotionally less important, but the degree of percepual familiarity is high and semantic knowledge is available. The results from Experiments 1 to 3 point towards late perceptual processing stages which might show a facilitation for personally familiar faces. Possibly, not emotional value or attachment, but semantic knowledge only and therewith a more widespread neural distribution within the brain is the necessary pre-requisit for a facilitative effect. If so, it seems reasonable to expect the same facilitative effect for famous faces. Like in Experiment 1 to 3 this facilitative interaction should be evident in behavioural as well as in ERP data. As suggested by Experiment 2, a possible facilitation of the expression discrimination for famous faces should be reflected in a shorter peak latency of the P300 for familiar german and international celebrities when compared to unfamiliar british ones. Neutral and happy portraits may have a different effect when compared to the observed interaction between facial familiarity and facial expressions for personally familiar faces displaying happiness and disgust. Because both expressions are probably the expressions most encountered, the facilitative effect might generalize to both facial expressions. Hence, a pure main effect of familiarity within performance and ERP data is expected.

2.6.2. Method

Participants. Sixteen participants (12 women and 4 men, mean age = 25,8 years, aged between 19 and 42 years) took part in the present experiment. They received either course credit or payment for participation. The mean handedness score (Oldfield, 1971) was +75,9 (ranging from +25 to +100). All participants had normal or corrected to normal vision.

Stimuli and Apparatus. Pictures of British, German and International celebrities with neutral and happy facial expressions were used. Since it was planned to conduct this study in Germany and Scotland as a cross national comparison of the familiarity effect, celebrities had to be famous either only in Germany, or Scottland, or in both countries. Frontal to three-quarter views were allowed. A set of 197 celebrities was collected, each of them was represented with two portraits showing a neutral and a happy expression. Pictures were edited in Adobe Photoshop to 8-bit black and white pictues with 170 by 216 pixels and a black backround. In the pre-rating, as well as in the experiment, they were presented on a 17 inch screen with a resolution of 800 x 600 pixels. At a viewing distance of 100 cm this corresponds to a size of 6,8 by 8,6 cm and a visual angle of 3,9o horizontal and 4,9 o vertical. ERTS® [page 77↓]served as experimental software for stimulus presentation and recording of behavioral responses.

Design and Procedure. In a two-choice reaction time task participants had to discriminate between neutral and happy facial expressions. In twelve consecutive blocks the whole stimulus set of 112 celebrities with 224 portraits, selected after a rating, was repeated three times. The possibility of participant determined breaks was given between blocks. Within each repetition of the stimulus set, the hand-to-key assignment was alternated once. At the beginning of the experiment and before every hand-to-key alternation participants performed a block of 24 practice trials. In each practice and experimental trial a fixation cross appeared in the middle of the screen for 1000 ms. Fixation was replaced by the stimulus which was presented for a maximum of 2000 ms or until the subject pressed the response key. For both, correct and incorrect responses, the next trial started after a fixed period of 1500 ms. In case of responses too early (under 100 ms) or too slow (above 2000 ms), feedback appeared for 800 ms after a blank screen of 500 ms. After the experimental blocks a familiarity rating was conducted. All 112 celebrities were presented to the participant with one picture at a time. The participants task was to rate the familiarity of each celebrity. No time limit was applied.

Electrophysiological recordings. The same experimental setup was used as in Experiments 2 and 3. The EEG was recorded from the same 31 electrode sites (see 2.2.2).

Offline the recording was cut into stimulus synchronized epochs of 2500 ms length (-500 ms prestimulus to 2000 ms poststimulus). Artefact treatment and ERP calculation were carried out with the same constraints as in Experiments 2 and 3. Response locked epochs were cut out of the stimulus locked epochs and averaged according to the point of time where the response occurred. An average reference was applied based on the same subset of 28 electrodes as in Experiments 2 and 3.

Data analysis. Statistical analysis of the behavioural data was performed by means of Huyhn-Feldt corrected repeated measures ANOVA including the within-variables familiarity (British, German, International) and expression (neutral vs. happy).

For peak latency measures of the P300 as well as onset latencies of the LRP, the jackknifing based method was used as outlined earlier. The F- and t-values of subsequent ANOVAs or two-tailed t-Tests were adjusted according to the method described by Miller et al. (1996), and by Ulrich and Miller (2001). Peak amplitude measures were derived from averaged event related potentials and ANOVAs were performed with the factors stimulus familiarity and expression. As in Experiments 2, and 3, mean amplitudes were derived from [page 78↓]the LhEOG for a 100 ms interval around the S-LRP, and LRP-R onsets. Huyhn-Feldt corrected repeated measures ANOVAs including the factors familiarity , and expression were calculated in order to assess possible effects on LRPs.

Topographical analysis was performed the same way as in Experiments 2 and 3. ANOVAs were calculated for amplitude measures and vector scaled data (McCarthy & Wood, 1985) for every 50 ms segment starting from 200 ms until 600 ms after stimulus onset. The within factors electrode site, familiarity , and expression were included. In case of significant interactions of electrode site by familiarity post hoc ANOVAs were calculated including any two levels of this factor.

2.6.3. Rating

Before the Experiment a rating was conducted in order to choose a subset of the stimuli with the celebrities best representing the neutral and happy expression as well as being most familiar in either one or in both countries. In Germany as well as in Glasgow ten participants (11 women; mean age = 24,7 years, ranging from 19 to 32 years) took part in the rating. Participants had to rate each picture first on familiarity and on facial expression immediately afterwards.

Familiarity was rated on a 4-point scale (ranging from 0 to 3) with the categories: unfamiliar (0; “unbekannt”), slightly familiar (1; “kommt mir bekannt vor”), familiar but don’t know the name (2; “bekannt, weiß was, aber nicht den Namen”) and, know the name (3; “namentlich bekannt”). For the latter category it was supposed that when the participant knows the celebrity by name also other semantic knowledge is available (Burton, & Bruce, 1992; Hay, Young, & Ellis, 1991).

Facial expression was rated on a 3-category scale (ranging from 1 to 3) with the categories: neutral (1), happy (2; “eher fröhlich”) and, very happy (3; “sehr fröhlich”). An additional category “0” or none at all (“weder noch”) was added for pictures that did not represent any of the two facial expressions, neutral or happy. All of the 394 pictures were presented in randomized order on a computer screen. No time limit was applied for any of the ratings. On each trial a picture was presented at the top of the screen together with the familiarity rating scale. After the participant pressed the corresponding key for the rating choice with a left hand finger, the scale disappeared and the face remained on the screen for 500 ms. Afterwards, the facial expression rating scale was added to the picture. Then, the participant had to rate the same picture on facial expression with a right hand finger on four different keys on the computer keyboard. Visual presentation was terminated after the second key press corresponding to the expression rating. The next trial started after a blank screen of [page 79↓]1000 ms duration. The disconnection of the response hands was done to prevent participants from mixing up the successions of ratings which were accomplished on one picture in a single trial.

Mean ratings for each picture were calculated for both countries together (except for familiarity ratings) as well as for each country separately. To select the pictures a sequential strategy was used. The familiarity ratings served as a first criterion. The rating scale ranging from 0 to 3 provided the range limit of the mean value. British and german celebrities were selected if the mean value of the familiarity ratings for each picture was at least 1.6 by his/her own fellow countrymen and at the most 0.9 in the other country. Secondly, the expression ratings were considered. Here, the ratings of all participants from both countries were combined. The possible range of the mean values varied between 1 and 3. The lowest category (0; none at all) did not belong to the scale and was omitted for the calculation of the mean value for the expression ratings. Neutral pictures were accepted if a mean value of 1.4 was not exceeded. Happy pictures ought to have a minimum mean value of 1.5. The category ‘none at all’ (0) served as a final criterion on the expression rating scale. A maximum of 12 entries of a single picture was allowed. As the selection procedure was applied to a single picture a celebrity was only selected if both pictures fell within these criterions.

According to the procedure, a total of 112 celebrities (34 british, 37 german and 41 international) were selected, each represented by a neutral and a happy picture, respectively. The mean values of the familiarity and expression ratings are summarized in Table 2, and 3.

Table 2. Mean familiarity ratings for the stimulus set consisting of famous faces of Experiment 6 separated for the ratings in Scotland and Germany.

 

Ratings in Scotland

Ratings in Germany

Type of familiarity

Brit.

Ger.

Int.

Brit.

Ger.

Int.

Neutral pictures

2.48

0.23

2.72

0.23

2.53

2.38

Happy pictures

2.46

0.21

2.68

0.22

2.51

2.31

Combined

2.47

0.22

2.70

0.23

2.52

2.35

As can be seen in Table 2 the mean values of the familiarity ratings for british celebrities in Scotland, for german celebrities in Germany, and for international celebrities in both countries did not differ much. In addition, the mean values of the expression ratings (Table 3) for the three different groups of celebrities (British, German, and International) have only minor deviations as well.


[page 80↓]

Table 3. Mean expression ratings for the stimulus set consisting of famous faces of Experiment 6 separated for the ratings in Scotland and Germany.

 

Ratings in Scotland

Ratings in Germany

Ratings combined

Type of familiarity

Brit.

Ger.

Int.

Brit.

Ger.

Int.

Brit.

Ger.

Int.

Neutral pictures

1.06

1.05

1.05

1.11

1.15

1.13

1.09

1.10

1.09

Happy pictures

2.31

2.20

2.27

2.26

2.38

2.35

2.28

2.29

2.31

2.6.4. Results

Reaction time and error rates. Figure 23 summarizes the RT and error rates for the expression discrimination task of Experiment 6. RT to happy faces was faster when compared to neutral faces (590 ms vs. 614 ms; F(1,15) = 4.5, p = .5). No other effect in RT was observed. Only the factor familiarity had an effect on error rates (F(2,30) = 4.7, p< .05). When compared to British and International celebrities, error rates in the expression discrimination task were lowest for German celebrities.

Event-related potentials. Figure 24 displays the N170 component at the electrode sites P10 for all conditions split up after familiarity and expression. As is evident from the figures the component peaks at exactly 170 ms after stimulus onset for all conditions. Statistical analysis of the peak amplitudes, and peak latencies revealed no difference between conditions.

Figure 23. Reaction time and error rate for the expression discrimination task of Experiment 6 separated for familiarity and expression.


[page 81↓]

Figure 24. The N170 component for the expression discrimination task of Experiment 6 at the electrode site P10 separated for familiarity and expression.

The P300 component averaged according to the familiarity by expression conditions is depicted in Figure 25. Calculation of the peak amplitude of the P300 component at the electrode site Pz revealed higher peak amplitudes for faces displaying happiness (M = 12.2 µV) when compared to the neutral expression (M = 10.9 µV; F(1,15) = 30.1, p < .001). There was no effect of facial familiarity (Fs < 1) nor an interaction of both factors (Fs < 1.6) for peak amplitude and peak latency of the P300 component.

Figure 25. The P300 component for the expression discrimination task of Experiment 6 at the electrode site Pz separated for familiarity and expression.

Figure 26 and 27 display the S-LRP and the LRP-R for all conditions, respectively. Both measures were not affected by horizontal eye movements, because there was no influence of experimental conditions on LhEOG (ps > .10). Despite small visible differences, [page 82↓]jackknife based averages revealed no effect of any of the two factors expression nor familiarity. The same holds true for the response locked LRP.

Figure 26. The stimulus-locked LRP for the expression discrimination task of Experiment 6 separated for familiarity and expression.

Figure 27. The response-locked LRP for the expression discrimination task of Experiment 6 separated for familiarity and expression.


[page 83↓]

Analysis of the mean amplitude distribution over the scalp revealed a significant interaction of expression by electrode site starting from 200 ms until 550 ms (Fs(27,405) > 5.5, ps < .001) after stimulus onset (for all results see Appendix 6.3.). However, vector scaled data revealed topographical differences in the time intervals from 200 until 350 ms post-stimulus for the happy versus neutral expressions but not for later time intervals. Concerning the different amplitude distribution between the two expressions, difference maps (Figure 28) point to parieto-occipital and frontal electrode sites as containing the highest amplitude differences. Starting later from 350 ms after stimulus onset, the amplitude distribution depends also on familiarity (Fs(54,810) > 2.8, ps < .006). Different amplitude distributions are evident between british and german celebrities in the timerange from 450 until 550 ms after stimulus onset (Fs(27,405)>5.0, ps < .001) as well as between british and international celebrities from 350 to 400 and from 500 until 600 ms post-stimulus (Fs(27,405) > 3.9, ps < .009). No topographical differences for familiarity was found when comparing the vector scaled data of the three groups of celebrities.

Figure 28. Differences of the mean amplitude distribution between pairs of the British (B), German (G), and International celebrities (I; top row), as well as between the neutral expression (N) and happiness (H; bottom row) for the expression discrimination task of Experiment 6 in all tested time intervalls; a grey shading equals a negative difference.

2.6.5. Discussion

No facilitative interaction was found for famous faces and the discrimination of facial expressions. This is in contrast to the RT results of Experiments 1 to 3, where a facilitation [page 84↓]was found for personally familiar faces displaying happiness. It was an unexpected result, since a facilitative effect for famous faces during an expression discrimination task was suggested by the results of other studies (Boudouin et al., 2000). The already mentioned differences between personally familiar and famous faces might possibly account for the absence of an interaction between facial familiarity and the discrimination of facial expressions in the present experiment. The most obvious differences between these faces are the lack of direct encounter and of associated emotional importance. The latter difference becomes obvious in studies using the SCR. Here, lower arousal levels are evident for famous faces when compared to personally familiar ones (Herzmann, Schweinberger, Sommer et al., in press; Tranel, Damasio, & Damasio, 1995). In addition, differences in the activation of the right amygdala were found for personally familiar and unfamiliar faces but not for famous and unfamiliar faces in studies using intracranial recordings (Seeck et al., 2001). Sugiura, Kawashira, Nakamura et al. (2001) presented corroborating results by showing differential PET activation of the amygdala, the hypothalamus, and the medial frontal gyrus for personally familiar and unfamiliar faces. These regions are linked to the behavioural significance of a recognized stimuli. It stands to reason that the lower emotional importance (or lower behavioural significance) of famous faces might explain the lack of an interaction between familiarity and the discrimination of facial expressions.

Another clue concerning the still unconfirmed hypothesis might be given by the different results concerning the amplitude and topographical distribution of the present and the previous experiments. For personally familiar versus unfamiliar faces (Experiment 2), topographical differences derived from vector scaled data were evident as early as 200 ms poststimulus. In contrast, in the present experiment different amplitude distributions for famous and unfamiliar faces started as late as 350 ms poststimulus. Most importantly, topographies remained the same for famous and unfamiliar faces in all tested time intervalls, because no significant effect was found for vector scaled data. Possibly, the discriminative information about familiarity within the present stimulus set is available later as for personally familiar and unfamiliar faces.

The only evident effect within the RT results were faster RTs for happy facial expressions when compared to neutral ones. That expressive faces are recognized faster within several paradigms is a finding often seen (e.g. Holmes, Vuilleumier, & Eimer, 2003; Läppänen et al., 2003). Other brain regions might be involved when recognizing facial expressions with emotional valence when compared to neutral faces (Kessler/West et al., 2001; Dolan, Fletcher, Morris et al., 1996). The topographical differences between the neutral [page 85↓]and happy portraits starting at 200 ms poststimulus in the present experiment corroborate these results. In addition, emotionally valenced faces may boost attention mostly through the involvement of e.g. the amygdala (Adolphs 2002; Allison, Puce, & McCarthy, 2000; Whalen, Rauch, Etcoff et al., 1998) and hence, faster RTs can be observed. Another hint of exceeded saliency or attention which is captured by the expressive happy faces when compared to the neutral expression, is the increased P300 peak amplitude for happy faces measured at the Pz electrode. This is in line with quite a few results showing increased P300 amplitudes for happy versus neutral faces (Carretie, & Iglesias, 1995; Marinkovic, & Halgren, 1998; Herrmann et al., 2002; Vanderplog, Brown, & March, 1987) or emotional arousing versus neutral scenes (Cuthbert, Schupp, Bradley et al., 2000). All these results suggest that happy faces are processed differently from neutral ones in the present expression discrimination task. As mentioned earlier, these differences might emerge because of higher saliency and increased attention to happy faces which might be based on differential involvement of brain areas that subserve these processes.

To summarize, the present experiment revealed no interaction between facial familiarity and the discrimination of facial expressions for famous and unfamiliar faces. This was an unexpected result and suggests that emotional importance may be a necessary prerequisite for an interaction of both processes. Since emotional importance cannot be premised for famous faces in this experiment, but surely for personally familiar faces in the previous section the just found independence of both processes might be explainable. In addition, differences in amplitude distribution between famous and unfamiliar faces started later than in Experiment 2 and 3. This might suggest that differential information about familiarity is available fairly late and therefore, familiarity cannot act as a facilitative for the discrimination of facial expressions as found for personally familiar faces.

2.7. Summary and Discussion of Part I

In Part I the question was raised, whether there is a facilitative interaction between facial familiarity and the discrimination of facial expressions. Recent data observed within different paradigms suggest this possibility (Schweinberger & Soukup, 1998; Baudouin et al., 2000, 2000a). They contrast the assumptions that can be drawn of the functional model of face recognition by Bruce and Young (1986). However, studies which found a facilitative interaction between facial familiarity and the discrimination of facial expressions only used performance data (Baudouin et al., 2000, 2000a; Schweinberger & Soukup, 1998). Thus, questions arise concerning the functional and neural processes that are the basis of such an [page 86↓]interaction. The experiments in Part I were conducted to replicate previous results with a simple two-choice RT paradigm. Participants had to discriminate between two facial expressions on successively presented portraits which depicted either familiar or unfamiliar faces. In addition, event-related potentials recorded in parallel should shed more light on the functional architecture of the processes involved.

The general hypothesis of the expression discrimination task expected that facial familiarity facilitates the discrimination of facial expressions especially when the discrimination of facial expressions is slowed down by a hard condition (c.f. Baudouin et al, 2000). To slow down the processing of facial expression recognition, the expressive intensity of the displayed expressions within the initial stimulus set (that is personally familiar vs. unfamiliar faces) was varied. Although normally the recognition of facial expession is very fast, an influence of familiarity might be possible in the hard condition and could facilitate the discrimination of facial expressions. Because expressive intensity did not affect an interaction between personal familiarity and the discrimination of facial expressions in an expected systematic way, this factor was not considered in the two other stimulus sets (experimentally familiarized vs. unfamiliar faces, and famous vs. unfamiliar faces).

In Experiment 1 participants had to discriminate the two facial expressions happiness and disgust on personally familiar and unfamiliar faces. The purpose was firstly to show a facilitation for personal familiarity on the discrimination of facial expressions within the stimulus set especially created for this experiment. Secondly, the possible explanation should be ruled out that the familiar faces used in the stimulus set were more expressive than the unfamiliar faces. This was accomplished by using an experimental group where all participants were personally familiar with half of the presented portraits. Here the facilitative effect of familiarity, mentioned above, was expected. In addition, in a control group - whose participants were unfamiliar with all presented portraits - no such effect was expected. The data of Experiment 1 suggested that the familiarity of a face facilitated the discrimination of facial expressions in general and especially when it displayed a happy expression. This effect was manifest in decreased RT and error rates for personally familiar faces when compared to unfamiliar ones. The variation of expressive intensity in order to slow down the perception of facial expressions and engender the facilitative effect of familiarity on this task, did not interact exclusively with familiarity. Contrary to the hypothesis, in the experimental group of Experiment 1 a strong facilitative effect of familiarity was found for happy faces with strong expressive intensity. Probably due to the mouth being closed in all portraits the task yielded fairly long RT in general. Therefore, the primary hypothesis of a facilitative familiarity effect [page 87↓]only when the discrimination of facial expressions takes long enough, may also account for the condition with strong expressive faces. On the other hand, higher error rates in the weak expressive intensity condition in concert with increased variability of RT may have blurred the facilitative effect of familiarity on the expression discrimination in the experimental group. Although there have been some problematic effects in the control group of Experiment 1, it was outlined that they do not stand against the hypotheses, because differential effects of familiarity are not present within specific conditions in both groups. For the experimental group the effect of personal familiarity for happy faces is most pronounced within the strong expressive intensity condition. In contrast, for the controls this effect is completely absent within the same condition. Hence, an explanation based on differences in expressive intensity between personally familiar and unfamiliar faces is excluded. Therefore, the conclusion is justifiable, that the facilitative effect is based on facial familiarity and not just on differences in expressive intensity between familiar and unfamiliar faces. Along these lines the stimulus set was appraised as a basically sound one to use in the following two experiments.

In Experiment 2 participants performed the same expression discrimination task as in Experiment 1. In addition, event-related potentials were recorded in order to assess the functional architecture of the underlying processes. Although smaller, the same facilitative effect of personal familiarity on the discrimination of facial expressions was found as in the preceeding experiment. Independent of expressive intensity, performance was facilitated for personally familiar faces. Again, this effect is almost completely ascribable to the facilitative effect of happy familiar faces.

In line with RT results, the facilitative effect of personal familiarity on the discrimination of a happy expression was also present in event-related potentials. For the interval between LRP-onset and response, which represents the duration of motoric processes, no effect of personal familiarity was found. In addition, the peak latency of the N170 component was the same for personally familiar and unfamiliar faces. Hence the confluence of both processes on early perceptual and on motoric processing stages beyond hand selection is unlikely, because familiarity did not influence these two measures mentioned last. In contrast, the interval between stimulus and LRP-onset – indicating pre-motor processes respective response selection – was shorter for personally familiar faces when compared to unfamiliar ones, when the task of the participants was to discriminate between facial expressions. Although the same facilitative effect is more visible and numerically even bigger in the P300 peak latency, which represents the time needs of late perceptual processing stages, ANOVA only revealed a trend. This may be due to the overall small effect in RT. In addition, [page 88↓]the fairly late peak latency of the P300 due to relatively slow RTs with high variability may have led to an increased variability of the elicitation point of the component. Hence the peak detection becomes less reliable through a broader apex. Due to less clear cut results concerning the P300 component, the response selection stage as an alternative facilitated cognitive process as indexed by the S-LRP results, cannot be ruled out. On the other hand, only if there is clearly no effect on the P300 peak latency could the response selection stage be safely adopted as the facilitated locus. Contrary, the numerical difference between happy familiar and unfamiliar faces is more pronounced within the P300 component when compared to the S-LRP interval. Therefore, the possibility of facilitated stimulus categorization time gains plausibility. Despite the reasons concerning the trend in the P300 peak latency and the numerically small effect in RT outlined above, I still consider late perceptual processes as the most likely functional process which is facilitated for personally familiar happy faces during the expression discrimination task.

In Experiment 3 it was attempted to decrease the RT variability by omitting the portraits with weak expressive intensity. In addition, a slightly changed design was introduced in order to have better control over stimulus familiarity. The SCR was recorded to personally familiar and unfamiliar faces. Preceeding the experimental blocks an additional block to refresh the memory was introduced. As expected, the facilitative effect of familiarity on RT was more pronounced than in Experiment 2. Unfortunately, no significant effect was observed in the ERP data although a numerical difference between happy familiar and unfamiliar faces was present for the P300 peak latency. Possibly, the reduced statistical power due to the omitted portraits with weak expressive intensity may have caused the lack of statistical significance.

The following Experiments 4 and 5 used only unfamiliar faces whereas half of the stimulus set was familiarized in a preceeding learning block. The advantage of using experimentally familiarized faces is to have a better control over familiarity and emotional expressiveness. In addition, the expression information which was encountered in the learning block was varied. Participants were familiarized with only one of the facial expressions which were presented in the following experimental blocks. Contrary to expectation no facilitative effect of familiarity on the discrimination of facial expressions was observed. There was also no advantage for the previously seen expression for familiarized persons.

To assess whether the lack of semantic knowledge might have been the reason for the unconfirmed hypothesis, Experiment 6 used a set of famous and unfamiliar faces. Again, no [page 89↓]facilitative interaction between familiarity and facial expression discrimination was present within behavioural data and ERPs.

Before trying to interpret the observed facilitative interaction between personal familiarity and the discrimination of facial expressions, I have to discuss the unexpected lack of effects in Experiment 3. With this experiment it was attempted to overcome the problem of high RT variability in Experiment 2 by omitting the portraits with weak expressive intensity and to get a better control over personal familiarity. Accordingly, an increased facilitative effect of personal familiarity on the discrimination of facial expressions was observed in Experiment 3. Unfortunately, this was only present in the performance data of the experiment. The effect was not reflected in ERPs at all. However, a numerical but not significant difference between happy familiar faces and unfamiliar ones was present for the P300 peak latency. As mentioned, the intention to lower the RT variability was not completely accomplished. In addition, by ommiting the portraits with weak expressive intensity the number of trials was diminished and hence statistical power also. Using more trials is not the best alternative, because unfamiliar faces will become more familiar with more representations. In order to increase the degrees of freedom, more participants would have been necessary. The disconfirmed hypothesis within ERP data of Experiment 3 suggests, that the facilitative interaction between familiarity and the discrimination of facial expressions is rather transient and weak.

In an attempt to explain the observed facilitative effect of familiarity in the ERP data of Experiment 2 (that are also present in the performance data of Experiments 1 and 3), I will return to the results of the second experiment. How could this facilitative effect of personal familiarity on the late perceptual processing stage within the expression discrimination task be explained? Baudouin et al. (2000a) claim that the familiarity of a face increases the «fluency of the processing» and therefore improves the recognition of its expression. This interpretation of their results is rather unclear as it does not refer to any functional processing stage which should be facilitated. Possibly, Baudouin et al. (2000) assume perceptual processing stages to be facilitated for familiar faces when expression processing is slowed down by a hard condition. However, a strong body of results shows that early percepual processing stages, as indexed by the N170 component, are unaffected by facial familiarity and facial expressions (Eimer & Holmes, 2002; Herrmann et al. 2002). The present results point to late perceptual processing stages as a possible locus of facilitation for personally familiar faces. On the other hand, motor processes beyond hand selection and also early perceptual processing stages as indexed by the N170 component can be excluded as being facilitated. It [page 90↓]was assumed, that the peak-latency of the P300 component represents the time needed for the perception of facial expressions. Although some results suggest, that the P300 latency is also influenced by response selection processes, it is only true when the task includes a response conflict (Leuthold & Sommer, 1998). This is not the case in the present task, making the latency to a pure perceptual measure for the task-relevant processing of facial expressions. Hence, based on present results, late perceptual processing stages are considered to be facilitated for personally familiar faces when participants discriminated facial expressions. An exception has been made in this sense, that it only holds true for the often seen facial expression of happiness. An explanation might be that, contrasting Bruce & Young (1986), the learned representation of a familiar face may depend on facial expressions that are encountered more often. For the stimulus set of Experiments 1 to 3 it can be assumed that personally familiar faces (lecturers of the University) are mostly encountered with neutral and happy expressions (when compared to disgust). Even if it is assumed, that the neutral expression is seen more often on personally familiar faces (Endo et al., 1992), happy expressions might share more overlapping features with a neutral expression than disgust. Hence representations might be more similar. In contrast, if it is not the characteristic of the stored representation of a face that might induce an interaction between facial familiarity and facial expressions, faster RT should also be present for familiar faces displaying disgust. This was clearly not the case. In their article Baudouin et al. (2000a) argue against the existence of expressive representations of familiar faces in the cognitive system. This is based on results with experimentally familiarized faces (Baudouin et al., 2000; Experiment 3). For faces that were familiarized with a happy in contrast to a neutral expression, the recognition of facial expressions independent of the presented expression (happy vs. neutral) was improved. This was not the case for faces which were familiarized with a neutral expression. However, recent data suggest, that at least the access to stored representations of familiar faces has an image specific influence. Increased priming effects are observed for same primes when compared to different portraits of the same person (Schweinberger et al., 2002; Jemel. Calabria, Delevenne et al., 2003). Admittedly, it can not clearly be decided by the data of Experiment 1 to 3, whether stored representations could affect an interaction between facial familiarity and the discrimination of facial expressions. The lack of an advantage within personally familiar faces displaying disgust indicates that stored representations of familiar faces may depend on the visual experience and hence, on the expression when a face is encountered. In this case, a learned representation of a familiar face might resemble the happy expression more and faster [page 91↓]access might be possible. In return, faster recognition of identity might help facilitating the discrimination of facial expressions through the vast cognitive system.

The results of Experiments 4, 5, and 6 using experimentally familiarized or famous and unfamiliar faces, stand against an explanation which considers a possible influence of representations of familiar faces that are dependent on often encountered expressions. For the expression discrimination task of Experiments 5, and 6 no facilitation was found for expressions that were already seen on experimentally familiarized faces. For famous faces (Experiment 6) it can be supposed, that they are often encountered with a neutral or happy expression. Hence, a facilitation for famous faces should have been observed. This was clearly not the case. Contrary to the experiments which used personally familiar faces the results of Experiment 4 to 6 do not suggest that possible expression dependent stored representations are relevant for an interaction between facial familiarity and the discrimination of facial expressions. For the experimentally familiarized faces which were used in Experiments 4, and 5 it is likely that FRU-like stored representations for experimentally familiarized faces were built, because recognition was very good as error rates in the test blocks of the learning session decreased to under 3%.

Another possible explanation might be, that personally familiar faces possess a greater emotional valence and personal importance than unfamiliar and famous faces. There have been many results showing that emotionally valenced pictures experience a faster processing by the brain than their neutral counterparts (Eger, Jedynak, Iwaki et al., 2003; Adolphs, 2002, Eimer & Holmes, 2002; Sato et al., 2001). During a passive watching task, Cuthbert et al. (2000) found an earlier positive slow wave starting around 200 ms after stimulus onset at midline electrodes for pleasant pictures when compared to unpleasant or neutral pictures. After 400 ms both types of emotionally arousing pictures evoked greater positivity than neutral pictures. Because pleasant pictures also yielded an increased skin conductance response it was suggested by Cuthbert et al. (2000) that they show earlier and increased affective arousal. Although the present experiments only used expressive faces it might be of additional importance if the expression is posed by an unfamiliar or familiar person. Hence personally familiar expressive faces may also evoke an earlier and increased affective arousal than unfamiliar faces. Indeed, in Experiment 2 and 3 they showed an enhanced positive slow wave starting at 200 ms after stimulus onset at centro-parietal and inferior parietal sites. This effect of personal familiarity is also evident in the amplitude and topographical distributions which diverge significantly beyond 200 ms after stimulus onset. In addition, a slightly enhanced SCR response was evident for personally familiar faces in Experiment 3. Through [page 92↓]the increased affective arousal of these faces the discrimination of facial expressions for personally familiar faces might be facilitated. Arousal may show a rather unspecific effect concerning the locus of interaction. In case of arousal modulation the affected locus might depend more on the temporal properties of involved processes and the task. By using simple visual and auditory stimuli of increased stimulus intensity results of Miller, Ulrich, and Rinkenauer(1999) pointed to shortened perceptual and pre-motoric processing stages, depending on the modality. Interestingly, no effect of stimulus intensity was observed on motor processes as indexed by the LRP-R. Motor processes were also unaffected in the present experiments, although the comparison between intensity of simple stimuli and an influence of arousal through complex stimuli like faces is questionable.

An effect of increased arousal may be modulated through the amygdala which is, inter alia, important for connecting faces to an emotional response (Rolls, 1999), and for directing attention to emotional valenced pictures (Krolak-Salmon et al., 2001; Vuilleumier, Armony, Driver et al., 2001; Breiter, Etcoff, & Whalen, 1996). It is important for learning conditioned responses and its rich connections are linked to brain areas which subserve the representation of reinforcer value - like the orbitofrontal cortex – or the response to emotional significant stimuli – like the anterior cingulate cortex (Cardinal, Parkinon, Hall et al., 2003). The amygdala was shown to be activated through facial expressions not only of fear but also of happiness (Whalen et al., 1998, Breiter et al., 1996). This shows, that the amygdala is also important for positively arousing pictures in general and for faces (Rolls, 1999). Thus it is not only relevant for pictures and faces with negative emotional valence or fear, as was thought previously (Morris, Friston, Buchel et al., 2001; Morris, DeBolis, & Dolan, 2001; Morris et al., 1998). Results of Gur, Schroeder, Turner et al. (2002) suggest, that the amygdala is not activated automatically but is dependent on the task. When using expressive and neutral faces, the amygdala was only activated during expression discrimination, but not during an age discrimination task. In contrast, Sugiura et al. (2001; see also Seeck et al., 2001) found the amygdala to be activated independently of the task during a familiar face detection task and a facial direction discrimination task for personally familiar faces when compared to unfamiliar faces. An expression discrimination task, or the posing of a facial expression per se may not be a prerequisite for amygdala activation. However, highly salient facial stimuli (as personally familiar faces are) may activate limbic structures and the amygdala. On the other hand, no activation of the amygdala was found for faces displaying disgust (Sprengelmeyer et al., 2003; Krolak-Salmon et al., 2001; Calder et al., 2001; Phillips et al., 1997). In addition, many results suggest, that the amygdala can modulate visual perception via back projections to the [page 93↓]visual cortices (Adolphs, 2002; Armaral et al., siehe Anderson ea; Vuilleumier et al., 2002). Also the superior temporal sulcus receives projections from the amygdala (Allison et al., 2000), an area which is also important for the perception of facial expressions (Haxby et al., 2000, Narumoto et al., 2001), and identity (Rolls, 1999). Referring to the present Experiments 1 to 3, an amygdala activation is very likely for personally familiar faces displaying happiness. This might not be true for disgust, because the amygdala is not involved in the perception of this expression (see above). Hence, if an increased saliency and arousal level may affect e.g. the superior temporal area via projections from the amygdala, the exclusively found facilitation for personally familiar happy faces within the expression discrimination task might be explainable.

Although the amygdala may also be activated by personal familiarity for faces displaying disgust this expression might act differently on the neurocognitive system. As mentioned earlier, the perception of disgust involves different brain areas excluding the amygdala when compared to other expressions. The basal ganglia and insula are highly linked to the perception of this expression (Calder et al., 2001; Sprengelmeyer et al., 2003). In the ERP data of Experiments 2 and 3 an increased positive slow wave was also found for faces displaying disgust, when compared to the happy expression. If it is assumed that for these faces the affective arousal is increased in general, an increased response speed should be evident. Indeed, independent of familiarity shorter RTs were found for faces displaying disgust in Experiment 2. As to the value of disgust in order to prevent physical harm it might be important to react to this expression on short notice independently of facial familiarity. Therefore, an exclusive facilitation for personally familiar faces displaying disgust may not be found.

In the experiments using famous or experimentally familiarized and unfamiliar faces, no facilitation was found for familiarity on the expression discrimination task. Certainly, the visual familiarity for famous or familiarized faces might be comparable to personally familiar faces. Thus, if visual familiarity with specific facial expressions would be the important factor for a facilitative interaction such an effect should have been observed in Experiments 4 to 6. This was clearly not the case. As already mentioned, the observed interaction between facial familiarity and the discrimination of facial expressions for personally familiar faces as well as its absence for famous and familiarized faces stand against the agument of expression dependent representations of familiar faces. The lack of personal encounter and most probably of personal importance of famous and experimentally familiarized faces might be more important reasons when trying to explain the present results. If, as it might be the case for [page 94↓]personally familiar faces, personal importance and hence, emotional arousal subserves an interaction via projections from the amygdala, the lack of an interaction for famous or experimentally familiarized faces is not surprising. In addition, when comparing the amplitude and topographical distributions for personally familiar or famous and unfamiliar faces, differences are evident concerning the effect of familiarity. Experiment 2 revealed differencial effects of personally familiar and unfamiliar faces for amplitudes and topographies as early as 200 ms poststimulus. This suggests, that the neural processing of personally familiar faces differs from the processing of unfamiliar faces. In Experiment 4, only different amplitude distributions emerged between famous and unfamiliar faces. In addition, they started as late as 300 ms poststimulus. No topographical difference was evident between both face types. The neural processing of famous and unfamiliar faces is probably more comparable. The observed late amplitude differences beyond 300 ms are possibly due to the recall of semantic knowledge about famous faces (Paller et al., 1999; 2000).

Based on the absence of a facilitative effect for famous and experimentally familiarized faces in Part I the question arises why an interaction was found elsewere (Baudouin et al., 2000, 2000a; Schweinberger and Soukup, 1998). The studies of Baudouin et al. (2000) and Schweinberger and Soukoup (1998) suffer from methodological problems. Concerning the asymmetric interaction between facial identity and facial expressions observed by Schweinberger and Soukup (1998), it has to be critically annotated that a small stimulus set was used. Facial expression and identity were only varied for two persons. It is possible that decreased RTs for the correlated condition when compared to the orthogonal one (see above) emerged only because expression discrimination could have relied on a picture based strategy. Due to different background shading or external facial features participants discrimination might have relied on these features instead on facial expression information (Kaufmann, 2002). Hence an interaction only emerged because both dimensions (facial expression and facial identity) were not varied independently as would be necessary for the Garner paradigm. In the study of Baudouin et al. (2000), an effect on RT was only found in Experiment 2. In the first experiment an interaction between facial familiarity and the discrimination of facial expressions emerged only for error rates. This shows that the interaction is transient and weak. In fact, in the easy condition of the expression discrimination task of Baudouin et al. (2000) famous faces were presented unconcealed and with a normal presentation time. Thus, it is comparable to the expression discrimination task of Experiment 6 which used also famous faces. As for the easy condition of Baudouin’s experiments, no facilitation of familiarity on the discrimination of facial expressions was [page 95↓]found. This suggests, that an interaction between facial familiarity and the discrimination of facial expressions emerges under special circumstances only, like slowed down expression processing, or the presentation of personally familiar faces.

In summary, evidence has been given suggesting a facilitative interaction between facial familiarity and the discrimination of facial expressions. This contrasts the functional model of face recognition by Bruce and Young (1986), as well as many results favouring the independence of both processes. Importantly, the facilitative effect was only observed for personally familiar faces displaying happiness, but not for famous or experimentally familiarized faces or disgust. Although other studies found an interaction for unfamiliar or famous faces, it has been argued above, that these studies suffer from methodological problems. The facilitation for personally familiar faces was consistently found in behavioural data. However, event related potentials gave a more inconsistent picture. The ERP results of Experiment 2 suggested late perceptual processing stages – as indexed by the P300 peak latency - as the functional process which is facilitated for personally familiar faces displaying happiness. Unfortunately, this result was not confirmed by Experiment 3, although a nummerical difference was present within the P300 peak latency. Different explanations of the results have been discussed. It was ruled out by the results of Experiments 4 to 6 using familiarized and famous faces, that a facilitative effect could be caused by expression dependent neural representations of familiar faces. More likely an increased arousal due to personal importance may explain the facilitation of expression discrimination for personally familiar faces. The amygdala is possibly involved in subserving this effect since it is relevant for gating autonomic and conditioned responses to arousing and important stimuli. It also possesses rich connections to brain areas that are important for representing reinforcer value, and it plays a role in responding to emotional significant stimuli.

Since an interaction between facial familiarity and the discrimination of facial expressions could be shown within the constraints above mentioned the question arises, as to whether this interaction does also hold for the opposite direction – an interaction between facial expressions and the discrimination of familiar faces. In the following Part II this question is addressed by reversing the task. Participants have to discriminate the presented faces according to facial familiarity, whereas facial expression is varied independently.


© Die inhaltliche Zusammenstellung und Aufmachung dieser Publikation sowie die elektronische Verarbeitung sind urheberrechtlich geschützt. Jede Verwertung, die nicht ausdrücklich vom Urheberrechtsgesetz zugelassen ist, bedarf der vorherigen Zustimmung. Das gilt insbesondere für die Vervielfältigung, die Bearbeitung und Einspeicherung und Verarbeitung in elektronische Systeme.
DiML DTD Version 3.0Zertifizierter Dokumentenserver
der Humboldt-Universität zu Berlin
HTML generated:
21.09.2004