I Didn't Quite Hear That

There are lots of studies that show we need a very short time to look at a face and make an assessment. When we see faces with an expression of anger, fear, happiness, or just blank like our kids when we ask them where they are going, our eyes spend more time on a fearful face. Clearly this makes sense as maybe we too should be fearful.

Now we don’t just look, we also listen and that introduces an interesting question. What happens when we hear a snatch of sound that also depicts an emotion if it is, or is not in synch with the visual glimpse?  In this week’s Public Library of Science, Rigoulot and Pell from McGill U have reported on a study to clarify this for us (1).

They dragooned 34 24-year old men and women who happened to be wandering around the campus and needed some excitement to glimpse a series of photos with fearful, angry, happy or blank faces just after they heard actors speak words which didn’t carry any sense, but were spoken with true emotion.

When the audible emotions clues fitted the visual, not surprisingly, the response was enhanced. The interesting part of the experiment was to see what happens when the visual and audible clues are out of synch. The staring at the pictures was detected electronically using a video camera and the pictures were shown after the voice was heard.

When the audible and visual clues were in synch, fear then happiness grabbed the most attention, but when out of synch, the staring time was less except when a blank face was presented with an emotional utterance.

Conclusion? We pay attention to what we hear and if we hear something out of synch with what we see, we look elsewhere for the frightened or happy person. Unless we see a poker face with a happy or angry voice, and then we get suspicious. Perhaps those students have spent too many evenings at the card tables.

Leave a Reply