Here is an excerpt from the IRB we submitted for review before beginning the experiment. The original document contains 20 pages of content. This section describes our justification for our experiment:
The McGurk effect, discovered by Harry McGurk and John MacDonald, is an audio illusion where visual information is powerful enough to cause a change in the perception of audio information. In the original study done by McGurk, different lip movements caused the perception of auditory stimuli to change. For example, if a person watched a video of a person mouthing the syllable “va”, but they played the syllable “ba”, they would hear “va” instead of “ba”. It was also recorded that sometimes the perceived sound was a fusion of both the auditory and the visual information given: playing the lip movements made for the syllable “ga” while playing the sound “ba” lead to the perception of the sound “da”. The McGurk effect is very curious because even after participants are informed about what the McGurk effect is, they still experience the same perceptual illusion.
Originally, the McGurk effect was evaluated using behavioral questions asking participants what syllable they heard. Another way to test for the McGurk effect is to use the ERP marker known as a Mismatch Negativity, or MMN. The MMN is an ERP component associated with higher order processing in the auditory cortex. The MMN has been successfully elicited in individuals from all ages, from newborns, to infants, to adults, and even in the womb. It is a well studied indicator of the auditory detection of changes in acoustic stimuli; in other words, it shows when the brain is detecting unusual changes in reliable, unchanging patterns. This type of stimuli uses the principle of the oddball paradigm, where in a sequence of the same stimuli there is is one deviant stimulus. For example, an MMN would be elicited to “va” in the following auditory stream: “ba, ba, ba, va, ba, ba,…” This is used to the benefit of studying the McGurk effect, since a deviant sound perceived to sound the same as the repeating syllable will not elicit the MMN even though the actual acoustic stimulus is different.
In the study done by Saint-Amour et al. (2006), the McGurk effect was used in conjunction with the MMN in the method outlined above. Participants listened to audio stimuli while watching lip movements while hooked up to an EEG cap recording the ERP information associated with each trial. In addition to looking at the ERP information the researchers used source analysis to determine from where in the brain these ERP components were being generated in order to give insight into the specific cortical areas that were associated with the tasks. This analysis allows a look into the systems needed to simultaneously process the visual and auditory information not only to stimulate the McGurk illusion, but also similar tasks like listening to music or participating in a conversation.
The goal of our study is to replicate the study done by Saint-Amour et al. using source analysis as well as their stimuli. We are interested in seeing if informing participants about the McGurk illusion will alter the strength of the MMN markers elicited during the experiment. We know that the effect is powerful enough to continue being perceived even after explanation, but we are curious as to how this knowledge might alter the MMN we record. Half of the participants will read an explanation on the McGurk effect (experimental group) while the other half will read a description of molecular gastronomy (control group). Both the MMN and the source analysis of each group will be compared to see if there is a significant difference between the two. We predict that individuals that read the McGurk illusion explanation will have a weaker MMN response than individuals who do not read the explanation of the illusion they are being presented with.
References
Kislyuk, D.S., Möttönen, R., and Sams, M. (2008). Visual processing affects the neural basis of auditory discrimination. Journal of Cognitive Neuroscience, 20, 2175-2184
McGurk, H., and MacDonald, J.(1976). Hearing lips and seeing voices. Nature, 264, 746-748.
Näätänen, R., and Kreegipuu, K. (2011). The Mismatch Negativity (MMN). In S. Luck & E. Kappenman (Eds.), The Oxford Handbook of Event-Related Potential Components. Oxford University Press.
Rosenblum, L., Schmuckler, M.A., and Johnson, J.A. (1997). The McGurk Effect in infants. Perception and Psychophysics, 59, 347-357.
Saint-Amour, D., De Sanctis, P., Molholm, S., Ritter, W., and Foxe, J. (2006). Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion. Neuropsychologia, 45, 587-597.
Skipper, J.I., van Wassenhove, V., Nusbaum, H.C., and Small, S.L. (2007). Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17, 2387-2399.
Yu, X., Lui, T., and Gao, D. (2015). The mismatch negativity: An indicator of perception of regularities in music. Behavioral Neurology, 2015, 1-12.
