Most of our percepts in daily life come to us through multiple sensory modalities. When you drink a cup of coffee, you can see the coffee, smell it, taste it, and perhaps feel the heat through the mug or hear it when it is poured. All of these inputs affect your perception of the coffee. My research focuses on how the brain fuses these separate sensory inputs into a multisensory percept.
In studying sensory integration, I am particularly interested in audiovisual interactions. We use fMRI and behavioral experiments to look at how different stimulus properties, such as SNR, temporal synchronization, or emotional content impact the way in which your brain combines audio signals with visual signals. Through collaborations with Sunah Kim, I am involved in a number of projects investigating the combination of visual and haptic information as well.
In this work, we have been implementing an additive-factors approach that looks for interactions between sensory modalities when one or more of these stimulus properties are varied. This approach allows us to sidestep many of the issues that are associated with pure subtraction methods and more common criteria used to assess multisensory integration.
My research also includes investigating speech perception through an object-recognition framework. While speech is often treated as a unique phenomena, the vast majority of the neural processes involved can be explained through the same processes that underpin simple object recognition. Most of our research in this direction is also performed from a multisensory point of view, comparing how speech and object percepts are integrated.
Finally, I am also interested in theories of emotion, work that I have continued since working with Drs. Mikels and Reuter-Lorenz at University of Michigan. While most relevant theories are based on behavioral or peripheral physiological measures (such as skin conductance and heart rate), most do not utilize evidence from the brain itself. With the current ability to garner such information through neuroimaging, these theories should be modified to account for how the brain itself processes emotional stimuli.
