Sensory Fusion Boosts Decision-Making Speed and Intelligence by Blending Vision and Audio
In a groundbreaking study, researchers from the Foxe's Cognitive Neurophysiology Lab at City College in New York City have uncovered the brain's intricate process of combining visual and auditory information for quicker and more accurate decisions in multisensory environments.
The research, led by first author John Egan from University College Dublin, was designed by Manuel Gomez-Ramirez to understand the integration of auditory, visual, and tactile inputs. The study involved two experiments with a total of 43 participants.
The findings reveal that the brain initially processes these sensory inputs separately in dedicated accumulators, before integrating their outputs in the motor system where the combined decision signal triggers a response. This integration allows reaction times to speed up and improves decision accuracy.
Using EEG recordings, the researchers observed that during tasks involving simultaneous visual (dot animation) and auditory (tones) changes, the auditory and visual decision processes develop in parallel within their own streams. However, these processes converge when their signals reach motor areas responsible for action initiation.
The team's computational modeling showed that a decision model where the sensory inputs integrate before triggering a motor response explains behaviour better than a "race" model where each sensory process independently competes. This is particularly true when one sensory signal is slightly delayed, demonstrating the brain's ability to synchronise multisensory information.
The study also found that auditory and visual evidence is accumulated in distinct processes during multisensory detection. Furthermore, the brain assigns different accumulators to auditory and visual inputs whose signals accumulate evidence separately but then merge downstream into a unified decision criterion.
The research, which was published in the journal Nature Human Behavior, highlights that this multisensory integration spans both cortical and subcortical brain regions to enhance perception, attention, and decision-making speed and precision. The weighting of sensory inputs during integration may depend on their reliability, consistent with frameworks like maximum likelihood estimation, where more reliable sensory cues influence the combined percept more strongly.
The study, conducted by a team that includes Simon Kelly, John Egan, Redmond O'Connell, and Manuel Gomez-Ramirez, answers long-standing questions about information integration and accumulation in multisensory conditions. The research was supported by the Science Foundation Ireland, Wellcome Trust, European Research Council Consolidator, Eunice Kennedy Shriver National Institute of Child Health and Human Development (UR-IDDRC), and the National Institute of Mental Health.
- Cognitive science, neuroscience, and health-and-wellness are interconnected fields that are now shedding new light on the brain's ability to process multiple sensory inputs simultaneously for quicker and more accurate decisions.
- The findings from a recent study by researchers in Manuel Gomez-Ramirez's lab at City College and colleagues from University College Dublin, published in Nature Human Behavior, indicate that the brain initially separates visual and auditory information before integrating it in motor areas responsible for action initiation.
- The research, which was supported by several organizations including the Science Foundation Ireland and the National Institute of Mental Health, suggests that the brain assigns different accumulators to auditory and visual inputs, whose signals accumulate evidence separately but then merge into a unified decision criterion.
- The study's findings have implications for mental health, as understanding the brain's multisensory integration process can offer potential insights into treating various cognitive disorders and improving overall fitness-and-exercise-related decision-making.
