Our senses of sight and sound play a crucial role in how we perceive and interact with the world. Though often thought of as separate, audio and visual inputs are deeply interlinked in our brains. Synchronizing what we see with what we hear allows for a richer multi-sensory experience. Control systems help integrate these senses by linking auditory and visual pathways. In this blog, we will explore the role control systems play in audio visual integration.

 

What is Audio Visual Integration?

 

Before discussing control systems, it's important to define what is meant by audio visual integration. Simply put, audio visual integration refers to the merging of what we see and what we hear into a unified perception. When our eyes and ears receive input simultaneously, the brain combines these streams of information. This allows us to locate sounds in space, recognize speech and nonverbal cues, and experience harmony between different sensory modalities. Audio visual integration is a fundamental part of how we perceive music, movies, conversations and many other daily events.

 

Low-Level Integration in the Brain

 

Some initial integration of auditory and visual inputs occurs at low processing levels in the brain. Once signals from the eyes and ears reach the thalamus, nearby neurons begin combining these streams. The superior colliculus, a midbrain structure involved in orienting attention, contains neurons that respond to both visual and auditory stimuli. These early convergence zones lay the groundwork for more complex integration higher up in the cortical areas. Control systems play an important modulatory role even at these initial stages of integration.

 

Cortical Control Systems for Audio Visual Processing

 

Much of the advanced integration between what we see and hear takes place in cortical regions of the brain. Let's explore some of the key control systems involved at this level:

 

Temporoparietal Junction

 

The temporoparietal junction (TPJ) sits at the border of the temporal and parietal lobes. This region contains multisensory neurons that integrate auditory and visual inputs about the location and motion of stimuli. The TPJ helps pair sounds with their visual sources to determine which stimuli come from the same event. It plays an important role in audio visual spatial processing and maintaining synchrony between the senses. Control pathways projecting to the TPJ help coordinate its multisensory integration functions.

 

Prefrontal Cortex

 

Higher-order association areas in the prefrontal cortex (PFC) are involved in complex audio visual scene analysis and object recognition. The PFC contains multisensory neurons tuned to different audio visual pairings. It receives feedback from visual cortical areas like the inferior temporal cortex (ITC) and auditory cortical regions like Heschl's gyrus. Control systems in the PFC help selectively pull together and associate auditory and visual features into unified perceptions and memories of objects and events.

 

Superior Colliculus

 

As mentioned earlier, the midbrain superior colliculus is an important early integration site. But it also serves as a cortical control system, receiving feedback projections from higher regions. Connections from the PFC and TPJ allow for context-dependent modulation of multisensory processing in the superior colliculus depending on tasks and expectations. This top-down control helps sustain synchronized audio visual perception.

 

Neuromodulatory Control Systems

 

In addition to anatomical pathways, several neuromodulatory systems help regulate audio visual integration in the brain. Let's review two key examples:

 

Dopamine

 

Dopamine is involved in attention, perception and reward processing. Areas of the PFC and TPJ connected with audio visual integration contain dopamine receptors. Fluctuations in dopamine levels can impact the strength of connections between auditory and visual regions. This neuromodulatory control helps tune integration according to behavioral states and perceptual salience. Dopamine may also promote multisensory plasticity during development.

 

Acetylcholine

 

The cholinergic system is important for arousal, learning and plasticity. Acetylcholine enhances neural responses to multisensory stimuli in areas like the superior colliculus and PFC. It likely aids integration by modulating the signal-to-noise ratio in these regions. Acetylcholine release is associated with attentional shifts and stimulus prediction errors - control processes that influence multisensory perception moment by moment.

 

Role of Experience and Plasticity

 

Our lifelong audio visual experience also shapes control systems over integration through plasticity mechanisms. Repeated pairings of synchronized input strengthen connections between sensory areas over time. Crossmodal plasticity further allows compensate following sensory loss.

 

Development

 

Infants initially show segregated processing which matures into adult-like multisensory skills through natural sensory experiences. Early audio visual exposure strengthens convergent zones via Hebbian plasticity principles. Control pathways underlying integration also undergo activity-dependent development.

 

Learning and Skills

 

Musical training enhances audiovisual synchronization areas in adults. Similarly, learning a new skill like lip-reading recruits associated control regions. Plasticity enables expertise-dependent changes to integration networks throughout the lifespan.

 

Crossmodal Compensation

 

Loss of one sense leads remaining areas to take on new functions. For instance, visual cortex activation increases following deafness, likely to help lipreading. Control systems guide adaptive crossmodal reorganization to maintain efficient perception.

 

Conclusion

 

In summary, control systems at multiple levels help orchestrate the merging of auditory and visual information in the brain. Anatomical pathways in cortical regions and neuromodulatory influences fine-tune multisensory integration according to cognitive and perceptual demands. Plasticity mechanisms further shape these control systems through lifelong audio visual experiences. Understanding these regulatory processes provides insight into our remarkable ability for audio visual perceptual abilities.