Dissertations, Theses, and Capstone Projects

Date of Degree

9-2023

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy

Program

Psychology

Advisor

Lucas Parra

Committee Members

Valerie Schafer

Ofer Tchernichovski

Edmund Lalor

Natalie Kacinik

Subject Categories

Cognition and Perception | Cognitive Neuroscience | Computational Neuroscience

Keywords

EEG, speech perception, noise

Abstract

Speech signals have strong and consistent effects on brain activity across individuals. Many previous studies have demonstrated a correlation between the amplitude envelope of ongoing speech and evoked responses measured through EEG or MEG. This correlation, also known as neural entrainment, appears to be modulated by attention, as well as other high-level factors, such as effort and motivation. The main hypothesis of this thesis is that these brain signals, alongside other bio-signals such as pupil size, can provide objective measures of listening effort and intelligibility in speech perception.

First, I examine neural entrainment in the case of minimally conscious patients, contrasting natural speech with reversed speech, which has identical frequency content and similar temporal dynamics, but is entirely devoid of semantic content. The basic question I ask is whether minimally conscious patients recognize speech. I find that neural entrainment varies between these conditions and also differs significantly from that of healthy participants for these stimuli.

Then, I examine whether modulation of neural entrainment may be due to changing dynamics of the stimuli or due to varying speech comprehension. I do this by modulating comprehension in healthy participants by varying the visuals in talking head type audiovisual stimuli. I find that neural entrainment does increase with comprehension, despite an identical auditory speech stimulus.

Some applications and refinement of the behavioral paradigm are discussed in the context of hearing aids and online experimentation.

Finally, I examine pupil size as a complementary measure alongside neural entrainment in studying effort and intelligibility. I hypothesize that neural entrainment and pupil size will co-vary but be differently influenced by intelligibility and effort. Instead, I found that the two can be separated and vary independently of each other, given the correct experimental design.

Because the ability to perceive speech is an integral part of everyday life, the degradation of this ability poses significant challenges to those living with hearing loss. Though interventions are available in the form of assistive hearing devices, these devices often come with costs in terms of fitting and adjustment that users find challenging. To that end, objective measures of a person's ability to perceive speech are desirable targets for engineering assistive devices such as hearing aids, as well as useful tools to understand the human speech perception ability. In this thesis, I will demonstrate novel experimental paradigms using noisy, naturalistic speech, that are able to capture behavior during perception with relatively high temporal resolution. I use these paradigms to demonstrate the ability to predict behavioral performance on speech perception tasks using EEG. Additionally, I demonstrate the effects of listening effort on pupil size and the interactions (or lack thereof) between the two.

In summary, these results demonstrate that objective measurement of psychological phenomena, such as speech intelligibility and listening effort, is possible using EEG, pupil size, and careful experimental design.

Download

This work is embargoed and will be available for download on Monday, September 30, 2030

Share

COinS