Dissertations, Theses, and Capstone Projects

Date of Degree

9-2021

Document Type

Dissertation

Degree Name

Ph.D.

Program

Educational Psychology

Advisor

Patricia Brooks

Committee Members

Bruce Homer

Irina Sekerina

Linnea Ehri

Robin O’Leary

Subject Categories

Educational Psychology

Keywords

Reading, Eye Tracking, Comprehension

Abstract

With the growing trend of using multimedia platforms such as YouTube to facilitate storytelling, understanding how and when to integrate text with visuals would benefit both the creators of these platforms and the young readers viewing them. The current study examined the effect of orthography on vocabulary acquisition and narrative comprehension in young readers (children in 2nd and 3rd grade, ages 6-9) during a computer-based storytelling task. We aimed to determine if having text available during storytelling benefits readers as predicted by Perfetti’s Lexical Quality Hypothesis (Perfetti & Hart, 2002), or hampers learning as predicted by Mayer’s Redundancy Effect (Moreno & Mayer, 2002). The dissertation also explored if having access to the text would affect children’s eye gaze patterns while viewing the story, which had been found during the pilot study. It was hypothesized that the group who had access to their native English text would perform better in measures of comprehension and vocabulary knowledge and that they would show a different eye gaze pattern during the computer-based storytelling task than the group presented with Greek symbols in place of English.

Children completed a battery of pretests to assess reading level and working memory ability and then were randomly assigned to conditions where picture books displayed either the story text in English (meaningful condition) or in Greek symbols (meaningless text condition). They then watched a PowerPoint presentation featuring Mercer Mayer’s “Frog, where are you?” (Mayer, 1969) while having their eye gaze monitored using Tobii 4C eye tracker. After viewing the presentation, the children retold the story and answered ten comprehension questions, assessing both direct and inferential comprehension. They then completed a vocabulary knowledge scale that used words from the stories to assess vocabulary comprehension, as well as a cloze task created by the experimenter using target words from the story.

Results found that the presence of text did not affect comprehension or vocabulary acquisition, as both groups had similar scores across measures. However, the study found significant differences in eye gaze patterns between the groups. In both conditions participants spent significantly more time viewing the text during the beginning of the story and then reduced their dwell time during the middle and the end. Participants in both conditions also spent a higher percentage of dwell time on the illustrations during the beginning and middle of the story before reducing their focus at the end. However, when compared to participants exposed to the Greek symbols, participants with access to their native text spent significantly less time viewing the story pictures overall.

Results also found that for each page of the story, all participants focused on the text more before and during the playing of the recorded audio narration than after the narration was completed. An interaction between condition and the time stage of narration on dwell time suggested that narration affected participants differently depending on whether they had access to the meaningful text. Specifically, the participants with access to their native English text showed variation in their dwell times between the three different time stages: before, during, and after narration, which was not seen in the group who saw the Greek symbols.

While the study needs to be replicated to confirm results due to limitations caused by sample size and data loss in eye tracking, the present findings could help educators better integrate multimedia story telling within their classrooms by designing the activities to better match their students’ attentional and cognitive needs.

Download

Share

COinS